AI and the ruler

I've seen this cautionary tale about putting too much faith in AI referred to in several places. It involves an AI program that had seemed to have "reached a level of accuracy comparable to human dermatologists at diagnosing malignant skin lesions." tells the rest:

However, a closer examination of the model’s saliency methods revealed that the single most influential thing this model was looking for in a picture of someone’s skin was the presence of a ruler. Because medical images of cancerous lesions include a ruler for scale, the model learned to identify the presence of a ruler as a marker of malignancy, because that’s much easier than telling the difference between different kinds of lesions.

I tracked down the original source of this story to an Oct 2018 article in the
Journal of Investigative Dermatology: "Automated Classification of Skin Lesions: From Pixels to Practice":

Dermatology images are the easiest to capture of all medical images, but also the least standardized. Standardization of images is difficult, even with dermoscopic images, as shown in Figure 1. Variability must be incorporated into training algorithms to create capacity to handle noisy data. This includes multiple camera angles, different orientations, blurry photos, multiple skin backgrounds, pen markings or rulers included in the photo, or variations in lighting. Otherwise, the algorithm will use features of nonstandardized photos to guide decision making. For instance, in our work, we noted that the algorithm appeared more likely to interpret images with rulers as malignant. Why? In our dataset, images with rulers were more likely to be malignant; thus the algorithm inadvertently “learned” that rulers are malignant. These biases in AI models are inherent unless specific attention is paid to address inputs with variability.
     Posted By: Alex - Thu Aug 17, 2023
     Category: Medicine | AI, Robots and Other Automatons

When all the talk about AI came out a while back, I read a story written by a reporter at some news org. He decided to check out one of the new AI's himself to find out its capabilities. He said it fell in love with him.
Posted by Virtual in Carnate on 08/17/23 at 06:28 AM
I wish more people - and especially more managers - would understand this. AI doesn't create. All AI does is regurgitate.
Posted by Richard Bos on 08/19/23 at 05:10 AM
Commenting is not available in this channel entry.