An accuracy figure is a statement about a cohort, not about a model

THE MODEL AND THE RADIOLOGIST CANNOT BE TOLD APART, AND THE CONTEST WAS NEVER FAIR. On 223 biopsied lesions of which 87 were malignant, a logistic regression on the radiologist's recorded descriptors scores AUC 0.837 against 0.851 for that radiologist's own BI-RADS assessment — a difference of -0.0136 with a 95% interval of [-0.070, +0.044]. The interval spans zero, so no difference is established. It could not have been: the model's inputs are the radiologist's judgement written down, recorded in the same reading that produced the assessment. A tie shows the descriptors carry the assessment, and nothing more.

THE COHORT DECIDES THE NUMBER BEFORE THE MODEL DOES. Every lesion here was biopsied — that is how a pathology label exists — so 39.0% of them are malignant. Hold this classifier's sensitivity (0.80) and specificity (0.69) fixed and move only the population: positive predictive value falls from 62.5% here to 1.29% at a screening rate of 0.5%, a factor of 48. Nothing about the model changed. A reported PPV without its cohort's prevalence beside it is not interpretable.

THE OPTIMAL THRESHOLD IS NOT A POINT AND OFTEN NOT IDENTIFIED. With 223 lesions the cost function is a step function, so an interval of thresholds achieves the same minimum. At equal costs every cut between 0.550 and 0.640 is indistinguishable — a range 0.090 wide. When a false negative is costed twenty times a false positive the optimum moves to 0.100 to 0.100. Publishing a threshold to three decimal places implies a precision this much data cannot support.

THE FIELD'S DEFAULT BENCHMARK IS BEATEN TO WITHIN 6.1 POINTS BY ONE DECISION RULE. On the 569-row Wisconsin diagnostic dataset, logistic regression on all 30 features reaches 95.43% under ten-fold cross-validation. A decision stump — a single threshold on worst radius, the simplest non-trivial classifier that exists — reaches 89.29%. A method reporting 97 or 98 per cent on this data has not demonstrated anything the stump does not, and the benchmark cannot distinguish methods that differ by less than that.

A LESION PHOTOGRAPHED TWICE IS ONE LESION AND TWO ROWS. The training file has 1,318 rows describing 782 lesions in 691 patients, because 536 lesions were imaged from two angles. Reporting at row grain inflates the apparent sample by 68.5%, and every row of a pair carries the same pathology label — so a random split puts the same outcome on both sides. This file ships with an official split whose patient overlap is exactly 0, which is the correct design and is worth noticing, because a split that does not do this is common.

THE PATTERN REPLICATES ON AN INDEPENDENT COHORT. On the UCI Mammographic Mass data — 961 lesions from a different institution and a different decade, 830 of them complete — a model on the recorded descriptors scores AUC 0.872 against 0.825 for the radiologist's BI-RADS, a difference of +0.047. The donors of that dataset flag BI-RADS as non-predictive and exclude it from modelling for exactly this reason. Two cohorts, same result: a model of a radiologist's description performs like the radiologist.