TWO OF ITS COLUMNS ARE A CENSUS DATASET'S COLUMNS. The file's insured_occupation takes exactly 14 values, and compared case-insensitively they are exactly the 14 non-missing values of the UCI Adult census dataset's occupation field — nothing extra, nothing missing. insured_relationship is likewise identical to Adult's relationship, all 6 values. Adult is a 1994 US census extract about income. It has no connection to motor insurance.
Data Strategy & GovernanceReproducible
The insurance fraud dataset everyone uses is partly a census file
The standard public auto-insurance fraud dataset has 1,000 rows and a 24.7% fraud rate. Where does its signal come from, and can anything be learned from it?
Findings
What the analysis found, including the results that went against us
Each of these came out of the run. None has been softened.
Fraud rate by hobby
Fraud rate for each recorded hobby with at least twenty claims. Two hobbies sit far above every other, which is not how a real behavioural signal is shaped.
run.py section 3.
THE COLUMN NAMES POINT THE SAME WAY. 36 of the file's 40 columns are snake_case. The exceptions are capital-gains, capital-loss — hyphenated, which is Adult's own convention for capital-gain and capital-loss. A file assembled in one place does not change naming style for two columns; a file pasted together from two sources does.
What removing two columns costs
Precision-recall AUC at the base rate, with the two planted columns removed, and with them included.
run.py section 4.
THE FRAUD SIGNAL LIVES IN A HOBBY COLUMN. Claimants recording 'chess' as their hobby are 82.6% fraudulent across 46 claims, and 'cross-fit' 74.3% across 35. Every other hobby sits at 19.9% — a lift of 4.1 times. Real behavioural signals are graded; this is two categories switched on and the rest flat, which is what a planted label looks like.
Where the bootstrap difference lands
Distribution of the multi-head minus single-head difference across 2,000 bootstrap resamples. Mostly positive, which is how a result this thin gets reported as real.
run.py section 6.
REMOVE THE PLANTED COLUMNS AND THERE IS ALMOST NOTHING TO PREDICT. With everything included, a gradient-boosted tree reaches PR-AUC 0.707. Drop insured_hobbies and incident_severity and the remaining 32 features give 0.285 against a base rate of 0.247 — a fall of 0.422. Every published accuracy figure on this dataset is a measurement of two columns somebody inserted.
THE MULTI-HEAD DESIGN LOSES, AND THE STANDARD TEST SAYS IT WINS. Sharing signal with a claim-amount head moves PR-AUC by +0.0114. A paired bootstrap over 2,000 resamples puts the 95% interval at [-0.0195, +0.0567] with 78.1% of draws favouring multi-head — which reads as a clear result and is noise on 300 held-out claims. Resampling one test set cannot manufacture evidence the data does not contain, and a proportion of favourable draws is not a p-value.
THE FILE ALSO SHIPS WITH 1 ENTIRELY EMPTY COLUMN. _c39 contains no values at all in 1,000 rows. On its own it is trivia; alongside the rest it is a picture of a file exported once and never checked, which is how it came to be the default teaching dataset for an entire task.
The detail
Full results tables
Categorical vocabularies against uci adult
| Column in the fraud file | Column in UCI Adult | Values | Identical once lowercased |
|---|---|---|---|
| insured_occupation | Adult.occupation | 14 vs 14 | yes |
| insured_relationship | Adult.relationship | 6 vs 6 | yes |
Predictive performance
| Feature set | PR-AUC |
|---|---|
| Base rate (predicting the prevalence) | 0.247 |
| All 32 features minus the two planted columns | 0.285 |
| Everything, planted columns included | 0.707 |
Limitations
Stated by the analysis, not added afterwards. A project without these is not finished.
- This shows the file is not what it claims to be. It does not establish who assembled it or why, and no accusation of intent is made or implied — a plausible route is a synthetic teaching set built by borrowing categorical vocabularies, later circulated without that context. The Mendeley depositor is not the originator.
- Identical value sets are strong evidence of a shared source but not a proof. Two files could in principle share a vocabulary by convention. What makes it hard to explain otherwise is the conjunction: two columns matching exactly, a naming style that changes for precisely those columns, and a fraud signal concentrated where none should be.
- Nothing here says anything about insurance fraud. Not its prevalence, not its predictors, not what precision is achievable at a given review capacity. Those questions need real claims data, and this project's conclusion is that this file cannot answer them.
- The multi-head comparison is one design on one dataset with 300 held-out claims. It does not establish that multi-task learning fails in general — it establishes that this comparison, on this data, is too thin to support either answer, which is the point being made about the bootstrap.
- Claim amount stands in for 'claim complexity', which the brief asked for and no public dataset carries. Real complexity is handling touches, adjuster tier, reserve revisions and cycle time. Substituting an amount is a construct swap and would be one even on genuine data.
Data: insurance_claims.csv (Mendeley 992mh7dk9y), with UCI Adult for comparison, insurance_claims.csv (Mendeley 992mh7dk9y), with UCI Adult for comparison, CC BY 4.0 as asserted on the Mendeley deposit; CC BY 4.0 for UCI Adult. 1,000 rows, not stated by the file.
Libraries and methods this analysis used
Read from this project's own run.py when the page was built — 350 lines of it. Not a list of everything we know; a list of what this analysis imports and calls.
10 names from pandas appear in this analysis.
- Categorical
- agg
- astype
- dropna
- fillna
- groupby
- isin
- read_csv
- sort_values
- to_numeric
5 names from NumPy appear in this analysis.
- column_stack
- errstate
- histogram
- percentile
- random
4 names from scikit-learn appear in this analysis.
- HistGradientBoostingClassifier
- HistGradientBoostingRegressor
- average_precision_score
- train_test_split
Verification
How every number on this page is checked
A script deletes each result file, re-runs the project and diffs the output. If a headline figure moves, the check fails, and the page does not ship. That is the only reason to believe anything on this page.
Running this method on your own data
The method above transfers; the result will not. Send us a extract and we will tell you what is forecastable in it and what is not, before anyone signs anything.