Every stage names the code that performs it, the libraries that code imports, and something that actually went wrong there. The incidents are the point — a pipeline diagram with tidy boxes says the work is orderly, and it is not.
Take the file as published and change nothing. Every later stage has to be able to point back at this and say what it did.
The retail export arrives as two spreadsheet tabs that overlap by nine days. Concatenate them, as the obvious first line of code does, and you inherit a $632,383 double count that survives naive de-duplication because the line values differ.
Data governanceshared/datasets.pyMeasure what is actually in the file before assuming anything about it — nulls, types, keys, duplicates, encoded missingness.
One scorecard over 26,304,951 cells across five public datasets. The bank set reports zero null cells and hides 12,718 missing values behind a token instead.
Cross-dataset qualityprojects/20-cross-dataset-quality/run.pyDeclare the rules the data must satisfy — grain, keys, types, ranges, sign conventions — before scoring it, so thresholds cannot be tuned to flatter the result.
Thirty-one rules written up front. 19 of 31 fail, nine at critical severity, and 65.4% of lines violate at least one. Twelve pass, which is worth stating as loudly as the failures.
Data governanceprojects/02-data-governance/run.pyBuild the tables the analysis reads: one row per grain, keys that hold, rejected rows accounted for rather than dropped quietly.
8 tables over 1,024,826 fact rows with 108 tests on every build. Every rejected row is accounted for, so the warehouse total ties back to the raw file rather than being asserted to.
Analytics engineeringshared/loader.pyprojects/03-analytics-engineering/run.pyThe modelling itself — forecasts, elasticities, classifiers, segments. The shortest stage, and the one everybody thinks is the whole job.
Weekly demand across 1,344 SKUs on a 13-week holdout: 0.6848 WMAPE against 0.7239 for a four-week moving average. A modest win, published as one.
Demand forecastingprojects/01-demand-forecasting/run.pyTest the result against the stupid version, and against what would happen if the effect were not there at all. Most methods do not survive this.
Our own exception rule captured 34.0% of revenue variance. A naive top-N rule captured 51.2% from a third of the rows. The loss is published, because a method never tested against the obvious one is a method you cannot price.
Decision intelligenceprojects/04-decision-intelligence/run.pyprojects/06-pricing-elasticity/run.pyEmit the result, the findings and what could not be established — then generate every page and figure from that one file.
emit.py refuses to write a result that carries no findings or no limitations. Across the portfolio that is 51 findings and 50 limitations, and every number on this site is read from those files rather than typed into a page.
All projectsshared/emit.pytools/build-project-pages.pyDelete the results, run everything again, and diff. Then check the prose, because reproducing a number says nothing about the sentence around it.
verify.py re-runs all eight projects and diffs every headline value and chart series. It passed twenty out of twenty while four published claims were factually wrong — three Manual adjustment lines where the data has seven, among others — because all four lived inside f-strings. lint_narrative.py exists to catch that class, and found them.
All projectsverify.pylint_narrative.pyEach file named above is real, and the tool list beside it is read from that code when this page is built. Where a stage is pure standard library, the tools it plainly uses are named instead — this stage writes the site's HTML and CSS, and emit.py stamps the git revision into every result.