A Model Validation Framework for Corrosion & Remaining-Life Models

Asset-integrity teams live with a recurring failure mode: a corrosion or remaining-life model arrives with impressive demo numbers, and then the conversation stops. What is the model actually doing? Is its rate within published literature for X65 carbon steel in atmospheric service? Does it hold up when you replay real survey history? Can your buyer’s engineer reproduce your numbers, not just trust a screenshot?

“Trust the vendor” is not a validation strategy. It’s how you get a surprise corrosion event three years after commissioning — and a very awkward conversation with whoever signs the integrity plan.

The Model Validation Framework (MVF) is a staged, evidence-producing pipeline that puts a degradation engine under test before it earns a seat in your integrity workflow:

Two design decisions matter more than any metric:

1. The engine is imported as a pinned library. MVF does not fork, wrap, or “improve” vendor math. The report names the exact engine commit, so there is no ambiguity about what was tested.
2. Every report carries a provenance block — framework version, git SHA, engine commit, dataset checksums, config hash, run stamp — and is deterministic for a given stamp. A buyer re-running the pipeline gets byte-identical numbers. That sentence is the whole business case in miniature.

The numbers we publish, not the ones we hope you believe

From the current pinned run:

• S0: 33/34 synthetic campaigns pass (the one “fail” is a genuine 2σ tail draw that the harness correctly flags — correctness including the flagging is the point)
• S1: 27/27 literature cells PASS across zones, materials, and min/typ/max rate bounds
• S2: per-vessel backtest with hit-rate, MAE, RMSE and class-level Brier scores — plus honest signal-to-noise caveats where template data is the limiting factor, not the engine

A validation framework that can’t show a failure is just report generation. Ours renders failures in red on the dashboard and explains them in the artifact.

Built to be re-run by someone who doesn’t trust you

That’s deliberate. The operational surface is small enough to audit:

python -m mvf generate --cases 50 --seed 42   # S0
python -m mvf run --stage S1                   # S1
python -m mvf run --stage S2                    # S2
python -m mvf ingest --campaign <dir>           # S3 — licensed campaigns
python -m mvf report                            # master report
python -m mvf serve                             # dashboard at :8765

Backed by a 132-test suite covering golden scoring values, per-band engine conformance, the hold-out protocol, provenance determinism, and the ingest validator. CI-gateable, because reproducibility claims without a gate are marketing.

What this means for you

• Procurement: vendor claims become reproducible artifacts instead of slide decks
• Integrity teams: a standing, auditable answer to “how do we know the model is right?”
• Regulators & clients: evidence trails with hashes, not adjectives

S3 is where this becomes part of your operating cycle: licensed field campaigns arrive through the NDA-ready data license template and the versioned campaign schema, run through the same scored pipeline, and land in the same provenance-stamped report. When the data is under contract, the evidence is a command away.

Bottom line

Model validation has three failure modes: it’s not done, it’s done once and forgotten, or it can’t be independently reproduced. MVF is designed against all three — staged evidence, deterministic artifacts, and a provenance block that makes “I ran it myself” the default posture of every stakeholder who sees it.