UK home insurance · market pricing
A model is trained on nine weeks of quotes, then asked to price three weeks it has never seen. This is what it got right, what it got wrong, and why.
Code, data pipeline and method: github.com/SMcode-source/uk-home-insurance-market-pricing
The headline
Across 7,353 quotes in three held-out weeks, the median prediction lands 4.1% away from what the insurer actually charged. The average miss is larger, at 6.0%, because a minority of quotes are badly wrong and pull the mean up.
Those are the figures for the system as it would actually run, re-reading the panel each week before pricing the next one. Held fully blind for all three weeks instead — no refresh at all — it degrades to 4.5% median, 7.0% mean and 77% within 10%. Both numbers appear throughout this page, and the difference between them is the subject of the section on Churchill.
“Accurate” has two meanings here and they do not move together. Getting the premium right is what this page measures. Getting the cheapest five in the right order is a different task, and the model identifies the actual cheapest provider only 58% of the time even while sitting within 4% on price — because when a dozen insurers quote within a few pounds of each other, a 2% error reshuffles the ranking.
Provider by provider
Five providers are predicted almost exactly. One is not. The bar is the average absolute error; the darker portion is what remains once the model is allowed to see each week before pricing the next.
Two very different failures hide inside a single accuracy number, and they call for different fixes. Bias is the model having a provider’s whole price level wrong — a correctable, systematic offset. Spread is what is left after that offset is removed, and it is the genuinely hard part.
| Provider | Quotes | Mean premium | Bias | Spread (SD) | Mean abs. error | Within 10% | Avg miss |
|---|---|---|---|---|---|---|---|
| Admiral | 395 | £208 | −0.5% | 5.2 | 3.11% | 94% | £6 |
| Policy Expert | 421 | £203 | −0.5% | 6.8 | 3.30% | 95% | £5 |
| Homeprotect | 352 | £422 | +1.3% | 6.5 | 3.47% | 95% | £18 |
| AXA | 724 | £214 | +0.7% | 7.2 | 4.05% | 90% | £8 |
| Aviva | 750 | £272 | +3.3% | 5.7 | 4.56% | 93% | £13 |
| Esure | 272 | £211 | −0.4% | 8.1 | 4.62% | 89% | £9 |
| NFU Mutual | 345 | £334 | +5.9% | 7.3 | 6.37% | 83% | £25 |
| Saga | 721 | £386 | −5.8% | 9.9 | 7.02% | 82% | £32 |
| Direct Line | 1,063 | £475 | −4.8% | 9.6 | 7.09% | 79% | £36 |
| Ageas | 500 | £273 | +6.8% | 7.7 | 7.23% | 78% | £23 |
| LV= | 726 | £300 | −4.0% | 9.4 | 8.26% | 58% | £32 |
| Churchill | 1,084 | £363 | −14.8% | 10.5 | 14.24% | 36% | £64 |
| All | 7,353 | £323 | −0.6% | 10.5 | 6.97% | 77% | £27 |
Where it breaks
Churchill is the worst-predicted provider by a wide margin, at nearly 15% average error against a field where the next worst is 8%. It is worth understanding exactly why, because the obvious explanations are all wrong.
This rules out the comfortable explanations. It is not that Churchill is a hard risk mix, or that the model under-fits it, or that the boundary week was averaged away — the fit at week 8 is essentially perfect. It is a structural break: Churchill changed its prices after the training window closed, and its fitted trend over weeks 0–8 pointed the other way, at −1.5% per week.
Weeks 0–8 contain no information about a week-9 break. Any change that improves Churchill’s holdout number using only that data is not a fix, it is fitting the answer. Two earlier attempts to extrapolate harder both made Churchill worse — the second pushed its error from 14.6% to 15.9% — precisely because they projected a downward trend into a market that was moving up.
The batch test withholds all three weeks at once. That is the right test of pure extrapolation and the wrong model of how the capability runs: the panel is re-collected every week, so week 9’s outturn is in hand before week 10 is priced. Feeding that back in is not cheating, it is the job.
The model now supports exactly that, and only that — it re-reads each provider’s price level from the newly observed week and carries it forward. The point model, the shape of every curve and every interaction are left untouched.
| Holdout week | Bias, blind | Bias, weekly refresh | Error, blind | Error, refresh |
|---|---|---|---|---|
| Week 9 | −9.5% | −9.5% | 8.51% | 8.51% |
| Week 10 | −19.3% | −11.6% | 16.20% | 11.93% |
| Week 11 | −23.3% | −6.0% | 17.99% | 11.71% |
How the number is produced
Nothing here is scored in-sample and no split is random. Two holdouts ask different questions, and the answers disagree — which is the point of running both.
| Approach | Temporal | Spatial | Within 10% (temporal) | Fit time |
|---|---|---|---|---|
| Boosted trees per brand, drift-corrected | 4.48 | 11.86 | 77% | 105s |
| Boosted trees per brand | 4.60 | 11.86 | 76% | 9s |
| Random forest | 5.30 | 12.61 | 69% | 32s |
| Boosted trees, pooled | 5.63 | 10.22 | 73% | 5s |
| CatBoost, pooled | 6.29 | 8.86 | 71% | 6s |
| Explainable boosting machine, per brand | 6.72 | 11.56 | 69% | 865s |
| Explainable boosting machine, pooled | 9.91 | 13.19 | 50% | 190s |
| Ridge on log premium | 13.47 | 11.88 | 37% | <1s |
| Per-brand geometric mean (baseline) | 42.31 | 39.51 | 11% | <1s |
| Market geometric mean (baseline) | 47.33 | 42.81 | 14% | <1s |
An explainable boosting machine is additive: it produces a readable table of one curve per rating factor, which is what an actuary can actually argue with. Unrestricted boosted trees can represent caps, collars and three-way interactions that an additive model structurally cannot. The gap between them is a measurement of how much non-additive structure the market really contains.
The sign flips. On unseen weeks there is real non-additive structure worth 2 points; on unseen geography the interpretable model costs nothing and is slightly better. That is a finding about the market, not a verdict on either model — and it means the choice between them is a decision about which risk you care about, not a race.
What to distrust
Everything above is measured on a generated sample extract. The generator was built to reproduce the structural features that make this problem hard — multiplicative rating, minimum-premium floors, brand-specific declines, tactical drift — so the measurements are meaningful as engineering. They are not observations about any real insurer.
The model is asked to price three weeks ahead. Nothing here says anything about three months. The forward drift correction is deliberately capped at a 13-week horizon and its strength was chosen on held-out weeks, which selected only a quarter of the fitted trend — the level series behaves like a random walk, and for a random walk the best forecast of next week is this week.
The model is fitted on log premium, so exponentiating gives a median-unbiased price, not a mean-unbiased one. That is why the pooled median error is −0.6% while the pooled mean error is −2.2%. Anything that needs expected spend rather than a typical price — reserving, budgeting — needs a smearing correction that is not applied here.
Sitting within 4% of every provider still only identifies the actual cheapest one 58% of the time, and reproduces the true cheapest-five list about 44% of the time. If the deliverable is a cheapest-N table rather than a price, these are the numbers that matter and they are far less flattering.
The weekly index runs on 46 risks present in every week, derived because no basket was declared. Which risks constitute your index is an editorial choice that changes the answer, and a vendor extract cannot make it for you.