The UK’s inflation figures were wrong and now we just have to live with that

It’s been two long weeks since we last wrote about the Office for National Statistics.

Luckily, the stats gang have done some posting of their own in the meantime, offering an update on efforts to integrate groceries scanner data into Britain’s inflation figures.

If you’ve been living under a rock and missed out on this major news, the ONS has been trying — for some time — to impute prices gathered from scanners in order to observe a lot of prices, rather than their traditional system of observing less of a lot of prices.

The other benefit of such a system is that it helps statisticians better understand the different quantities of similar products being bought, allowing them to weight items more appropriately within the overall index calculation.

The other, other benefit is this: scanner data would take into account prices actually paid, rather than the current system of tracking listed.

To understand why this matters, consider the Tesco Clubcard. As we recently noted, Britain’s biggest supermarket chain is a big deal when it comes to national statistics. And as any regular Tesco shopper knows, having its loyalty card — the Clubcard — is more or less obligatory to unlock its best offers (caveats).

According to Kantar, Tesco currently has a 27.8 per cent share of the UK’s grocery market. According to Tesco, Clubcard sales penetration in the UK is 84 per cent. Ergo, just under a quarter of all UK grocery spending is done at Clubcard prices, and those prices are hugely important if you want to measure the real rate of inflation.

But the ONS ignores Clubcard prices. Here’s an extract from its Consumer Prices Indices Technical manual, the inflation bible:

Discounted and subsidised prices are only recorded if available to anyone with no conditions of sale, otherwise the non-discounted or unsubsidised price is recorded. Money-off coupons and loyalty cards are excluded.

The logic here makes some sense. As the ONS puts its elsewhere, “our basic collection practice is to only consider discounts if they are available to all”. And of course Clubcard prices are only available to the great many people who join that club (basically trading their data for discounts).

But in terms of capturing inflation in terms of the change in how much people are actually paying for goods, it’s a mess.

The ONS was asked about this in a freedom of information request summer before last. In response, they said:

[At] the end of 2022 we started collecting loyalty card prices alongside the prices used for live production of our consumer price statistics. We recently used this data to conduct an informal (internal) analysis to judge the effect loyalty card discounts would have on our inflation measures if they were included. This exercise did not produce evidence that a hypothetical index created by using the discount prices would be materially different to published CPI at the headline or divisional level.

In short, they found Clubcard prices don’t matter.

The scanner data was supposed to be introduced in March this year, but this is the ONS we’re talking about, so because reasons they’re now scheduled to land in March next year.

The ONS offered some initial indications of the change’s impact in January (written up by Chris Giles here). And this morning, it offered a little more.

We have produced indicative estimates of the impact of introducing groceries scanner data into UK consumer price statistics; estimates are subject to change following further quality assurance.

So says “Transformation of UK consumer price statistics, groceries scanner data analysis: April 2025”. It adds:

The average indicative change to the Consumer Prices Index (CPI) annual rate from the introduction of groceries scanner data between January 2019 and June 2024 was negative 0.04 percentage points; this already adjusts for the changes we introduced in February 2025.

Now, 0.04 percentage points doesn’t seem like much. And, sure, often the difference seems to have been quite small overall across that period.

But we have to take issue with this.

Firstly, showing an absolute change over a long period of time seems bound to produce a fairly small number, given you might expect adjustments to occur in both directions. Also, a 2019–2024 window slightly dilutes the whole intense inflation wave (with its weird dynamics) we had in the middle of the period studied.

Secondly — as the ONS data shows but doesn’t really dwell upon — even if the overall average change is small, some of the changes have been pretty big.

Here’s a chart from the ONS release:

The version in their article is responsive, so it’s not hard to see the precise data on-site, but to us this still qualifies as a certified Axis of Evil™. By showing the two compounded 12-month rates (the lines) on the same axis as the spread between them (the columns), you make those columns look small.

And as another chart further down the article shows, those columns ought to feel big:

Headline CPI would have been a quarter of a percentage point lower in March 2024 using this data. Let’s cast our minds back to how mainFT covered that month’s CPI figure at the time:

Consumer prices rose at an annual rate of 3.2 per cent in March, down from 3.4 per cent in February, the Office for National Statistics said on Wednesday. The figure was slightly higher than the 3.1 per cent forecast by economists polled by Reuters and the Bank of England and above the BoE’s 2 per cent target. 

Traders in swaps markets are now betting that the BoE will begin reducing its benchmark rate from a 16-year high of 5.25 per cent in either September or November, having fully priced in a cut for September before the ONS release.

With its “improved methods”, the ONS now thinks the year-on-year change in CPI that March was 3.1 per cent. Adjust for the scanners, and that becomes 2.8 per cent — 0.4pp below the originally-reported figure.

That is a big swing! This is a big deal! It’s clearly materially different*, and — assuming it is actually truer to the reality of inflation — it remains materially significant that the ONS is taking so long to get this improved methodology in place.

As FTAV Friday charts quiz T-shirt owner Gregory Boggis points out on X:

Let’s just be glad the UK never makes major spending decisions based on a few billion pounds of fiscal swing.

* OK, OK, obviously it wouldn’t have mattered because markets are perfectly efficient and therefore have already priced in any and all errors that might occur in the ONS’s CPI calculations.