How often is our data right?

Method

• What is sampled. Every brand-and-retailer listing marked as on sale in the public data on the day the sample is drawn. Announced launches are not in the pool.
• How it is drawn. For each listing, key = slug|retailer|country|kind. We sort the keys by the SHA-256 hash of seed|key (lowercase hex, ascending) and take the first 30, or all of them if there are fewer. The seed is the edition, for example 2026-10: it is set in advance and cannot be chosen after the fact. The sample is drawn from the public data as published after the last daily update of the month, and the audit file records which data snapshot was used.
• How each listing is judged. We open the linked source and ask one question: does it support this brand being on sale at this retailer, in this country, in the way we count it (in store or online)? Correct if it does, incorrect if it does not, could not verify if the page cannot be opened or is not specific enough. We do not guess. Each source is opened and read individually; the checks are done by our operator agent with web tools, not by an automated pass/fail script.
• What correct means. The source supports the listing as published. It does not confirm that the product is on the shelf today: some sources are several years old, and the basis for each listing gives the source date.
• Precision = correct / (correct + incorrect), with a Wilson score interval at 95%. Listings we could not verify are reported but left out of the calculation.
• Recall = brands on a retailer's official list that we also list, divided by all brands on that list. It is calculated only where the retailer itself publishes a list.

Redraw the sample yourself

Download the audit data file. It contains the seed and every key in the pool. This Python 3 snippet should reproduce the sample exactly:

import hashlib, json
pub = json.load(open("2026-10.json"))   # the audit data file linked above
seed, keys = pub["seed"], pub["population_keys"]
h = lambda k: hashlib.sha256(f"{seed}|{k}".encode()).hexdigest()
sample = sorted(keys, key=lambda k: (h(k), k))[:pub["sample_size"]]
assert sample == [f"{s['slug']}|{s['retailer']}|{s['country']}|{s['kind']}" for s in pub["sample"]]

Audit, corrections and rankings

• The audit does not change any score or rank. Rankings follow only the published formula.
• A listing found incorrect goes through the same corrections process as a reader's report and will be corrected in the data for the next edition. Frozen editions are not edited.
• A finished audit is final. Its file carries a SHA-256 digest over the sample, verdicts, recall and finalisation date, and our checks refuse any change to a final audit except corrections appended at the end. If we later find a mistake in the audit itself, the original verdict stays and the change is listed under that audit's corrections, with the date and reason.