Skip to main content

Tune thresholds

The two thresholds decide what merges automatically, what a person reviews, and what is ignored. This page sets them from evidence rather than guesswork.

Thresholds are log2 evidence units, not a score

They are not on a 0–1 scale and not percentages. Each whole point doubles the strength of the evidence required, so a threshold of 12 asks for roughly four thousand times more support for a match than for a coincidence. Most other platforms use a bounded score, which makes this the commonest source of mis-tuning. See how probabilistic scoring works.

Before you start​

  • An active match profile with blocking and comparison ladders configured.
  • Labelled pairs — pairs you have confirmed as matches or non-matches. This is the input everything here depends on; without it you are guessing.
  • Permissions for each step: match.labeled-pair.create to build the labelled set, match.profile.evaluate to evaluate, match.profile.calibrate-weights to calibrate, match.profile.optimize-thresholds to optimise, and match-profile.update to save the result. Missing any one fails that step alone.

1. Build a labelled set​

Label from your own data, not synthetic examples. The pairs that matter are the hard ones:

  • Confirmed matches, especially awkward ones — a name change, a transcription error, a missing field.
  • Confirmed non-matches that look similar — a father and son at one address, two people sharing a common name and city. These are what stop the auto-link threshold from being set too low.

A labelled set of only easy cases produces thresholds that fail on exactly the cases you built the system for.

2. Evaluate the current profile​

Evaluation scores every labelled pair with the active configuration and reports precision, recall, and the score distribution.

Read the distribution, not just the headline numbers: where do known matches cluster, where do known non-matches cluster, and how much do the two overlap? The overlap is your clerical review band. If it is enormous, no threshold will save you — the discriminating power has to come from better blocking, better ladders, or better weights.

3. Calibrate weights​

Rather than hand-setting the two probabilities per level, fit them from the labelled set: how often each level occurs among true matches, and how often among true non-matches.

Calibration is guarded — a fitted set that would lower measured quality is reported and not applied. Trust the guard; a calibration that degrades results usually means the labelled set is unrepresentative, not that the fit is wrong.

4. Set the thresholds​

The Thresholds tab with the score distribution The Thresholds tab. The distribution shows how many pairs fall either side of each line.

The two are chosen on different principles.

Auto-link is set by precision. It must sit above every known non-match, so nothing the labelled data says is not a match can merge unattended. A balanced measure like F1 cannot help here: it counts auto-linked and clerically-reviewed pairs alike, so it is structurally blind to where the auto-link line sits.

Clerical review is set by capacity. Lower it and you catch more true duplicates but queue more pairs. This is an operational decision — set it to the volume your stewards can actually work through, then revisit as the queue drains.

5. Add a minimum-agreement rule​

Threshold tuning cannot protect against a single attribute carrying an entire match on its own. Requiring agreement on several distinct attributes before auto-linking is threshold-independent — it holds even if a threshold is later mis-set.

Set it on any entity type with rich identity data. Records that genuinely match almost always agree on more than one thing.

6. Re-evaluate and verify​

Re-run evaluation and confirm precision on the auto-link tier is what you require. Then dry-run against real records and inspect the tiers.

Two sanity checks worth doing every time

The threshold must be reachable. If the auto-link threshold exceeds the total weight available from all your ladders combined, nothing can ever auto-link and the entity type will silently never merge. Add up the maximum weight across every ladder and leave real headroom — frequency adjustment only ever weakens evidence, so a threshold needing flawless agreement everywhere is unreachable in practice too.

The review band must not be empty. If the clerical review threshold is at or above the auto-link threshold there is no review band, and every pair either merges or vanishes.

Next​

Last verified against commit 315eb047 (2026-08-03)