Standardizing data
Standardization turns a value into its canonical form on the way in. It runs before storage, so the canonical value is what everything downstream sees: matching compares it, data quality scores it, search indexes it.
Why it comes first
Standardization rules, grouped by attribute. Rules on one attribute run in the order shown.
Matching compares canonical values. If one system sends (555) 123-4567 and
another sends +1 555 123 4567, an unstandardized comparison sees a
disagreement and scores it as evidence against a match — the exact opposite
of the truth.
Standardization is not cosmetic. It is what makes the rest of the pipeline correct.
One canonical value
The standardized value is the stored value. There is no parallel raw column shadowing it.
This is deliberate. Two representations of one fact drift, and every consumer then has to decide which to trust. The original input remains recoverable from the audit trail, so nothing is lost — but there is exactly one value in play.
What can be standardized
| Standardizer | Turns | Into |
|---|---|---|
| Name | john SMITH | Consistent casing and spacing |
John.Doe+news@Example.COM | A normalized address | |
| Phone | (555) 123-4567 | A canonical international format |
| Date | Jan 15, 2026 | An unambiguous date |
| Address | A free-text address | Structured, expanded components |
| Case | Mixed input | Upper, lower, title, or sentence case |
| Strip | 123-45-6789 | Only the characters you keep |
| Composite | A structured value | Per-sub-field standardization |
Rules are configured per attribute and run in a defined order, so you can strip separators and then uppercase, for example.
Addresses
Addresses are the hardest case, because the same address has many valid written forms. Address standardization parses free text into components and expands abbreviations, which is what lets matching compare postal codes exactly while comparing street lines loosely.
An address kept as one joined string cannot support that.
Configuration fails loudly
A standardization rule whose configuration the engine cannot honour is rejected when you save it — not accepted and silently skipped at ingest.
This matters more than it sounds. A silently ignored rule produces data that looks standardized, matches badly, and gives no signal about why. Failing at configuration time makes the problem visible while you are still looking at it.
Advice
- Standardize everything you match on. An unstandardized matching attribute is a source of false negatives.
- Standardize before loading in bulk. Values are standardized on write; a rule added afterwards does not retroactively rewrite what is already stored without a recompute.
- Keep rules simple and composable. Several small ordered rules are easier to reason about than one elaborate one.
Next
Last verified against commit 315eb047 (2026-08-03)