Standardization at a glance
Where it sits
raw input → STANDARDIZE → validate → store → match / score / search
Everything downstream reads the standardized value. Matching compares canonical forms, so standardizing is not cosmetic — it is what makes the rest correct.
What each standardizer produces
| Standardizer | Produces |
|---|---|
| Name | Consistent casing and spacing; common abbreviations expanded |
| A normalized address, lowercased | |
| Phone | A canonical international format |
| Date | An unambiguous date from many input forms |
| Address | One canonical expanded line, lowercased — not split into components |
| Uppercase / lowercase / titlecase / sentencecase | The value recased |
| Address line | A single address line, normalized |
| Strip | The value with matching characters removed |
| Unicode fold | Accents and compatibility characters normalized |
| Nickname expand | A canonical given name from a familiar form |
| Composite | Per-sub-field standardization |
One canonical value
The standardized value is the stored value. There is no parallel raw column — two representations of one fact drift, and every consumer then has to decide which to trust.
Standardization is one-way. The audit trail records the standardized value and whether standardization changed it — not what you sent. Nothing anywhere stores the raw input.
Keep your own copy at the source if you need to reproduce what was submitted, and test a rule on sample values before applying it to loaded data.
Order matters
Rules run in a configured order per attribute. Stripping separators then uppercasing gives a different result from the reverse. Normalize shape first, then case.
Two traps
Rules run on write. A rule added after loading does not retroactively rewrite stored data. Recompute, then match.
Phone assumes US when you do not set a default country. It does not fail — a number without a country prefix is normalized as if it were American, which produces a plausible-looking wrong number rather than a visible error. Set the default country to the region your data actually comes from.
Fails loudly
A rule the engine cannot honour is rejected when you save it, not skipped at ingest. A silently ignored rule produces data that looks standardized, matches badly, and gives no signal why.
Next
Last verified against commit 1e1de1ad (2026-08-03)