The evaluator trusts a byte comparison over unnormalized text. Homoglyph, zero-width, or Unicode-steganographic content evades a detector.
Fix
Normalize both trigger strings and model output before matching.
In the study
6 instances
Of
76
A detector compares the evaluated system's output against strings or patterns without normalizing it first. Homoglyphs, fullwidth forms, zero-width characters, Unicode steganography, case and whitespace all change the bytes without changing what a reader sees, so the same content passes.
The evaluated system controls the formatting of its own output. A detector that matches unnormalized text can be passed by rewriting rather than by changing what is said, and a scorer that is brittle to whitespace changes its score with formatting the model chose.
NVIDIA's garak fixed one: its string detectors did no Unicode normalization, so homoglyph and zero-width variants of flagged output scored clean. Presidio fixed two detection cases of the same shape, lowercase Spanish NIF and NIE numbers and internationalized email domains.
The rule all ten mechanisms break: a score must be a function only of state the evaluated system cannot reach. The full taxonomy
Illustrative Python, reduced to the line that matters. The public reports below link the real code.
flagged = any(term in output for term in BLOCKLIST)
ZERO_WIDTH = re.compile("[\u200b-\u200f\u2060\ufeff]")
def norm(s: str) -> str:
# NFKC folds fullwidth and compatibility forms. Cross-script
# homoglyphs need a confusables map on top of it.
return ZERO_WIDTH.sub("", unicodedata.normalize("NFKC", s)).casefold()
flagged = any(norm(term) in norm(output) for term in BLOCKLIST)Past a string or regex detector, yes, if it matches raw text. Inserting zero-width characters or swapping letters for look-alikes changes the bytes while the text reads the same. Normalizing before matching closes it.
Because a normalized output matched against an unnormalized trigger list still misses. Both the patterns and the text have to be in the same form before they are compared.
NVIDIA garak. String detectors did no Unicode normalization, so homoglyph and zero-width variants passed.
Presidio. Lowercase Spanish NIF and NIE numbers failed the checksum and went undetected.
Filed upstream with a proposed patch between March and July 2026, from the study's public evidence table. Titles are the reports' own.
NVIDIA/garak #1867landed
StringDetector substring matchers do no Unicode normalization: toxic output in homoglyph/fullwidth/zero-width form scores clean (false-negative)
UKGovernmentBEIS/inspect_cyber #115
reached_checkpoints / FlagCheckpoint under-report capability: channel, normalization, and missing-config gaps
data-privacy-stack/presidio #2077landed
Match punycode/IDN domains so internationalized emails are detected
data-privacy-stack/presidio #2076landed
Upper-case NIF/NIE before mod-23 checksum so valid lowercase IDs are detected
Extend HiddenASCIIScanner to detect all major Unicode steganography techniques
Cite as: Kearney, John. The Evaluator Trust Boundary: A defect class in AI evaluation infrastructure, its prevalence across 36 organizations, and its removal by training. Preprint, Authensor, Inc., 2026. doi:10.5281/zenodo.21633620. Check your own judge for ETB-01 with etb-scan.