Why Validation Gates Matter in AI Memory Systems

The first time you build AI memory, the temptation is to save everything. Every fact the model extracts, every detail the user mentions, straight to storage. It feels comprehensive.

A week later, the memory store is a swamp. Dozens of slightly-different copies of the same fact. Contradictions. Outdated information sitting next to fresh information. The model is more confused than it would have been with no memory at all.

The fix is a validation gate.

What a Validation Gate Does

A validation gate sits between the fact extraction step and the persistent store. Every new fact goes through it before being saved. The gate runs three checks:

1. Duplicate detection — is this fact already known?
2. Contradiction check — does this conflict with an existing high-confidence fact?
3. Confidence scoring — how sure are we that this fact is accurate?

Based on the results, the fact is either:

• Auto-approved and promoted to long-term memory
• Rejected as a duplicate or contradiction
• Queued for review if it's ambiguous

How Noomachy Implements It

Noomachy's validation gate runs as a Firestore trigger. When a new fact gets written to the staging collection, the trigger fires automatically:

1. Embed the new fact (Vertex AI textembedding-gecko)
2. Vector search existing memories for the closest match
3. If cosine similarity > 0.92 → mark as duplicate, reject
4. Run Claude with a "does X contradict Y?" prompt for any near-matches
5. If contradiction → flag for human review
6. If confidence > 0.85 and no conflicts → auto-approve, promote to semantic memory
7. Otherwise → leave in staging for the user to review

The whole pipeline runs in under a second.

Why Auto-Approval Has a Threshold

The auto-approval threshold (default 0.85) is the key knob. Set it too low and noise gets through. Set it too high and the user has to manually approve every fact, defeating the purpose.

0.85 turns out to be a sweet spot:

• Above 0.85: facts that the model is genuinely confident about (extracted from clear user statements)
• Below 0.85: speculation, inference, or ambiguous wording

Users can adjust this in Agent Settings → Memory if they want stricter or looser auto-approval.

What Gets Filtered Out

Real examples of facts that the validation gate rejects:

• "User likes coffee" (already known with confidence 0.95)
• "User lives in Beirut" vs "User lives in Dubai" (contradiction — flagged for review)
• "User mentioned they might be considering thinking about starting a project" (confidence too low)
• "The current date is X" (transient, not worth long-term storage)

What gets through:

• Concrete personal facts ("Joseph works at KodeFoundry")
• Strong preferences ("Prefers dark mode in all apps")
• Project details ("Building an AI agent platform called Noomachy")
• Relationships ("Sarah is the user's CTO")

The Validation Queue UI

For facts that don't auto-approve, Noomachy shows them in a Validation Queue in the Memory Explorer. You can:

• Approve them manually
• Reject them with a reason
• Edit the wording before approving

This puts you in control of what your agent learns. After a few weeks of curating, the memory store becomes a high-signal record of the things that actually matter.

Why This Matters Long-Term

A memory store with no validation degrades over months. The signal-to-noise ratio drops. The model spends its context budget on irrelevant facts. Eventually it's worse than no memory at all.

A memory store with good validation gets better over time. The longer you use it, the smarter your agent becomes — because every memory in the store is one that survived the gate.

Try It

Sign up free → and watch your agent's Memory tab fill up with auto-validated facts. Anything that needs review will show up in the validation queue.