TrueVectors SuiteAgent Behavioral Reliability
TrueVectors Auditor
The TrueVectors Auditor checks an agent's claims against the realized result and proves what it can — with math, not LLM opinions.
What it does
AI agents now do more work than anyone can re-check — shipping code, but also crunching data, building financial models, running migrations, and writing research and reports. So the review question changed — from "does this look right?" to "did the agent actually do what it said?" The TrueVectors Auditor answers that: it checks the agent's claims, its scope, and the real result it produced, and where it can it proves the answer instead of guessing. See the full pipeline →
We audit outcomes, not optics
Most tools ask whether the work looks correct. The Auditor asks whether it actually does the right thing — checking an agent's claims against the realized result and the invariants that must hold. The failure that ships is the one where the output looks clean while the numbers, the logic, or the data are quietly wrong.
The AI suggests. The math decides.
An AI on its own is a confident guesser — ask it twice, get two answers. The Auditor pairs the model with an engine that does real math. The AI spots what might be wrong; the math proves or disproves it and can overrule the AI outright. Same input, same verdict, every time. The term of art is neuro-symbolic — the model proposes, the math disposes — and it's the moat as AI models become a commodity.
Confidence scores are theater
A 0.87 confidence on a hallucinated finding is still a hallucination with a number on it. The Auditor refuses to dress up guesses — every finding is stamped as proved, corroborated, or model judgment, and a deterministic check can overrule the model outright. If we can't back a verdict, we say so rather than fake certainty.
See the Outcome Auditor at work
The Outcome Auditor proves the rules your data or agent must obey — checked with math, zero LLM tokens. Below is one real case end to end: an AI-modernized COBOL→Python monthly-interest job.
Reads the realized artifact, then rules on it
The schema is read straight from the output file — not declared by hand. Each declared law gets a verdict on the real data: here the legacy control total is broken by $0.26, while "one interest row per balance" holds.
A plain-English verdict, and the value of the audit
A receipts-only narrator turns the proven outcomes into one headline and a business-impact line — no source code, no guessing. The Value of this audit block is exposure-gated: it reports the measured exposure computed from your artifact, never a fabricated number.
Traced to the one line that disagrees
The legacy COBOL and the AI-modernized Python are shown side by side. The drift localizes to a single choice — the port rounds ROUND_HALF_UP where the mainframe truncates the sub-cent remainder. A defect that passed code review and unit tests, caught by math.
See the Code Auditor at work
The Code Auditor audits the final code on disk against a stated reviewer intent. It is a hybrid by design: AI reviews every file and raises review leads, and a deterministic static-analysis layer independently re-checks the patterns it can verify — only those are tagged proved / corroborated and counted as verified findings. Everything else is clearly labeled an unverified AI review lead. We never conclude with an LLM: every verdict in the receipt is mechanically checkable.
A workspace view of every run
Latest findings, cost, and 30-day spend at a glance, plus a trends panel showing proved vs unverified per run. Descriptive counts only — no composite "reliability score", and dry-runs are excluded.
Every run summarized by evidence tier
Verified findings and unverified AI review leads are reported as separate counts — never one combined total — with severity counts. The evidence-tier legend is explicit: confirmed, proved, failing-case, checked, or AI lead — and unverified AI leads are hidden by default behind a "Verified only" view.
Findings that show the proof and the fix
Each finding names what's wrong, why it matters, and a suggested fix — pinned to the exact lines, with the proving code shown alongside. Grade each one as accurate, actionable, not actionable, or a false positive to sharpen precision over time.
Proved tier first, everywhere
The TrueVectors Auditor is designed to run where you work. Authorized enterprise evaluations are delivered CI-first through private pipeline or container references. CLI, editor, PR-comment, and optional desktop surfaces drive the same engine, but no public Action, image, or installer is currently published. On every surface the proved tier (everything except a bare llm guess) is the default view. Unproved llm findings are never hidden — they're one click (or --all) away, and their count is always shown:
CLI
The JSON summary leads with proved-tier findings; --all reveals the llm tier.
PR comments
Inline review comments are posted only for proved-tier findings, and llm findings are collapsed behind a summary line.
Every verdict the Auditor produces is persisted to the TrueVectors Hub — the durable system of record for what was proved, what was corroborated, and what stayed model judgment.
Built to be trusted
Honest by design
TrueVectors' copy follows a fixed claims vocabulary, enforced in CI: "proved" is used only for the proved tier, results are always reported misses-first (what was missed alongside what was caught), and the deterministic layer's precision claim is stated exactly: zero deterministic false positives on the published corpus, enforced in CI. The ABR Taxonomy — the agent-failure-mode vocabulary and findings JSON Schema — is published as a standalone, semver-versioned spec.
✔ 0 deterministic false positives
✔ Misses-first reporting
✔ Fixed claims vocabulary
✔ Published taxonomy specPrivate by default
TrueVectors is environment-local and uses zero product telemetry: no usage ping, no analytics, no phone-home. Your work, findings, file paths, and prompts stay inside the authorized CI/VPC boundary except for explicitly enabled LLM calls.
$ tv audit --workspace .
# 100% local execution
# 0 bytes sent to TrueVectors
# 0 telemetry pingsSee it prove something
TrueVectors is an independent research project in early access. Reproduce the public checks or request an authorized CI-first evaluation.
TrueVectors verifies what your agents actually did. Trust through proof.