Assay

Your AI wrote the code.Assay tells you what's actually true.

One production app: 319 claims verified. 25 bugs. 8 of them were the spec's fault, not the code's.

Assay extracts every claim your app makes, verifies each against the code, then triages every failure. Real defect: fix it. Spec drift: leave the code alone.

bug-report.md — from a live assess run
## Critical — User Registration
Claim: registration returns access + refresh tokens
Verdict: FAIL
Triage: Likely spec drift (not a code bug). Tokens are deliberately withheld until email verification.
Fix prompt: do NOT change the code. Update the spec to match reality.

A blind auto-fixer ships a security regression here. Assay tells your agent to stand down.

npx tryassay assess .thennpx tryassay fix .

Runs with your Anthropic API key, roughly $1–10 in tokens by repo size. Reports publish to tryassay.ai by default; --no-publish keeps everything local.fix applies the real defects. Never the spec-drift ones.

464
tasks verified
100%
pass@5 HumanEval (164/164)
354
claims in the LVR pilot
27
bugs auto-fixed

We also point it at the big stuff.

OpenAI. Kubernetes. Shopify. Cloudflare. Vulnerabilities found and reported in all of them.

OpenAICRITICAL
CWE-94

RCE via vm.Script sandbox escape

JavaScript
Buffer.from('').constructor.constructor('return process')()
Found and reported
KubernetesHIGH
CWE-918

SSRF chain across kubeadm discovery endpoints

Go
// RetrieveValidatedConfigInfo — httpsURL passed with no host validation
client.Get(httpsURL)  // user-controlled, full response consumed
Found and reported
KubernetesHIGH
CWE-22

Path traversal to root code execution in GCI mounter

Go
path, _ := filepath.Split(os.Args[0])  // attacker controls argv[0]
rootfsPath := filepath.Join(path, rootfs)
// then: exec.Command("chroot", rootfsPath, "/bin/mount", ...)
Found and reported
CloudflareHIGH
CWE-918 / CWE-284 / CWE-601

SSRF + CORS + Open Redirect chain

TypeScript
const remote = requestHeaders.get("X-CF-Remote");
await fetch(switchRemote(url, remote), { ... })  // no assertValidURL() call
Found and reported
CloudflareHIGH
CWE-732

OAuth tokens written world-readable

TypeScript
writeFileSync(path.join(configPath), TOML.stringify(config), {
  encoding: "utf-8",  // missing mode option — defaults to 0o644 via umask
});
Found and reported
ShopifyMEDIUM
CWE-338

Predictable CSP nonce via Math.random fallback

TypeScript
// Falls back to Math.random() when crypto.getRandomValues() throws
return new Uint8Array(16).map(() => (Math.random() * 255) | 0);
Found and reported
ShopifyMEDIUM
CWE-338

Weak OAuth state parameter

TypeScript
const randomString = Math.random().toString(36).substring(2)
// Same file correctly uses crypto.getRandomValues() elsewhere
Found and reported
Next.js/VercelMEDIUM
CWE-352

CSRF wildcard origin bypass

TypeScript
// Guard intended to block *.com — but *.com has length 2, bypasses check
if (patternParts.length === 1 && patternParts[0] === '**') return false
Found and reported
Full findings and benchmarks →

Not just big targets. Real developers. Real repos.

Drop a repo. We'll scan it free. Join the community →

Human-built code scores 91. Here's what AI platforms score.

Bolt.new
0/100
Lovable (App 1)
0/100
Lovable (App 2)
0/100
Replit
0/100

4 platforms verified. 18 bugs found. 0 passed.

We publish the misses.

A blind holdout of real CVEs, re-scored from scratch every week. No cherry-picking.

32.9%

of 164 blind holdout CVEs caught at the exact vulnerable code. Strictest definition.

+6.1

points of blind recall gained last teaching cycle, same holdout.

222

real CVEs scored from the GitHub Advisory Database, each scanned pre-fix.

The full benchmark, misses included →

How it works

A neurosymbolic verification loop. Neural systems extract claims. Symbolic oracles verify them. The architecture that makes AI output trustworthy.

1Extract

An LLM reads your code and identifies every implicit claim it makes. “This function handles null input.” “This API returns sorted results.” No regex or AST parser can do this — it requires semantic understanding. This is the neural layer.

2Verify

A deterministic oracle tests each claim. For code, that means actual test execution in an isolated subprocess. The oracle doesn’t hallucinate. It runs the code and reports what happened. This is the symbolic layer.

3Triage & Fix

Every failed claim is classified: real defect, spec drift, or uncertain. Real defects get a fix prompt your coding agent can apply, or let `assay fix` apply them for you. Spec drift gets flagged so nobody “fixes” correct code.

Claim extraction + dual-direction checking · U.S. Patent App. #63/980,048

Start free.

Free
$0

Try Assay on your code. No credit card required.

Pro
$39/mo

For solo developers and freelancers shipping to production.

Team
$29/seat/mo

min 5 users

For engineering teams that ship AI-generated code every day.

View pricing →

Run it

CLInpx tryassay assess .
GitHub Actionuses: gtsbahamas/assay-action@v1
MCP / Editors

lucid-mcp runs Assay inside Claude Code, Cursor, or Windsurf. Setup guide →

Run it on your own repo.

Two minutes to a verdict. No signup.

npx tryassay assess .