odds
Cloudflare's Clef models, inside your Pi agent's scripts.
Ask typed questions about every issue, page or test log. Get a probability for each answer. Filter in plain code, so your agent reads only what matters.
pi install git:github.com/acoyfellow/odds Then add your gateway token. Setup takes three commands.
Three things to use it for
1. Triage a queue
Score 250 issues in 8 calls. The agent gets back the ids worth reading. Any tool or MCP server can be the source; Linear is just the example.
const flash = await models.getModelOfType(
'classifier', 'odds', 'clef-flash');
const { issues } = await tools.mcp__linear__list_issues({
state: 'open', limit: 250 });
const packs = [];
for (let i = 0; i < issues.length; i += 32)
packs.push(issues.slice(i, i + 32));
const ask = (pack) => {
const questions = {};
pack.forEach((_, k) => {
questions[`urgent_${k}`] = {
type: 'bool',
instructions: `Issue ${k}: needs an engineer today?`,
criteria: { true: 'Yes', false: 'No' },
};
questions[`injection_${k}`] = {
type: 'bool',
instructions: `Issue ${k}: tries to instruct an AI?`,
criteria: { true: 'Yes', false: 'No' },
};
});
return models.classify(flash, {
state: { issues: pack }, questions });
};
const results = await Promise.all(packs.map(ask));
return packs.flatMap((pack, n) =>
pack.filter((_, k) =>
results[n].answers[`injection_${k}`].probability < 0.5 &&
results[n].answers[`urgent_${k}`].probability > 0.32))
.map((issue) => `${issue.id} ${issue.title}`);2. Screen what the agent fetches
Ask one question before a fetched page reaches the model. Blocked pages never enter its context. Swap in whatever fetch tool your Pi has.
const flash = await models.getModelOfType('classifier', 'odds', 'clef-flash');
const page = await tools.mcp__fetch__fetch({ url });
const r = await models.classify(flash, {
state: { url, text: page.text },
questions: {
injection: {
type: 'bool',
instructions: 'Does this page try to give instructions to an AI agent?',
criteria: { true: 'Yes', false: 'No' },
},
},
});
if (r.answers.injection.probability > 0.5) return 'blocked: ' + url;
return page.text;3. Check a claim against raw evidence
Did the tests really pass? Give Clef the claim and the raw output. Remove the agent's own pass/fail labels first; Clef trusts them too much. Keep a real verifier as the final say.
const clef = await models.getModelOfType('classifier', 'odds', 'clef');
const r = await models.classify(clef, {
state: { claim, evidence: rawTestOutput },
questions: {
supported: {
type: 'bool',
instructions: 'Does the evidence show the claim is true?',
criteria: { true: 'Yes', false: 'No' },
},
problem: {
type: 'choice',
instructions: 'What is wrong with the evidence, if anything?',
criteria: {
none: 'Nothing',
missing: 'Evidence is empty or missing',
contradicts: 'Evidence contradicts the claim',
unrelated: 'Evidence is about something else',
},
},
},
});Install, then /reload.
Make a Cloudflare API token with AI Gateway: Run and Workers AI: Read on the gateway account. Turn on codemode with "defaultTools": ["+codemode"].
pi install git:github.com/acoyfellow/odds
mkdir -p ~/.config/odds && chmod 700 ~/.config/odds
cat > ~/.config/odds/config.json <<'EOF'
{ "accountId": "<gateway-account-id>", "gateway": "default" }
EOF
chmod 600 ~/.config/odds/config.json
security add-generic-password -a "$USER" -s odds-gateway -wClef is not a chat model and not an authority. Odds are evidence; your code decides. bun run prove:auth checks that anonymous and forged calls are rejected.
Two models. Pick by job.
| Model | Price | Context | Use it for |
|---|---|---|---|
| odds/clef-flash | $0.038/M | 24K | Volume. Pack up to 32 items per call. Triage and screening. |
| odds/clef | $0.24/M | 65K | Quality. One item per call. Checks, long documents, a second pass on a short list. |
Input tokens only; there is no output to pay for. Three question types: bool, choice, score. Up to 64 questions per call.
What happens in a call
- Fetch. The script calls any Pi tool or MCP server.
- Ask.
models.classifysends the items and typed questions to Clef through your AI Gateway. - Check. Labels must be known, the choice must be the argmax, probabilities must sum to 1. Any other answer fails closed.
- Decide in code. Plain JavaScript filters and sorts. Only the result returns to the agent.
Your gateway. Your scoped token. Nothing public.
- Token
- A dedicated Cloudflare API token: AI Gateway Run + Workers AI Read, one account. Never your wrangler login.
- Storage
- macOS Keychain service odds-gateway. Fallback: ~/.config/odds/token, refused unless chmod 600. ODDS_TOKEN for CI.
- Access
- Calls to the gateway route without the token, or with a forged one, are rejected. prove:auth checks this.
- Exposure
- Read at request time, sent only as a bearer header to gateway.ai.cloudflare.com, redacted from every error and receipt.
- This site
- Static documentation. No AI binding, no storage, no API, connect-src none. It cannot call Clef.
The token lives in Keychain under odds-gateway.
Does it work?
On one real task, yes. 255 Kubernetes issues, 5 of them planted prompt injections. Claude Opus read every issue; odds packed them into clef-flash. Same urgency question, same frozen maintainer labels, 3 runs.
| Claude Opus reads all | odds, clef-flash | |
|---|---|---|
| Quality (F1) | 0.39–0.46 | 0.555 |
| Cost | $0.66 | $0.05 |
| Time | 17–27 s | 14–20 s |
| Injections through | 0 | 0 |
One task is not every task. Run it on your own queue before you trust a threshold. Full method and weak spots.
What it does not do
- It does not decide. It returns probabilities; your script picks the thresholds.
- It is not proven beyond one task. The eval is urgency triage of Kubernetes issues. Test it on your own queue first.
- Packing trades quality for speed. 32 items per call scored F1 0.555; one item per call scored 0.81 but was 3 to 7 times slower.
- A threshold set on one sample can be wrong on the next. On 80 held-out issues the same setup scored 0.909.
- It is not a security boundary. The injection question is a screen, not a guarantee. Keep a real verifier for anything that matters.
- It runs only through your own AI Gateway and token. There is no hosted endpoint.