How We Built a Fact-Checker Where the Model Proposes and the Code Decides

TL;DR

Is that claim someone just forwarded on WhatsApp actually true? Checking it properly takes 25 minutes of searching and reading, so most people never do. So we built TruthLens.

TruthLens is a fact-checking tool for viral claims, built for Azinhack 2026. You paste a claim in English or Hindi, and it plans searches, pulls real articles, and returns a verdict with quotes you can check against the source. It's aimed at people who see a forwarded claim on WhatsApp or Telegram and want a sourced answer in under a minute, not a 25-minute research session.

We're proudest of one design rule: the AI is never allowed to invent evidence. Gemini can only propose quotes, and plain code checks that each one actually appears in the fetched page before a verdict is computed. Across 20 claims, 72 of 76 quotes passed that check. The other 4 were thrown out.

The problem

Checking a single claim properly is slow. You write a few searches, skim results, open pages, find the paragraph that actually says something, and then decide whether it supports or contradicts the claim. It's tedious enough that most people simply don't do it, and a quick AI answer isn't a fix, because language models will happily sound confident while citing something that was never written.

Indian news adds two more problems. Claims often come in Hindi or a Hindi-English mix, and sources vary a lot in how reliable they are.

What we built

From the user's side it's simple. You enter a claim, and the page streams each stage live: the queries being run, the sources found, the pages being read, the evidence picked out, and finally the verdict. You can watch it work instead of staring at a spinner, and every quote in the result links back to the page it came from.

Behind that, a few things we cared about:

• One of the generated queries deliberately looks for evidence against the claim, so the search isn't just confirming whatever was typed in.
• Sources are ranked by tier (official, established outlets, then everything else), and we cap how many pages can come from a single outlet so one site can't dominate.
• It works in English and Hindi.

TinyFish handles discovery and reading, Gemini plans and judges, and plain code does the ranking and the final quote check. Most of the 52 seconds goes to collecting evidence.

Where TinyFish fit

TinyFish does the part that needs to be grounded in the real web.

• TinySearch finds candidate sources for each query. Across the 20 claims, all 100 searches succeeded.
• TinyFetch reads the full page text, so the model works from the article itself and not a search snippet. 77 of 100 fetches succeeded.
• Gemini plans the queries and judges the evidence. It never gets to decide what the sources say; it only points at text that TinyFetch returned.
• Our own code verifies every cited quote against the fetched text, then computes the verdict.

What worked

Making the code the judge of evidence

This is the decision that paid off most. Quote verification takes about 4 milliseconds, and it caught 4 quotes out of 76 that didn't match the source. Without it, those would have shown up in a verdict looking perfectly legitimate.

Each thing Gemini produces has a code check on the other side, and the final verdict and score come from the code.

Searching against the claim

Of the evidence stances, about 32% contradicted the claim and 16% were context. A pipeline that only searched for support would have missed most of that.

Tiering and per-outlet caps

The 20 claims drew on 99 unique URLs across 55 domains, so the evidence wasn't just one or two outlets repeated.

Streaming every stage

It makes the wait feel shorter and, more importantly, makes the reasoning visible. People trust a verdict more when they can see where it came from.

Measured outcome

We ran 20 real claims through the deployed pipeline and logged everything.

About the time saved. We did also estimate how long this would take by hand, and we want to be upfront that it's an estimate, not a measurement. We priced each step (writing queries, reading pages, finding quotes, writing a verdict) and got roughly 14 to 27 minutes per claim depending on how fast the person is, against under a minute with TruthLens. That's about 4.5 to 9 hours saved over 20 claims.

We're deliberately not quoting a headline percentage. Those numbers rest on our own per-step assumptions, and a careful human would not necessarily read exactly 3.85 pages either. The part we're confident in is the measured one: a sourced, quote-checked verdict in about 52 seconds on average. If we wanted a hard comparison, we'd time ourselves fact-checking five of these claims manually.

About accuracy. We can't give an accuracy figure yet, because we don't have independent, human-labelled verdicts to compare against. The internal consensus score averaged 98, but that measures agreement within the pipeline, not correctness, so we haven't treated it as accuracy.

What we'd do differently

• Build the labelled test set first. The biggest gap is that we can't say how often TruthLens is right. Next time we'd label 50 claims by hand before touching the prompts.
• Keep "misleading" as a real verdict. Gemini judged two claims (the Delhi Police/Ramlila one and the Cannes one) as MISLEADING, but our final logic turned them into REFUTED. Final matched the judge on 18 of 20, and those two are exactly the cases where nuance mattered.
• Fill in Tier 4 and widen the official lists. Tier 4 was empty in our runs (0 sources), and half the evidence came from Tier 3. For Indian news especially, better source lists would make verdicts more reliable.
• Handle failed fetches better. 23% of fetches failed, mostly timeouts. Retries, or a fallback to the next-best source, would help, and bot-blocking by some sites is something we'd want to handle more gracefully.
• Test Hindi and Hinglish properly. The one Hinglish claim was our slowest at 125 seconds, but with a sample of one we can't conclude anything about languages.

Advice for other builders

1. Don't let the model be the final authority on facts. Let it propose, then verify with code. It's cheap, fast, and it genuinely catches mistakes.
2. Ask for evidence against your hypothesis, not just for it.
3. Log everything from day one. Writing this log was easy because every stage already stored its own numbers.
4. Be honest about what you didn't measure. Reviewers can tell the difference, and it makes your real numbers more believable.
5. Expect fetching to be your flakiest stage and design for failures.

Try it

TruthLens

• Live app: truth-lens-ten-ruddy.vercel.app
• Code: GitHub
• Demo video: Watch the demo
• Build Library page: TruthLens

Build your own

Want an agent that reads the live web for you? Sign up free at agent.tinyfish.ai.

More student builds are in the TinyFish Build Library.

Building as a student? Join the TinyFish Student Program with your school email for $35 in free credits.