How TinySet works: Building a dataset from a simple prompt

Ok so TinySet does one thing: you describe the dataset you want in plain English, and a bunch of agents go out and build it from the live web while you watch. Something like "Data centers under construction in the US, with operator, location and planned capacity" came back with 200+ projects across 7 columns.

If you just wanna try it, it's live at agent.tinyfish.ai/playground/datasets.

What TinySet can do today

If you're just here to figure out what it does, here's the quick version:

• Type a prompt, get a dataset. Describe what you want, check the columns it proposes, tweak whatever you need, and hit Build.
• It finds the stuff that's scattered everywhere. It searches across public sites for the things that belong in your table, so you're not the one hunting them down.
• Row 200 gets the same care as row 2. Every row gets its own research, so quality doesn't fall off as the table grows.
• Values come from pages the agents actually read. That's checked in code, not just asked for nicely in a prompt.
• You watch it build live. Names pop in first, then the cells, and you can close the tab and come back without losing your run.
• Use it wherever you work. TinyFish Playground, the ChatGPT plugin, the Claude connector, CLI, SDKs, REST API or MCP, then export to CSV or Excel, or share a read-only link.

Quick reality check: it's up to 250 rows per run, public web only, a dataset is a snapshot rather than a live feed, and it collects stuff rather than deciding what's "best."

That's what it does today. How it got here is a whole other story lol.

The engineering problems, in 30 seconds

Quick backstory first, because TinySet didn't start out as a product at all. Back in May a few of us built a lowkey side quest in public, and what we didn't expect was for people to actually use it, and for a few companies to start running real work through it. So for the last couple of months I've been rebuilding it from the ground up as a proper TinyFish product.

A lot of stuff sounded great in theory and absolutely did not work in practice. Here's what broke, so pick your favorite disaster:

If you're here for the engineering details, this is where it gets fun.

Under the hood: what broke and what we fixed

Every row gets its own agent

The first version was one agent, researching row after row. At 20 rows it was great. Past that, it fell apart in a really specific way: by entity 30 the agent was forgetting what it found for entity 5. Rows came back with wrong values or missing fields, and it wasn't the model being dumb, its context was just stuffed with stale research from every row before.

The fix was to stop asking one agent to remember everything. Every row that needs research gets its own agent, with one job, a tool budget, and zero knowledge of the other rows. That's still the core of how TinySet fills a table today, and it's why row 200 gets the same care as row 2.

Discovery works in smaller pieces

Isolated row agents fixed quality, but we were still stuck at around 200 rows, and the data teams who wanted TinySet needed a lot more than that. The cap turned out to be a harness problem, not a web problem, and it came from discovery: one agent trying to find every entity in a big ask loses track of what it's already found and starts handing back duplicates.

So discovery got split up too. A coordinator agent breaks a big ask into pieces that don't overlap (a region into cities, a program into batches or years, a market into categories, a long list into letter ranges), and each piece gets its own worker that only lists names, never researches them.

"Bay Area coffee shops" becomes San Jose, Sunnyvale, Santa Clara, Mountain View, Palo Alto and so on. Every new row then gets checked against the primary key before it goes in, ignoring case and stray spaces, so the same data center showing up on five listing sites becomes one row, not five.

Planning takes seconds instead of minutes

For a while the planner went and searched the web before proposing your columns, so it could show you where each one had been spotted. Cool idea in theory. In practice planning took 2 to 7 minutes, and staring at a screen where nothing happens for 7 minutes is not it.

So I cut it way down. Planning is now a single model turn with no web access at all, so your columns come back in seconds, and the agents that actually go out on the web figure out the rest during the build.

Every value has to be backed by something real

Ok this is the part I really wanted to get right. Agents misreport their sources, and not on purpose. A row agent sees a URL in a search snippet and cites it as a page it read, or a listing agent remembers a company from training and "finds" it on a page that never mentioned it.

You can tell a model not to do that in the prompt, and it'll still do it often enough that a prompt isn't a guarantee. So the rules live in code, and two of them carry most of the weight:

• A name has to be on a page the worker read. Every name a worker sends back gets checked against the text of the pages and search results it actually pulled, and anything that doesn't show up there gets dropped. So a model that remembers a bookstore from training literally cannot put it on your sheet.
• A source has to be a page the row agent actually opened. If a row agent cites a URL it never fetched, the write gets refused, and the only way through is to go fetch that page or cite the one it really read.

The most common refusal in production is exactly that second one, and once the agent is told what went wrong, it usually fixes it on the next turn.

Different jobs need different models

Row agents make most of a run's model calls, so that's where the model choice matters most. We test on datasets where we already know the right answer, like US national parks, EU member states, Nobel physics laureates since 2000, the S&P 100 and Premier League clubs, and score every run against the real list.

The coordinator is a different story, because a cheaper coordinator split jobs into overlapping pieces, found duplicate entities and stalled for ages before the first name showed up, so it stays on the stronger model.

Every agent also has a fallback model, so one bad response from a provider doesn't end your run.

TinySet collects. You decide how to rank.

People kept asking for things like "the top 10 gyms," and the honest answer is the harness has no idea what "top" means. Any 10 gyms are not the top 10.

So we leaned into what TinySet is actually good at: getting you the ones that match, with columns like rating or price that let you sort it however you want. It's a collection tool, and we're cool with that.

What actually happens when you hit Build

Here's the current flow, start to finish. Say you type "independent bookstores in Portland that sell used books, with website."

1. Plan. Your prompt becomes a spec: what one row is, the column that identifies it, 3 to 8 typed columns (text, number, URL, date, yes/no), and a one-line hint per column about where that value usually lives on the web. "That sell used books" becomes a yes/no column that works as a filter, so only rows where it's true are kept. "In Portland" is just scope, so it doesn't get a column.
2. You edit. Rename stuff, drop a column, change the key, and pick how many rows you want (1 to 250).
3. Split. The coordinator splits the job into smaller pieces. For the bookstores that might be one Portland bookstore directory, plus a search per neighborhood. The coordinator never sees a web page itself, everything it knows comes back as short reports from its workers.
4. List. Partition workers go through their piece with TinySearch and TinyFetch and send back names the moment a page shows them, along with anything the listing already printed next to each name (a website, an address). Names land in your table right away, so the sheet fills in front of you, names first, then cells.
5. Fill. Any row the listing didn't already complete gets its own row agent, which reads pages about that one entity and fills in the rest. A value only goes in if it came off a page that agent actually read.
6. Stream. Rows land in your table as they're written, so you watch the dataset build itself instead of a spinner. Same stream is in the API, so an agent can start working with the first rows while the rest are still coming in.

Every agent also gets a fixed budget of web calls and time, and those limits live in code, so no agent can talk its way into more. A full run usually takes 5 to 20 minutes, and you can close the tab and come back. Runs sit on a queue, so neither one of our deploys nor your wifi dropping halfway through will kill it, and the stream picks up right where it left off.

What's still missing

It's a first release, so here are the real edges:

• Public web only, so nothing behind logins or paywalls, and a business with no web presence won't show up.
• Up to 250 rows per populate for now, and a run can come back with fewer rows than you asked for, depending on what's actually on the web.
• A dataset is a snapshot of the moment you build it. You can add more rows, but it doesn't refresh itself yet.
• It won't rank for you (see above, no top 10s).

When a run's done, you can share the dataset with a read-only link or export it to CSV or Excel. Next up is keeping datasets fresh on their own and making them bigger.

Go build something

Pricing is simple: 10 cents per row, paid from your TinyFish wallet, and every new account starts with $8 in it. You can build a dataset wherever you already work:

From the CLI, plan it, tweak the spec if you want, then build:

tinyfish dataset plan "US data centers under construction, with operator, location and planned capacity" > spec.json
tinyfish dataset create --from-spec spec.json --rows 100 --pretty

Or in one call over the API:

curl https://agent.tinyfish.ai/v1/datasets \
  -H "X-API-Key: $TINYFISH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "US data centers under construction, with operator, location and planned capacity",
    "target_rows": 100
  }'

Need ideas first? We put real datasets and reports, with the exact prompts, at tinyfish.ai/deep-work. And if you build something cool with it, or it gets something wrong, tag @Tiny_Fish and tell us. We read everything, lets gooo.