Blog
Ideas and Insights
Latest news, updates, and insights from TinyFish.
Someone needs their agent to answer a question about the web. They add a search API. A few weeks later, the agent needs to navigate a JavaScript-heavy site, so they add a headless browser service. Then someone discovers the agent is burning through LLM tokens on raw HTML, so they add a fetch API to get clean markdown instead. Then the workflows get more complex: authenticated portals, multi-step flows, session state. So they wire in a separate agent framework to manage it all.
Four tools. Four API keys. Four billing relationships. Four sets of rate limits to track. Four failure modes to debug at 2am when the pipeline goes down.
This is the default architecture for AI agents that interact with the web in 2026, and it has a name: infrastructure fragmentation. It's not a design choice anyone makes deliberately. It's what happens when you solve each problem as it arrives, reaching for the best available point solution each time.
The question worth asking is whether this is still the right architecture, or whether it made sense only because a unified alternative didn't exist yet.
Before examining the fragmentation problem in detail, it helps to be precise about what web-enabled AI agents actually need to do. Most of the capability requirements reduce to four primitives:
Search — the ability to query the web and get back fast, structured results an agent can act on directly. Not a human-facing results page, but a clean JSON response — titles, URLs, snippets, metadata — designed for programmatic consumption. Real search results, available mid-reasoning without breaking the agent's flow.
Browser — the ability to operate a real browser: load JavaScript-heavy pages, interact with dynamic content, navigate multi-step flows, handle authentication. A raw HTTP request returns the HTML skeleton; a browser gets what a human user actually sees.
Fetch — the ability to retrieve the content of a specific URL and return it in a format the agent can reason over. Clean markdown, structured JSON, or raw HTML depending on the use case. This is distinct from search (you already know the URL) and from browser (you don't need full interaction, just the content).
Agent execution — the ability to complete multi-step, goal-directed tasks on the web. Navigate to a site, log in, fill out a form, extract a result, handle unexpected states. This is what turns the other three primitives into workflows.
These are not exotic requirements. They're the baseline for any agent that needs to interact with the live web. The problem is that in the current ecosystem, each one is served by a completely separate product, built by a completely separate company, with completely separate pricing, authentication, and operational overhead.
The friction is obvious: four API keys instead of one, four dashboards to monitor, four rate limits to track. But the deeper costs are less often quantified.
When an agent makes a decision, it often needs multiple web operations in sequence: search for context, fetch the relevant page, navigate to verify a detail, execute a final action. In a fragmented stack, each of those operations goes through a separate API, with its own cold starts, its own network hops, and its own authentication overhead.
The latency adds up in ways that matter. Exa's Instant search API runs at sub-200ms. Browserbase's browser sessions have a observed initialization times of 5 to 10 seconds in independent testing. Firecrawl's page fetches return in 2 to 6 seconds for static pages, longer for dynamic content. In a single agent turn that requires all three, you're looking at 10 to 20 seconds of infrastructure latency before the agent's logic runs a single line. For any real-time workflow, that's not a minor inconvenience — it's a constraint that shapes what's possible.
We broke down exactly where each of those seconds goes in The Hidden Latency Tax in AI Agent.
When an agent fails in a fragmented stack, the failure could be in any layer. Did the search API return stale results? Did the browser session time out? Did the fetch API clean the page in a way that stripped the data the agent needed? Did the agent framework lose session state partway through a multi-step flow?
Each failure mode requires different debugging tools, different logs, different retry strategies. A developer debugging a production failure in a four-tool stack is triaging four different systems simultaneously. This is not a hypothetical edge case — it's a routine experience for any team running agents at scale.
Exa charges per search request. Firecrawl charges per scraped page with a separate billing track for its AI extraction feature. Browserbase charges per browser-minute. Agent frameworks like Browser Use are open-source, so the browser infrastructure cost falls back on you anyway, just via a different route. When you're trying to understand the actual cost of running your agent, you're reconciling four different billing models, none of which use the same unit.
This matters most when you're trying to optimize. You can't easily compare "should I use more search calls and fewer fetch calls?" when search is measured in requests, fetch is measured in pages, and browser sessions are measured in minutes. The units don't talk to each other.
The most underappreciated cost of fragmentation is what gets lost at the boundaries between tools.
When you search with Exa and get back a list of URLs, then fetch those URLs with Firecrawl, then navigate one of them with Browserbase, you're moving data between three systems that have no shared context. The search results don't carry authentication state. The fetch output doesn't carry session information. The browser session doesn't know what the agent already searched for. Each tool operates in isolation, and the agent has to explicitly maintain state across all of them.
In a unified system, that context is preserved automatically. A search result can flow directly into a browser session without a serialization boundary. A fetched page can inform the agent's navigation decisions without being re-parsed. The infrastructure knows what the agent is trying to do because the agent told it once, at the beginning.
Understanding why fragmentation emerged requires looking at what each category's leading tool was actually built for.
Exa was built as a neural search engine for AI. It's excellent at what it does: semantic search with sub-200ms latency on its Instant tier, real-time results, and a clean API. It was not built to operate browsers, fetch pages, or run multi-step agent workflows. It solves search, and only search, intentionally.
Firecrawl was built to turn websites into LLM-ready data. Send it a URL, get back clean markdown or structured JSON. It handles JavaScript rendering, proxy rotation, and anti-bot measures for the fetch use case. It was not built to search the web or execute goal-directed agent workflows. Firecrawl's free tier gives you 500 lifetime credits — enough to prototype, not enough for production, and the billing switches to a per-page credit model with no pay-as-you-go option at the Hobby tier.
Browserbase was built to host remote browser sessions that developers can connect to via CDP — compatible with Playwright, Puppeteer, and Selenium. It gives you a cloud browser without managing your own infrastructure. Cold starts on browser sessions run 5 to 10 seconds, which is acceptable for many workflows and a blocker for real-time ones.
Browser Use was built as an open-source agent framework that gives developers control over agent behavior in code. It's genuinely powerful for teams that want to customize agent logic, and it's free. The infrastructure overhead — proxies, fingerprint maintenance, detection avoidance — is entirely your responsibility.
Each of these is a real product with real users and genuine strengths. The issue is not that any of them is bad. The issue is that none of them was designed to work with the others, and integrating all four into a coherent production stack is work that doesn't show up in any of their pricing pages.
Before looking at what exists, it helps to be precise about what a unified web infrastructure platform would need to satisfy — not as a wishlist, but as a functional checklist.
Cover all four primitives natively. Search, browser, fetch, and agent execution can't be bolt-ons. If any one of them goes through a third-party integration, you've just reintroduced a fragmentation boundary.
Use a single billing model. Not "one dashboard that aggregates four bills." One credit pool, one pricing unit, one line on your monthly invoice. This is not just a convenience — it's what makes cost optimization possible, because you can compare the cost of different approaches to the same task.
Minimize cold-start latency across all primitives.Browser sessions are the worst offender in fragmented stacks, with initialization times ranging from sub-250ms to 10+ seconds depending on the platform. A platform that routes agent operations through the same underlying browser infrastructure for both the Browser API and the Web Agent eliminates the penalty for switching between them mid-task.
We cover why this single metric predicts pipeline performance better than any other in The Hidden Latency Tax.
Handle the web as it actually is. Bot protection, authenticated portals, JavaScript-heavy pages — these aren't edge cases. A platform that only handles the easy parts of the web solves the easy problem, not the production one.
Publish verifiable real-world performance data. Synthetic benchmarks are easy to design in your own favor. A platform worth trusting on production infrastructure should be willing to publish live-web results with individual task traces you can inspect.
These are demanding requirements. No point solution meets all of them — by definition, a point solution is optimized for one. The question is whether a platform exists that treats all six as design constraints rather than nice-to-haves.
This is what TinyFish is built to be, and it's worth checking against the checklist above.
The four capabilities map directly to four TinyFish products: a Search API for real-time web queries, a Browser API for stealth remote browser sessions, a Fetch API for clean content extraction, and a Web Agent for multi-step goal-directed task execution. One API key grants access to all four. One step-based credit system covers all operations — browser, proxy, and AI inference costs are included in every plan, with no separate infrastructure bills.
The integration matters beyond billing convenience. When a TinyFish web agent needs to fetch a page, the Fetch API runs on the same underlying browser infrastructure as the Browser API — no cold-start penalty for switching between operations mid-task. The agent doesn't need to manage state across four different systems because there's only one system.
Checking against the requirements stated above:
The practical question is whether unification matters yet, or whether fragmentation is a manageable cost that only becomes painful at scale.
The honest answer is that it depends on what you're building.
For a simple agent that searches the web and summarizes results, a single search API is enough. Firecrawl or Exa is the right call; you don't need a platform. For a one-off scraping task on a static page, a Chrome extension or a requests library script is the right call. You don't need any of this.
The calculus changes when your agent needs to do more than one thing on the web: search for context and then navigate to verify it, fetch a page and then log in to access the data behind it, run parallel operations across dozens of sites while managing session state across all of them. That's when the integration overhead of a fragmented stack starts showing up in your engineering time, your debugging cycles, and your monthly billing reconciliation.
The teams who are experiencing this most acutely right now are the ones building agents for enterprise workflows: competitive intelligence pipelines that need to search and navigate and authenticate, healthcare data workflows that need to access authenticated portals at scale, financial monitoring systems that need real-time data from live sites. These are not experimental prototypes. They're production systems where infrastructure fragmentation has a direct cost in reliability and engineering overhead.
There's also a less tangible argument. The web is becoming more dynamic, more authenticated, and more actively resistant to automated access. The percentage of valuable web data that sits behind JavaScript rendering, login walls, or bot-protection has been growing consistently and will continue to. A fragmented stack built on point solutions for the current web will require ongoing reintegration work as each layer evolves independently. A unified stack owned by a single team can evolve coherently.
What is agentic web infrastructure?
Agentic web infrastructure refers to the underlying systems that enable AI agents to interact with the live web: searching for information, operating browsers, fetching page content, and executing multi-step workflows. Unlike traditional web scraping infrastructure, agentic infrastructure is designed for goal-directed operation by autonomous agents, not just data extraction by humans writing scripts.
Why can't AI agents just use existing scraping tools?
Traditional scraping tools were built for extracting data from known URLs at scale. AI agents need to discover URLs through search, navigate dynamic pages that require browser execution, maintain session state across multi-step flows, and handle authenticated portals. These are categorically different requirements that most scraping tools weren't designed to address. That's why the current ecosystem has evolved toward specialized point solutions for each capability and why integration overhead has become a real engineering cost.
What's the difference between a web agent and a web scraper?
A web scraper extracts data from pages using predefined patterns — selectors, XPath, HTML parsing. It assumes you already know where the data is and what structure to expect. A web agent navigates to find data, handles unexpected states, completes multi-step tasks, and maintains context across operations. Scraping is a subset of what a web agent can do, but not the other way around.
Is unified infrastructure actually cheaper than using separate tools?
It depends on what costs you're counting. Per-request API costs may be comparable. The difference shows up in engineering time for integration, debugging overhead when things fail, and the opportunity cost of latency in real-time workflows. For teams at low volume with simple workflows, point solutions are often more economical. For teams running production agents that need all four capabilities, the overhead of maintaining four separate integrations is typically the larger cost.
For a detailed cost comparison across different stack configurations, see our pricing page.
What should I look for when evaluating web infrastructure for AI agents?
Five things: whether it covers all four primitives (search, browser, fetch, agent) natively; cold-start latency for browser sessions; how it handles authenticated portals and bot-protection; whether the billing model reflects actual compute costs or arbitrary product-line pricing; and whether the vendor has published real-world benchmark results, not just synthetic evaluations.
No credit card. No setup. Run your first operation in under a minute.
TL;DR: TinyFish is now an n8n community node. Drop it into any workflow, point it at a URL, tell it what you want, and get clean JSON back. The web just became another input in your automation pipeline.
Most AI web agents collapse under production load. Not because the models are weak, but because the architecture is fragile. TinyFish proposes "codified learning" as a solution. Codified learning structures workflows as typed decision graphs, isolating ambiguity to the minority of steps that need it and keeping everything else deterministic, cacheable, and observable.