Lessons from Building a Deep Research System
Over the past few months, I built a deep research system: an agent that searches the web, browses websites, gathers evidence from dozens of sources and produces long-form reports with citations. While the system itself was built for research, most of the challenges we encountered had surprisingly little to do with research.
They were problems of context management, orchestration and product design. We spent far more time deciding where reasoning should happen, what responsibilities belonged in code rather than the model, how information should move through the system and how the harness should align with the way its users make decisions.
This blog uses our deep research system as a case study to explore those ideas. The architecture is specific to research, but the lessons are not. Most of them generalize to building reliable agent harnesses of any kind.
How We Built Our Deep Research System
Before diving into the lessons, it is worth briefly explaining how the system worked.
Unlike a traditional chatbot, a deep research system is expected to investigate a topic rather than answer it immediately. Given a research query, the system may need to search for information, browse websites, gather evidence from dozens of sources and continuously refine its understanding of the problem before producing a report.
The diagram below shows a simplified, high-level view of the system architecture.
At a high level, the orchestrator drives the research process by planning the investigation, invoking tools and deciding what information to investigate next. Search discovers candidate sources, while fetch and browse extract evidence from static and dynamic websites. As the investigation progresses, the system periodically generates partial reports so users can see what has been learned so far before the final report is complete.
While the architecture evolved significantly over time, the most interesting lessons came from the assumptions that turned out to be wrong.
Reasoning Across Context Boundaries
The first version of our system used a dedicated planner. Given a research question, the planner would generate sub-questions, search queries and a research strategy. The orchestrator would then execute that plan, invoking search, fetch and browser tools to gather evidence from the web.
The decomposition seemed natural. Planning and execution are different tasks, so separating them into different components looked like a clean design. As new evidence came in, the planner could be called again to revise the strategy and generate new search directions.
The problems appeared once research became iterative. Researchers rarely ask a question once and accept the answer. They read the report, narrow their scope, ask follow-up questions and gradually refine what they're looking for. The system needed to build on everything it had already learned.
The orchestrator naturally accumulated that understanding. It had access to the conversation history, previous reports, intermediate findings and the user's evolving intent. The planner, however, was a fresh LLM call every time it was invoked. Only the structured agent state crossed the boundary. The richer understanding that existed inside the orchestrator stayed behind.
At first glance, this seems easy to fix: just give the planner everything the orchestrator knows. In practice, that quickly becomes impractical. The orchestrator's understanding wasn't simply a collection of facts that could be serialized and passed across. It was the result of an entire reasoning process built up over many turns. Reconstructing that understanding inside a fresh LLM call is fundamentally different from continuing the reasoning in the same one. No matter how much context we passed, the planner was always rebuilding a mental model from a snapshot, while the orchestrator already had one.
This became most obvious on follow-up questions. A report might cover a company's overall market position, and the user would ask to focus only on pricing risks in Southeast Asia. The orchestrator understood exactly what "focus only" meant because it had participated in the entire conversation. The planner received only a compressed representation of that understanding and frequently expanded the research back to the broader market analysis.
Eventually we removed the planner and moved planning into the orchestrator itself.
Planning still exists in the system today, but it happens inside the same reasoning loop that executes the research, against the same context rather than a copy of it.
When we add reasoning nodes, we tend to focus on the capabilities they introduce and overlook the information boundaries they create. Boundaries are not free. Every one requires context to cross from one component to another, and every crossing is a chance for something to be lost, flattened, or read differently on the other side. When two components are reasoning about the same objective, that cost can outweigh whatever decomposition was supposed to buy you.
Not Every Decision Needs the Same Context
Removing the planner was supposed to make the system simpler, and at first it did. But we introduced a different problem almost immediately. Our instinct was to parallelize aggressively. Each search query got its own subagent: it would search the web, classify the results, fetch the pages, and extract evidence, then hand everything back to the orchestrator. On paper this looked like good architecture - independent units of work running concurrently, each responsible for its own slice of the research.
In practice, it was a disaster for latency and cost. Spinning up a subagent per query meant the orchestrator was constantly context-switching: dispatching work, waiting, ingesting each subagent's output, merging it into its own state. The overhead compounded with every iteration.
The deeper problem was that we had reached for subagents to solve a problem that didn't require them. None of these steps were judgment calls. Ranking a set of search results, classifying a source, extracting claims from a page - it was a deterministic flow with fixed inputs knowable ahead of time. No orchestrator needed to be involved.
So the subagents went away. In their place: tool calls that did the post-processing deterministically and handed back something already distilled.
The orchestrator never saw the intermediate mess. It received ranked sources and extracted evidence claims. Its context shrank, latency dropped, cost dropped, and it could focus on the only two things it actually needed to do: plan, and decide what to look at next.
What became clear was that it is tempting to make every stage of the pipeline an agent because it feels more flexible. In practice, flexibility comes at the cost of latency, context and non-determinism. If a step has a deterministic flow, it is usually better expressed as pre- or post-processing around a tool call rather than another reasoning node.
The interesting design problem is deciding where that boundary lies. A completely deterministic pipeline would be fast and predictable, but it would also remove the model's ability to adapt its strategy as new information emerges. A pipeline that reasons everywhere has the opposite problem: every decision becomes slower, more expensive and harder to control. Building a good harness is largely about finding the boundary between deterministic execution and model flexibility.
Learning from Researchers
While these lessons improved the architecture, they did not necessarily make the product more useful. That part only came from spending time with the people who would actually use it.
When we started showing reports to researchers, we realized our idea of a "good" report was incomplete. We had been optimizing for coverage: more sources, more evidence and more comprehensive write-ups. Researchers were evaluating something different.
Our system mostly cared whether a claim could be supported at all. Researchers cared about the strength of that support. They were constantly judging whether a source was appropriate for the question, how much confidence should be placed in a conclusion and whether different pieces of evidence deserved equal weight.
Consider two research tasks. If you are investigating public sentiment toward a new product, thousands of social media posts may be more valuable than a carefully written government report. If you are estimating unemployment rates, the opposite is true. Source quality is not absolute; it depends on the objective of the research.
This changed what we optimized for. Source ranking started accounting for the research objective instead of relying on a single notion of quality. The synthesizer was prompted to reason about the strength of evidence rather than treating every supported claim equally. We also gave researchers direct control over which domains should be prioritized, excluded or exclusively considered during research.
To validate these changes, we built an internal evaluation harness that used an LLM judge to score each report across dimensions researchers consistently cared about, including relevance, evidence strength, trust in citations and source diversity. While not a replacement for human evaluation, it gave us a scalable way to measure whether the harness was actually improving.
The improvements reflected what we had been hearing from researchers throughout development. Reports were more focused on the original objective, drew conclusions from stronger evidence and inspired greater confidence in their citations.
The striking part was that none of this came from a better model. We had spent the previous lessons making the model reason more cleanly, and the thing that made reports actually useful was something else entirely: matching how researchers themselves decide what to trust.
The best harness is not the one that does the most - it is the one that thinks the way its users do.
Conclusion
We thought the hard parts would be search, browsing and retrieval. They mostly weren't.
By the end, most of what improved the system weren't changes to the prompts or the model. They were decisions about what to take away from the model: which jobs to move into code, which context to stop feeding it, which reasoning to pull back into a single loop. The work was in deciding what the model shouldn't be responsible for.
The other half came from watching people use it. A harness that scores well on a benchmark isn't the same as one a user trusts, and the gap between the two was where a lot of the real work turned out to be. We stopped optimizing for the benchmark and started optimizing for the users.
