Search API Benchmark 2026: What Artificial Analysis Shows

Search APIs are becoming core infrastructure for AI agents, but per-query price and latency do not capture how they perform inside an autonomous search loop. Agents may search repeatedly, inspect results, fetch pages, and reformulate queries, so the search provider can affect answer quality, token usage, task latency, and total cost.

Artificial Analysis measures these tradeoffs by holding the candidate model and agent harness constant while changing the Search API provider. Its benchmark covers 20 Search API products across 10 providers. This article focuses on 12 of them, comparing Search Index performance, cost, and latency using the September 8, 2026 results.

TL;DR

• Perplexity Search (medium), 80: Highest overall Search Index and highest BrowseComp accuracy at 87.
• Perplexity Search (high), 79: Nearly the same quality as Medium, without improving on it in this benchmark.
• Octen Search (highlights), 77: Lowest end-to-end task latency at 16.1 seconds.
• Perplexity Search (low), 77: Strong BrowseComp result, but slower and more expensive per completed benchmark task than Medium or High.
• Parallel Search (advanced), 75: Highest-scoring Parallel configuration and tied for the highest DeepSearchQA F1 at 81.
• Brave Search (LLM context), 75: Top-six quality with 22.9-second task time, but the highest total task cost among the 12 configurations analyzed.
• You.com Search (highlights), 74: One of the faster configurations at 19.7 seconds per task.
• Exa Search (auto), 74: Relatively balanced scores across DeepSearchQA, BrowseComp, and AA-Omniscience.
• Firecrawl Search, 73: Highest AA-Omniscience accuracy at 73, but slowest total task time at 61.7 seconds.
• Parallel Search (basic), 73: Faster than Advanced, but substantially higher total benchmark cost.
• Parallel Search (fast), 73: Same overall score as Basic, with lower search cost and lower task latency.
• TinyFish Search (web), 71: $0 measured search cost, the lowest calculated total task cost, and a +38-point lift over the model-only baseline.

Why agentic search needed its own benchmark

The most important result is the gap between search and no search. Using the same candidate model, GPT-5.6 Luna at medium reasoning, the model-only baseline scores 33 on the Search Index.

Every search-enabled configuration in the benchmark improves substantially on that baseline. Perplexity Search Medium reaches 80, a +47-point lift, while TinyFish reaches 71, a +38-point lift.

That gap shows why agentic search needs to be evaluated at the task level. Search changes more than retrieval quality. It can materially affect whether the model finds enough evidence to solve the task at all.

The reason a dedicated agentic-search benchmark matters is that the Search API does more than return a set of links.

The benchmark gives the candidate model access to web_search and web_fetch. The provider being evaluated powers web_search; the model decides whether to search, inspect the response, search again, fetch a returned URL, or finish the task. Because those decisions happen dynamically, two Search APIs with similar request prices can produce very different completed-task economics.

One provider may return evidence that lets the model answer after a few searches. Another may lead to additional queries, more input context, or more reasoning tokens. The benchmark explicitly notes that candidate-model cost can vary by search provider because returned payloads, the number of searches performed, and reasoning-token usage can all change.

That is why this benchmark measures the agent working with the Search API, not just the Search API in isolation.

What an agentic search loop actually looks like

The benchmark uses the Stirrup agent harness for search-enabled runs. The candidate model receives two tools:

• web_search, powered by the Search API under evaluation
• web_fetch, a shared text-only extraction tool

It then has up to 25 agent turns, with unlimited tool calls inside that turn budget.

A simplified task can look like this:

1. Read the question.
2. Form a search query.
3. Inspect the Search API response.
4. Decide whether the evidence is sufficient.
5. Search again, change the query, or fetch a result.
6. Continue until enough evidence has been gathered.
7. Call finish and submit the answer.

If the model exhausts all 25 turns without calling finish, the task receives a zero. Each search returns up to 10 results, and the model sees the provider’s native response payload rather than an Artificial Analysis-normalized result format.

Because those payloads can differ in structure, amount of text, highlights, snippets, and other returned context, the search provider can influence what the model does next. Although the model and agent harness remain fixed, different responses may lead to more searches, more context consumption, or additional reasoning.

This is why per-request Search API cost and completed-task cost are not directly interchangeable. The number of searches, amount of model context, and reasoning required can all vary by provider.

Where agentic search differs from RAG and conventional search APIs

Agentic search, RAG, and conventional Search APIs all support retrieval, but they differ in how retrieval is controlled.

RAG grounds model output in information retrieved from an external knowledge source. The original RAG architecture combines a parametric language model with non-parametric memory retrieved from an external index, while modern RAG systems can support multiple retrieval steps, reranking, and other more advanced retrieval patterns.

A conventional Search API provides ranked web results or search metadata in response to a query. The application decides how those results are used and whether another retrieval step is needed.

Agentic search shifts more of that control to the model. The model can evaluate the evidence it has already collected, decide what information is still missing, reformulate its query, search again, fetch additional sources, and determine when it has enough evidence to answer.

In practice, these approaches can be combined. A RAG system can use live web search as one of its retrieval sources, while a Search API can serve as a tool inside an agentic workflow. This benchmark evaluates this agentic setup by holding the model and agent harness constant while changing the web_search provider.

What this benchmark actually controls

The benchmark describes as a provider-swap comparison. Across provider results, it holds the following constant:

• Candidate model: GPT-5.6 Luna (medium)
• Reasoning effort: Medium
• Temperature: 0.6
• Grader model: GPT-5.6 Luna (medium)
• Turn budget: 25
• Tool calls: unlimited within those 25 turns
• Maximum search results: 10 per search
• Available tools: web_search and web_fetch
• Contamination filtering: enabled

The Search API provider is the variable being changed. This makes the benchmark useful for comparison, but it also sets clear limits on what the results mean.

It tests search, not each vendor’s complete product stack

Only web_search changes with the provider. The same shared text-only web_fetch extraction layer is used across all results. Native fetching, scraping, browser automation, deep-research agents, and other provider products are therefore outside the scope of this score.

For example, the benchmark tests Firecrawl with scrape_format=none, TinyFish with domain_type=web, Exa Auto with highlights enabled, and Octen with highlights enabled. Perplexity’s three public configurations change search_context_size, while Parallel’s rows use Fast, Basic, and Advanced modes.

So a company’s Search Index result should not be interpreted as a score for its entire web infrastructure product.

It uses a single answer model

GPT-5.6 Luna at medium reasoning is fixed across the benchmark. That isolates the search-provider variable, but it also means a different model or agent implementation may interact differently with the same search payload.

Small score differences should not be overinterpreted

The benchmark reports point scores, but does not publish confidence intervals alongside the Search Index rows.

An observed score of 80 versus 79 is therefore the result under this benchmark configuration. It should not automatically be interpreted as evidence of a universally meaningful one-point quality difference across every workload.

How the Search Index is scored

The Search Index is the equal-weighted mean of three evaluations:

The formula is:

Search Index = (DeepSearchQA F1 + BrowseComp accuracy + AA-Omniscience accuracy) ÷ 3

The same grader model is used across all three evaluations.

What DeepSearchQA measures

DeepSearchQA is designed around broad research questions that can require many searches.

The benchmark uses the complete 900-task public evaluation split. Answers can contain one item or a set of items, and performance is measured using LLM-graded F1 over the expected answer items.

What BrowseComp measures

BrowseComp focuses on difficult-to-find facts requiring multi-hop web browsing.

BrowseComp uses a hard 200-sample subset from the larger BrowseComp evaluation pool and scores the final response using exact-answer accuracy.

What AA-Omniscience measures here

AA-Omniscience is a factual QA benchmark focused on correctness and calibration. The evaluation uses 600 private held-out samples balanced across six domains.

Importantly, the Search Index uses AA-Omniscience accuracy, not the benchmark’s standard Omniscience Index or hallucination rate.

The methodology uses accuracy as the more directly comparable signal when a search provider is paired with the candidate model. In this setting, the question is effectively: given access to evidence through this Search API, does the model ultimately answer correctly?

That means the values in the Omniscience column should be read as accuracy inside the Search API benchmark, not as standalone AA-Omniscience Index scores.

What a completed search task actually costs

The benchmark separates Search API cost from candidate-model cost. Search cost is based on the provider’s list price for the searches the agent actually runs. Model cost includes input, cached, reasoning, and output tokens generated while completing the task.

That produces a much more useful comparison than list price alone:

*Total is calculated by adding Artificial Analysis’s published Search API and candidate-model cost columns.

Parallel Basic and Advanced, for example, generate very different model-token costs even though the search provider comes from the same company. Perplexity Low produces a higher total task cost than Medium or High. TinyFish generates no measured Search API cost in the benchmark, but still incurs candidate-model cost.

That is the core economic lesson of an agentic-search workload: what matters is not only what one search costs, but what the agent has to do after receiving it.

How to read cost efficiency

Cost efficiency in an agentic search workflow depends on more than the price of a single search request. Different providers can change how often the model searches, how much context it consumes, and how much reasoning is required. The useful comparison is therefore the relationship between Search Index, completed-task cost, and latency, rather than per-request price alone.

12 Search API configurations compared

The configurations are ordered by Search Index from highest to lowest. Where scores are tied, the original benchmark order is preserved.

1. Perplexity Search (medium), 80

Benchmark profile: Search Index 80; baseline lift +47; DeepSearchQA 81; BrowseComp 87; AA-Omniscience 72; total benchmark cost $91.39/1K tasks; 27.7 seconds/task.

Perplexity Medium records the highest overall Search Index at 80. It also leads BrowseComp with an accuracy of 87 and ties Perplexity High and Parallel Advanced for the highest DeepSearchQA score at 81. Its AA-Omniscience score is 72, with cost and latency remaining the main tradeoffs.

Verdict: Perplexity Medium delivers the strongest composite quality result among the 12 configurations analyzed.

2. Perplexity Search (high), 79

Benchmark profile: Search Index 79; baseline lift +46; DeepSearchQA 81; BrowseComp 86; AA-Omniscience 71; $91.37/1K total task cost; 28.5 seconds/task.

The benchmark uses search_context_size=high for this configuration. It matches Medium on DeepSearchQA at 81 but scores one point lower on both BrowseComp and AA-Omniscience. Task latency is also slightly higher, while total cost is effectively the same.

Verdict: The High setting does not improve on Medium for this benchmark workload despite using the larger search-context configuration.

3. Octen Search (highlights), 77

Benchmark profile: Search Index 77; baseline lift +44; DeepSearchQA 80; BrowseComp 86; AA-Omniscience 66; total cost $58.22/1K tasks; 16.1 seconds/task.

Octen records the lowest completed-task latency at 16.1 seconds and the lowest individual search latency at 0.21 seconds. Its DeepSearchQA and BrowseComp results remain strong at 80 and 86, while AA-Omniscience drops to 66, the lowest result in that component among the 12 configurations.

Verdict: Octen offers the strongest measured latency result while retaining a top-four Search Index, but its factual QA component is noticeably weaker.

4. Perplexity Search (low), 77

Benchmark profile: Search Index 77; baseline lift +44; DeepSearchQA 76; BrowseComp 85; AA-Omniscience 70; total cost $104.78/1K tasks; 35.8 seconds/task.

The benchmark uses search_context_size=low for this configuration. Low scores below both Medium and High, takes longer to complete a task, and produces the highest total task cost of the three Perplexity settings.

This result applies to the fixed benchmark harness and should not be interpreted as evidence that lower context is inherently less efficient.

Verdict: Within this benchmark, Perplexity Low offers a weaker quality, cost, and latency tradeoff than Medium or High.

5. Parallel Search (advanced), 75

Benchmark profile: Search Index 75; baseline lift +42; DeepSearchQA 81; BrowseComp 77; AA-Omniscience 67; total cost $83.51/1K tasks; 41.6 seconds/task.

Advanced is the highest-scoring Parallel configuration, with a DeepSearchQA score of 81, tied with Perplexity Medium and High for the highest result in that component. The main tradeoff is latency, with a 41.6 seconds/task time compared with Parallel Basic at 26.5 seconds and Fast at 18.5 seconds.

Verdict: Parallel Advanced delivers the strongest quality result within Parallel’s configurations, particularly on DeepSearchQA, at the cost of higher latency.

6. Brave Search (LLM context), 75

Benchmark profile: Search Index 75; baseline lift +42; DeepSearchQA 78; BrowseComp 77; AA-Omniscience 69; total cost $129.53/1K tasks; 22.9 seconds/task.

Brave matches Parallel Advanced on overall Search Index while completing tasks considerably faster. Its main tradeoff is cost: combining the benchmark’s search and candidate-model cost produces a total of $129.53 per 1,000 tasks, the highest among the 12 configurations analyzed. This is the cost of the completed benchmark workload, not the price of a single Brave Search request.

Verdict: Brave combines strong overall quality with relatively low task latency, but at the highest calculated total task cost in this comparison.

7. You.com Search (highlights), 74

Benchmark profile: Search Index 74; baseline lift +41; DeepSearchQA 77; BrowseComp 77; AA-Omniscience 69; total cost $117.48/1K tasks; 19.7 seconds/task.

You.com’s strongest result is latency. At 19.7 seconds per task, it is the third-fastest configuration after Octen Highlights and Parallel Fast. Its component scores are relatively balanced, with 77 on both DeepSearchQA and BrowseComp and 69 on AA-Omniscience.

Verdict: You.com occupies a relatively fast, middle-high quality position rather than leading a single quality benchmark.

8. Exa Search (auto), 74

Benchmark profile: Search Index 74; baseline lift +41; DeepSearchQA 78; BrowseComp 74; AA-Omniscience 70; total cost $127.15/1K tasks; 31.9 seconds/task.

The benchmark tests Exa with type=auto and highlights enabled. Its component scores are relatively balanced at 78 on DeepSearchQA, 74 on BrowseComp, and 70 on AA-Omniscience. The main tradeoff is cost, with a calculated total of $127.15 per 1,000 tasks, the second highest among the 12 configurations.

Verdict: Exa Auto is relatively balanced across the three quality components, but its calculated total task cost is among the highest in the comparison.

9. Firecrawl Search, 73

Benchmark profile: Search Index 73; baseline lift +40; DeepSearchQA 74; BrowseComp 74; AA-Omniscience 73; total cost $75.42/1K tasks; 61.7 seconds/task.

Firecrawl records the highest AA-Omniscience accuracy at 73, despite a lower overall Search Index. Its main tradeoff is latency, with the longest completed-task time in the comparison at 61.7 seconds.

The benchmark runs Firecrawl with scrape_format=none, so this result evaluates Firecrawl Search rather than its broader scraping workflow.

Verdict: Firecrawl performs particularly well on AA-Omniscience, but its overall score and task latency are less competitive in this benchmark.

10. Parallel Search (basic), 73

Benchmark profile: Search Index 73; baseline lift +40; DeepSearchQA 79; BrowseComp 73; AA-Omniscience 68; total cost $114.83/1K tasks; 26.5 seconds/task.

Parallel Basic completes tasks faster than Advanced but scores two points lower on the Search Index. It also produces a higher total task cost at $114.83 per 1,000 tasks versus $83.51 for Advanced, driven largely by higher candidate-model cost.

Verdict: Parallel Basic improves latency over Advanced, but gives up quality while producing a higher total task cost in this benchmark.

11. Parallel Search (fast), 73

Benchmark profile: Search Index 73; baseline lift +40; DeepSearchQA 80; BrowseComp 72; AA-Omniscience 68; total cost $68.08/1K tasks; 18.5 seconds/task.

Parallel Fast matches Basic on overall Search Index while reducing both cost and latency. It scores 80 on DeepSearchQA versus 79 for Basic, while BrowseComp falls slightly from 73 to 72. Task time drops from 26.5 to 18.5 seconds, and total cost falls from $114.83 to $68.08 per 1,000 tasks.

Advanced still leads Parallel overall at 75, but Fast gives up only two points while materially reducing cost and latency.

Verdict: Parallel Fast offers the strongest cost and latency tradeoff across Parallel’s three configurations.

12. TinyFish Search (web), 71

Benchmark profile: Search Index 71; baseline lift +38; DeepSearchQA 67; BrowseComp 75; AA-Omniscience 71; Search API cost $0; model cost $34.55/1K tasks; total cost $34.55/1K tasks; 58.5 seconds/task.

TinyFish’s clearest strength is cost efficiency. The benchmark records $0 in Search API spend, and its $34.55 candidate-model cost results in the lowest calculated total task cost among the 12 configurations analyzed.

That efficiency still produces a meaningful quality improvement over the no-search baseline. TinyFish raises the Search Index from 33 to 71, a +38-point lift. Its component results are stronger on BrowseComp and AA-Omniscience, scoring 75 and 71 respectively. The BrowseComp result is higher than Exa, Firecrawl, Parallel Basic, and Parallel Fast.

DeepSearchQA is the weaker component at 67, while the 58.5-second task time places TinyFish toward the slower end of the benchmark. These results point to deeper, multi-step research and task latency as the clearest areas for improvement.

Verdict: TinyFish stands out on efficiency, combining a +38-point improvement over the no-search baseline with $0 measured Search API spend and the lowest calculated total task cost in the comparison. Its BrowseComp and AA-Omniscience results are stronger than its overall Search Index suggests, while DeepSearchQA and latency remain the main areas for improvement.

Picking by what matters most to the build

The benchmark presents quality, cost, and latency as separate decision dimensions. The results therefore produce different leaders depending on the constraint:

These results show why the phrase “best Search API” needs a workload attached to it: the highest quality, lowest latency, and lowest cost belong to different configurations.

For a production agent, the useful question is therefore:

Which tradeoff matches the bottleneck in this workload?

• For a deep-research agent, DeepSearchQA performance may matter most.
• For an interactive application, 16 seconds versus 60 seconds can be decisive.
• For an agent expected to issue very large numbers of searches, task cost may dominate.

And for workloads with a mixture of these constraints, the best answer may not be the provider at the top of any single column.

What the benchmark tells us

The results show that Search API performance is a tradeoff across quality, latency, and cost. Perplexity Medium leads on overall Search Index, Octen Highlights is fastest, and Firecrawl leads AA-Omniscience.

TinyFish’s clearest strength is efficiency. It raises the model from 33 to 71, scores 75 on BrowseComp and 71 on AA-Omniscience, while recording $0 in Search API spend and the lowest calculated total task cost among the 12 configurations analyzed.

DeepSearchQA and task latency remain the clearest areas for improvement, but the benchmark still establishes a meaningful position for TinyFish: substantial quality lift without adding Search API cost. The right Search API ultimately depends on the workload and whether the priority is maximum quality, lower latency, or cost-efficient search at scale.

Frequently asked questions

What is agentic search?

Agentic search is search used inside a model-controlled loop rather than as a one-shot retrieval call. In this benchmark, the model can issue web_search calls, inspect the evidence, fetch returned URLs, search again, and decide when it has enough information to finish. It receives 25 agent turns with unlimited tool calls inside that budget.

How is this different from a normal Search API benchmark?

A conventional search benchmark can evaluate individual queries or result relevance in isolation.

Artificial Analysis instead measures the Search API as part of an agent task. That means the provider can influence final answer quality, the number of searches, model-token consumption, completed-task cost, and latency.

How is agentic search different from RAG?

RAG refers broadly to grounding generation with retrieved external information. Agentic search describes a workflow where the model can dynamically decide what to search for next based on previously retrieved evidence.

The two can be combined. An agentic system may search the live web, retrieve from a private corpus, or use both during the same task.

Which configuration scores highest?

Perplexity Search Medium records the highest Search Index at 80, with a +47-point lift over the same model without search.

Which configuration is fastest?

Octen Search Highlights has the lowest completed-task time at 16.1 seconds. It also reports the lowest per-query search latency for Octen at 0.21 seconds.

Which Search API is cheapest in the benchmark?

TinyFish Search has the lowest measured Search API cost at $0 per 1,000 benchmark tasks. It still incurs candidate-model cost, so a completed TinyFish-enabled benchmark task is not free. It reports $34.55 in model cost per 1,000 tasks.