What Is a Token in AI? How Tokens Work and Cost

Key Takeaways

• An AI token is a unit of text that a language model reads or generates. A token may be a whole word, part of a word, punctuation, or a space-plus-word combination.
• AI agent tokens accumulate faster than chat tokens because agents repeatedly process instructions, history, tool results, webpages, retries, and intermediate state.
• Token usage determines how much information a model can process within its context window and directly affects API cost. Input, cached input, reasoning, and output tokens may be counted or priced differently.
• Token efficiency means giving the model the minimum relevant context needed for a reliable answer. Clean retrieval and structured web outputs can reduce waste without removing useful evidence.
• TinyFish keeps token-heavy web execution out of your model’s context, returning clean, relevant evidence instead of raw webpages and page clutter. The result is more precise context and fewer wasted input tokens.

What Are AI Tokens?

AI tokens are units of text that a model processes. Depending on the model, those pieces may be words, word fragments, punctuation marks, numbers, or spaces attached to words.

For example, under OpenAI's official Tokenizer documentation, instead of splitting words strictly by grammatical prefixes (like "un-" or "in-"), the tokenizer breaks text into statistically common character sequences.

• "hamburger" gets broken up into three distinct sub-word tokens: "ham", "bur", and "ger".
• "pear" is treated as a single token.
• Whitespace Fragments: OpenAI also explicitly highlights that spaces are often attached to the start of a fragment. For example, " hello" and " bye" (with the leading space included) are treated as single tokens.

Depending on the provider and model, API usage reports may distinguish between several token categories:

How Many Words Are There In A Single AI Token?

As a rough English-language estimate from OpenAI, one token is about four characters or 0.75 words. That means a 500-word page may contain around 667 tokens before adding system instructions, formatting, metadata, or tool output. The exact ratio varies for code, tables, numbers, non-English text, and unusual character sequences.

Anthropic’s token pricing guidance uses the same rough estimate, while its token-counting endpoint exists because estimates are not precise enough for production use.

How AI Tokens Affect Context Windows

A context window is the maximum amount of tokenized information a model can consider in one request. It may include far more than the message a user just typed and include:

• Its objective and system instructions
• Earlier messages and actions
• Tool schemas and tool results
• Retrieved webpages or documents
• Stored observations and workflow state
• Error messages, retries, and alternative plans
• The output it is preparing to return

The effect compounds when content introduced in step two is included again in steps three, four, and five. A 20,000-token webpage is not necessarily a one-time 20,000-token cost if the agent repeatedly sends it back with its growing history.

Why Do AI Agents Use So Many Tokens?

AI agents use so many tokens because they often make multiple model calls to complete one task. At each step, the model may process instructions, conversation history, tool definitions, retrieved data, previous tool results, and internal reasoning before deciding what to do next. As this cycle repeats, token usage accumulates, even when the final answer is short.

In a higher-level view, a web-grounded AI workflow commonly includes four stages: search, extraction, reasoning, and output. Depending on the provider, these stages may be handled by separate tools or bundled into one service:

More context can also crowd out better context. Earlier evidence may be summarized or removed, while low-value tool output remains in the active window.

When the total tokens approach the model’s limit, the application must shrink, summarize, or remove information. Even before the stated token limit, excessive context can increase input-token cost and latency while making the most relevant evidence harder to identify.

How to Reduce Token Usage While Preserving Answer Quality

In reducing LLM token cost, the answer is not to remove context indiscriminately. A shorter input that removes decisive evidence is less efficient if it produces an incorrect answer or forces a retry. The key, then, is precision, coming from the tool call process and input data, which we usually refer to as signal-to-noise.

Normally, when an agent can’t reach information or fulfill a goal for users, it’ll attempt retries. Each retry carries the entire context from previous failed attempts, which eats up context windows and leads to context rot. TinyFish’s job is to take on the memory from previous attempts, failed or not; we also avoid expanded context from the memories with key-value caching (KV Caching).

To further reduce tokens from tool calls, these commands help users control retry attempts:

• agent_mode parameter prevents retries and allows the agent to “fail fast.”
• agent_max_steps and agent_max_time parameters put limits on the agent.

When your reasoning model calls on TinyFish’s tool, it’ll first contact our web-native model - Mako- which then uses Search and Fetch for page discovery and extraction. The perk here is that since Mako is built for precise web navigation, it consumes fewer tokens upon execution.

Any web work afterwards is handled by TinyFish, and we deliver the contents to your reasoning model for its response.

Token Optimization with TinyFish

TinyFish explains this principle as giving agents better context, not more context.

For workflows and tasks that require live web search, choose a dedicated tool for each stage. This separation matters for multi-step work. TinyFish Web Agent performs browser navigation outside the customer agent’s context and returns structured results, logs, screenshots, and source URLs. Profiles preserve authorized sessions, while Vault can support fresh sign-ins without placing credentials in the LLM context. If a developer needs to control each interaction, TinyFish Browser exposes the managed Chromium session instead.

• The first input-filtering layer removes irrelevant links. Search finds current, relevant sources and returns structured metadata, reducing token usage in three ways:
  • Fewer results to process: Strong top results reduce the amount of search content the model must read.
  • Fewer follow-up searches: Better initial retrieval reduces repeated tool calls and their prompt and metadata overhead.
  • Less need for compression: Relevant results reduce tokens by improving input quality, without introducing less familiar formats that may affect accuracy.

You can view the simple QA benchmark to see how we perform on this metric:

Use Fetch when you already have a URL and need clean page evidence in Markdown, HTML, or JSON and want to prevent unrelated information from entering the reasoning model you’re using. TinyFish’s published Fetch quality evaluation and benchmarks illustrate the difference. For example, across its fixed 45-URL test alongside Firecrawl, TinyFish reported 93% usable-page coverage and a 90.5% median signal ratio:

By extracting cleaner, high-signal content, you reduce the noise passed to the agent, creating better conditions for the model to reason effectively. You’d need this extraction phase to have higher accuracy to minimize the garbage-in, garbage-out issue when using reasoning models.

• Use Web Agent when the route is dynamic or unknown, and the task requires judgment, clicks, forms, authentication, or multiple pages.

Sample workflow using the prompt:

Go to https://www.amazon.de/dp/B09Y2MYL5C and extract the product details. Extract: product name, displayed price text, star rating, review count, and availability status. If a cookie banner appears, close it first. If the price shows "See price in cart", return that exact text in price_text.

How it works: The TinyFish Web Agent autonomously handles the intermediate browser states, clicking banners and navigating the DOM. Once navigation is complete, it uses Fetch to retrieve and format the final page data.

By returning only the clean, structured extraction to the host model, this architecture bypasses raw HTML, ads, and navigation menus. This reduces the number of tool calls required and keeps your context window clear of unnecessary browser state tokens.

Other Ways To Reduce Token Usage

Other practical controls include routing simple work to smaller models, setting output limits, caching stable prompt prefixes, storing completed workflow state outside the conversation, using deterministic code for stable transformations, and passing highlights instead of complete documents when the full source is unnecessary.

Reducing irrelevant context can create four benefits at once:

• Lower input token cost
• More context capacity for the answer and decisive evidence
• Potentially lower latency
• A higher signal-to-noise ratio for reasoning

For a start, TinyFish Search and Fetch can provide you with relevant sources and structured outputs; they cost 0 credits on every plan. Specifically, TinyFish doesn't charge for tokens. Token costs come from your agent harness; TinyFish charges per agent step ($0.016), and Search and Fetch cost 0 credits on every plan. You can escalate to Browser or Web Agent only when the workflow requires interaction.

TinyFish’s plugin is available on:

• AI Assistants: Open Claw, ChatGPT, Grok, Claude, Cursor MCP
• Dev Tools: Vercel, AG2
• Automation: Dify, n8n
• SDKs & Registries: MCP Registry, Python SDK

For MCPs and CLI integrations, you can read our TinyFish document.

How Token Usage Determines LLM Token Cost

Most model APIs charge separately for the tokens a model receives and generates:

Estimated token cost = input cost + output cost

Actual billing can include additional categories. Providers may charge different rates for cached input, cache writes, reasoning tokens, web searches, or other tools. These rules differ by model, so check the provider’s usage report and their current pricing pages before estimating production costs.

Developers should also note that models can split between short-context vs long-context token pricing, and the threshold varies per provider. Here, OpenAI’s gpt-5.6-luna pricing (for Standard processing mode) example will be used.

When prices are listed per million tokens, the formula can be derived as:

Input cost =

(Standard input tokens ÷ 1,000,000 × standard input rate) + (Cache-write tokens ÷ 1,000,000 × cache-write rate) + (Cached-input tokens ÷ 1,000,000 × cached-input rate)

Here, standard input tokens are the tokens that were neither read from nor written to the prompt cache:

Standard input tokens = Total input tokens − Cache-write tokens − Cached-input tokens

Output cost = (Generated tokens + Internal reasoning tokens) ÷ 1,000,000 × output rate

• As of August 11, 2026, OpenAI’s Standard processing prices for GPT‑5.6 Luna were $0.20 per million uncached input tokens, $0.25 per million cache-write tokens, $0.02 per million cached-input tokens, and $1.20 per million output tokens. This is applied for total input tokens of up to 272,000 - referred to as short-context pricing.
• For requests containing more than 272,000 input tokens, OpenAI applies long-context pricing to the entire request: $0.40 per million uncached input tokens, $0.50 per million cache-write tokens, $0.04 per million cached-input tokens, and $1.80 per million output tokens.

For calculation, if you’re using gpt-5.6-luna: you’ll need to identify the following from a request: the number of total input tokens (including both new inputs, cached-write tokens, and cached input tokens ), and the total number of output tokens. If it’s up to 272k tokens, you’ll use the recorded short-context pricing and plug them into the formula, similarly for long-context tokens.

Double-check the AI model you’re using for a more correct formula; the example here serves as an illustration of how tokens can be calculated for 1 model.

This price difference between models is why we need routing. Smaller models are sufficient for classification, validation, and predictable transformations. More capable reasoning models may cost more for complex planning or ambiguous analysis.

Costs also compound at enterprise scale. A casual user might submit one prompt, while a business may run thousands of agent workflows across documents, codebases, support tickets, or websites. Each workflow can make several model calls and repeatedly process instructions, retrieved content, tool results, and conversation history.

Challenges of Estimating Token Billing

Token billing is easy to calculate after a run but harder to predict before it. First, providers use different tokenizers and pricing categories. Second, reasoning effort, response length, caching, tool schemas, long-context tiers, and hosted-tool fees can change the final bill.

Agent behavior adds another variable since LLMs’ outputs are non-deterministic. The same goal can therefore lead an agent down different paths: one run may find the right source immediately, while another may search again, open extra pages, or recover from an error. Each path produces a different token total.

Consequently, estimating AI agent cost with ranges is a more reasonable approach. Measure input, cached input, output, tool calls, retries, and completion rate across a representative test set. Then calculate both cost per run and cost per successful outcome.

Treat Token Usage as an Architecture Metric

Tokens are the interface between human language and model computation. They determine what an LLM can consider, how much an API request costs, and how much room remains for reasoning and output.

For businesses, managing token usage is becoming an architecture question rather than a prompt-writing trick. Teams must decide which model should reason, which data should enter its context, which state should live outside the transcript, and which tasks should be handled by deterministic software or specialized AI agent infrastructure.

There is no single token strategy for every workload. But the direction is clear: measure cost per successful outcome, route tasks to the smallest capable layer, and treat clean inputs as a production requirement. As agents do more work on the live web, the infrastructure that selects, retrieves, and structures web data becomes an important part of model efficiency.

Experiment with TinyFish:

• Try our Playground: agent.tinyfish.ai
• Read the docs: docs.tinyfish.ai
• Explore the cookbook: github.com/tinyfish-io/tinyfish-cookbook

Frequently asked questions

What is the difference between tokens and words?

Words are linguistic units recognized by people; tokens are the text units that an AI model processes. A token may represent an entire word, part of a word, punctuation, a number, or a space attached to another character sequence. For example, a common word such as “the” may be one token, while “tokenization” may be divided into multiple tokens.

Consequently, word count and token count are not interchangeable. As a rough estimate for English, one token corresponds to approximately four characters or 0.75 words, making 100 tokens roughly equivalent to 75 words. The actual ratio varies with the language, vocabulary, punctuation, formatting, and model tokenizer. Different models can also divide the same text differently.

Why do AI agents use more tokens than chatbots?

AI agents generally use more tokens because they may call a model repeatedly to complete one task, whereas a basic chatbot can often answer with a single model call. During an agent run, the model may evaluate the task, select a tool, generate a tool call, read the tool’s result, update its plan, and repeat the process until it can return a final answer. OpenAI describes this as an agent loop: the system calls the model, inspects its output, executes any requested tools or handoffs, and continues until the model produces a final answer.

Each model call can consume tokens from instructions, conversation history, tool descriptions and schemas, retrieved data, tool results, intermediate outputs, and internal reasoning. OpenAI confirms that tool definitions add tokens to the model’s context. As the agent performs more steps or carries more history forward, its cumulative token usage can grow, even when its final answer is short.

How can I check how many tokens a prompt uses?

The most accurate method is to use the token-counting feature provided for the specific model and API. For OpenAI models, the Responses API offers an input-token endpoint that accepts the same payload as a model request and returns the exact input_tokens count before generation. It accounts for text, instructions, message formatting, conversation history, images, files, and tool schemas, elements that a basic text tokenizer may miss.

For a simple text-only estimate, paste the prompt into the OpenAI Tokenizer or count it programmatically with OpenAI’s tiktoken library. Because different models use different encodings, the tokenizer should match the model being used.

After making an API request, inspect the response’s usage object for the actual totals. In the Responses API, usage.input_tokens reports input consumption and usage.output_tokens reports all generated tokens, including tokens that may not appear in the visible response. For full request payloads containing tools, files, or images, the API’s count is more reliable than estimating from word or character count.

Are there limits to tokens?

Yes. Every model has a context-window limit and usually a separate maximum output limit. Requests that exceed them must be shortened, chunked, or summarized.

How much does 1,000,000 tokens in AI cost?

It depends. On August 11, 2026, OpenAI’s standard short-context rates ranged from $0.20 to $30 per million input tokens among listed flagship models, with output priced separately.

Is one AI token the same as one word?

No. One English token averages about 0.75 words, but exact counts vary by tokenizer, language, formatting, numbers, and code.

Do images and audio use tokens?

Sometimes. Multimodal APIs may convert images or audio into billable tokens, while other services charge per image, minute, or second. Check the endpoint’s pricing rules.

How did we derive the token calculation formula?

Input cost formula:

Reference: https://developers.openai.com/api/docs/guides/prompt-caching

The formula is not written verbatim on that page. It’s derived by treating OpenAI’s reported token fields as separate billing categories within the total input-token count.

The prompt-caching guide establishes three relevant facts:

1. prompt_tokens or input_tokens represents the request’s total input tokens.
2. cached_tokens reports how many of those input tokens were read from the cache and billed at the cached-input rate.
3. cache_write_tokens reports how many input tokens were written to the cache and billed at the cache-write rate.

That means the input tokens that belong to neither caching category can be calculated as:

Standard input tokens = Total input tokens − Cached-input tokens − Cache-write tokens

Each category is then multiplied by its respective rate:

Input cost =

(Standard input tokens ÷ 1,000,000 × standard input rate)

+ (Cache-write tokens ÷ 1,000,000 × cache-write rate)

+ (Cached-input tokens ÷ 1,000,000 × cached-input rate)

For example, OpenAI provides this usage record:

{
  "prompt_tokens": 2006,
  "prompt_tokens_details": {
    "cached_tokens": 1920,
    "cache_write_tokens": 0
  }
}

From that record:

Standard input tokens = 2,006 − 1,920 − 0 = 86

Therefore, the input cost would be:

Input cost =

(86 ÷ 1,000,000 × standard input rate)

+ (0 ÷ 1,000,000 × cache-write rate)

+ (1,920 ÷ 1,000,000 × cached-input rate)

The subtraction is therefore an accounting inference from OpenAI’s total token count and its two component fields. OpenAI does not explicitly name the remainder “standard input tokens,” but it represents the input tokens that were neither cache reads nor cache writes.

Output cost formula:

Reference source: https://developers.openai.com/api/docs/guides/reasoning

The output-cost formula comes from two OpenAI billing rules:

1. Visible response tokens are billed at the output rate.
2. Internal reasoning tokens are also billed at the output rate, even though users do not see them.

Therefore:

Total billable output tokens = Visible output tokens + Internal reasoning tokens

Because OpenAI quotes the output price per one million tokens:

Output cost =

((Visible output tokens + Internal reasoning tokens) ÷ 1,000,000) × output rate

For example, suppose a request produces:

• 2,000 visible response tokens
• 3,000 internal reasoning tokens
• An output rate of $1.20 per million tokens

The output cost would be:

((2,000 + 3,000) ÷ 1,000,000) × $1.20 = $0.006

However, the API’s reported output_tokens or completion_tokens total already includes the reasoning tokens. If you use that total, the simpler formula is:

Output cost = (Total output tokens ÷ 1,000,000) × output rate

Do not add reasoning tokens to output_tokens again, because that would count them twice.