Token Counter

What a token is

A language model does not read characters or words. It reads tokens: chunks of text drawn from a fixed vocabulary that the model's tokenizer learned from a large corpus. Common English words are usually one token each. Rarer words are cut into pieces, and a long identifier, a URL or a word in a less common language can take many tokens for very few visible characters.

Whitespace and punctuation count too. A leading space is often folded into the token that follows it, and a newline or a run of indentation can be its own token. This is why a screen of minified JSON and a screen of prose of the same length cost very different amounts.

Why counts differ between tokenizers

Each model family ships its own tokenizer with its own vocabulary. The same sentence can come out as 20 tokens in one and 24 in another, because the vocabularies were built from different data and with different merge rules. A count is only meaningful for the tokenizer it was made with.

That is why this page shows two numbers. For OpenAI models the encoding is public, so the count is exact. For Claude the tokenizer is not available to run locally, so the count is an estimate. The Claude and OpenAI pages explain each case in detail.

Why the count matters

Tokens are the unit for two limits you will hit. The first is the context window: prompt, retrieved documents, conversation history and the model's reply all share one budget, and going over it either errors or silently drops content. The second is cost, since providers meter input and output in tokens.

Counting before you send is cheap insurance. It tells you whether a document fits, how much room is left for the answer, and whether a prompt template has quietly grown as people added instructions to it over time.

A worked example

Take the example prompt above. Most of its words are common, so each lands on one token, and the punctuation adds a few more. A word like 'authentication' may be one token or two depending on the vocabulary, which is exactly the kind of difference that makes the OpenAI and Claude numbers diverge slightly. Try swapping in a rare word, a file path or a snippet of code and watch the count rise faster than the character count does.

Exact versus estimated

Use the exact count when a hard limit is involved and leave a margin when it is an estimate. An estimate is good for sizing and sanity checks, such as whether a prompt is closer to two thousand tokens or twenty thousand. It is not good for squeezing the last few hundred tokens out of a window. For that, ask the provider to count, or leave headroom.

Nothing you paste here is sent anywhere. Counting runs in your browser tab.

Frequently asked questions

Related tools

• Claude Token Counter

  Estimate Claude tokens and input cost for a prompt, with the limits of estimating spelled out.

• OpenAI Token Counter

  Exact o200k_base token counts for GPT-style prompts, computed locally.

• Agent Framework Picker

  Five questions to decide between a plain loop, an SDK, or a framework.

• LLM Cost Calculator

  Estimate what an LLM feature costs per request, per day and per month from your own token counts and prices.

• Agent Run Budget Simulator

  See how an AI agent's cost grows turn by turn and where your budget cap would stop it.

• JSON to Zod

  Paste a JSON sample or a JSON Schema and get a Zod schema and TypeScript types, in your browser.

• Tool Schema Validator

  Check an OpenAI, Anthropic or MCP tool definition for common mistakes and convert it to the other two formats.