OpenAI Token Counter

What o200k_base is

o200k_base is a byte-pair encoding published by OpenAI, with a vocabulary of roughly two hundred thousand entries. It is the encoding used by the recent GPT-4o family and later models. Byte-pair encoding starts from bytes and repeatedly merges the most frequent adjacent pairs found in training text, so frequent strings become single tokens and rare strings are assembled from smaller pieces.

A larger vocabulary than the older cl100k_base encoding generally means fewer tokens for the same text, especially for non-English languages and code. Because the encoding is public, anyone can reproduce the count exactly without calling an API.

Exact, local counting

The count here is produced by a JavaScript implementation of o200k_base, loaded in your tab the first time you count. It is the same algorithm the API uses to tokenize your text, so for a given string the result matches. No text is sent to a server, which makes it safe to paste prompts that contain private data.

Why chat requests count a little more

The number above covers the raw text. A chat completion request is not raw text. Each message is wrapped in framing tokens that mark its role and boundaries, and the reply is primed with a few more. This is a small, roughly constant overhead per message, plus a few tokens per request, rather than something that scales with length.

For a single long prompt the overhead is negligible. For a conversation of many short messages, or a request that carries many tool definitions, it adds up and the API's reported input usage will sit above the sum of your pasted text. Function and tool schemas are serialized into the prompt too, so they count as well.

A worked example

The example above mixes an instruction with a quoted error string. The instruction words are common and mostly one token each. The error string is where it gets interesting: 'ECONNRESET' is an uppercase identifier that the encoding splits into several pieces, and the punctuation around it adds more. Paste it and compare the token count to the word count to see the gap.

If you send it as a single user message, expect the API to report a few tokens beyond this page's number, from the message framing.

Small edits change counts in ways that surprise people. A word with a leading space is a different token from the same word at the start of a line, and 'Token' and 'token' are different entries in the vocabulary. So reformatting a prompt, without changing a single word, can move the total by a few tokens.

Use this page to check that a prompt fits before you send it, to compare two phrasings of the same instruction, or to see which part of a large input is expensive. For a full request, add the per-message overhead and leave room for the reply.

Frequently asked questions

Related tools

• Token Counter

  Count tokens for any text, exactly for OpenAI models and estimated for Claude, in your browser.

• Claude Token Counter

  Estimate Claude tokens and input cost for a prompt, with the limits of estimating spelled out.

• Agent Framework Picker

  Five questions to decide between a plain loop, an SDK, or a framework.

• LLM Cost Calculator

  Estimate what an LLM feature costs per request, per day and per month from your own token counts and prices.

• Agent Run Budget Simulator

  See how an AI agent's cost grows turn by turn and where your budget cap would stop it.

• JSON to Zod

  Paste a JSON sample or a JSON Schema and get a Zod schema and TypeScript types, in your browser.

• Tool Schema Validator

  Check an OpenAI, Anthropic or MCP tool definition for common mistakes and convert it to the other two formats.