How Does AI Token Counting Work?
AI models like GPT-4o, Claude, and Gemini don't read text the way humans do. Instead, they use a tokenizer to break text into tokens — small chunks that can be whole words, word pieces, or even individual characters. The word “tokenization” becomes three tokens: “token”, “ization” — while common words like “the” are single tokens.
This matters because API pricing is based on token count, not word count. Our token calculator uses the same tiktoken library that OpenAI uses internally, running entirely in your browser via WebAssembly — so your text is never sent to any server.
Use the Token Visualizer above to see exactly how your text is split into tokens. Each colored chip represents one token — tap or hover over any chip to see its token ID. Compare token counts across different models to find the most cost-effective option for your use case.