You can use the tool below to understand how a piece of text might be tokenized by a language model, and the total count of tokens in that piece of text.
This is also the only tokenizer which uses the official API. This was written hours after the API was released on the 25th of November 2024.
0
0
text only
