openai/tiktoken
tiktoken, на сайте с March 31, 2023 00:28
tiktoken is a fast BPE tokeniser for use with OpenAI's models.