zurawiki/tiktoken-rs

by @github

[[tiktoken-rs](https://crates.io/crates/tiktoken-rs)] - Library for tokenizing text with OpenAI models using tiktoken. [](https://github.com/zurawiki/tiktoken-rs/actions/workflows/ci.yml)