Modern multilingual tokenizers often fragment Ukrainian and other underrepresented Cyrillic-script languages more heavily than English, resulting in disparities in cost and context capacity. Researchers analyzed nine production tokenizers and five languages with standardized Cyrillic and Latin representations, covering over 8 million word forms. The study found that Ukrainian shows a significant token overhead on modern tokenizers, ranging from 68% to 121%, compared to older tokenizers like cl100k. The researchers also evaluated two mitigation strategies: LLMLingua-2 reduced Ukrainian input length by 47-49% on an e-commerce benchmark, and a balanced byte-level BPE tokenizer reduced the held-out UK/EN ratio from 2.22x to 1.30x. Romanization increased Ukrainian token counts by 2-19% on most tokenizers. --- Why it matters: This study matters because it highlights the disparities in cost and context capacity between Cyrillic-script languages like Ukrainian and English, which can impact AI model performance and efficiency. Understanding these disparities is crucial for developing more inclusive and accurate multilingual AI systems. Source: https://arxiv.org/abs/2608.21384