Researchers have found that different language models, such as Transformers and Linear RNNs, learn to represent numbers in a similar way despite being trained on natural text. They identified a two-tiered hierarchy of features used by these models, with some learning geometrically separable features that can be used for linear classification. The study suggests that the data, architecture, optimizer, and tokenizer all play key roles in determining whether a model learns these features. --- Why it matters: This research matters to AI engineers because it highlights the phenomenon of convergent evolution in feature learning, where different models learn similar features from varying training signals. Understanding how this occurs can inform the development of more efficient and effective language models. Source: https://arxiv.org/abs/2604.20817