Researchers have proposed a new framework called ClientMorpher for federated instruction fine-tuning of Large Language Models (LLMs). The framework uses routing signatures from pretrained Mixture-of-Experts (MoE) models to organize client collaboration prior to aggregation. This approach is designed to improve personalized performance in heterogeneous client distributions, where indiscriminate aggregation can lead to negative transfer. ClientMorpher includes two clustering strategies: ClientMorpher-C and ClientMorpher-E, which directly cluster clients using expert activation profiles or first cluster experts based on their cross-client usage signatures, respectively. The framework is evaluated on the Databricks Dolly-15K dataset with pathological and Dirichlet-based heterogeneous client distributions across multiple instruction-following tasks. --- Why it matters: This matters to AI engineers because it provides a new approach for personalized federated instruction fine-tuning of sparse MoE LLMs, which can improve performance in decentralized, privacy-sensitive data environments. The framework's ability to organize client collaboration prior to aggregation can also reduce the risk of negative transfer and improve overall model efficiency. Source: https://arxiv.org/abs/2608.15311