A new approach has been proposed for efficiently approximating the Pareto set in large deep neural networks. The method uses a mixture of experts (MoE) based model fusion to capture trade-offs between multiple objectives and approximate the entire Pareto set. This is achieved by ensembling weights from specialized single-task models, reducing computational cost during inference. Experimental results on vision and language tasks demonstrate the efficiency and scalability of this approach. --- Why it matters: This matters to AI researchers because it provides a scalable solution for multi-objective optimization problems in large neural networks, enabling more efficient trade-off analysis and multi-task learning. Source: https://arxiv.org/abs/2406.09770