TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs
Researchers have developed a new framework called Test-Time Self-Distillation with Fisher-Anchored Restoration (TTSD-FAR) to improve the performance of large video-language models on multimodal tasks such as emotion recognition. The framework addresses the challenge of missing or noisy modalities at test time by using a frozen teacher model and an adaptive student model that updates only a small number of parameters through self-distillation. The Fisher-Anchored Restoration c
Researchers have developed a new framework called Test-Time Self-Distillation with Fisher-Anchored Restoration (TTSD-FAR) to improve the performance of large video-language models on multimodal tasks such as emotion recognition. The framework addresses the challenge of missing or noisy modalities at test time by using a frozen teacher model and an adaptive student model that updates only a small number of parameters through self-distillation. The Fisher-Anchored Restoration component monitors the stability of the student model and restores it to the teacher's anchor when distributional shifts are detected. Experiments on several datasets show that TTSD-FAR outperforms other adaptation methods, including entropy-based adaptation and retrieval-augmented generation, especially in scenarios with long adaptation horizons.
---
Why it matters: This matters because current state-of-the-art models for multimodal tasks like emotion recognition often struggle to adapt to missing or noisy modalities at test time. TTSD-FAR provides a more robust solution that can handle arbitrary missing-modality patterns and remain effective during continual adaptation, which is crucial for real-world deployment.
Source: https://arxiv.org/abs/2608.18386
This article was originally published at: https://arxiv.org/abs/2608.18386