Researchers have investigated whether using marginal coverage can ensure that zero-shot vision-language models (VLMs) are safe in real-world scenarios. They tested three popular VLMs - CLIP, OpenCLIP, and SigLIP - on various datasets and found that while marginal coverage remained relatively high, the worst-case performance of these models collapsed. The study suggests that relying solely on marginal coverage may not be sufficient to guarantee class-conditional safety for zero-shot VLMs under deployment shift. --- Why it matters: This research matters because it highlights a potential flaw in the current approach to ensuring the reliability and safety of AI models, particularly those used for vision-language tasks. The findings have implications for the development and deployment of these models, which are increasingly being used in real-world applications. Source: https://arxiv.org/abs/2608.19376