AI

You Are What You Prompt: Prompt Quality, Domain Shift, and Uncertainty in Agrifood Vision-Language Models

Researchers have studied how well zero-shot classification works in agrifood vision-language models when using different types of prompts and datasets. They found that weighting prompts by their discriminative signal can improve performance under domain shift, but only if the prompts are specific to the new domain. A generic pool of prompts does not work as well. The study also introduced a method for detecting epistemic uncertainty in these models based on prompt disagreemen
Researchers have studied how well zero-shot classification works in agrifood vision-language models when using different types of prompts and datasets. They found that weighting prompts by their discriminative signal can improve performance under domain shift, but only if the prompts are specific to the new domain. A generic pool of prompts does not work as well. The study also introduced a method for detecting epistemic uncertainty in these models based on prompt disagreement. --- Why it matters: This matters because it shows that current zero-shot classification methods can be improved by using more tailored prompts, which could increase their reliability and accuracy in real-world applications. It also provides insights into how to better detect when these models are uncertain or failing. Source: https://arxiv.org/abs/2608.18116

This article was originally published at: https://arxiv.org/abs/2608.18116