ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models
Large vision-language models often generate content that isn't supported by the input image. To prevent this 'hallucination', researchers propose a method called ReWEIGH to measure how strongly an image supports each token in the generated text. ReWEIGH uses a training-free decoding intervention that aggregates evidence from visual positions and compares it with a reference estimated from unlabeled images. This approach reduces hallucinated object mentions by up to 21.3% whil
Large vision-language models often generate content that isn't supported by the input image. To prevent this 'hallucination', researchers propose a method called ReWEIGH to measure how strongly an image supports each token in the generated text. ReWEIGH uses a training-free decoding intervention that aggregates evidence from visual positions and compares it with a reference estimated from unlabeled images. This approach reduces hallucinated object mentions by up to 21.3% while preserving or improving descriptive performance, across various architecture families.
---
Why it matters: This matters because large vision-language models are widely used in applications like image captioning and visual question answering, where accuracy is crucial. ReWEIGH's method can help improve the reliability of these models by reducing hallucinations and providing more accurate results.
Source: https://arxiv.org/abs/2608.19075
This article was originally published at: https://arxiv.org/abs/2608.19075