Researchers evaluated the effectiveness of large language models (LLMs) as data quality annotators. They tested LLMs on two e-commerce tasks: entity matching and brand mislabeling. The results showed that LLMs performed similarly to rule-based baselines in entity matching, but outperformed them in brand mislabeling due to their ability to access background knowledge. However, the consistency of LLM judgments was high, with 99.7% agreement across repeated runs. This suggests that LLMs may offer little advantage when strong lexical signals already exist, but can be beneficial when tasks require background knowledge. --- Why it matters: These findings are important for AI engineers and researchers because they highlight the limitations and potential benefits of using LLMs as data quality annotators. Understanding when LLMs excel or falter can help optimize their use in real-world applications. Source: https://arxiv.org/abs/2608.18158