Researchers have developed a new framework to evaluate the performance of document parsers in extracting mathematical formulas from PDFs. The benchmarking system generates synthetic PDFs with precise LaTeX ground truth, allowing for controlled evaluation of layout, formulas, and content characteristics. The team applied a novel evaluation metric, 'LLM-as-a-judge', which assesses semantic equivalence of parsed formulas beyond surface-level notation differences. A human study validated this approach, showing high correlation with human judgment. The framework evaluated 20+ contemporary PDF parsers, revealing significant performance disparities. This work provides actionable guidance for practitioners selecting parsers for downstream applications. --- Why it matters: This research matters to AI engineers and researchers because it addresses a critical challenge in training large language models and building scientific knowledge bases from academic literature. Accurate mathematical formula extraction is essential for these tasks, and this framework provides a robust evaluation metric and benchmarking system to support parser development. Source: https://arxiv.org/abs/2512.09874