ProofJudge: Tool-Grounded LLM Evaluation of Formal Proof Quality in Mathlib
Researchers have developed ProofJudge, a tool that uses an artificial intelligence model to evaluate the quality of formal mathematical proofs in Lean 4. The system assesses five aspects of proof quality beyond mere correctness and is trained on a dataset of 218 declarations from Mathlib PRs. Initial results show that the AI judge can accurately recover human preferences for proof quality, with some models achieving up to 80.8% accuracy.
Researchers have developed ProofJudge, a tool that uses an artificial intelligence model to evaluate the quality of formal mathematical proofs in Lean 4. The system assesses five aspects of proof quality beyond mere correctness and is trained on a dataset of 218 declarations from Mathlib PRs. Initial results show that the AI judge can accurately recover human preferences for proof quality, with some models achieving up to 80.8% accuracy.
---
Why it matters: This matters because it shows promise in developing more effective tools for evaluating formal proofs, which could improve the efficiency and reliability of mathematical research.
Source: https://arxiv.org/abs/2608.20432
This article was originally published at: https://arxiv.org/abs/2608.20432