Teaching agentic AI to learn expert reasoning for rare disease diagnosis
Researchers have developed a method to teach artificial intelligence systems to reason like experts in diagnosing rare diseases. The approach, called liteOdyssey, uses a process called Policy Iteration with Human Feedback (PIHF) to improve the accuracy of large language models (LLMs). In tests on over 1,200 cases of rare diseases, the system ranked the correct disease first about 59% of the time, compared to around 26% without the policy. The method allows experts to inspect
Researchers have developed a method to teach artificial intelligence systems to reason like experts in diagnosing rare diseases. The approach, called liteOdyssey, uses a process called Policy Iteration with Human Feedback (PIHF) to improve the accuracy of large language models (LLMs). In tests on over 1,200 cases of rare diseases, the system ranked the correct disease first about 59% of the time, compared to around 26% without the policy. The method allows experts to inspect and revise the AI's reasoning, and can be transferred across different models. This could potentially improve diagnosis rates for rare diseases.
---
Why it matters: This work matters because it addresses a significant challenge in medical diagnosis: rare diseases often require expert-level reasoning that is difficult to transfer or replicate using traditional machine learning approaches. By developing an agentic AI system that can learn and adapt this expertise, researchers are taking a crucial step towards improving diagnosis rates for these conditions.
Source: https://arxiv.org/abs/2606.16149
This article was originally published at: https://arxiv.org/abs/2606.16149