FDA Awards $1.29M to Rad Partners to Test 'LLMs as a Jury' for AI Radiology Reports

Rad Partners' Cognita Imaging received a $1.29M FDA grant to test an 'LLMs as a jury' framework for evaluating AI-generated radiology reports. The 18-month project applies the method to ~1 million exams, with radiologists reviewing discordant cases to identify failure modes.

• The framework compares judgments from multiple large language models rather than relying on a single LLM, aiming to reduce dependence and improve consistency.
• Radiologists will adjudicate only clinically important disagreements, determining whether errors originate from the AI report, the LLM jury, or the original radiology findings.
• Cognita will deliver software code, guidance for building LLM juries, and validation cohort analyses to the FDA.

Automated summary

RadPigeon summaries are original and for information only. They are not clinical advice.