LLMs Nearly Perfect for Low CAD-RADS, Struggle with High-Grade Stenosis and Management

Claude 3.7 achieved almost perfect CAD-RADS agreement (κ=0.997) on synthetic CCTA reports, but all LLMs struggled with high-grade stenosis (4A/4B) and management recommendations. Performance varied by prompt identity (radiologist vs cardiologist).

• Four LLMs were tested on synthetic CCTA reports; sample size not specified.
• Management recommendation agreement was lower than CAD-RADS scoring, with greatest variability at CAD-RADS 3.
• Limitation: synthetic data may not reflect real-world clinical variability; external validation absent.

Automated summary

RadPigeon summaries are original and for information only. They are not clinical advice.