LLMs in Alzheimer's Disease: Systematic Review Flags Performance Drop on External Validation

Current evidence does not support LLM use in AD/MCI without clinician oversight. Systematic review of 11 studies: diagnostic models had high internal accuracy (0.94-0.97) but dropped on external validation (3-way accuracy -7% pts; MMSE R² 0.90→0.25).

• Systematic review of 11 studies evaluating LLMs for diagnosis (n=3), treatment (n=2), and education (n=8).
• Diagnostic models internal accuracy 0.94-0.97, but external validation saw three-way classification accuracy drop ~7% pts and MMSE-prediction R² fell from 0.90 to 0.25.
• Educational outputs lacked source attribution and exceeded reading levels; only 2/11 studies measured hallucination.

Small heterogeneous evaluations, sparse hallucination measurement, and absence of prospective clinical validation limit generalizability.

Automated summary

RadPigeon summaries are original and for information only. They are not clinical advice.