LLMs Extract NSCLC Metastasis from Clinical Notes: Comparative Study of Three Models

Large language models can extract metastasis status from clinical notes in NSCLC: MedGemma achieved F1=0.80 for overall metastasis in a 579-patient cohort, and F1=0.93 at the note level in a radiology cohort. Llama3 reached F1=0.79 for brain/CNS metastasis.

• For brain/CNS metastasis extraction, zero-shot Llama 3.1-70B achieved F1-scores of 0.79 (cohort 1) and 0.93 (patient-level, cohort 2).
• Exploratory recovery in 675 patients with missing registry labels showed agreement accuracy of 0.90 and F1 0.89 against manual annotations.
• The fine-tuned encoder model may have been limited by inaccuracies in the registry-derived reference standard.

Automated summary

RadPigeon summaries are original and for information only. They are not clinical advice.