Public chest X-ray AI datasets show major label errors, bias, and poor external performance

Two board-certified radiologists reviewing public chest X-ray datasets found significant label disagreement. Cross-dataset testing of AI models revealed sharp drops in external performance (AUPRC and F1 scores) versus internal tests, and a source-classification model could ident…

• Expert radiologist review frequently disagreed with the automatically extracted labels in datasets such as MIMIC-CXR, ChestX-ray14, PadChest, and CheXpert.
• Subgroup analyses revealed reduced model performance for minority age and sex groups, demonstrating population bias.
• The study did not report whether any models underwent prospective or external clinical validation.

Automated summary

RadPigeon summaries are original and for information only. They are not clinical advice.