Vision-Language AI for Chronic Tuberculosis on Chest X-Rays
A vision-language model that reads chest X-rays together with patient histories and treatment records to screen for chronic tuberculosis, and writes its findings up as a diagnostic report a radiologist can check.
Key findings
- Combines chest X-rays with clinical context such as patient histories and treatment records
- Pre-trained on 5 million paired medical images and texts, then fine-tuned on 100,000 chronic-TB chest X-rays
- 94% precision and 94% recall for key chronic TB findings, including fibrosis, calcified granulomas and bronchiectasis
- AUC above 0.93 and intersection-over-union above 0.91 for detecting and localizing TB-related abnormalities
Plain-English summary
- The question
- Can a vision-language model that combines the chest X-ray with the patient's history and treatment records make chronic tuberculosis screening more consistent than manual reading alone?
- The data
- Pre-training on 5 million paired medical images and texts, followed by fine-tuning on 100,000 chest X-rays specific to chronic TB.
- The method
- A Vision Transformer (SIGLIP encoder) encodes the image and a transformer text encoder processes the clinical context. Cross-modal attention aligns the radiographic features with the text, and a Gemma-3b decoder generates the diagnostic report.
- The result
- 94% precision and 94% recall for key chronic TB pathologies, including fibrosis, calcified granulomas and bronchiectasis. AUC scores exceeded 0.93 and intersection-over-union values were above 0.91.
Why it matters for hospitals and radiologists
The paper targets the inconsistency of manual interpretation. Because the model writes its reasoning out as a report tied to localized findings, a radiologist can check what it saw and where, instead of accepting or rejecting a single score.
Related at 5C: Generalized medical AI (GM AI)
Citation
Praveen Shastry, Sowmya Chowdary Muthulur, Naveen Kumarasami, Anandakumar D, Mounigasri M, Keerthana R, Kishore Prasath Venkatesh, Bargava Subramanian, Kalyan Sivasailam, Revathi Ezhumalai, Abitha Marimuthu. Advancing Chronic Tuberculosis Diagnostics Using Vision-Language Models: A Multi modal Framework for Precision Analysis. arXiv preprint arXiv:2503.14536, 2025. https://doi.org/10.48550/arXiv.2503.14536
