Interpretable Medical Imaging Pipeline — Chest X-ray

Problem

A raw disease probability is not enough to trust a medical model. A clinician needs to know what the model saw, where it looked, to have it explained in the language of a radiology report, and to be able to ask follow-up questions. Can one pipeline classify thoracic pathologies, visually justify each prediction, write a structured report, and answer questions grounded in both the scan and medical knowledge?

Approach

An end-to-end pipeline. An uploaded chest X-ray is preprocessed (with source-aware normalization) and passed through a DenseNet121 multi-label classifier, trained from scratch on the CheXpert dataset, to produce per-pathology probabilities. Every prediction is explained with an on-demand Grad-CAM heatmap (click a pathology to see where the model looked, with an opacity slider over the scan). Confidence-thresholded findings are handed to a local LLM (Llama 3.1 via Ollama) that writes a structured, editable radiology report exportable to PDF. A retrieval-augmented "Ask about this X-ray" chatbot answers follow-up questions, grounding scan-specific questions in the actual findings and general medical questions in a curated knowledge base via cosine-similarity retrieval. The whole thing is a React + FastAPI app, fully self-hosted.

Result

A working end-to-end system built around a DenseNet121 trained on CheXpert reaching a mean AUROC of 0.847 across five pathologies (Atelectasis, Cardiomegaly, Consolidation, Edema, Pleural Effusion). It goes from an uploaded X-ray to per-pathology probabilities, per-class Grad-CAM explanations, a generated radiology report, and a grounded chatbot, all running locally so no image or PHI ever leaves the machine. The evaluation view is deliberately honest about the limitation: AUROC is strong, but at a fixed 0.5 cutoff sensitivity is low on the rarer findings (class imbalance), so the model ranks well before it decides well. Per-condition thresholds tuned on validation are the next step.

Highlights

• Self-trained DenseNet121 on CheXpert, mean AUROC 0.847 across 5 pathologies
• On-demand per-class Grad-CAM: click a finding to see where the model looked
• Local LLM radiology report (Llama 3.1 via Ollama), editable, with PDF export
• RAG chatbot grounded in scan findings + a curated medical knowledge base
• Evaluation view with ROC curves and per-label AUROC, including where the model is weak
• React + FastAPI, fully self-hosted for PHI privacy. A research and education demo, not a diagnostic tool