Learning Ophthalmologist Clinical Reasoning for Glaucoma Diagnosis from Fundus Images

Abstract

Clinical reasoning before diagnosis

Glaucoma is a leading cause of irreversible blindness worldwide. Ophthalmologists diagnose it through a structured reasoning process — sequentially evaluating optic-nerve-head characteristics before reaching a final diagnosis — whereas existing AI systems typically perform direct image classification without clinically meaningful reasoning. We present the first clinically annotated fundus reasoning dataset, 1,077 fundus photographs paired with expert-authored six-step diagnostic reports, and a reasoning-driven vision–language framework that explicitly models the ophthalmologist's diagnostic workflow by generating structured clinical reasoning prior to diagnosis. The generated reports are clinically validated, achieving the best performance across all evaluated clinical findings — a cup-to-disc-ratio MAE of 0.070, an ISNT Kendall distance of 1.73, and the highest semantic agreement with expert reports (BERTScore-F1 = 0.874). The framework also improves diagnosis, reaching a balanced accuracy of 94.7% and precision of 94.8%, showing that explicitly modeling expert clinical reasoning simultaneously improves interpretability and diagnostic performance.

1,077

expert-annotated fundus images

94.7%

balanced accuracy (diagnosis)

0.070

cup-to-disc-ratio MAE

0.874

BERTScore-F1 vs. experts

artificial intelligence retinal fundus imaging glaucoma screening chain-of-thought multimodal language model

Framework

Structured clinical evidence, then reasoning, then diagnosis

Conventional screening maps a fundus image directly to a label. Our framework keeps the same frozen RETFound backbone but inserts explicit clinical reasoning: the backbone first produces structured clinical evidence, a vision–language model then writes a six-step chain-of-thought, and only then is the binary diagnosis made. The improvement therefore comes from added clinical reasoning, not from a stronger visual backbone.

Overview of the framework: conventional AI vs. clinical-reasoning diagnosis, and construction of the expert glaucoma CoT dataset. — **Figure 1 — Overview.** (a) Compared with conventional glaucoma screening, our method incorporates structured clinical reasoning before diagnosis while using the same RETFound backbone. (b) Construction of the expert glaucoma CoT dataset: 1,077 fundus images annotated with six-step clinical reasoning and binary glaucoma labels.

The six-step clinical chain-of-thought

Step 1

Image quality

Assess focus, contrast, artifacts and disc-margin clarity for reliable analysis.

Step 2

CDR evaluation

Estimate vertical & horizontal cup-to-disc ratio against the glaucomatous cutoff.

Step 3

ISNT rule

Rank inferior–superior–nasal–temporal rim widths and flag violations.

Step 4

Glaucomatous signs

Check notching, RNFL defect, bayoneting, PPA / β-zone atrophy, disc hemorrhage.

Step 5

Structural summary

Integrate the findings and weigh the automated probability against them.

Step 6

Final classification

Reason to a binary glaucoma / non-glaucoma decision with justification.

Model output

A predicted report, verbatim

A Papila test-split fundus photograph is passed to the model, which outputs the six-step report below. Steps 1–5 are the reasoning; step 6 is the final classification — here matching the ground-truth label.

Predicted six-step chain-of-thought report for a glaucomatous eye. — **Figure 2 — Predicted CoT report (glaucomatous eye).** The report is the model's output verbatim; the prediction matches the ground-truth label.

BIOVLM dataset

The first expert glaucoma chain-of-thought dataset

1,077 color fundus photographs from the public LAG and Papila datasets, each paired with a complete expert-authored six-step reasoning report (average length 321.6 ± 46.3 words) and a binary label.

Dataset composition pie chart and vertical/horizontal cup-to-disc ratio distributions by diagnosis. — **Figure 4 — Composition & cup-to-disc ratio.** Expert-annotated CDR cleanly separates the classes: median vertical CDR 0.63 vs. 0.33, horizontal 0.64 vs. 0.35 (glaucoma vs. non-glaucoma).

**Table 1 — Dataset composition.**
Characteristic	Value
Fundus images	1,077
Glaucoma	430 (39.9%)
Non-glaucoma	647 (60.1%)
LAG database	659 (343 / 316)
Papila dataset	418 (87 / 331)
Training split	825 (315 / 510)
Validation split	92 (35 / 57)
Test split	160 (80 / 80)
Avg. report length	321.6 ± 46.3 words

Six reasoning indicators are annotated per image (steps 2–4 of the CoT):

Vertical & transverse cup-to-disc ratio (continuous 0–1)
ISNT rim ranking across the four quadrants (order)
Per-quadrant rim abnormality (3 ordinal levels × 4)
Six glaucomatous structural signs (presence × 6)

Every image is paired with an expert-authored reasoning report, making this the first large-scale glaucoma fundus dataset with multi-step reasoning annotations.

Results

Best clinical findings, best diagnosis

Against a fine-tuned Qwen3.5-VL and the proprietary foundation models Claude and GPT-5.5, our framework achieves the strongest performance on every evaluated clinical indicator and the highest semantic agreement with expert reports.

Bar charts of clinical-finding correctness and report-quality metrics. — **Figure 8 — Report correctness & quality.** Lower is better for CDR MAE and ISNT Kendall distance; higher is better for macro-F1, BERTScore-F1 and ROUGE-Lsum.

**Report generation** — clinical findings & report agreement (test split).
Metric	Ours	Qwen3.5-VL	Claude	GPT-5.5
CDR MAE ↓	0.070	0.102	0.135	0.127
ISNT Kendall ↓	1.73	2.07	2.27	n/a
Rim-level macro-F1 ↑	0.656	0.512	0.510	0.373
Signs macro-F1 ↑	0.613	0.501	0.465	0.375
BERTScore-F1 ↑	0.874	0.867	0.869	0.866
ROUGE-Lsum ↑	0.454	0.416	0.461	0.439

**Diagnosis** — balanced accuracy vs. foundation & language models.
Model	Bal. Acc.	95% CI
Ours (VLM)	94.7%	90.6–98.1
RetiZero	93.6%	88.1–98.0
RETFound	93.1%	87.6–97.5
DINOv2-L	84.7%	77.1–91.2
Qwen3.5-VL	84.3%	78.8–89.5
Claude	74.4%	67.5–81.0
GPT-5.5	71.9%	65.1–78.2

Precision and recall comparison across retinal foundation models and vision-language models. — **Figure 10 — Diagnosis performance.** Our framework attains the highest overall precision (94.8%) and recall (94.7%) while staying balanced across glaucoma and non-glaucoma cases — for example 96.5% glaucoma precision with 96.5% non-glaucoma specificity, avoiding the class imbalance seen in the language-model baselines.

Ablation studies

Reasoning is what carries the diagnosis

Removing the reasoning stage, or replacing structured evidence with only an initial diagnostic signal, collapses the balanced diagnosis — confirming that the structured clinical evidence and intermediate reasoning, not the backbone alone, drive reliable performance.

**Pipeline ablation.** Balanced accuracy with sensitivity / specificity.
Configuration	Bal. Acc.	Sens.	Spec.
Full framework	94.69%	92.88%	96.50%
Abl. 1 — image only	83.75%	97.50%	70.00%
Abl. 2 — image + diagnosis	70.63%	98.75%	42.50%

**Leave-one-indicator-out.** Impact of removing each clinical cue.
Variant	Bal. Acc.
Ours (all indicators)	94.69%
− per-quadrant rim status	94.38%
− ISNT rim ordering	94.37%
− glaucomatous signs	93.75%
− cup-to-disc ratio	92.50%

The cup-to-disc ratio is the single most informative cue (largest drop when removed), while every indicator contributes complementary diagnostic information.

Citation

BibTeX

@article{zhou2026glaucoma,
  title   = {Learning Ophthalmologist Clinical Reasoning for Glaucoma Diagnosis from Fundus Images},
  author  = {Zhou, Kaichen and Chen, Yuzhen and Yildiz, Elif and Shi, Min and Dai, David
             and Chen, Grace and Zheng, Jiale and Wang, He and Zhan, Fangneng and Saini, Chhavi
             and Shen, Lucy Q. and Guo, Yike and Liang, Paul Pu and Wang, Mengyu},
  journal = {npj Digital Medicine (under review)},
  year    = {2026}
}