Pathology copilot learns from 1,000 minutes of gaze and slide navigation

Zhi Huang@ZhiHuangPhD

🚀 Introducing #PathCowork: Learning from real-world pathology practice to build AI copilots!

Most pathology AI learns what pathologists conclude, but rarely how they actually practice.

Diagnosis is more than recognizing patterns. It requires knowing where to look, when to zoom in, how to connect evidence, and what to examine next.

Using our #TissueLab (t.co/x6kD2XHdou) platform, we captured synchronized eye gaze, voice narration, and slide navigation from 16 pathologists across 48 sessions and over 1,000 minutes of real-world diagnostic examinations at @PennPathLabMed, under an active IRB.

We explored whether AI could learn directly from this clinical diagnostic expertise.

🔍 Learning where to look: Visual grounding improved from 5.6% to 52.2%, enabling AI to better locate diagnostic features in tissue images.

🧭 Learning how to examine: Our agents outperformed Claude Code and Codex across 5 pathology tasks, running ~5× faster at ~10× lower cost.

💻 We are releasing PathCowork-1T and PathCowork-27B, along with code and model weights!

Our vision is to turn everyday clinical practice into a continuous source of learning for medical AI.

The next breakthrough may come not just from scaling models and datasets, but from fundamentally rethinking what AI learns and how it learns.

📄 Paper: t.co/4mrw0AJpod
💻 Code & weights: t.co/WJhf99W1KO
#DigitalPathology #MedicalAI #AgenticAI
@PennPathLabMed @PennMedicine