AfaraBook a call
← All articles

November 18, 2026

Suvrankar Datta and the Case for Clinical AI Evaluation Built in India

Suvrankar Datta and the Case for Clinical AI Evaluation Built in India

Abstract radiology review panels, annotation grids and a human handover decision point in a muted clinical-tech palette

Meta description: Clinical AI validation means testing accuracy, reliability and safe handover. Suvrankar Datta’s work in India shows how to examine each.

Clinical AI validation is the process of checking whether an AI system performs reliably and safely for a defined clinical task. Radiologist and physician-scientist Suvrankar Datta offers a useful example: his work tests medical AI against expert readers, repeats evaluations to assess reliability, and asks when a system should defer to a clinician.

Datta is based in New Delhi, India. Across radiology benchmarks, imaging models and multilingual clinical tools, his work asks a practical question: what evidence is needed before an AI system can be trusted in a real clinical workflow?

Who he is and where the work comes from

Datta completed his MBBS at JIPMER Puducherry, where he was a gold medallist, and his MD in Radiodiagnosis and Interventional Radiology at AIIMS New Delhi. He is the founder and Group Lead of the Centre for Responsible Autonomous Systems in Healthcare (CRASH Lab) at the Koita Centre for Digital Health, Ashoka University, and a Simons Ashoka Early Career Fellow. His profile and the CRASH Lab team page describe his clinical and research work.

His evaluation questions have roots in two very different care settings. During a two-month residential posting at a rural primary health centre in Ramanathapuram, he shared emergency coverage, ran non-communicable disease clinics and camps, and referred patients onward. The setting had medicine shortages and no operative facilities.

Later, in high-volume public-sector radiology at AIIMS New Delhi, he independently interpreted more than 200 examinations during emergency shifts, mostly CT and ultrasonography. These experiences do not stand in for every Indian clinical setting. They do help explain why his research looks beyond a model’s answer: clinical tools must be evaluated in relation to the people, resources and workflows around them.

What clinical AI validation looks like in RadLE: the finding that reframes the debate

Datta is co-first author and benchmark co-lead of Radiology’s Last Exam (RadLE), a pilot benchmark submitted on 29 September 2025. It brought together 50 expert-level “spot diagnosis” cases across multiple imaging modalities and tested five frontier AI models through their native web interfaces: OpenAI o3, GPT-5, Gemini 2.5 Pro, Grok-4 and Claude Opus 4.1.

The researchers compared model results with board-certified radiologists and radiology trainees. Blinded experts scored diagnostic accuracy, and the team assessed reproducibility across three independent runs. In the paper’s abstract, board-certified radiologists achieved 83% accuracy, compared with 45% for trainees. The best-performing AI model, GPT-5, achieved 30%.

Those figures are a clear result for this benchmark, not a universal estimate of performance across all radiology cases or clinical settings. RadLE is a pilot built around difficult, expert-level cases. Its value lies partly in making the task and comparison explicit, rather than treating a broad claim of “expert-level” performance as sufficient evidence.

The repeated runs add another dimension. In the paper’s findings, reliability was substantial for GPT-5 and o3, moderate for Gemini 2.5 Pro and Grok-4, and poor for Claude Opus 4.1. An accuracy score describes how often answers were correct in the test. Repeating the evaluation helps ask whether a model gives stable results, rather than producing a good score only in one run.

The work was accepted at RSNA 2025 as a Cutting Edge Oral Presentation. Its central point is not that one number settles whether AI belongs in radiology. It is that medical AI evaluation needs a defined task, a human comparison, expert review and a way to examine consistency.

Why a taxonomy of reasoning errors matters more than a leaderboard rank

A leaderboard can order systems by a score, but a rank alone says little about the nature of their mistakes. RadLE also proposes a taxonomy of visual reasoning errors. That gives researchers a structured way to examine failures rather than recording each one only as an incorrect answer.

For product and technical teams, this distinction matters. A model’s overall accuracy does not show whether its errors are consistent, whether they could be caught in review, or whether the model should have passed the case to a human. A useful evaluation therefore examines not only whether an answer is correct, but also what the model did when it was uncertain.

RadLE 2.0 extends the benchmark to 200 expert-level cases and adds uncertainty-aware measures covering accuracy, reliability, safety, confidence and handover readiness. The technical report describes this broader evaluation without reducing readiness to a single accuracy score.

Handover readiness is a particularly important question for clinical AI. If a model is not ready to make a diagnosis independently, a workflow needs to establish when a clinician should take over. Evaluation can test whether the system’s confidence supports that decision, rather than assuming that every case should receive a model-generated answer.

Three repeated abstract imaging-evaluation panels connected by a review loop to a human checkpoint

What MedVersa adds to the picture

Datta is also a co-author of MedVersa, a generalist foundation model for diverse medical imaging tasks, published in NEJM AI in 2026 and led by Harvard investigators.

The reported finding is specific to draft reporting: MedVersa’s generalist-model draft reports reduced radiologists’ reporting time and clinically important discrepancies while supporting accurate reporting. This matters because it looks at the model as part of a radiologist’s work, rather than treating image interpretation as a stand-alone contest between a person and a system.

It also clarifies the difference between assistive and autonomous use. Evidence that a model can support radiologists with draft reports is not, by itself, evidence that it should report independently. Those are distinct uses, and they call for distinct evaluation questions.

The unglamorous infrastructure work

A benchmark can test systems on defined cases. But clinical AI also needs data and evaluation methods that reflect the populations and workflows where it may be used.

Datta is principal investigator on a three-year DST-A*STAR India-Singapore programme developing indigenous radiology datasets, subgroup evaluations and culturally congruent vision-language models for South Asian populations. The focus is not simply whether a model performs well on average. It is whether evaluation can reveal differences across relevant patient groups.

That distinction matters because an overall result can conceal variation between subgroups. Without subgroup evaluation, a strong average score cannot answer whether performance is consistent for the people a system is meant to serve. This programme makes local data and subgroup analysis part of the research question, rather than an afterthought.

Other projects extend that attention to clinical context. Datta supervises clinical research and technical work on PrAImaan, a Gates Foundation-supported programme for blinded, reproducible and locally relevant evaluation of health AI systems in India, under principal investigator Prof. Anurag Agrawal.

He also supervises clinical evaluation on V2DD, which examines multilingual AI scribes and voice-to-structured-data systems. Its evaluation areas include transcription fidelity, critical omissions, hallucinations, clinician correction burden, privacy, usability and interoperability. These measures look at more than whether a transcript reads fluently: they ask what information is captured, what is missed, and what work remains for clinicians.

Datta is also principal investigator on PREDICT-AI, an IndiaAI CATCH-supported consortium with Tata Memorial Hospital and Manentia AI. The programme focuses on decision support for post-chemotherapy RP-LND planning in testicular cancer. Together, these projects show why medical AI evaluation is not one test repeated everywhere. Imaging benchmarks, draft reports, voice systems and decision support each raise different questions about performance and use.

What Afara takes from this

The work offers a clear lesson: evaluation should reflect clinical tasks, local context and the consequences of a wrong or unstable output. That is why Afara puts credentialed healthcare professionals from the global south (including physicians, nurses, pharmacists and radiographers) in the evaluation loop, rather than treating clinical review as a checkbox.

If you are working through how to evaluate a clinical AI system for a specific workflow, a useful next step is to define the task, the relevant users and the failures that matter.

FAQ

How accurate are AI models at reading medical images compared with radiologists?

In the 50-case RadLE pilot, board-certified radiologists achieved 83% diagnostic accuracy, while the best-performing AI model, GPT-5, achieved 30%. These figures describe that benchmark’s expert-level cases; they should not be treated as a general score for every imaging task.

What is clinical AI validation?

Clinical AI validation is the process of testing whether an AI system performs reliably and safely for a defined clinical task. Depending on the system, evaluation may examine accuracy, reproducibility, uncertainty, errors, workflow fit and whether the model should hand a case to a clinician.

Why do medical AI models fail on local patient populations?

A result measured across a dataset may not reveal how performance varies across patient groups or settings. Datta’s India-Singapore programme addresses this evaluation gap by developing indigenous radiology datasets and subgroup evaluations for South Asian populations.