SymptomAI improves differential diagnosis by asking questions

On July 22, Google Research presented SymptomAI, a prototype built on Gemini 2.0 Flash that interviews people about symptoms and proposes a differential diagnosis: a list of possible causes, not a confirmed diagnosis. The paper had appeared as a preprint in May, but it merits a reading now because it replaces prepared clinical vignettes with something harder and more realistic: 13,917 conversations in which Fitbit users described health problems in their own words.

The advance is not simply that the model knows about diseases. It is an investigation of whether the model can obtain the information it lacks. Between June 2025 and April 2026, participants were randomly assigned to five conversation designs. The baseline resembled ordinary chat: the user decided what to disclose and the model responded. Four others required the system to ask about details such as onset, location, severity, and progression, either by following a clinical script or by dynamically choosing the next question.

The result is in the interview

On average, the four strategies that required follow-up questions achieved 27.57% higher accuracy than the baseline under the study's measure. There was no statistically significant difference between the two dynamic approaches—in which the model chose what to ask—and the two based on canonical clinical questions. This does not show that improvisation is always as safe as following a protocol. It does show that the interaction design may matter more than opening the same model and expecting a good first answer.

That changes a practical decision for anyone building health AI. A system receiving “my abdomen hurts” should not treat brevity as the whole available history. It needs to know what information is missing, request it without unduly leading the response, and recognize when conversation is insufficient. The relevant capability is a loop: ask, update possibilities, and decide whether it can continue to guide or should escalate. In that architecture, the prompt and conversation flow are part of the clinical instrument being evaluated.

The comparison with doctors is the most striking result and the one requiring the most precision. Of the 13,917 people, 1,228 reported a diagnosis received from a healthcare professional. Manual analysis narrowed this to 517 eligible conversations reviewed by three family physicians. For every case, three lists were produced: SymptomAI's and two from physicians who had read the same transcript. An independent clinical rater, blinded to authorship, ranked SymptomAI's list first in 53.3% of evaluations; the participant-reported diagnosis also appeared more often among its first five possibilities.

The comparison is not a complete consultation

“SymptomAI outperforms doctors” would be too broad a conclusion. The baseline physicians did not interview the person: they received a conversation led by the system itself. Nor was there a verified medical record serving as ground truth. The final label was what the participant remembered and entered about a healthcare visit, and only a fraction of the cohort supplied one. The test measures the quality of differential lists over shared text, not physical examination, test selection, treatment, patient outcomes, or triage safety.

Top-five accuracy also has a specific purpose. It rewards placing the reported cause somewhere among five possibilities; it does not require ranking it first or measure whether uncertainty is communicated well. A broad list can help avoid overlooking a possibility, but in a patient's hands it can also create undue alarm or reassurance. Before turning this result into a product, consequential errors would need measurement: missed warning signs, unnecessary escalation, false reassurance, and performance differences across languages, demographic groups, and rare conditions.

The study adds a second exploration using Fitbit data. The authors used SymptomAI's first diagnosis as a label for all 13,917 conversations and looked for coincident changes in sleep, heart rate, respiration, and other signals. They found coherent associations around acute infections. This is evidence that some model-defined groupings capture a real physiological signal, but it is not independent validation of each diagnosis: the model itself defines the groups that are then compared, and temporal correlation is neither causation nor clinical confirmation.

My reading is that the work identifies an improvement that looks small but runs deep in system design: do not require users to know in advance which medical detail matters. An agent can turn a vague concern into a more useful account for reasoning or for preparing a visit. That could widen access to initial guidance and improve the information reaching a professional.

The next trial should compare complete pathways: real patients interviewed by the system and by clinicians, with appropriate access to prior history, subsequent tests, and outcome follow-up. It should also evaluate abstention and escalation, not only diagnoses included in a list. Until then, SymptomAI is exactly what the paper says it is: a research prototype, not a medical device, a commercial feature, or authorization to replace professional care. Its strongest contribution is not automating the doctor, but showing that a good agent begins by knowing how to ask.

Sources

• Google Research, SymptomAI: Towards a conversational AI agent for everyday symptom assessment, July 22, 2026.
• Joseph Breda et al., SymptomAI: Towards a Conversational AI Agent for Everyday Symptom Assessment, preprint revised May 10, 2026.
• Tao Tu et al., Towards conversational diagnostic artificial intelligence, Nature, April 9, 2025.