TRIPOD-LLM: Standardized Reporting for LLMs in Healthcare

TRIPOD-LLM is now published in Nature Medicine

The TRIPOD-LLM reporting guideline for studies using large language models

Jack Gallifant^1,2,3Harvard Medical School, Department of Biomedical Informatics, Majid Afshar^4,20Healthcare Analytics and Clinical Informatics, Saleem Ameen^1,5,6,29Harvard Medical School, Yindalon Aphinyanaphongs^7,29NYU Langone Health, Shan Chen^8,9,28University of Oxford, Giovanni Cacciamani^8,10,29Keck School of Medicine USC, Dina Demner-Fushman^11,29National Library of Medicine, Dmitriy Dligach^12,29Loyola University Chicago

Roxana Daneshjou^13,14,29Stanford University School of Medicine, Chrystinne Fernandes^1,29Harvard Medical School, Lasse Hyldig Hansen^15,29Aarhus University, Adam Landman^16,20Brigham and Women's Hospital, Lisa Lehmann^16,29Brigham and Women's Hospital, Liam G. McCoy^17,29University of Toronto, Timothy Miller^18,29Boston Children's Hospital, Amy Moreno^19,29Duke University School of Medicine

Nikolaj Munch^15,29Aarhus University, David Restrepo^1,20,29Harvard Medical School, Guergana Savova^18,29Boston Children's Hospital, Renato Umeton^21,29Dana-Farber Cancer Institute, Judy Wawira Gichoya^22,29Emory University School of Medicine, Gary S. Collins^23,24University of Oxford, Karel G. M. Moons^25,26University Medical Center Utrecht, Leo A. Celi^1,27,28Harvard Medical School, MIT, Danielle S. Bitterman^1,8Harvard Medical School, Dana-Farber Cancer Institute

Nature Medicine Paper

Interactive Website

Standardized Reporting for LLMs in Healthcare

TRIPOD-LLM provides standardized reporting guidelines for large language models in healthcare applications. This extension of the TRIPOD framework addresses the unique challenges of LLMs through a comprehensive checklist of 19 main items, covering key aspects from research design to clinical applicability. Developed through expert consensus, TRIPOD-LLM emphasizes transparency, human oversight, and task-specific performance reporting, supported by an interactive website for guideline completion.

Why Standardized Reporting Matters

Large language models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. Current LLM research often lacks consistency in how methods and results are reported, making it difficult to assess and compare different approaches.

TRIPOD-LLM extends the TRIPOD + artificial intelligence statement, addressing the unique challenges of LLMs in biomedical applications.

TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories.

Key Features of TRIPOD-LLM

Modular Framework

Accommodates various LLM research designs and tasks with a flexible structure that can be adapted to different use cases while maintaining reporting consistency.

Comprehensive Checklist

19 main items and 50 subitems covering all aspects of LLM research from title and abstract to methods, results, and discussion.

Task-Specific Performance

Emphasizes the importance of reporting performance metrics that are relevant to the specific healthcare task being addressed.

Interactive Website

Interactive tool for easy guideline completion and PDF generation for submission with research papers.

Interactive Website

To facilitate the adoption of TRIPOD-LLM, we developed an interactive website that guides researchers through the process of completing the checklist, providing context and examples for each item.

Visit TRIPOD-LLM Website

How To Cite

When using TRIPOD-LLM for your research, please cite the following paper:

Bibliography

Gallifant, J., Afshar, M., Ameen, S. et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med (2025). https://doi.org/10.1038/s41591-024-03425-5

BibTeX

@article{gallifant2025tripod,
  title={The TRIPOD-LLM reporting guideline for studies using large language models},
	author = {Gallifant, Jack and Afshar, Majid and Ameen, Saleem and Aphinyanaphongs, Yindalon and Chen, Shan and Cacciamani, Giovanni and Demner-Fushman, Dina and Dligach, Dmitriy and Daneshjou, Roxana and Fernandes, Chrystinne and Hansen, Lasse Hyldig and Landman, Adam and Lehmann, Lisa and McCoy, Liam G. and Miller, Timothy and Moreno, Amy and Munch, Nikolaj and Restrepo, David and Savova, Guergana and Umeton, Renato and Gichoya, Judy Wawira and Collins, Gary S. and Moons, Karel G. M. and Celi, Leo A. and Bitterman, Danielle S.},
  journal={Nature Medicine},
  year={2025},
  publisher={Nature Publishing Group},
  doi={10.1038/s41591-024-03425-5}
}