An expert-moderated benchmark for evaluating AI exercise advice Status: Pilot study protocol. Data collection not started.
- Pilot protocol
- Author: Dr Alan Ruddock
ExerciseBench is a planned benchmark for evaluating how large language models answer exercise questions, built on HealthBench but with transparent theme development, expert-moderated rubrics and checks of the AI grader against experts. This repository holds the pilot protocol.
Ruddock, A. (2026). ExerciseBench: Development of an expert-moderated benchmark for evaluating AI exercise advice, protocol for a pilot study (Version 0.2). https://github.com/aprofrudd/ExerciseBench
© 2026 Alan Ruddock. The ExerciseBench protocol is licensed under CC BY 4.0. You may share and adapt it for any purpose, provided you credit the author and indicate any changes. Full terms are in LICENSE.
The source document is maintained in Word with Zotero references and converted to protocol.md for each release. Please do not edit protocol.md directly.
- v0.2 (7 October 2026): adds Khasentino et al. (2025) on PH-LLM and clarifies the gap ExerciseBench addresses.
- v0.1 (3 October 2026): first public pilot protocol.