Issues

• Request for per-row upstream provenance of MMLU auxiliary_train members

  hendrycks/test#30 · Filter by author Egor5647 opened on Sep 13, 2026

• Test split: 78 items shared across two subjects, 27 exact duplicates, 4 repeated-keyed-option items

  hendrycks/test#29 · Filter by author collapseindex opened on Aug 28, 2026

• My first repository

  hendrycks/test#27 · Filter by author UnusersOp opened on Jul 20, 2025

• Incorrect answers

  hendrycks/test#26 · Filter by author cinjon opened on Sep 23, 2024

• What are expected to submit for the leaderboard integration?

  hendrycks/test#25 · Filter by author zhimin-z opened on Mar 4, 2024

• Suggestion: appropriate use of medical word: catheter

  hendrycks/test#24 · Filter by author hiroyaiizuka opened on Feb 26, 2024

• Why is this repository called test?

  hendrycks/test#22 · Filter by author nikhilweee opened on Jan 31, 2024

• Duplicate Answers in Validation Set

  hendrycks/test#21 · Filter by author riedgar-ms opened on Jan 16, 2024

• can not download

  hendrycks/test#20 · Filter by author fourfireM opened on Jan 5, 2024
  1

• Hrvatski

  hendrycks/test#19 · Filter by author Deni7s opened on Dec 22, 2023

• Incorrect answer for Q5 in high_school_computer_science_dev.csv

  hendrycks/test#18 · Filter by author TenType opened on Dec 9, 2023

• Seems that setting logprobs=100 is not useful now.

  hendrycks/test#14 · Filter by author KL4805 opened on Sep 17, 2023
  1

• Answers A, B, C, D are not all equally likely - is it really accurate to use random baseline as comparison?

  hendrycks/test#11 · Filter by author bmosaicml opened on Apr 19, 2023

• Dismatch dataset categories

  hendrycks/test#10 · Filter by author zhichengg opened on Mar 31, 2023

• Fetching encoder json and bpe does not work (fixed by removing a typo)

  hendrycks/test#9 · Filter by author kovacgrgur opened on Feb 28, 2023