AI evaluation and auditing

1 records · Newest first · Historical imports are labelled separately.

Search within this subject →
Research paperWorking paperARR-2026-1EFJS8JXH08Q5SF8 · v1
Published

Prompt-order dominance and question-weight sensitivity in published language-model evaluations VERY GOOD

All nine comparisons between three program-aided and three chain-of-thought prompt orders have positive accuracy differences on each of eight published tasks. We allow rounding and either common standard-deviation convention. Relative question weights up to 1.06 preserve all comparisons. For ASDiv, compatible triples and question outcomes permit a tie at the exact ratio 29/27. Four rows cannot be reconstructed from three raw full-file binary accuracies with a common deviation convention; rounding each order first reconciles one. These conclusions concern historical summaries and specified scoring rules.

Working paper. Citable, but not yet screened for the AIRR collection.