Published
Prompt-order dominance and question-weight sensitivity in published language-model evaluations VERY GOOD
All nine comparisons between three program-aided and three chain-of-thought prompt orders have positive accuracy differences on each of eight published tasks. We allow rounding and either common standard-deviation convention. Relative question weights up to 1.06 preserve all comparisons. For ASDiv, compatible triples and question outcomes permit a tie at the exact ratio 29/27. Four rows cannot be reconstructed from three raw full-file binary accuracies with a common deviation convention; rounding each order first reconciles one. These conclusions concern historical summaries and specified scoring rules.
Working paper. Citable, but not yet screened for the AIRR collection.