I had no idea things were this bad
"Secret societies"

Must reading for social scientists who care at all about public policy

Read Bruce D. McCullough and Ross McKitrick's new paper, "Check the Numbers: The Case for Due Diligence in Policy Formation". (I thank Professor McCullough for the pointer.)

From the beginning, where the authors classify replication in the social sciences as an under-provided public good, to the last sentence, "Disclosure of data and code for the purpose of permitting independent replication in no way intrudes on or imperils academic freedom; instead, it should be seen as essential to good scientific practice, as well as a contribution to better public decisionmaking," it is superb.

In between, the authors discuss some recent horrible examples, including the following:

--The Harvard Six Cities study

--The Boston Fed study

--The "hockey stick" graph

--The U.S. obesity epidemic (JAMA paper)

--The Bellesiles affair

--The JPE file sharing paper

Calls for more replication in social science research are now astonishingly old--I was in graduate school or shortly out of it when I read a paper by the then editor of Economic Inquiry calling forcefully for more, and his wasn't the first. There's been some improvement since then, but shamefully little. 

This paper deserves a wide audience.

Comments