A fabulous coinage
"Secret societies"

Must reading for social scientists who care at all about public policy

Read Bruce D. McCullough and Ross McKitrick's new paper, "Check the Numbers: The Case for Due Diligence in Policy Formation". (I thank Professor McCullough for the pointer.)

From the beginning, where the authors classify replication in the social sciences as an under-provided public good, to the last sentence, "Disclosure of data and code for the purpose of permitting independent replication in no way intrudes on or imperils academic freedom; instead, it should be seen as essential to good scientific practice, as well as a contribution to better public decisionmaking," it is superb.

In between, the authors discuss some recent horrible examples, including the following:

--The Harvard Six Cities study

--The Boston Fed study

--The "hockey stick" graph

--The U.S. obesity epidemic (JAMA paper)

--The Bellesiles affair

--The JPE file sharing paper

Calls for more replication in social science research are now astonishingly old--I was in graduate school or shortly out of it when I read a paper by the then editor of Economic Inquiry calling forcefully for more, and his wasn't the first. There's been some improvement since then, but shamefully little. 

This paper deserves a wide audience.