It would be interesting to see Dani Rodrik respond to this article (pdf) by Richard Jong-a-Pin and Jakob de Haan in the latest issue of Econ Journal Watch:

Economists treat replication the way teenagers treat chastity—as an ideal to be professed but not to be practiced (Hamermesh 2007, 1).

HPR’s [Hausmann, Pritchett, and Rodrik’s] finding that a political regime change increases the probability of an economic growth acceleration is wrong and the result of a data error. When we correct for this error and stick to the definition of political regime change as a three-unit change in Polity, we find that regime changes do not affect the probability that a growth acceleration occurs. We also find some evidence that economic liberalization increases the probability of a growth acceleration (sustained or otherwise)…

The work represented here was submitted, of course, to the Journal of Economic Growth, although in that version of the paper we had not yet pinpointed the data-description error in the Polity IV manual. The paper was rejected on the basis of the argument that our note is a “welcome correction, however, of limited significance for the main contribution of the original paper.” However, in their abstract, HPR state that one of their main conclusions is that “Political regime changes are statistically significant predictors of growth accelerations.”

Jakob de Haan blogs about the experience:

As our paper was a comment on a previously published paper in the Journal of Economic Growth, it is unlikely to be accepted by another journal. However, a relatively new electronic journal called Econ Journal Watch, recognizes the importance of replication in economics. The editor of that journal, Dan Klein, was therefore happy to publish our paper. It will be published in the first issue of 2008. Of course, HPR get the opportunity to reply to our critique.

Even though I am very happy with this new outlet, I feel that editors of all scientific journals should pay much more attention to replication. A starting point is that authors of published empirical research should commit to make their data available to anyone interested. Unfortunately, even this is not common practice.

Admittedly, the primary achievement of the Growth Accelerations paper was to change how we think about identifying economic growth in a relevant manner. I certainly didn’t recall the regime change finding when I thought of the article. Nonetheless, a data coding error seems like a substantive correction, and I haven’t seen any reply from Hausmann, Pritchett or Rodrik.

I should also note that the Hamermesh paper is interesting in itself.