02 October 2012

Reviewing peer review

I submitted "When worlds collide: business meets bureaucracy in the water sector" to Water Alternatives in April, and it was rejected five months later. I've decided to put the paper in my "finished" section since it's not worth revising and submitting elsewhere.

For the sake of completeness (and perhaps some insight into the world of academic peer-review), I include here the results of five months of referee effort -- plus my replies.

Reviewer #1
Does the abstract reflect the contents and conclusions of the paper? Insufficient evidence to support the conclusions presented.

Is the article original or creative? An interesting idea, but not presented to its full advantage.

Are the evidence and arguments sound and convincing? No; insufficient support for the central hypothesis; lack of current scholarly references and bold assertions without convincing evidence in support.

Are the figures necessary and of good quality? Simplistic and could use references to more detailed studies.

Is the language clear and correct? Needs a good English language edit and work on grammar. Inconsistent use of terms.

Does the article reference appropriate citations in the relevant literature? This is largest shortcoming – needs current and relevant research citations, including current World Bank and related reports on this topic.

General comments: Interesting topic, but not a scholarly work; more a provocative think-piece that fails to convince due to lack of concrete supporting evidence for some pretty bold assertions. Perhaps more of a policy piece; which still requires considerable grounding in practice.

My reply: Reviewer #1 wanted more citations of other academic works but did not provide any references (something that's both polite and a best practice, even when referencing the reviewer's own work), which makes it hard to know what I was missing. In my experience, these objections mean either that my points have been made already or that I need to integrate others' work into my point. I'll never know.

Reviewer #2
The general premise of the manuscript, that institutions must adapt to changing water scarcity conditions in order to effectively manage water resources, is sound. The four domains identified in Table 1 need to be fully developed. A paragraph or two on the decision frameworks within each domain would be helpful. The argument for market-based allocation strategies should discuss “greater good” criteria in more detail. Is it better to allocate water resources for municipal or agricultural use, where resources are limited? Why? Explore the underlying ethic of greater good to support the argument. Water rights should be at least discussed if not addressed directly. See Mike Acreman,, 2001 (Water Policy 3(3):257-265)

My reply: Reviewer #2 is asking for clarifications without rejecting the paper.

Reviewer #3
I have taken a look at the Zetland paper, and I don’t think it is worth including in this issue. Much of it is a primer on economic theory; most of it makes sweeping statements without evidence to back them up. For example: “Market pricing signals are already very effective in allocating other goods and inputs among trading countries, and they could be used for sustainable water allocation. “ This has nothing to back up the assertion, and ignores the research that has been done showing that market allocation of water is very problematic. The examples that are there are scatter-shot, rather than building a solid foundation for the arguments.

My reply: Reviewer #3 appears to misunderstand the nature of markets, trade and the meaning of "could." S/he also fails to provide any references. I've never seen evidence that "market allocation of water is very problematic" except in circumstances that the initial allocation is also flawed. That's because there are, to my knowledge, no instances of "market allocation;" there are only instances of "market RE-allocation" of water rights that were originally allocated through political means.

Overall, I am surprised that these comments are so brief, vague and dismissive, but Water Alternatives is known to have a political slant that is less accommodating to market-friendly discussions. Rejection is easier than engagement.

Bottom Line: Social scientists engineers spend more time arguing that they ARE right (biased citations, theoretical models, and empirics that support their pre-conceived notions) than analyzing problems and leaving conclusions to readers. Back to blogging.

No comments:

Post a Comment

Spam will be deleted. Comments on older posts must be approved.
If you're having problems posting, email your comment to me