Reviewing for Reading Research Quarterly

Reading Research Quarterly publishes the highest quality research in literacy for and by scholars throughout the world. As such, the journal fosters connections among researchers to build a coherent knowledge base in literacy across geographic and intellectual borders. The emphasis on the highest quality research is not only a tradition across editorial teams since the journal’s inception; it also represents the optimism of people everywhere that insights into the human condition and solutions to some of the most intractable problems of our times can be productively addressed through the intellectual endeavors of individuals and communities. There is no more important responsibility than protecting the longstanding commitment of this journal to the highest quality research.

Reviewers should familiarize themselves with the guidelines below, and then read the “how to review” information.

Guidelines

At its core, research involves making an argument. Researchers make knowledge claims on the basis of theory and evidence linked through a clear chain of reasoning that constitutes the warrant for those claims. The arguments are principled, in that the researcher follows a systematic line of inquiry that is accessible to the scrutiny of others in the research community. The logic of inquiry should also adhere, broadly speaking, to the rules of evidence particular to the disciplinary perspectives (e.g., psychology, sociology, anthropology) being brought to bear on the research question(s).

With this view of research in mind, please consider the following questions as you review a manuscript:

How have the authors located their inquiry against a background of extant knowledge and assumptions?

How well have they made connections to relevant research?

How good is the fit between research question(s) and methodology?

Have the authors conducted the data collection and analysis appropriately?

How strong is the warrant for authors’ claims? Is there a clear, coherent chain of reasoning by which the author(s) reached their conclusion(s)? Do you find the authors’ argument convincing? Have the authors sufficiently considered alternative explanations and perspectives? Have the authors been open-minded and thorough?

So what? How is the research significant for informing and improving theory and/or practice?

Some of these questions are more appropriate than others, depending on the nature of the inquiry. Nevertheless, in the final analysis, a piece of research is only as good as the quality of the chain of reasoning by which the researchers organize the extant knowledge and assumptions, their observations and analyses, and their knowledge claims “to create a cogent, persuasive argument” (Shulman, 1997, p. 26).

For further explanation on this view of research, you may wish to read Howe and Eisenhart (1990), Scientific Research in Education (Committee on Scientific Principles for Education Research, National Research Council, 2002), and Shulman (1997).

Some Considerations for Crafting a Good Review

A good review makes explicit the bases for your comments and recommendation. Make your thinking public. For example, if you state that “one simply cannot use these studies to reach this conclusion,” say why. If you state that “the research makes a compelling case,” say why. Make explicit the line of reasoning by which you reached your overall recommendation concerning publication, including the bases or criteria you used to evaluate the manuscript. Your critique should be consistent with the direction you suggest for revision, and the seriousness of the issues you raise should be consistent with the overall recommendation you make concerning publication.

A good review is well organized. Use a logical structure to organize your review. We do not wish to be prescriptive, but we offer some suggestions. Some reviewers like to start by briefly summarizing what the authors did, what they found, and how the authors interpreted the findings. It is often helpful to give your overall evaluation of the manuscript and the main reasons for it early in your review. Some reviewers separate out methodological or technical issues from conceptual ones. This is often helpful to authors and to editors as technical issues call for one set of actions (e.g., revising the tabulation of results) whereas conceptual issues call for another set of actions (e.g., reframing the argument). Stage your comments in a way that makes clear what the big issues of concern are as opposed to small issues. All research has flaws, but only some flaws have substantive consequences for the authors’ claims.

A good review is constructive. A good review should be formative as well as summative in its evaluation. If you think a manuscript should be rejected, give feedback that will help the authors improve their work in future manuscripts or studies. Even if a manuscript is outstanding in all aspects, help the authors understand the range of potential responses from members of the research community. Be specific and to the point.

Please write your review as if you are addressing the author(s) rather than the editors. You may wish to write your review using the second person. By writing as if you are addressing the author(s), you emphasize the importance of reviews in helping authors strengthen their scholarship and writing, regardless of whether their manuscript is accepted for publication. It will also help establish a collegial and friendly tone.

References

Howe, K., & Eisenhart, M. (1990). Standards for qualitative (and quantitative) research: A prolegomenon. Educational Researcher, 19(4), 2–9. dx.doi.org/10.2307/1176381

Committee on Scientific Principles for Education Research, National Research Council. (2002). Scientific research in education. Washington, DC: National Academy Press. newton.nap.edu/catalog/10236.html

Shulman, L.S. (1997). Disciplines of inquiry in education: A new overview. In R.M. Jaeger (Ed.), Complementary methods for research in education (2nd ed., pp. 3–29). Washington, DC: American Educational Research Association.