by Richard L. Allington
Evidence-based education (EBE) is hot! EBE is the new phrase used to describe one of the key aspects of recent U.S. federal education legislation. The phrase seems to be replacing the older and narrower phrase, scientifically based reading research (used repeatedly in both the Reading Excellence Act and No Child Left Behind Act). EBE is a less bulky but still broader term, perhaps signaling an intent to broaden EBE to subjects other than reading. EBE, or some version of it, seems to be rising in popularity in education internationally as well.
I cannot imagine anyone opposing EBE in principle. Of course our efforts should be guided by evidence. Where would alternative sources of guidance come from? The horoscope? Hucksterism? A roll of the dice? What has guided educators historically if not evidence?
Well, for one instance, there is what the U.S. Department of Education (ED) has termed professional wisdom. In fact, the Department.has included professional wisdom in defining EBE, noting that EBE involves decisions made using professional wisdom integrated with the best available empirical evidence (www.ed.gov/nclb/methods/whatworks/eb/edlite-slide003.html).
I like this definition of EBE for two reasons. First, it honors the evidence derived from experience. Second, while it points to an important role for evidence gathered using “scientific”methods, it also suggests there are limits in that evidence base. And there are limits.
Currently, the ED website (www.ed.gov/rschstat/research/pubs/rigorousevid/guide_pg3.html) lists only two findings as meeting the “gold standard”for empirical evidence: Reducing class size and providing expert one-to-one tutoring for struggling readers. These are the only areas where findings are supported by the results from multiple independent experimental studies. Thus far, none of the findings of the National Reading Panel have been included in this list, but perhaps the efforts of the new What Works Clearinghouse (www.whatworks.ed.gov) will one day qualify one or more of NRP findings as meeting the empirical test.
With this as background, we might consider current events that suggest we should be using evidence in the selection of core reading programs and supplemental reading interventions. If governmental education agencies cannot locate evidence to support many broad principles of educational design (e.g., instructional groupings, scheduled instructional time, necessary explicitness of instruction, nature of professional development, and so on), how should state education agencies and school districts and teachers make decisions about very specific aspects of instruction such as what sort of emphasis should be the focus of core reading instruction or reading interventions? Without good evidence on these issues, how can schools be expected to use research to select specific instructional materials or activities?
The truth is that research will not be of much help in identifying effective core or supplemental reading programs. There is no set of independent studies that meets the federal gold standard that supports the use of any specific core or supplemental reading program. But states and school districts are touting attention to the research as some programs are identified as allowable purchases and others are not listed. If it isn’t gold standard evidence, then just what sorts of evidence are being used in the decisions?
For core program selection, many states and schools used a checklist entitled A Consumer’s Guide to Evaluating a Core Reading Program Grades K–3: A Critical Elements Analysis, developed at the University of Oregon. But that guide includes several criteria that are not supported by the research (e.g., inclusion of decodable texts), and no one has ever compared the effectiveness of core programs that rated well with others that rated less well. Basically, the guide is a couple of individual researchers’ ideas of what they think the research suggests might be included in a core reading program. But that isn’t empirical evidence; that is, at best, professional judgment marketed as empirical evidence.
Instead of relying on checklists or lists prepared by someone else, I hope that state education agencies and school districts will turn instead to the empirical research that is available. There have been a number of independent experiments assessing the potential of many commercial supplemental program products (but I know of only one study comparing the effects of core programs, and none of the programs assessed in that study seem to be on the approved list). There have been more studies published on the impact of various supplemental programs. But almost all of these studies were published after the approved program lists were first crafted. Sometimes these studies suggest that a product does significantly enhance the development of one or more of the elements of effective reading. But a general theme in these reports is that none actually improves reading proficiency generally.
It may seem as though there is more research available to support commercial products than I am suggesting. However, if you examine the ad copy and other advertising materials for core and supplemental reading programs, I’ll suggest that you will not find citations of published, peer-reviewed research studies offered as evidence. You won’t find multiple citations of independent, randomized field trials (independent as in studies completed by researchers with no financial ties to the product). Yet this is the gold standard that has been identified as central for decisions involving the expenditure of federal funds in the U. S. Reading First program.
Instead of rigorous research, these advertising materials offer testimonials, cherry-picked case reports, or simple assertions that the product design was influenced by the report of the National Reading Panel. It is not hard to find lots of assertions of evidence backed by no actual gold standard evidence. Or any independently produced evidence.
If rigorous, independent evidence is almost never available, how then might we select from the broad array of commercial materials available?
First, I’d ask about a product’s educative potential. In other words, what evidence is available that suggests that using the product develops teachers’ expertise about effective reading instruction? Since study after study points to teacher expertise as the critical variable in effective literacy instruction, why would we purchase products that have little, if any, potential to develop teaching expertise?
Second, ask yourself what role you intend the product to play in the total literacy curriculum. In other words, I’ve never encountered any product that, by itself, comprises even a full reading curriculum, much less a full literacy curriculum. Some products provide support for some very narrow, specific element or elements of a full curriculum. The product might provide a short-term plan for developing various skill and strategy proficiencies. While those products might assist teachers in planning skill and strategy instruction, they typically provide little opportunity for students to develop the autonomous, automatic, and appropriate application of those proficiencies while actually reading. In other words, they provide scant opportunities to actually read.
It is during independent reading that readers consolidate the skills and strategies they have been taught and come to own them. Some products may assist in fostering a few very specific skills or strategies (e.g., phonemic awareness, decoding, fluency, vocabulary, or narrative text comprehension). But we have to ask whether these products sop up so much instructional time that our curriculum becomes unbalanced such that we develop readers who can complete a story map but cannot decode big words. Or who can decode but exhibit little comprehension of what they have read. Or who can read passages fluently with repeated readings but whose vocabulary deficit grows every year in part because they are reading and rereading the same texts. And so on.
No product currently available provides a complete reading curriculum. We can use commercial products in our design of an effective literacy curriculum, but we cannot expect any vendor to produce the complete curriculum for us.
Finally, we need to ask ourselves about the interest level of the product and about its potential to engage minds and foster an interest in reading. The now-defunct National Reading Research Center spent a decade studying the design of reading instruction from an “engagement”perspective and provided the profession with an array of powerful findings. Key to reading proficiency is engagement in reading. Key to engagement is how interesting the texts and the curriculum topics are.
I wish EBE had greater potential for improving reading instruction and achievement. I wish we had the many rigorous studies that are now simply absent from the research literature. Hopefully, the emphasis on EBE will stimulate the needed research and the funding to support it. For now, though, we must pay attention to the research while also relying upon the professional wisdom that experience has provided the field.
Richard L. Allington is a professor of education at the University of Tennessee.
What counts as evidence in evidence-based education? (December 2005). Reading Today, 23(3), 16.
Journals
Reading Today