Abstract of

Validity, Reliability, and Utility of the Observation Survey of Early Literacy Achievement

Carolyn A. Denton, The University of Texas at Austin

Dennis J. Ciancio, The University of Texas Health Science Center at Houston

Jack M. Fletcher, The University of Texas Health Science Center at Houston

This study evaluated the validity, reliability, and utility of five of the six subtests of Clay's (2002) Observation Survey of Early Literacy Achievement (OS), including Letter Identification, Word Reading, Writing Vocabulary, Hearing and Recording Sounds in Words, and Text Reading. In addition to a review of studies providing assessment of the reliability and validity of the OS, we analyzed (a) the concurrent validity of the OS subtests relative to other instruments with known psychometric properties, (b) their validity and accuracy as predictors of reading development in grade 1, and (c) interadministrator reliability for the Text Reading and Writing Vocabulary subtests. The score distributions of most of the OS subtests indicated inadequate floors and ceilings, suggesting that when the OS subtests are scored as recommended in the manual, they are not suitable for program evaluation. Analyses of predictive validity indicated that decisions based on the attainment of established benchmarks in OS Word Identification, Text Reading, and Writing Vocabulary would be valid for early screening purposes and to evaluate student outcomes, but more work needs to be done in setting benchmarks for these subtests. In addition to this evidence for predictive validity, there was evidence for concurrent validity of the OS. Interassessor reliability was high for both the Text Reading and Writing Vocabulary subtests in our sample. Our review of the characteristics and administration procedures of each subtest suggests cautions regarding the utility of some of the measures for diagnostic assessment and progress monitoring. Overall, with some limitations, the OS can be validly implemented to assess components of early reading development.

Abstract from Denton, C.A., Ciancio, D.J., & Fletcher, J.M. (2006, January/February/March). Validity, Reliability, and Utility of the Observation Survey of Early Literacy Achievement. Reading Research Quarterly, 41(1), 8–34. doi: 10.1598/RRQ.41.1.1