Classroom Quality in the Age of Accountability: Using a Comprehensive Multidimensional Rasch Approach to Investigate the Validity of the Early Childhood Environment Rating Scale-Revised
Foster, Brandon.
2017
-
Abstract: The Early Childhood Environment Rating Scale-Revised (ECERS-R) is
the most widely used measure of classroom quality, and has been implemented into numerous
states' Quality Rating and Improvement Systems (QRISs). However, the ECERS-R was never
designed for implementation into these systems or to be used for the purposes of
accountability. As such, its validity for determining benchmarks ... read moreon key indicators of
classroom quality and suitability for identifying low or high performing preschools has yet
to be established. Findings from a handful of validity studies which utilized classical
test theory (CTT) have highlighted information pertaining to the factor structure of the
measure. However, the CTT approach provides very limited diagnostic information about the
functioning of the ECERS-R (e.g., Fan, 1998; Kieffer, 1998; Macdonald & Paunonen, 2002;
Traub, 1997; Schumacker & Smith, 2007), and studies investigating the psychometric
properties of the ECERS-R using more rigorous methods from item-response theory have hardly
emerged (Gordon et al., 2013; 2015). This dissertation provides the first comprehensive
account of the psychometric properties of the ECERS-R, utilizing the ECLS-B dataset, a
nationally representative sample of early childhood environments. In particular, three
proposed dimensional specifications of the measure that have emerged from the literature
were examined. Psychometric analyses were carried out using Andrich Rating Scale Models and
Multidimensional Random Coefficient Multinomial Logit (MRCML) models. To organize this
information, this dissertation adopted Wolfe and Smith's (2007) Rasch validity framework,
which ensured that analyses provided psychometric information about each component of
Messick's seminal (1989) construct validity framework. Results illuminate the psychometric
trade-offs that occur when different specifications of the ECERS-R are utilized to measure
classroom quality. Additional information, which builds off the work of Gordon et al.
(2013; 2015b), is provided to guide policymakers in the use of total scores from the
measure in policy applications. Further, a promising new dimensional structure of the
ECERS-R is posited and confirmed. Results from these analyses suggested that the ECERS-R
functioned as a risk assessment, capable of precisely measuring classrooms with low levels
of classroom quality, but functioned poorly for classrooms with average or above average
levels of classroom quality. These results raise serious concerns about the use of the
ECERS-R in most states' early-childhood assessment systems, and call into question some
basic assumptions for how this measure is assumed to have performed in the prior research
studies.
Thesis (Ph.D.)--Tufts University, 2017.
Submitted to the Dept. of Child Development.
Advisor: Christine McWayne.
Committee: Tama Leventhal, Shelagh Peoples, and Eric Dearing.
Keywords: Early childhood education, Quantitative psychology, and Public policy.read less - ID:
- 0r967g40b
- Component ID:
- tufts:22401
- To Cite:
- TARC Citation Guide EndNote
