2,234
Views
31
CrossRef citations to date
0
Altmetric
Articles

A “conditional” sense of fairness in assessment

, , , , , , , , , & show all
Pages 121-140
Published online: 18 Mar 2013
 

Standardizing aspects of assessments has long been recognized as a tactic to help make evaluations of examinees fair. It reduces variation in irrelevant aspects of testing procedures that could advantage some examinees and disadvantage others. However, recent attention to making assessment accessible to a more diverse population of students highlights situations in which making tests identical for all examinees can make a testing procedure less fair: Equivalent surface conditions may not provide equivalent evidence about examinees. Although testing accommodations are by now standard practice in most large-scale testing programmes, for the most part these practices lie outside formal educational measurement theory. This article builds on recent research in universal design for learning (UDL), assessment design, and psychometrics to lay out the rationale for inference that is conditional on matching examinees with principled variations of an assessment so as to reduce construct-irrelevant demands. The present focus is assessment for special populations, but it is argued that the principles apply more broadly.

Acknowledgements

Research findings and assessment tasks described in this article were supported by the Principled Assessment Science Assessment Designs for Students with Disabilities (Institute of Education Sciences, US Department of Education, R324A070035). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies. We are grateful to the guest editor Hossein Karami, Heather Buzick, Eric Hansen, Shelby Haberman, and two anonymous reviewers for helpful suggestions, to John Poggio (University of Kansas), Richard Vineyard (Nevada State Department of Education), and Abel Leon (CAL Testing) for their generous support during the implementation of assessment tasks at schools and school districts, to Robert Dolan as co-PI at the start of the project and advisor thereafter, and to Eric Hansen for advice on evidence-centered design and task accommodations.

Notes

Hansen et al. (2007) Hansen, E. G., Mislevy, R. J. and Steinberg, L. S. 2007. Accessibility of testing within a validity framework (U.S. Patent # 7217134). Washington, DC: U.S. Patent and Trademark Office [Google Scholar] and Hansen et al. (2005) Hansen, E. G., Mislevy, R. J., Steinberg, L. S., Lee, M. J. and Forer, D. C. 2005. Accessibility of tests within a validity framework. System: An International Journal of Educational Technology and Applied Linguistics, 33: 107133. (doi:10.1016/j.system.2004.11.002)[Crossref] [Google Scholar] give an alternative specification using Bayes nets.

Released item in the “7th Grade Science Formative Test” by CAL Testing (formerly Kansas Computerized Assessments). Retrieved April, 2009 from http://kca.cete.us, Center for Educational Testing and Evaluation.