Abstracts Category : Other

Impact of Composite Population Priors on Computer Adaptive Test Proficiency Estimates

by Kristin M Morrison

Testing has existed for thousands of years and has evolved from all examinees receiving the same test to adaptive testing, in which the test is tailored to the individual examinee. These adaptive testing designs have shown to be improvements over fixed-length, conventional tests in terms of proficiency measurement, reduced testing time, and faster scoring. A popular method of test administration is computer adaptive testing (CAT) using expected a posteriori (EAP) estimation. This Bayesian estimation approach utilizes previous information known about the examinee to obtain more precise estimates of the individuals ability. An appropriate prior will generally increase estimation precision, decrease outlier influences, and provide an estimate for all possible response patterns. An inappropriate prior, however, may result in biased estimates (Embretson & Reise, 2000). Previous studies have used collateral information (i.e., additional information) concerning the examinee, such as demographic variables, age, grade, or previous test scores, to aid in estimation. Several studies have used previous test scores (Matteucci & Veldkamp, 2013; Veldkamp & Matteucci, 2013; van der Linden, 1999), but none have looked directly at priors based on group membership. This study examined the influence of various group priors, such as composite priors (i.e., priors created from combining groups) and individual priors (i.e., priors specific to the group), on estimation in CAT designs. Results of the study show group-specific priors perform best; however, it is impossible to know true group membership. Thus, results of the study support the use of priors based on the population because priors based on demographics may adversely impact some high ability groups.Advisors/Committee Members: Embretson, Susan E (advisor), Roberts, James S (committee member), Spieler, Daniel (committee member), Templin, Jonathan (committee member), Thomas, Rick (committee member).