Computerized Multistage Testing: Theory and Applications

Free Standard Shipping

Purchasing Options

ISBN 9781466505773
Cat# K14586



SAVE 20%

eBook (VitalSource)
ISBN 9781466505803
Cat# KE16087



SAVE 30%

eBook Rentals

Other eBook Options:


  • Represents one of the first books on the design, psychometrics, implementation, and operational use of MST
  • Details the steps in the implementation process of an MST design
  • Describes the advantages of MST over linear tests and CATs, including shorter tests and exposure control for items
  • Emphasizes the flexibility of MST in providing consistently accurate assessment for rapidly changing populations
  • Explores the most recent practical applications and their challenges


Devising tests that evaluate a nation’s educational standing and implement efficacious educational reforms requires a careful balance among the contributions of technology, psychometrics, test design, and the learning sciences. Unlike other forms of adaptive testing, multistage testing (MST) is highly suitable for testing educational achievement because it can be adapted to educational surveys and student testing. Computerized Multistage Testing: Theory and Applications covers the methodologies, underlying technology, and implementation aspects of this type of test design.

The book discusses current scientific perspectives and practical considerations for each step involved in setting up an MST program. It covers the history of MST, test design and implementation for various purposes, item pool development and maintenance, IRT-based and classical test theory-based methodologies for test assembly, routing and scoring, equating, test security, and existing software. It also explores current research, existing operational programs, and innovative future assessments using MST.

Intended for psychologists, social scientists, and educational measurement scientists, this volume provides the first unified source of information on the design, psychometrics, implementation, and operational use of MST. It shows how to apply theoretical statistical tools to testing in novel and useful ways. It also explains how to explicitly tie the assumptions made by each model to observable (or at least inferable) data conditions.

Table of Contents

Test Design, Item Pool, and Maintenance
Overview of Computerized Multistage Tests Duanli Yan, Charles Lewis, and Alina A. von Davier
Linear Tests and Computerized Adaptive Tests (CATs)
Multistage Tests (MSTs)
MST Designs for Different Purposes
Implementation Schemes
Designing MST
Content Balance and Assembly
Exposure Control
Scoring, Linking, and Equating
Reliability, Validity, Fairness, and Test Security
Current and Future Applications
Logistic Challenges

Multistage Test Designs: Moving Research Results into Practice April L. Zenisky and Ronald K. Hambleton
The MST Design Structure
The State of Research: MST Development and Design Considerations
Conclusions and Next Steps

Item Pool Design and Maintenance for Multistage Testing Bernard P. Veldkamp
Designing an Item Pool Blueprint
Applications in Item Writing

Mixed-Format Tests: Issues and Methods Jiseon Kim and Barbara G. Dodd
Literature Review on Design Components in Mixed-Format MST
Comparing Other Testing Approaches
Issues and Future Research Suggestions for Mixed-Format MST

Design and Implementation of Large-Scale Multistage Testing Systems Richard Luecht
MST Design and Implementation Considerations
Conclusions: A Research Agenda

Test Assembly
Overview of Test Assembly Methods in Multistage Testing
Yi Zheng, Chun Wang, Michael J. Culbertson, and Hua-Hua Chang
MST Framework
MST Assembly Design
Automated Assembly for MST
Setting Difficulty Anchors and Information Targets for Modules
"On-the-Fly" MST (OMST) Assembly Paradigm
MST, CAT, and Other Designs—Which Way to Go?

Using a Universal Shadow-Test Assembler with Multistage Testing Wim J. van der Linden and Qi Diao
Solving Shadow-Test Assembly Problems
Basic Design Parameters
Different Testing Formats
Relative Efficiency of Formats
Empirical Study
Concluding Comments

Multistage Testing by Shaping Modules on the Fly Kyung (Chris) T. Han and Fanmin Guo
MST by Shaping
MST-S versus MST-R versus CAT
Discussion and Conclusion

Optimizing the Test Assembly and Routing for Multistage Testing Angela Verschoor and Theo Eggen
Optimizing MST Assembly: A Nonexhaustive Search
Limited Item Pools, Two- and Three-Parameter Models

Routing, Scoring, and Equating
IRT-Based Multistage Testing
Alexander Weissman
Motivation for Tailored Testing
Routing Rules
Scoring and Classification Methodologies
Final Comments

A Tree-Based Approach for Multistage Testing Duanli Yan, Charles Lewis, and Alina A. von Davier
Regression Trees
Tree-Based Computerized Adaptive Tests
Tree-Based Multistage Testing
An Application
Limitations and Future Research

Multistage Testing for Categorical Decisions Robert Smith and Charles Lewis
Computer-Mastery Methods
Information Targeted at Cut versus at Ability
Influence of Multiple Cut Scores
Factors That Can Reduce Optimal Solutions
Example Based on Smith and Lewis (1995)

Adaptive Mastery Multistage Testing Using a Multidimensional IRT Model C.A.W. Glas
Definition of the Decision Problem
Computation of Expected Loss and Risk Using Backward Induction
Selection of Items and Testlets
Simulation Studies
Conclusions and Further Research

Multistage Testing Using Diagnostic Models Matthias von Davier and Ying (Alison) Cheng
The DINA Model and the General Diagnostic Model
Experience with CD-CATs

Considerations on Parameter Estimation, Scoring, and Linking in Multistage Testing Shelby J. Haberman and Alina A. von Davier
The Item Response Model
The Test Score
Approaches to Parameter Estimation

Test Reliability, Validity, Fairness, and Security
Reliability of Multistage Tests Using Item Response Theory
Peter W. van Rijn
Test Reliability
Application: IRT Reliability for MST in NAEP

Multistage Test Reliability Estimated via Classical Test Theory Samuel A. Livingston and Sooyeon Kim
The Estimation Procedure
Testing the Accuracy of the Estimation Procedure
How Accurate Were the Estimates?

Evaluating Validity, Fairness, and Differential Item Functioning in Multistage Testing Rebecca Zwick and Brent Bridgeman
Content Balancing
Opportunities for Item Review and Answer Changing
Skipping Strategies
MST Routing Algorithms
The Digital Divide
Comparability of Computer Platforms
Accommodations for Students with Disabilities and English Language Learners
Differential Item Functioning Analysis in MSTs
Application of the Empirical Bayes DIF Approach to Simulated MST Data

Test Security and Quality Control for Multistage Tests Yi-Hsuan Lee, Charles Lewis, and Alina A. von Davier
An Overview of a Three-Component Procedure
Tools to Evaluate Test Security and Quality Control
A Simulation Study Using CUSUM Statistics to Monitor Item Performance

Applications in Large-Scale Assessments
Multistage Test Design and Scoring with Small Samples
Duanli Yan, Charles Lewis, and Alina A. von Davier
Small Data Sample
Item Pool
Various MST Module Designs
Routing and Scoring
Comparisons of the Six MST Designs
Limitations and Future Research

The Multistage Test Implementation of the GRE Revised General Test Frédéric Robin, Manfred Steffen, and Longjuan Liang
From CAT to MST
MST Design

The Multistage Testing Approach to the AICPA Uniform Certified Public Accounting Examinations Krista J. Breithaupt, Oliver Y. Zhang, and Donovan R. Hare
Research on Multistage Testing
Item Bank Development for MST
Content Security Monitoring for MST
Inventory Exposure Planning for MST

Transitioning a K–12 Assessment from Linear to Multistage Tests Carolyn Wentzel, Christine M. Mills, and Kevin C. Meara
Administering CTP Items Online
Creating a New MST Scale Using IRT
Multistage-Adaptive Test Development
Score Reporting

A Multistage Testing Approach to Group-Score Assessments Andreas Oranje, John Mazzeo, Xueli Xu, and Edward Kulick
Targeted Testing
Goals of the Study

Controlling Multistage Testing Exposure Rates in International Large-Scale Assessments Haiwen Chen, Kentaro Yamamoto, and Matthias von Davier
Item Exposure Rate Control for Multistage Adaptive Assessments
Method: How to Compute and Adjust the Item Exposure Rates

Software Tools for Multistage Testing Simulations Kyung (Chris) T. Han and Michal Kosinski

Closing Remarks
Past and Future of Multistage Testing in Educational Reform
Isaac I. Bejar
Future of MST
A Model-Based Three-Stage Design
Item Generation and Automated Scoring and Broadly Accessible Test Content
Summary and Conclusions