González / Wiberg Applying Test Equating Methods

Using R
1. Auflage 2017
ISBN: 978-3-319-51824-4
Verlag: Springer International Publishing
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 217 Seiten

Reihe: Methodology of Educational Measurement and Assessment

This book describes how to use test equating methods in practice. The non-commercial software R is used throughout the book to illustrate how to perform different equating methods when scores data are collected under different data collection designs, such as equivalent groups design, single group design, counterbalanced design and non equivalent groups with anchor test design. The R packages equate, kequate and SNSequate, among others, are used to practically illustrate the different methods, while simulated and real data sets illustrate how the methods are conducted with the program R. The book covers traditional equating methods including, mean and linear equating, frequency estimation equating and chain equating, as well as modern equating methods such as kernel equating, local equating and combinations of these. It also offers chapters on observed and true score item response theory equating and discusses recent developments within the equating field. More specifically it covers the issue of including covariates within the equating process, the use of different kernels and ways of selecting bandwidths in kernel equating, and the Bayesian nonparametric estimation of equating functions. It also illustrates how to evaluate equating in practice using simulation and different equating specific measures such as the standard error of equating, percent relative error, different that matters and others.

Autoren/Hrsg.

González, Jorge

Wiberg, Marie

Inhaltsverzeichnis

1;Foreword;7
1.1;References;9
2;Preface;10
2.1;References;12
3;Contents;13
4;Acronyms;18
5;List of Symbols;20
6;1 General Equating Theory Background;24
6.1;1.1 Introduction;24
6.1.1;1.1.1 A Conceptual Description of Equating;25
6.1.2;1.1.2 A Statistical Model View of Equating;25
6.2;1.2 Statistical Models;26
6.2.1;1.2.1 General Definition, Notation, and Examples;26
6.2.2;1.2.2 Types of Statistical Models;27
6.2.3;1.2.3 Mathematical Statistics Formulation of the Equating Problem;29
6.2.4;1.2.4 Mathematical Form of the Equating Transformation;30
6.2.5;1.2.5 Continuization;31
6.2.6;1.2.6 Requirements for Comparability of Scores;32
6.2.7;1.2.7 Assessing the Uncertainty of Equating Results;32
6.3;1.3 Collecting Data in Equating;33
6.3.1;1.3.1 Data Collection Designs in Equating;34
6.3.1.1;1.3.1.1 Single Group Design;34
6.3.1.2;1.3.1.2 Equivalent Groups Design;34
6.3.1.3;1.3.1.3 Counterbalanced Design;34
6.3.1.4;1.3.1.4 Non Equivalent Groups with Anchor Test Design;35
6.3.1.5;1.3.1.5 Non Equivalent Groups with Covariates Design;35
6.4;1.4 Some Examples of Equating Transformations;36
6.4.1;1.4.1 The Equipercentile Equating Function;36
6.4.2;1.4.2 The Linear Equating Function;37
6.4.3;1.4.3 The Kernel Equating Function;37
6.5;1.5 R Packages That Are Used in This Book;38
6.6;1.6 Summary and Overview of the Book;38
6.7;References;39
7;2 Preparing Score Distributions;42
7.1;2.1 Data;42
7.1.1;2.1.1 Data from Ch2:kolenbrennan2014;42
7.1.2;2.1.2 Data from Ch2:vondavieretal2004;43
7.1.3;2.1.3 The ADM Admissions Test Data;43
7.1.4;2.1.4 The SEPA Test Data;44
7.2;2.2 Preparing the Score Data;44
7.2.1;2.2.1 Functions to Create Score Frequency Distributions;45
7.2.2;2.2.2 Score Data in the EG Design;45
7.2.3;2.2.3 Score Data in the SG Design;50
7.2.4;2.2.4 Score Data in the NEAT Design;53
7.3;2.3 Presmoothing the Score Distributions;56
7.3.1;2.3.1 Polynomial Log-Linear Models for Presmoothing;56
7.3.2;2.3.2 Polynomial Log-Linear Smoothing in equate;58
7.3.3;2.3.3 Examples;59
7.3.3.1;2.3.3.1 Smoothing Univariate Distributions;59
7.3.3.2;2.3.3.2 Smoothing a Bivariate Distribution;60
7.3.4;2.3.4 Choosing the Best Log-Linear Model;61
7.4;2.4 Using Other Arguments, Packages and Functions;64
7.5;2.5 Summary;65
7.6;References;65
8;3 Traditional Equating Methods;66
8.1;3.1 Equipercentile, Linear, and Mean Equating Transformations;66
8.2;3.2 Assumptions in the Different Designs;67
8.2.1;3.2.1 Assumptions in EG, SG, and CB Designs;67
8.2.2;3.2.2 Assumptions in the NEAT Design;68
8.3;3.3 Traditional Equating Methods for the EG, SG and CB Designs;69
8.4;3.4 Traditional Equating Methods for the NEAT Design;69
8.4.1;3.4.1 Linear Equating Methods for the NEAT Design;70
8.4.1.1;3.4.1.1 Tucker Equating;70
8.4.1.2;3.4.1.2 Nominal Weights Equating;71
8.4.1.3;3.4.1.3 Levine Observed-Score Equating;71
8.4.1.4;3.4.1.4 Levine True-Score Equating;72
8.4.1.5;3.4.1.5 Chained Linear Equating;73
8.4.2;3.4.2 Equipercentile Equating Methods for the NEAT Design;73
8.4.2.1;3.4.2.1 Frequency Estimation;73
8.4.2.2;3.4.2.2 Chained Equipercentile Equating;74
8.4.2.3;3.4.2.3 Braun-Holland Equating;74
8.5;3.5 Examples with the equate Function;75
8.5.1;3.5.1 The equate Function;75
8.5.2;3.5.2 Examples Under the EG and SG Designs;76
8.5.3;3.5.3 Examples Under the NEAT Design;83
8.5.3.1;3.5.3.1 Linear Methods;83
8.5.3.2;3.5.3.2 Equipercentile Methods;83
8.5.3.3;3.5.3.3 Comparison Between Linear and Equipercentile Methods;84
8.5.4;3.5.4 Examples Using the ADM Data Under the NEAT Design;86
8.6;3.6 Additional Features in equate;86
8.7;3.7 Performing Traditional Equating Methods with SNSequate;87
8.8;3.8 Comparing Traditional Test Equating Methods;88
8.8.1;3.8.1 Bootstrap Standard Errors of Equating;88
8.8.2;3.8.2 Bias and RMSE;89
8.8.3;3.8.3 Examples Using equate;90
8.8.4;3.8.4 Additional Example: A Comparison of Traditional Equating Methods;91
8.9;3.9 Summary;94
8.10;References;94
9;4 Kernel Equating;96
9.1;4.1 A Quick Overview of Kernel Equating;96
9.2;4.2 Step 1: Presmoothing;97
9.2.1;4.2.1 Presmoothing with SNSequate;97
9.2.1.1;4.2.1.1 Presmoothing Under the EG Design;98
9.2.1.2;4.2.1.2 Presmoothing Under the SG Design;99
9.2.1.3;4.2.1.3 Presmoothing Under the CB Design;101
9.2.1.4;4.2.1.4 Presmoothing Under the NEAT Design;101
9.2.1.5;4.2.1.5 Modeling Complexities in the Data;102
9.2.2;4.2.2 Presmoothing with kequate;104
9.2.2.1;4.2.2.1 Presmoothing Under the EG Design;104
9.2.2.2;4.2.2.2 Presmoothing Under the SG Design;105
9.2.2.3;4.2.2.3 Presmoothing Under the CB Design;106
9.2.2.4;4.2.2.4 Presmoothing Under the NEAT Design;106
9.2.2.5;4.2.2.5 Modeling Complexities in the Data;107
9.2.2.6;4.2.2.6 Presmoothing Under the NEC Design;108
9.2.3;4.2.3 Assessing Log-Linear Model Fit;109
9.2.3.1;4.2.3.1 Assessing Log-Linear Model Fit in SNSequate;110
9.2.3.2;4.2.3.2 Assessing Log-Linear Model Fit in kequate;111
9.3;4.3 Step 2: Estimation of Score Probabilities;113
9.3.1;4.3.1 Estimation of Score Probabilities with SNSequate;113
9.3.2;4.3.2 Estimation of Score Probabilities with kequate;114
9.4;4.4 Step 3: Continuization;115
9.4.1;4.4.1 Bandwidth Selection;116
9.4.2;4.4.2 Choosing the Kernel;116
9.4.3;4.4.3 Continuization Choices in SNSequate;117
9.4.4;4.4.4 Continuization Choices in kequate;117
9.5;4.5 Step 4: Equating;118
9.5.1;4.5.1 Equating in SNSequate;118
9.5.2;4.5.2 Equating in kequate;122
9.6;4.6 Step 5: Computation of Accuracy Measures;125
9.6.1;4.6.1 Calculating the Standard Error of Equating;126
9.6.2;4.6.2 Standard Error of Equating Difference;126
9.6.3;4.6.3 Percent Relative Error;126
9.6.4;4.6.4 Obtaining SEE, SEED, and PRE in SNSequate;127
9.6.5;4.6.5 Obtaining SEE, SEED, and PRE in kequate;129
9.7;4.7 Different Features in kequate and SNSequate;132
9.8;4.8 Summary;132
9.9;References;132
10;5 Item Response Theory Equating;134
10.1;5.1 IRT Models;134
10.1.1;5.1.1 Scoring Using IRT Models;135
10.2;5.2 Equating IRT Scores;136
10.2.1;5.2.1 Parameter Linking;136
10.2.1.1;5.2.1.1 Moments Methods to Estimate Equating Coefficients;137
10.2.1.2;5.2.1.2 Characteristic Curves Methods to Estimate Equating Coefficients;138
10.2.1.3;5.2.1.3 IRT Parameter Linking Using SNSequate;138
10.2.1.4;5.2.1.4 IRT Parameter Linking Using equateIRT;139
10.3;5.3 Equating Observed Scores Under the IRT Framework;142
10.3.1;5.3.1 IRT True-Score Equating;143
10.3.2;5.3.2 IRT Observed-Score Equating;143
10.3.3;5.3.3 IRT True-Score and Observed-Score Equating Using SNSequate;144
10.3.4;5.3.4 IRT True-Score and Observed-Score Equating Using equateIRT;149
10.4;5.4 Other Equating Methods for IRT Scores;151
10.4.1;5.4.1 Concurrent Calibration;151
10.4.1.1;5.4.1.1 Concurrent Calibration Using ltm;152
10.4.2;5.4.2 Fixed Item Parameter Calibration;154
10.4.2.1;5.4.2.1 Fixed Item Parameter Calibration Using mirt;154
10.5;5.5 Other R Packages for IRT Analysis;156
10.6;5.6 Summary;157
10.7;References;157
11;6 Local Equating;160
11.1;6.1 The Concept of Local Equating;160
11.1.1;6.1.1 True Equating Transformation;161
11.2;6.2 Performing Local Equating;162
11.3;6.3 Local Linear Equating Transformations;162
11.3.1;6.3.1 Local Linear Equating Conditioning on Anchor Test Scores: NEAT Design;163
11.3.2;6.3.2 Local Linear Equating Method of Conditional Means: SG Design;163
11.3.3;6.3.3 Local Linear Equating Examples in R;163
11.3.3.1;6.3.3.1 Implementing Local Linear Equating Conditioning on Anchor Test Scores;164
11.3.3.2;6.3.3.2 Implementing the Local Linear Equating Method of Conditional Means;167
11.4;6.4 Local Equipercentile Equating Transformations;167
11.4.1;6.4.1 Local IRT Observed-Score Equating;168
11.4.2;6.4.2 Local Observed-Score Kernel Equating Conditioning on Anchor Test Scores;169
11.4.3;6.4.3 Local IRT Observed-Score Kernel Equating;169
11.4.4;6.4.4 Local Equipercentile Equating Examples in R;170
11.4.4.1;6.4.4.1 Local IRT Observed-Score Equating Using SNSequate;170
11.4.4.2;6.4.4.2 Local Observed-Score Kernel Equating Using kequate;172
11.4.4.3;6.4.4.3 Local IRT Observed-Score Kernel Equating Using kequate;174
11.5;6.5 Other Local Equating Methods;177
11.6;6.6 Summary;177
11.7;References;177
12;7 Recent Developments in Equating;179
12.1;7.1 Alternative Kernel Equating Transformations;179
12.1.1;7.1.1 Epanechnikov Kernel;179
12.1.2;7.1.2 Adaptive Kernels;180
12.1.3;7.1.3 Examples of Epanechnikov and Adaptive Kernel Equating in SNSequate;181
12.2;7.2 Bandwidth Selection in Kernel Equating;183
12.2.1;7.2.1 Rule-Based Bandwidth Selection;183
12.2.2;7.2.2 Bandwidth Selection with Double Smoothing;184
12.2.3;7.2.3 Examples of the Rule-Based and Double Smoothing Bandwidth Selection Methods Using kequate;184
12.3;7.3 Item Response Theory Kernel Equating;185
12.3.1;7.3.1 Two Polytomous IRT Models;185
12.3.2;7.3.2 Performing IRT Kernel Equating with kequate;186
12.3.3;7.3.3 Examples of IRT Kernel Equating for Binary Scored Items Using kequate;187
12.3.4;7.3.4 Examples of IRT Kernel Equating for Polytomous Scored Items Using kequate;189
12.4;7.4 Bayesian Nonparametric Approach to Equating;190
12.4.1;7.4.1 Bayesian Nonparametric Modeling;190
12.4.2;7.4.2 BNP Model for Equating;191
12.4.3;7.4.3 An Illustration of the BNP Model for Equating in SNSequate;192
12.5;7.5 Assessing the Equating Transformation;194
12.5.1;7.5.1 An Illustration of Assessing (x) in Kernel Equating Using SNSequate;196
12.6;7.6 Summary;199
12.7;References;199
13;Appendix A Installing and Reading Data in R;201
13.1;A.1 Installing R;201
13.1.1;A.1.1 R Studio;201
13.2;A.2 Installing and Loading R Packages;202
13.3;A.3 Working Directory and Accessing Data;202
13.4;A.4 Loading Data of Different File Formats;203
13.5;Reference;204
14;Appendix B Additional Material;205
14.1;B.1 Design Functions;205
14.2;B.2 C C C C Matrices;207
14.3;B.3 Calculation of the SEE;207
14.4;B.4 Score Distributions Under the NEAT Design;208
14.5;B.5 The Lord-Wingersky Algorithm;209
14.6;B.6 Other Justifications for Local Equating;210
14.7;B.7 Epanechnikov Kernel Density Estimate and Derivatives;211
14.8;B.8 The Double Smoothing Bandwidth Selection Method in Kernel Equating;212
14.9;B.9 The DBPP Model;213
14.10;B.10 Measures of Statistical Assessment When Equating Test Scores;213
14.11;References;214
15;Index;216