Title: Discrepancies in the Structural Change Test in Applications to the BTL
and MPT Model Related to the Covariance Estimator

Authors: Claudia Glemser, Florian Wickelmaier

Affiliation: University of Magdeburg; University of Tübingen

Abstract:
Structural change tests are a useful tool to check for parameter instability
in a parametric model, but they have mostly been applied to regression models
so far. Zeileis (2005) introduced a theoretical framework for general
structural change tests applicable to any parametric model and an
implementation in the R package strucchange (Zeileis, 2006; Zeileis, Leisch,
Hornik & Kleiber, 2002). We wrote a series of simulations to systematically
examine the applicability of this general approach to Bradley-Terry-Luce (BTL)
and multinomial processing tree (MPT) models, focusing on the effect of the
covariance estimator. The simulations are set up to detect a structural change
in the parameter set along a two-factor group variable. Unlike in a standard
logistic regression, in BTL and MPT models the OPG, Hesse and sandwich
estimators for the covariance matrix produce more discrepant structural change
test statistics with higher effect sizes. Even though we simulated no
individual difference (i.e. the observations for all participants were drawn
from the same distribution), the discrepancies in the structural change test
statistic for the BTL and MPT model decreased with the number of observations
contributed per participant and practically disappeared when simulating only
one observation per participant, while keeping the number of total
observations fixed. Across all numbers of observations per participant, the
Hesse estimator yields consistent test statistics, which are close to the
(asymptotically) equivalent Likelihood ratio test statistic. On the other
hand, the OPG estimator yields lower and the sandwich estimator higher test
statistics for more repeated measures. Fortunately, these discrepancies only
had a tangible effect on the test's power for small sample sizes. Current
simulation results and possible causes and conclusions will be discussed at
the workshop.

References:
Zeileis, A. (2005). A Unified Approach to Structural Change Tests Based on ML
Scores, F Statistics, and OLS Residuals. Econometric Reviews, 24, 445-466.
doi:10.1080/07474930500406053 

Zeileis, A. (2006). Implementing a Class of Structural Change Tests: An
Econometric Computing Approach. Computational Statistics and Data Analysis,
50, 2987-3008. doi:10.1016/j.csda.2005.07.001 

Zeileis, A., Leisch, F., Hornik, K., & Kleiber, C. (2002). strucchange: An R
Package for Testing for Structural Change in Linear Regression Models. Journal
of Statistical Software, 7, 1-38. doi:10.18637/jss.v007.i02