Title: Applying Generalizability Theory to Interdependent Network Data:
Estimating Interrater Reliability

Authors: Debby ten Hove, Terrence D. Jorgensen, L. Andries van der Ark

Affiliation: University of Amsterdam

Abstract:
Current definitions and measures of interrater reliability (IRR) do not suit
the interdependent nature of dyadic network data. These data involve several
nested components of which a researcher might want to know the IRR. We
extended the social relations model (SRM) for variance decomposition of
interdependent network data with rater components using generalizability
theory, resulting in the rater-extended SRM (RESRM). We estimated the variance
components of the RESRM and derived the IRR from these variance components
using Bayesian hierarchical linear modeling with Stan in the R package rstan.
In this talk, we will discuss the difficulties faced and decisions made in the
modeling procedure for a dataset on social mimicry. We will suggest possible
extensions of this approach, such as assessing scale-score reliability of
interdependent network data measured with multiple indicators.