Classical Test/Item Response Theory

Measurement Theory: Latent Variables in CTT

Assessment can be described mathematically in terms of probability theory as a random experiment (Steyer):

1. A person \(u \in \Omega_U\) is drawn from a population (manifest unit random variable \(U : \Omega \rightarrow \Omega_U\)).
2. An observation \(y \in \Omega_o\) is made of this person \(u\) (manifest random variables \(Y : \Omega \rightarrow \mathbb R\))

Measurement of a Latent Variable

A well-defined numerical value for the attribute \(Y\) of object \(u\) is the object conditional expectation \(E(Y | U=u)\) (Steyer).

The regression

\begin{align} \eta := & E(Y | U) \end{align}

is the latent random variable (a reflective measurement).

Measurements are unreliable

For a given person \(U=u\) the observation of \(Y\) is still a random variable and has a probability distribution \(P(Y | U=u)\).

The numerical value \(E(Y | U=u)\) is unknown and needs be estimated from assessed observations.

Reliability in Classical Test Theory

The reliability of a measurement in CTT is the proportion of variance in \(Y\) that is “explained” when knowing the unit \(\rho_Y= 1-\frac{Var(Y | U)}{Var(Y)}\), the remaining residual variance is the measurement error \(Var(Y | U)\).

Aggregating Information from multiple items

Multiple observations attenuate measurement errors.

Random Experiment

1. A person \(u \in \Omega_U\) is drawn.
2. An multivariate observation \(\ym \in \mathbb{R}\) is made of this person \(u\) (manifest random variables \(Y_1,\ldots,Y_m : \Omega \rightarrow \mathbb \mathbb{R}\))

Assumptions \(\Ym : \Omega \rightarrow \mathbb{R}\) are τ-equivalent (parallel)

1. \(\tau\) -equivalence¹ \(\tau := E(Y_1 | U) = \ldots = E(Y_m | U)\)
2. Error variables \(\epsilon_i=Y_i-\tau\) are uncorrelated with measurement error \(\sigma_{\epsilon}^2=Var(\epsilon_i)\).

Reliability of Aggregated Observations

Reliability of Aggregation

In classical test theory a test of length \(m\) displays higher reliability when considering the items’ mean:²

\begin{align} \label{defYbar} \bar{Y} := & \frac{1}{m}\sum_{i=1}^m Y_i\\\ \rho_{\bar{Y}} := & \frac{Var(\tau)}{Var(\tau)+\frac{i}{m}\sigma_{\epsilon}^2} \end{align}

Random Experiment of the the Rasch Model

For items with success/failure, also multiple observations attenuate measurement errors.

Random Experiment

1. A person \(u \in \Omega_U\) is drawn from a population (manifest unit random variable \(U : \Omega \rightarrow \Omega_U\)).
2. An multivariate observation \(\ym \in \Omega_o\) is made of this person \(u\) (manifest random variables \(Y_1,\ldots,Y_m : \Omega \rightarrow \mathbb \{0,1\}\))

Assumptions: Rasch Homogeneity

Assumptions and Latent Variables of the Rasch Model

ENTSCHEIDE Ist IRT ein benötigtes Beispiel für Aggregation?

Reference logit latent True-score variables

In the Rasch-Modell the logit of the \(U\) -conditional expectations of the first observable \(Y_1\):

\begin{align} \eta := \tau_1 := \ln \frac{E(Y_1 | U)}{1-E(Y_1 | U)} \end{align}