Longitudinal Factor Analysis and Measurement Invariance Across Time

18.1 The Longitudinal Measurement Model

A construct measured by several items at several occasions is represented by a longitudinal confirmatory factor model: one latent factor per occasion, each measured by the same set of indicators, with the factors free to covary across time. Figure 18.1 draws the structure for two occasions, and it generalizes to any number. The essential feature that distinguishes the longitudinal model from a stack of separate cross-sectional factor models is the treatment of the uniquenesses, the indicator residuals. The same item measured at two occasions shares more than the common factor: it carries an indicator-specific component, a stable idiosyncrasy of that particular question that persists across time and has nothing to do with the construct. A vocabulary item may be reliably easy for reasons unrelated to verbal ability; a survey item may be consistently read in a particular way. This shared indicator-specific variance induces a correlation between the same item’s residuals across waves, and the longitudinal factor model must include these correlated uniquenesses. Omitting them does not merely lose a nuisance parameter; it forces the indicator-specific stability to be absorbed into the only pathway left for it, the correlation among the factors, and so inflates the estimated stability of the construct itself.

Note. One factor per occasion, each measured by the same indicators, with the factors free to covary. The dashed arrows are the correlated uniquenesses that link each indicator’s residual to itself across time, capturing indicator-specific stable variance. Omitting them forces that stability into the factor covariance and inflates the apparent stability of the construct. The diagram generalizes to any number of waves.

The bias is not hypothetical, and Figure 18.2 quantifies it on the running dataset, a simulated four-wave, three-indicator engagement scale with a known measurement model. When the correlated uniquenesses are included, the estimated factor correlations track their true values; when they are omitted, every factor correlation is inflated, so that a researcher would overstate how much a person’s standing on the construct at one wave predicts their standing at the next. Correlated uniquenesses are therefore the default backbone of a longitudinal factor model, not an optional refinement.

Note. Estimated wave-to-wave factor correlations from the engagement data, with and without correlated uniquenesses, against their true values. Omitting the correlated residuals (red) pushes every factor correlation upward, overstating the stability of the construct. The indicator-specific stable variance has nowhere else to go.

Before the factors can be compared across time, the model must be identified, and identification in a mean-and-covariance-structure model is a system of bookkeeping that trips many users. The latent variables have no inherent scale or origin, so each factor needs a scale and a zero point fixed by convention, and there are three conventions in use: the marker-variable method fixes one loading to one and its intercept to zero; the fixed-factor method fixes the factor variance to one and the factor mean to zero; and the effects-coding method constrains the loadings to average one and the intercepts to sum to zero. The conventions interact with the invariance constraints, because what a given constraint identifies depends on what has already been fixed, and Table 18.1 lays out the grid. The practical rule is to choose one convention, apply it consistently, and account for every degree of freedom at every step, which is the discipline this chapter models throughout. Fit is evaluated with the indices of Chapter 8, the comparative fit index, the root mean square error of approximation, and the standardized root mean square residual, read against their conventional benchmarks with the standing caution that those benchmarks are rules of thumb from particular simulation conditions, not laws, and are used here as diagnostics rather than verdicts.

18.2 The Invariance Ladder

Measurement invariance is tested as a sequence of nested models, each adding a set of equality constraints across time and each licensing a further kind of comparison. Figure 18.3 shows the sequence run on the engagement scale, and the pattern is the object of study. Configural invariance requires only that the same factor structure, the same items loading on the same factor, holds at every occasion; it establishes that the construct has the same form over time but licenses no quantitative comparison. Metric (or weak) invariance adds equality of the factor loadings across time, and it is what licenses comparing covariances and regression slopes involving the factor, because equal loadings mean a unit of the factor has the same meaning at every wave. Scalar (or strong) invariance adds equality of the indicator intercepts, and it is the crucial level for the study of change, because only when both loadings and intercepts are invariant can the factor means be compared across time, which is to say only then does a change in the latent mean reflect change in the construct rather than drift in the measurement. Strict invariance adds equality of the indicator residual variances, which licenses the comparison of observed composite scores, since only then do equal factor standings imply equal expected observed scores. Table 18.2 states what each level constrains, means, and licenses.

Note. Comparative fit index across the four nested invariance models on the engagement scale. Configural and metric invariance fit well, so the loadings are invariant, but imposing scalar invariance, equal intercepts, collapses the fit, because one indicator’s intercept drifts. Strict invariance adds equal residuals to the already-failing scalar level. The change in fit at the scalar step, not its absolute value, carries the message.

On the engagement data the sequence tells a clear story. The configural model fits well, with a comparative fit index of \(1.00\) and a root mean square error of approximation of \(.006\). Metric invariance holds: constraining the loadings equal across the four waves changes the chi-square by only \(1.9\) on six degrees of freedom, far from significant, and leaves the fit indices unchanged. Scalar invariance fails decisively: constraining the intercepts equal worsens the chi-square by \(374\) on six degrees of freedom and drops the comparative fit index by \(.079\), from \(1.00\) to \(.92\). The story is that the items load on the construct identically over time, so the construct has the same meaning, but at least one item’s intercept has drifted, so the observed scores are not on a common footing and the latent means cannot yet be compared. Adjudicating this requires decision criteria, and the chapter’s policy, following the reporting literature, is to report several. The chi-square difference test is exact but sensitive to sample size, flagging trivial noninvariance in large samples; the change in the comparative fit index, with a heuristic threshold near \(.01\), and the change in the root mean square error of approximation are less sample-dependent but were calibrated for multi-group rather than longitudinal comparisons, a transfer that should be noted; and an effect-size measure of the magnitude of noninvariance completes the picture. Table 18.3 collects the criteria with their cautions. Here every criterion agrees, because the planted violation is large, but the honest analyst reports the set and does not lean on a single cutoff.

18.3 When Invariance Fails

A failure of scalar invariance is the beginning of the analysis, not its end. The first task is to locate the noninvariant parameter. Because a model with equality constraints hides the offending parameter from the ordinary modification index, which is computed for a single model, the forensic tool is the score test that evaluates releasing each constraint, or equivalently the transparent procedure of refitting the model while freeing one indicator’s intercepts at a time and seeing which restores fit. Figure 18.4 shows the result: freeing the second indicator’s intercepts returns the comparative fit index to \(1.00\), while freeing either other indicator does not, identifying the second item as the culprit. The location should be interrogated with theory rather than chased greedily, because releasing constraints purely to improve fit capitalizes on chance and can free parameters that are noninvariant only by sampling noise. Figure 18.5 shows the anatomy of the violation directly: the second indicator, which has the smallest loading, nonetheless shows the largest change from the first wave at waves three and four, a pattern impossible under invariance, where an item’s change should be proportional to its loading times the factor’s change. The item has drifted upward in its baseline, contributing more to the observed score at later waves than the construct warrants.

Note. Comparative fit index of the scalar model when each indicator’s intercepts are freed across waves. Freeing the second indicator restores fit to \(1.00\); freeing either other indicator leaves the model misfitting. This identifies the second item as the noninvariant one, the forensic use of the score test rather than a greedy search.

Note. Mean change from the first wave for each indicator. Under invariance an item’s change equals its loading times the factor’s change, so the highest-loading item should change most. Instead the second item, with the smallest loading, changes most at waves three and four (shaded), the signature of an upward intercept drift rather than true growth.

Having located the violation, the analyst adopts partial invariance, freeing the offending intercept while holding the others equal, an approach introduced by Byrne, Shavelson, and Muthén (1989). Freeing the second indicator’s intercepts at waves three and four restores the fit to a comparative fit index of \(1.00\), and the latent means can now be compared, resting on the invariant remainder of the scale as anchor items. The honest questions are how much partiality is too much and which items may serve as anchors, and they have no mechanical answer: a scale on which most items have drifted cannot support a credible comparison, and the defensible practice is to keep a substantial majority of items invariant, to justify the anchor set on substantive grounds, and to report a sensitivity analysis showing that the conclusions hold under alternative choices of which items to free. Crucially, the noninvariance is itself a substantive finding. An item that drifts has changed its relationship to the construct, which may reflect recalibration by respondents, a shift in what the item measures, or a genuine change in the construct’s manifestation, the heterotypic continuity of Chapter 3, and in clinical contexts it connects to the response-shift literature on how patients reinterpret a scale as their condition evolves. When full invariance is untenable, more flexible alternatives exist, including approximate invariance through Bayesian small-variance priors that allow measurement parameters to differ slightly rather than exactly, and the alignment method that optimizes a compromise across many groups or occasions (Asparouhov & Muthén, 2014), though their guidance for longitudinal use is still developing.

18.4 Categorical Indicators

Psychological items are often ordinal, a handful of ordered response categories rather than a continuous scale, and invariance testing must then work on the latent response distribution underlying the categories. Each ordinal item is modeled as a coarsening of a latent continuous response by a set of thresholds, and Figure 18.6 draws the idea: a five-category item arises from four thresholds carving a latent normal response into ordered bands. Invariance for categorical items therefore concerns the thresholds as well as the loadings, and the two are coupled in a way that requires a specific testing sequence and specific identification constraints, worked out by Wu and Estabrook (2016) and adapted to the longitudinal case by Liu and colleagues (2017). Estimation is typically by a weighted least squares method, robust to the non-normality of ordinal data, rather than maximum likelihood. Table 18.4 contrasts the continuous and categorical procedures. A practical consideration links back to Chapter 6: the weighted least squares estimator handles missing data less gracefully than the full-information maximum likelihood of the continuous model, using a pairwise-style treatment that assumes a more restrictive missingness mechanism, so with substantial missingness a maximum-likelihood approach with a robust correction, or a fully Bayesian treatment, may be preferable. On the engagement items recoded to five categories, the configural and threshold-and-loading-invariant models both fit, illustrating the machinery; the full categorical sequence, with its careful identification, is developed in the references.

Note. A five-category ordinal item is modeled as a latent continuous response variable coarsened by four thresholds. Invariance testing for categorical items constrains the loadings and the thresholds together, on the latent response scale, and is estimated by robust weighted least squares rather than maximum likelihood.

18.5 Why This Labor Matters

The motivation for the entire apparatus is that ignoring noninvariance biases the study of change, and a simulation makes the cost concrete. Figure 18.7 shows the recovered latent-mean trajectory of the engagement construct under three analytic choices, against the known truth. The truth is a steady growth of \(0.4\) per wave. The composite score, the average of the items, which silently assumes strict invariance, overstates the later growth, because the drifting item inflates the composite at waves three and four. A factor model that wrongly imposes full scalar invariance does the same, attributing the item’s drift to the factor mean. Only the partial-invariance model, which frees the drifting intercept, recovers the true trajectory. The bias is not a rounding error: at the final wave the composite estimates a change of \(1.44\) against the true \(1.20\), a twenty-percent overstatement of the construct’s growth, and Figure 18.8 shows that the overstatement grows without bound as the drift increases, while the partial-invariance estimate stays near the truth regardless. This simulation is the argument for the measurement-first sequence of the structural equation part: a growth model, however sophisticated, inherits the biases of the scores it is fit to, and the second-order growth model of Chapter 20, which fits the trajectory to the invariant latent factors rather than to the composites, is the proper remedy. Factor scores extracted from an invariance analysis and then fed into a growth model are a tempting shortcut but carry their own hazards, because the scores are estimated with error that the second stage ignores, a plug-in problem best avoided by fitting the measurement and growth models jointly.

Note. Recovered latent-mean change of the engagement construct under three analytic choices, against the known truth. The composite score and the wrongly-full-scalar factor model overstate the later growth, because the drifting item inflates them; only the partial-invariance model recovers the true trajectory. This is the argument for measurement-first analysis of change.

Note. The estimated wave-one-to-wave-four latent change against the magnitude of the intercept drift, for the composite score and the partial-invariance model. The composite’s estimate inflates as the drift grows; the partial-invariance model’s estimate stays near the truth. Noninvariance is not self-correcting, and larger drift means larger bias.

18.6 Reporting an Invariance Analysis

An invariance analysis is reported so that a reader can follow every constraint and reproduce every decision, and the field has documented how uneven this reporting has been (Putnick & Bornstein, 2016). The report states the longitudinal factor model, including the correlated uniquenesses and the scaling convention; presents the sequence of models as a table with the fit of each and the fit difference at each step under the several criteria; names any noninvariant parameters and the evidence locating them; states the partial-invariance decisions and the anchor logic; and reports a sensitivity analysis showing that the substantive conclusions survive alternative partial specifications. Table 18.5 is the checklist and the template for the sequence table that every such paper needs. The write-up interprets noninvariance substantively where it occurs rather than burying it as a technical caveat, because a drifting item is a finding about the construct.

18.7 Running the Invariance Sequence in R

The longitudinal factor model and the invariance constraints are specified in lavaan by labeling the parameters that are to be held equal across waves. The correlated uniquenesses are added as residual covariances between each indicator and itself across time.

library(lavaan)
config <- '
  f1 =~ 1*y1w1 + L2_1*y2w1 + L3_1*y3w1     # loadings free per wave (configural)
  f2 =~ 1*y1w2 + L2_2*y2w2 + L3_2*y3w2
  f3 =~ 1*y1w3 + L2_3*y2w3 + L3_3*y3w3
  f4 =~ 1*y1w4 + L2_4*y2w4 + L3_4*y3w4
  y1w1 ~~ y1w2 + y1w3 + y1w4                # correlated uniquenesses ...
  y2w1 ~~ y2w2 + y2w3 + y2w4                # ... same indicator across waves
  y3w1 ~~ y3w2 + y3w3 + y3w4  '
m_config <- sem(config, data = engage_long, meanstructure = TRUE)

Metric invariance gives the loadings a shared label across waves (L2, L3); scalar invariance additionally shares the intercept labels and frees the latent means; strict invariance additionally shares the residual labels. Adjacent models are compared with anova and their fit with fitMeasures.

anova(m_config, m_metric, m_scalar, m_strict)          # nested chi-square tests
fitMeasures(m_scalar, c("cfi","rmsea","srmr"))          # fit of each level
# Locate a violation by freeing one indicator's intercepts and refitting,
# or with lavTestScore(m_scalar) for the score tests on the equality constraints.
# Ordinal items: sem(..., ordered = c("oy1w1", ...), estimator = "WLSMV")

The complete analysis, including the correlated-uniqueness bias demonstration, the full ladder with degree-of-freedom accounting, the location and partial-invariance repair, the latent-mean recovery, the consequences simulation, and the ordinal fit, is the shipped script ch18_analysis_V01.R, with figures drawn by ch18_figures_V01.R and the dataset generated by gen_engage_long_V01.R. The analysis writes the equality constraints explicitly for transparency; in practice the semTools package automates the entire sequence.

18.8 Common Misconceptions

Several beliefs about invariance mislead. The first is that invariance is a preliminary hurdle rather than science; a failure of invariance is a substantive finding about how a construct is measured and understood over time, and item drift can be the most interesting result in a study. The second is that the chi-square difference test is too sensitive and should be ignored entirely; it is sensitive to sample size, which is a reason to report it alongside effect sizes and fit-difference indices, not to discard it. The third is a pair of opposite errors, that freeing a single loading invalidates the whole analysis and that one should free whatever the modification indices suggest; the truth is that partial invariance with a well-justified minority of freed parameters supports valid comparison, while greedy freeing capitalizes on chance. The fourth is that composite scores sidestep the invariance question; they do not, they silently assume the strict level, and the consequences simulation shows the cost when that assumption is false. The fifth is that latent means can be compared once the model fits; they can be compared only once scalar invariance, at least partial, is established.

Where to Go Next

This chapter is the measurement foundation on which the rest of the structural equation part builds. Chapter 19 fits the latent growth curve model, revealing it to be the same growth model as Chapter 14 in structural-equation form, and Chapter 20 develops the second-order growth model that fits the trajectory to the invariant latent factors of this chapter rather than to composite scores, the proper remedy for the bias demonstrated here. Chapter 21 makes the invariance assumptions inside the cross-lagged panel models explicit, and Chapter 22 extends the question to invariance across latent classes. In the intensive-longitudinal part, Chapter 25 confronts the within-person factor structure that dynamic models assume. The vertical-scaling problems of large-scale assessment in Chapter 34 are invariance problems in another guise. The discipline this chapter models, accounting for every constraint and every degree of freedom, and interpreting a failure of invariance as a finding rather than a nuisance, is the working habit of measurement-conscious longitudinal research.

Exercises

1. 18.1 Specify and count. Write the syntax for a three-indicator, four-wave longitudinal factor model under two scaling conventions, and produce the degree-of-freedom accounting for the configural through strict models.
2. 18.2 Run the ladder. On a dataset with a planted noninvariant intercept, run the full sequence, locate the violation, free it with justification, and report the sequence table.
3. 18.3 Ordinal invariance. Fit configural and threshold-invariant models to ordinal items with a robust weighted least squares estimator, and interpret the comparison.
4. 18.4 Extend the consequences. Vary the magnitude of the intercept drift and plot the resulting bias in the estimated latent change, reproducing the slope-bias curve.
5. 18.5 Write the section. From provided output, write the measurement-and-results text for an invariance analysis, interpreting the noninvariance substantively.

References