SEM APPROACH FOR THE DETECTION OF RESPONSE SHIFT
EXAMPLE SYNTAX LISREL
! We use an example of a three-factor model with on nine indicators, measured at two occasions
! S1.t1, S2.t1, and S3.t1 are three measures of social health measured at time 1
! S1.t2, S2.t2, and S3.t2 are the same three measures of social health measured at time 2
! M1.t1, M2.t1, and M3.t1 are three measures of mental health measured at time 1
! M1.t2, M2.t2, and M3.t2 are the same three measures of mental health measured at time 2
! P1.t1, P2.t1, and P3.t1 are three measures of physical health measured at time 1
! P1.t2, P2.t2, and P3.t2 are the same three measures of physical health measured at time 2
! The data is captured in a [covariance matrix], [means vector], and [number of observations]
STEP 1 : ESTABLISHING A MEASUREMENT MODEL
! Specification of the data
! ni = number of observed variables
da ni=18 no=[number of observarions] ma=cm
cm fi=[covariance matrix]
me fi=[means vector]
LA
S1.t1 S2.t1 S3.t1 M1.t1 M2.t1 M3.t1 P1.t1 P2.t1 P3.t1
S1.t2 S2.t2 S3.t2 M1.t2 M2.t2 M3.t2 P1.t2 P2.t2 P3.t2
! Specifications of the model-matrices
mo ny=18 ne=6 ly=fu,fr ps=sy,fr te=sy,fr al=fu,fi ty=fu,fr
LE
SOCIAL.T1 MENTAL.T1 PHYSICAL.T1
SOCIAL.T2 MENTAL.T2 PHYSICAL.T2
! SPECIFICATION OF COVARIANCE STRUCTURE
! matrix of common factor loadings
! all free to be estimated (no restrictions over time)
pa ly
1 0 0 0 0 0
1 0 0 0 0 0
1 0 0 0 0 0
0 1 0 0 0 0
0 1 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 1 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 1 0 0
0 0 0 1 0 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 0 1
0 0 0 0 0 1
0 0 0 0 0 1
! matrix of common factor (co)variances
! includes longitudinal relations
! identification is done through fixing the factor variances at 1 (at both occasions)
ma ps
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
0 0 0 0 0 1
pa ps
0
1 0
1 1 0
1 1 1 0
1 1 1 1 0
1 1 1 1 1 0
! matrix of residual (co)variances
! includes covariances of the same residual factors over time
pa te
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
0 0 0 0 0 1
0 0 0 0 0 0 1
0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 1
1 0 0 0 0 0 0 0 0 1
0 1 0 0 0 0 0 0 0 0 1
0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
! SPECIFICATION OF MEAN STRUCTURE
! vector of common factor means
! identification through fixing common factor means at zero (at both occasions)
pa al
0 0 0 0 0 0
! vector of intercepts
pa ty
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
pd
ou ml mi so
! -> When model has adequate fit and is interpretable, continue with STEP2
STEP 2 : OVERALL TEST OF RESPONSE SHIFT
! Specification of the data
da ni=18 no=[number of observarions] ma=cm
cm fi=[covariance matrix]
me fi=[means vector]
LA
S1.t1 S2.t1 S3.t1 M1.t1 M2.t1 M3.t1 P1.t1 + P2.t1 + P3.t1
S1.t2 S2.t2 S3.t2 M1.t2 M2.t2 M3.t2 P1.t2 + P2.t2 + P3.t2
! Specifications of the model-matrices
mo ny=18 ne=6 ly=fu,fr ps=sy,fr te=sy,fr al=fu,fr ty=fu,fr
LE
SOCIAL.T1 MENTAL.T1 PHYSICAL.T1
SOCIAL.T2 MENTAL.T2 PHYSICAL.T2
! SPECIFICATION OF COVARIANCE STRUCTURE
! matrix of common factor loadings
! all factor loadings are constrained to be equal across occasions using the 'eq' command
pa ly
1 0 0 0 0 0
1 0 0 0 0 0
1 0 0 0 0 0
0 1 0 0 0 0
0 1 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 1 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 1 0 0
0 0 0 1 0 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 0 1
0 0 0 0 0 1
0 0 0 0 0 1
eq ly 1 1 ly 10 4
eq ly 2 1 ly 11 4
eq ly 3 1 ly 12 4
eq ly 4 2 ly 13 5
eq ly 5 2 ly 14 5
eq ly 6 2 ly 15 5
eq ly 7 3 ly 13 6
eq ly 8 3 ly 14 6
eq ly 9 3 ly 15 6
! matrix of common factor (co)variances
! identification is done through fixing the factor variances at 1, only at the first occasion
ma ps
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
0 0 0 0 0 1
pa ps
1
1 1
1 1 1
1 1 1 0
1 1 1 1 0
1 1 1 1 1 0
! matrix of residual (co)variances
pa te
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
0 0 0 0 0 1
0 0 0 0 0 0 1
0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 1
1 0 0 0 0 0 0 0 0 1
0 1 0 0 0 0 0 0 0 0 1
0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
! SPECIFICATION OF MEAN STRUCTURE
! vector of common factor means
! identification through fixing the common factor means at zero, only at the first occasion
ma al
0 0 0 0 0 0
pa al
0 0 0 1 1 1
! vector of intercepts
! all intercepts are constrained to be equal across occasions using the 'eq' command
pa ty
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
eq ty 1 ty 10
eq ty 2 ty 11
eq ty 3 ty 12
eq ty 4 ty 13
eq ty 5 ty 14
eq ty 6 ty 15
eq ty 7 ty 16
eq ty 8 ty 17
pd
ou ml mi so
! -> Compare the model fit of the model from STEP2 with the model fit of the model from STEP1
! -> When the omnibus test for response shift is significant, this indicates the (overall) presence of response shift
! -> Continue with STEP3 to identify specific indications of response shift
STEP 3 : DETECTION OF RESPONSE SHIFT
! -> Use an iterative procedure to modify the model from STEP2 (where all possible response shifts are considered one at a time) to identify specific response shift effects.
! -> Test whether response shift is significant
! -> Look at parameter estimates to interpret detected response shift
! Calculate effect-size for impact of response shift on change in the observed variable
! -> See Verdam, Oort, & Sprangers, 2017 (JCE, 85, 37-44)
STEP 4: TRUE CHANGE ASSESSMENT
! -> In the final model from STEP3: Look at the parameter estimates of the common factor means at the second occasions (i.e. SOCIAL.T2, MENTAL.T2 and PHYSICAL.T2)
! -> Calculate effect-size for the estimated true change using Cohen's d: mean_post - mean_pre / sqrt(variance_post + variance_pre - 2correlation_postpre*sd_post*sd_pre)
! -> To evaluate the impact of response shift on true change, compare these estimates to the same estimates of the 'no response shift model' from STEP2