This supplementary document illustrates the use of the accompanying R package equaltestMI. The sample statistics from Table 1 of Lee and Al Otaiba (2015) are used as an example. The description of the data can be found in the original article and the results obtained from equaltestMI are discussed at length in the published article “Advances in Measurement Invariance and Mean Comparison of Latent Variables: Equivalence Testing and A ProjectionBased Approach” <DOI: 10.3389/fpsyg.2017.01823>.
The R package equaltestMI is available on CRAN and can be downloaded for use on any R platform with version higher than 3.1.0. Users can pass different arguments to the main function eqMI.main() to examine measurement invariance using the conventional multiplegroup approach (NHT) or equivalence testing (ET) approach. Under the framework of equivalence testing, users obtain the minimum tolerable size (Tsize) and adjusted cutoff values to evaluate the goodnessoffit of each invariance test. The projection method is also available for testing the equality of latent means.
This package is developed for complete dataset with two groups/time points. Use of saturated models might lead to problems in calculation of adjusted RMSEA thresholds. Please consider alternative methods if you have datasets that do not satisfy the requirements.
Users can install the package from CRAN:
## load package
# install.packages("equaltestMI")
library(equaltestMI)
or install the most recent version from the maintainer’s GitHub repository:
# install.packages("devtools")
# library(devtools)
# devtools::install_github("gabriellajg/equaltestMI", force=TRUE)
library(equaltestMI)
## sample statistics where M1 and M2 are sample means
## Cov1 and Cov2 are sample covariance matrices;
## group 1 = boys ineligible for freereduced lunches
## group 2 = boys eligible for freereduced lunches
Group1 < read.table('Group1.txt', header = TRUE)
Group2 < read.table('Group2.txt', header = TRUE)
Group1 < as.matrix(Group1)
Group2 < as.matrix(Group2)
M1 < Group1[1,]
M2 < Group2[1,]
Cov1 < Group1[2:7,]
Cov2 < Group2[2:7,]
## lavaan model syntax
model < '
AlphabetKnowledge =~ Letter_Name+ Letter_Sound
PhonologicalAwareness =~ Blending + Elision
Spelling =~ Real_Words + Pseudo_Words
'
## the results using equivalence testing and projection method
## full R output will be presented in Part 3
test < eqMI.main(model = model,
sample.nobs = c(78, 174),
sample.mean = list(M1, M2),
sample.cov = list(Cov1, Cov2),
meanstructure = TRUE,
output = 'both',
quiet = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE, bootstrap = FALSE)
Sample means:
#> Letter_Name Letter_Sound Blending Elision Real_Words Pseudo_Words
#> 45.26 40.45 10.91 6.51 23.88 14.12
Sample Covariance Matrix:
#> Letter_Name Letter_Sound Blending Elision Real_Words Pseudo_Words
#> Letter_Name 207.360 159.097 32.589 25.805 61.776 45.075
#> Letter_Sound 159.097 280.228 42.888 36.748 76.123 60.204
#> Blending 32.589 42.888 18.233 10.713 19.051 14.219
#> Elision 25.805 36.748 10.713 20.070 20.372 16.709
#> Real_Words 61.776 76.123 19.051 20.372 73.616 47.429
#> Pseudo_Words 45.075 60.204 14.219 16.709 47.429 44.356
Sample means:
#> Letter_Name Letter_Sound Blending Elision Real_Words Pseudo_Words
#> 41.32 34.88 9.08 4.45 19.24 11.07
Sample Covariance Matrix:
#> Letter_Name Letter_Sound Blending Elision Real_Words Pseudo_Words
#> Letter_Name 295.840 232.200 38.996 20.174 67.593 57.771
#> Letter_Sound 232.200 324.000 43.164 22.824 77.954 60.458
#> Blending 38.996 43.164 19.010 9.260 23.428 16.272
#> Elision 20.174 22.824 9.260 10.049 15.254 11.042
#> Real_Words 67.593 77.954 23.428 15.254 64.320 38.411
#> Pseudo_Words 57.771 60.458 16.272 11.042 38.411 38.688
#>
#>  Equality of Population Covariance Matrices under NHT 
#> Chisq Df pvalue
#> fit.pop.cov 48.85006 21 0.0005261173
#>
#>  ChiSquare and ChiSquareDifference Test under NHT 
#> Chisq Df pvalue Chisq.diff Df.diff pvalue
#> fit.pop.cov 48.850 21 0.001
#> fit.configural.g1 4.408 6 0.622
#> fit.configural.g2 10.641 6 0.100
#> fit.combine.groups 15.049 12
#> fit.metric 20.033 15 0.171 4.984 3 0.173
#> fit.residuals 42.512 21 0.004 22.479 6 0.001
#> fit.varfactor 54.175 27 0.001 11.663 6 0.070
#> fit.scalar 23.732 18 0.164 3.699 3 0.296
#> fit.strong.means 41.066 21 0.006 17.334 3 0.001
#> fit.strict.residuals 45.968 24 0.004 22.237 6 0.001
#> fit.strict.means 63.630 27 0.000 17.662 3 0.001
#>
#>  Tsize epsilon, RMSEA, and Adjusted Cutoff Values under ET 
#> epsilon_t RMESA_t cut.01 cut.05 cut.08 cut.10
#> fit.pop.cov 0.209 0.141 0.076 0.097 0.121 0.139
#> fit.configural.g1 0.028 0.097 0.116 0.133 0.157 0.175
#> fit.configural.g2 0.071 0.154 0.116 0.133 0.157 0.175
#> fit.metric 0.049 0.181 0.151 0.164 0.187 0.205
#> fit.residuals 0.140 0.216 0.116 0.133 0.157 0.175
#> fit.varfactor 0.078 0.161 0.116 0.133 0.157 0.175
#> fit.scalar 0.040 0.163 0.151 0.164 0.187 0.205
#> fit.strong.means 0.125 0.289 0.151 0.164 0.187 0.205
#> fit.strict.residuals 0.138 0.215 0.116 0.133 0.157 0.175
#> fit.strict.means 0.127 0.291 0.151 0.164 0.187 0.205
#> goodnessoffit
#> fit.pop.cov poor
#> fit.configural.g1 excellent
#> fit.configural.g2 fair
#> fit.metric fair
#> fit.residuals poor
#> fit.varfactor mediocre
#> fit.scalar close
#> fit.strong.means poor
#> fit.strict.residuals poor
#> fit.strict.means poor
#>
#>
#>  Means of Latent and Specific Factors by the Projection Method and under NHT 
#> Chisq Df pvalue
#> fit.mvmean 19.906793 6 0.0028771810
#> fit.common 18.672371 3 0.0003195302
#> fit.specific 4.163034 3 0.2443890413
#> Validity Index is 0.98856
#>
#>  Means of Latent and Specific Factors by the Projection Method and under ET 
#> epsilon_t RMESA_t cut.01 cut.05 cut.08 cut.10
#> fit.mvmean 0.126 0.205 0.116 0.133 0.157 0.175
#> fit.common 0.133 0.298 0.151 0.164 0.187 0.205
#> fit.specific 0.043 0.170 0.151 0.164 0.187 0.205
#> goodnessoffit
#> fit.mvmean poor
#> fit.common poor
#> fit.specific fair
#>
test1 < eqMI.main(model = model,
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = FALSE, adjRMSEA = FALSE)
test2 < eqMI.main(model = model,
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = FALSE, adjRMSEA = FALSE,
projection = TRUE)
test3 < eqMI.main(model = model,
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = FALSE)
test4 < eqMI.main(model = model,
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE)
test5 < eqMI.main(model = model,
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE)
test6 < eqMI.main(model = model, structure = 'mean',
sample.nobs = c(78, 174), sample.cov = list(Cov1, Cov2),
sample.mean = list(M1, M2), meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE)
test7 < eqMI.main(model = model, data = literacy.dat,
group = "FRL", meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE)
test8 < eqMI.main(model = model, data = literacy.dat,
group = "FRL", meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE, bootstrap = TRUE)
test9 < eqMI.main(model = model, data = literacy.dat,
group = "FRL", meanstructure = TRUE,
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE, bootstrap = FALSE,
quite = TRUE)
test10 < eqMI.main(model = model, data = literacy.dat,
group = "FRL", meanstructure = TRUE,
group.partial = c("Spelling=~Real_Words", "Blending~1"),
equivalence.test = TRUE, adjRMSEA = TRUE,
projection = TRUE)
so that the loadings of ‘Spelling’ on ‘Real_Words’ and the intercept of ‘Blending’ are allowed to vary across groups.
For a complete view of the help page of function eqMI.main(), please install R package printr and type ?eqMI.main in R console:
#> Registered S3 method overwritten by 'printr':
#> method from
#> knit_print.data.frame rmarkdown
eqMI.main  R Documentation 
Test measurement invariance with equivalence testing, projection methods, and adjusted RMSEA cutoffs for two groups.
eqMI.main( ..., output = "both", equivalence.test = TRUE, adjRMSEA = TRUE, projection = FALSE, bootstrap = FALSE, quiet = TRUE, B = 100, seed = 111 )
…

The same arguments as for any lavaan model. See Users must explicitly specify the name of the input elements for this function to catch. For example, specify ‘data = HolzingerSwineford’ instead just ‘HolzingerSwineford’. 
output

If the function prints out results of covariance structure, mean structure, or both. The value of 
equivalence.test

If 
adjRMSEA

If 
projection

If 
bootstrap

If 
quiet

If 
B

The number of boostrap samples used in bootstrap approach. 
seed

The initial seed to generate bootstrap samples. Default at 111. 
An allinone function with several added options to conduct a sequence of tests needed to evaluate MI. The chisquare statistics, except the one for testing the equality of covariance structure, are obtained based on lavaan::sem
function. The test statistic of the covariance structure equality is obtained via the method of Lagrangian multiplier. Equivalence testing is enabled by setting equivalence.test=TRUE
and this function will calculate Tsize, RMSEA, and adjusted RMSEA cutoff values, and provide the goodnessoffit.
A list is returned with:
AnnotatedOutput
Annotated outout that will be printed to the console if quiet==FALSE.
eqMI.stat
Test statistics, degrees of freedom, pvalues, ncp, Tsizes, RMSEAs, their cutoff values, and the goodnessoffit under equivalence testing. A formated version of eqMI.stat
will be printed if quiet=FALSE
.
convention.sem
Results of conventional multiplegroup SEM using Lavaan. Returned object of eqMI.semtest
.
projection.res
Results of projection methods on tests of latent means. Returned object of eqMI.projection
and eqMI.bootstrap
.
Deng, L., & Yuan, K. H. (2016). Comparing Latent Means Without Mean Structure Models: A ProjectionBased Approach. Psychometrika, 81(3), 802829. https://doi.org/10.1007/s1133601594918
Jiang, G., Mai, Y., & Yuan, K. H. (2017). Advances in Measurement Invariance and Mean Comparison of Latent Variables: Equivalence Testing and A ProjectionBased Approach. Frontiers in Psychology, 8, 1823.
Yuan, K. H., & Chan, W. (2016). Measurement invariance via multigroup SEM: Issues and solutions with chisquaredifference tests. Psychological methods, 21(3), 405426. https://doi.org/10.1037/met0000080
data(HolzingerSwineford) semmodel<' L1 =~ V1 + V2 + V3 L2 =~ V4 + V5 + V6 L3 =~ V7 + V8 L4 =~ V9 + V10 + V11 ' # If raw data are available; test < eqMI.main(model = semmodel, data = HolzingerSwineford, group = "school", meanstructure = TRUE, output = 'both', quiet = FALSE, equivalence.test = TRUE, adjRMSEA = TRUE, projection = TRUE, bootstrap = FALSE) # when only sample statistics are available; # sample.cov need to be provided for tests of covariance structure; # sample.mean need to be provided for tests of mean structure; school1 < subset(HolzingerSwineford, school==1)[,12] school2 < subset(HolzingerSwineford, school==2)[,12] test < eqMI.main(model = semmodel, sample.nobs = c(nrow(school1), nrow(school2)), sample.cov = list(cov(school1), cov(school2)), sample.mean = list(colMeans(school1), colMeans(school2)), meanstructure = TRUE, output = 'both', quiet = FALSE, equivalence.test = TRUE, adjRMSEA = TRUE, projection = TRUE, bootstrap = FALSE)