![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
fandaalThe interpretation of group differences on observed scores, in terms of psychological attributes,
depends on the invariance of measurement models across the groups that figure in the
comparison.
In psychometrics, a significant array of theoretical models and associated techniques
has been developed to get some grip on this problem (Mellenbergh, 1989; Meredith, 1993; Millsap
& Everson, 1993). In practice, however, group differences are often simply evaluated through
the examination of observed scores—without testing the invariance of measurement models that
relate these scores to psychological attributes.
Tests of measurement invariance are conspicuously lacking, for instance, in some of the
most influential studies on group differences in intelligence. Consider the controversial work
of Herrnstein and Murray (1994) and Lynn and Vanhanen (2002). These researchers infer latent
intelligence differences between groups from observed differences in IQ (across race and
nationality, respectively) without having done a single test for measurement invariance. (It is
also illustrative, in this context, that their many critics rarely note this omission.) What these
researchers do instead is check whether correlations between test scores and criterion variables
are comparable (e.g., Lynn & Vanhanen, 1994, pp. 66–71), or whether regressing some
criterion on the observed test scores gives comparable regression parameters in the different
groups (e.g., Herrnstein & Murray, 2002, p. 627). This is called prediction invariance. Prediction
invariance is then interpreted as evidence for the hypothesis that the tests in question are
unbiased.
In 1997 Millsap published an important paper in Psychological Methods on the relation
between prediction invariance and measurement invariance. The paper showed that, under realistic
conditions, prediction invariance does not support measurement invariance. In fact, prediction
invariance is generally indicative of violations of measurement invariance: if two groups differ in
their latent means, and a test has prediction invariance across the levels of the grouping variable,
it must have measurement bias with regard to group membership. Conversely, when a test is
measurement invariant, itwill generally showdifferences in predictive regression parameters. One
would expect a clearly written paper that reports a result, which is so central to group comparisons,
to make a splash in psychology. If the relations between psychometrics and psychology were
in good shape, to put forward invariant regression parameters as evidence for measurement
invariance would be out of the question in every professional and scientific work that appeared
after 1997.
depends on the invariance of measurement models across the groups that figure in the
comparison.
In psychometrics, a significant array of theoretical models and associated techniques
has been developed to get some grip on this problem (Mellenbergh, 1989; Meredith, 1993; Millsap
& Everson, 1993). In practice, however, group differences are often simply evaluated through
the examination of observed scores—without testing the invariance of measurement models that
relate these scores to psychological attributes.
Tests of measurement invariance are conspicuously lacking, for instance, in some of the
most influential studies on group differences in intelligence. Consider the controversial work
of Herrnstein and Murray (1994) and Lynn and Vanhanen (2002). These researchers infer latent
intelligence differences between groups from observed differences in IQ (across race and
nationality, respectively) without having done a single test for measurement invariance. (It is
also illustrative, in this context, that their many critics rarely note this omission.) What these
researchers do instead is check whether correlations between test scores and criterion variables
are comparable (e.g., Lynn & Vanhanen, 1994, pp. 66–71), or whether regressing some
criterion on the observed test scores gives comparable regression parameters in the different
groups (e.g., Herrnstein & Murray, 2002, p. 627). This is called prediction invariance. Prediction
invariance is then interpreted as evidence for the hypothesis that the tests in question are
unbiased.
In 1997 Millsap published an important paper in Psychological Methods on the relation
between prediction invariance and measurement invariance. The paper showed that, under realistic
conditions, prediction invariance does not support measurement invariance. In fact, prediction
invariance is generally indicative of violations of measurement invariance: if two groups differ in
their latent means, and a test has prediction invariance across the levels of the grouping variable,
it must have measurement bias with regard to group membership. Conversely, when a test is
measurement invariant, itwill generally showdifferences in predictive regression parameters. One
would expect a clearly written paper that reports a result, which is so central to group comparisons,
to make a splash in psychology. If the relations between psychometrics and psychology were
in good shape, to put forward invariant regression parameters as evidence for measurement
invariance would be out of the question in every professional and scientific work that appeared
after 1997.
