Back to [Archive] Post-review discussions

Canadian IQ on WAIS IV
Admin
The author has sent me some more information regarding the study. The problem is that he received some material from Pearson on the condition that he does not share it publicly. This sort of conflicts with the mandatory data sharing policy. In this case, however, the only relevant data are in the table found in the paper as well as a few numbers also mentioned in the paper, so there is no conflict in this case. But future cases may cause problems.

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:
The American and Canadian standardisation samples were then matched in terms of education, ethnicity and sex to give two matched samples of 488 (aged 16 to 69) and 101 (aged 70 - 90) which were 92% white and 8% 'Asian.' This reduced the Canadian advantage to 2.1 points. This implies that of the 4.5 IQ points advantage of Canada, 2.4 IQ points are attributable to racial differences.


This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?
The author has sent me some more information regarding the study. The problem is that he received some material from Pearson on the condition that he does share it publicly. This sort of conflicts with the mandatory data sharing policy. In this case, however, the only relevant data are in the table found in the paper as well as a few numbers also mentioned in the paper, so there is no conflict in this case. But future cases may cause problems.

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:
The American and Canadian standardisation samples were then matched in terms of education, ethnicity and sex to give two matched samples of 488 (aged 16 to 69) and 101 (aged 70 - 90) which were 92% white and 8% 'Asian.' This reduced the Canadian advantage to 2.1 points. This implies that of the 4.5 IQ points advantage of Canada, 2.4 IQ points are attributable to racial differences.


This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?


I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.
Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.
Admin
I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.


Sure.

Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.


Matching can skew results even if systems are the same. Matching US blacks and whites on SES reduces CA d to ~.67. But this doesn't mean they are equally smart, that is the sociologist's fallacy. One may make the same fallacy with these data, altho of course much smaller in effect size.

I would be satisfied about this issue if the author simply wrote that this reduction is expected based on racial composition data and their known IQ's, but that matching on educational attainment can bias results.

Better would of course be if Pearson would give us numbers matched on age, sex, race but not educational attainment.
I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.


Sure.

Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.


Matching can skew results even if systems are the same. Matching US blacks and whites on SES brings reduces d to ~.67. But this doesn't mean they are equally smart, that is the sociologist's fallacy. One may make the same fallacy with these data, altho of course much smaller in effect size.

I would be satisfied about this issue if the author simply wrote that this reduction is expected based on racial composition data and their known IQ's, but that matching on educational attainment can bias results.

Better would of course be if Pearson would give us numbers matched on age, sex, race but not educational attainment.


All US or Canadian patriots please forgive me but Canada and the US are pretty much identical with regards to culture or educational systems. Blacks and whites live in very different communities with different access to educational opportunities so of course matching them on education is gonna skew the results. But US Whites and White Canadians are pretty much the same culture, with a few differences here and there (a Canadian friend of mine once told me that the US is Coca Cola and Canada is Pepsi) so matching them on education is gonna have very small effect on the results.
Admin
As I wrote before, that may be true but it is irrelevant.
Can you articulate your argument and explain why it is irrelevant? And explain logically the sociologist's fallacy?
Admin
For the sociologist's fallacy, see:
Jensen, Arthur. 1973. Educability and Group Differences. Chapter 11 "Equating for socioeconomic variables"
Also discussed in Jensen, A. 1998. The g Factor: The Science of Mental Ability. p. 449ff.

The goal is to find the relative g difference between two populations. The usual sociological practice is to equate them for socioeconomic variables as was done by Pearson with the data presented in this paper. This usually reduces the difference between groups (also in this case). Then it is interpreted as showing that these environmental factors are causally responsible for the difference. This is not right. It may be that g causes the environmental difference in SES, and so controlled for/equating SES means controlling/equating for g indirectly and partly.

In this study, Pearson has matched two subsamples for age, sex, race (white) and educational attainment. This means that they are possibly indirectly controlling for g.

The matter is made worse because the persons are from two different countries which certainly have some differences in what the standards are for awarding people degrees (cf. academic inflation).
For the sociologist's fallacy, see:
Jensen, Arthur. 1973. Educability and Group Differences. Chapter 11 "Equating for socioeconomic variables"
Also discussed in Jensen, A. 1998. The g Factor: The Science of Mental Ability. p. 449ff.

The goal is to find the relative g difference between two populations. The usual sociological practice is to equate them for socioeconomic variables as was done by Pearson with the data presented in this paper. This usually reduces the difference between groups (also in this case). Then it is interpreted as showing that these environmental factors are causally responsible for the difference. This is not right. It may be that g causes the environmental difference in SES, and so controlled for/equating SES means controlling/equating for g indirectly and partly.

In this study, Pearson has matched two subsamples for age, sex, race (white) and educational attainment. This means that they are possibly indirectly controlling for g.

The matter is made worse because the persons are from two different countries which certainly have some differences in what the standards are for awarding people degrees (cf. academic inflation).


Ok I see your point. To geneticists, this is known as gene-environment correlation.
Edit 1: I had interpreted differently what they meant by matching for educational level. I had assumed they had compared each educational level in turn (e.g. Americans with a hs degree vs Canadians with a hs degree, etc.). Instead what they are doing is different: e.g. if a country has more % people with university degrees than the other, they'll match the numbers so that both samples have the same % people with a university degree.
It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.
It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.


Can you also state that the higher Canadian vs US IQ is probably due to more blacks living in the US than in Canada?
It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.


Can you also state that the higher Canadian vs US IQ is probably due to more blacks living in the US than in Canada?


I thought I kind of said that. I can make it more explicit if you like.
Admin
I converted the data to a useful file. See here.

I found some of the g-loadings for the tests in this paper: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0074980

Then I used MCV to calculate the Jensen effect. It is 0.83. P is 1e-04. This probably isn't a fluke. Scatterplot attached.

Where do you want to go from here? Presumably you want to include this analysis. It is a good finding. Jensen effects between two white populations.

The R code used to do this analysis is:
library(car)

CANUS.d = c(0.24,0.23,0.31,0.2,0.26,0.29,0.11,0.22,0.14,0.17,0.23,0.21,0.21,0.06,0.03) #input d data
CANUS.g = c(0.6474,0.7044,0.6859,0.7158,0.6796,0.7704,0.4853,0.6571,0.6487,0.5367,0.6656,0.778,0.6985,0.3472,0.4562) #input g loadings

rcorr(CANUS.d,CANUS.g) #correlation results

DF = as.data.frame(cbind(CANUS.d,CANUS.g))
scatterplot(CANUS.d ~ CANUS.g, DF,smoother = F)
I converted the data to a useful file. See here.

I found some of the g-loadings for the tests in this paper: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0074980

Then I used MCV to calculate the Jensen effect. It is 0.83. P is 1e-04. This probably isn't a fluke. Scatterplot attached.

Where do you want to go from here? Presumably you want to include this analysis. It is a good finding. Jensen effects between two white populations.

The R code used to do this analysis is:
library(car)

CANUS.d = c(0.24,0.23,0.31,0.2,0.26,0.29,0.11,0.22,0.14,0.17,0.23,0.21,0.21,0.06,0.03) #input d data
CANUS.g = c(0.6474,0.7044,0.6859,0.7158,0.6796,0.7704,0.4853,0.6571,0.6487,0.5367,0.6656,0.778,0.6985,0.3472,0.4562) #input g loadings

rcorr(CANUS.d,CANUS.g) #correlation results

DF = as.data.frame(cbind(CANUS.d,CANUS.g))
scatterplot(CANUS.d ~ CANUS.g, DF,smoother = F)


Oh, I see. I thought you'd found this on the internet! I will rewrite again and include this
It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.


Can you also state that the higher Canadian vs US IQ is probably due to more blacks living in the US than in Canada?


I thought I kind of said that. I can make it more explicit if you like.


You wrote "This reduction is expected based on racial composition data because Northeast Asians have a significantly higher average IQ than African Americans or Hispanics". But you didn't say that the US has more Hispanics and African Americans (you left this implicit). It should be made explicit with figures from census data. After that I'll approve.
I converted the data to a useful file. See here.

I found some of the g-loadings for the tests in this paper: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0074980

Then I used MCV to calculate the Jensen effect. It is 0.83. P is 1e-04. This probably isn't a fluke. Scatterplot attached.

Where do you want to go from here? Presumably you want to include this analysis. It is a good finding. Jensen effects between two white populations.

The R code used to do this analysis is:
library(car)

CANUS.d = c(0.24,0.23,0.31,0.2,0.26,0.29,0.11,0.22,0.14,0.17,0.23,0.21,0.21,0.06,0.03) #input d data
CANUS.g = c(0.6474,0.7044,0.6859,0.7158,0.6796,0.7704,0.4853,0.6571,0.6487,0.5367,0.6656,0.778,0.6985,0.3472,0.4562) #input g loadings

rcorr(CANUS.d,CANUS.g) #correlation results

DF = as.data.frame(cbind(CANUS.d,CANUS.g))
scatterplot(CANUS.d ~ CANUS.g, DF,smoother = F)


Oh, I see. I thought you'd found this on the internet! I will rewrite again and include this


I understand this as meaning, correct me if i'm wrong, that the IQ of white Canadians is genuinely likely to be higher than that of white Americans. Is this so?
I converted the data to a useful file. See here.

I found some of the g-loadings for the tests in this paper: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0074980

Then I used MCV to calculate the Jensen effect. It is 0.83. P is 1e-04. This probably isn't a fluke. Scatterplot attached.

Where do you want to go from here? Presumably you want to include this analysis. It is a good finding. Jensen effects between two white populations.

The R code used to do this analysis is:
library(car)

CANUS.d = c(0.24,0.23,0.31,0.2,0.26,0.29,0.11,0.22,0.14,0.17,0.23,0.21,0.21,0.06,0.03) #input d data
CANUS.g = c(0.6474,0.7044,0.6859,0.7158,0.6796,0.7704,0.4853,0.6571,0.6487,0.5367,0.6656,0.778,0.6985,0.3472,0.4562) #input g loadings

rcorr(CANUS.d,CANUS.g) #correlation results

DF = as.data.frame(cbind(CANUS.d,CANUS.g))
scatterplot(CANUS.d ~ CANUS.g, DF,smoother = F)


Oh, I see. I thought you'd found this on the internet! I will rewrite again and include this


I understand this as meaning, correct me if i'm wrong, that the IQ of white Canadians is genuinely likely to be higher than that of white Americans. Is this so?


Not really, it doesn't tell you anything about the racial origin of the IQ difference. It just says that this difference is based on g.
Admin
It indicates that the IQ difference is on g, not on the non-g variance (well, most of it is on g). This may be the first time I've seen a white-white country-country comparison used for a Jensen analysis.
It indicates that the IQ difference is on g, not on the non-g variance (well, most of it is on g). This may be the first time I've seen a white-white country-country comparison used for a Jensen analysis.


92% white and 8% 'Asian.'
I submit yet another version. I have referenced the data file.
Admin
There is no attached datafile. You should write something like "See supplementary material". Furthermore, you should probably add the scatterplot. Adding the scatterplot saves you from having to add the number of subtests (N=15) used for the MCV analysis. The other reason to add the scatterplot, is that a visual inspection of the data is important given that the data may be clearly non-linear. Correlations are the strength of the linear relationship.

Finally, there is no credit to the reviewer who actually suggested and did the analysis.

Otherwise, it is fine.