In general, I don't see any large problems with this study. An idea was tested with the relevant datasets in an appropriate way.

All quotes are from the authors/the paper unless otherwise stated.

However, we cannot completely comply with your submission requirements, as one column in our data set (Wonderlic IQs by job) is copyrighted / proprietary.

One cannot copyright datasets (https://en.wikipedia.org/wiki/Sui_generis_database_right#United_States ). This column is not copyrighted. Note that this information has already been published elsewhere, e.g. by Gottfredson (1997). This was however the old version (1992 as I recall). If you don't dare publish it, please ask Wonderlic for permission to share this dataset.

Speaking of which, did you cross-check the IQ by job with the old Wonderlic data? It may not use exactly the same job types, however. Jobs change over time, of course.
There is also a large collection of IQ by job types here: http://emilkirkegaard.dk/en/?p=5644 Most of them are not as extensive as yours.

Aside from that one column, I note that the remaining dataset is not published attached either. All data that can be shared, must be shared for publication in this journal, so I'd like the authors to please provide the dataset. The best solution is to use OSF and make a public repository: http://osf.io/
However, I will do this for the authors, if they do not have the time or expertise.

Scores on diverse sets of mental abilities tests correlate positively.

I would cite Carroll 1997 and/or Dalliard 2013 for this.
http://www.cambridge.org/us/academic/subjects/psychology/cognition/human-cognitive-abilities-survey-factor-analytic-studies
http://humanvarieties.org/2013/04/03/is-psychometric-g-a-myth/
The first being the conservative choice.

Biologically, g likely reflects the speed and efficiency with which brains process information (Jensen, 1998, 2011; Pesta & Poznanski, 2008).

Perhaps take a look at: http://www.nature.com/nrn/journal/v11/n3/full/nrn2793.html
There are more studies too. There is a brief summary of these findings in http://emilkirkegaard.dk/en/?p=5034 (unfinished research project)

Galton (1892) first documented Black / White (e.g.) differences on cognitive tests, and these differences have persisted in the literature since then (Roth et al., 2001).

I find that hard to believe. Galton did not have any IQ tests since the first was invented by Binet some years later. He did make some rough estimates of the race groups. Perhaps that's what you have in mind?

This century, Roth et al., (2001) conducted a massive, meta-analytic summary of race / IQ differences (N = 6,246,729 people). They reported an overall Black / White effect size of 1.10.

Perhaps the most comprehensive and recent analysis is that unpublished study by Fuerst at
http://humanvarieties.org/2013/01/15/secular-changes-in-the-black-white-cognitive-ability-gap/
I told him to get it published. Let's see what happens! :)

Other racial and ethnic groups show different mean IQs. The Asian IQ mean is estimated at 106 (Rushton & Jensen, 2005).

This is ”Asian” in the US. Asian is a broad category that includes e.g. Indians. Indians in the US do very well (https://en.wikipedia.org/wiki/Indian_Americans#Socioeconomic ), and may be the top scoring group, but in India, they do very poorly (mean IQ around low 80s). You should clarify.

The digit span task is a classic example illustrating this effect. Here, the tester vocalizes a random number string (e.g., 5, 4, 7, 1, 9) at the rate of one number per second. The respondent then immediately attempts to recall each number in either forward (i.e., 5, 4, 7, 1, 9) or backward (i.e., 9, 1, 7, 4, 5) serial order. The backward version is roughly twice as g-loaded as the forward version, and it produces roughly twice as large a difference across Black / White groups (Jensen & Figueroa, 1976).

This claim was also made last time and it is not right. See:
http://openpsych.net/forum/showthread.php?tid=239

Lower scores on a DOT element indicate higher complexity and / or stature.

Think you mean status? Stature is height. Perhaps you mean it in the extended meaning of “social standing” or so.

Tying together IQ, race, and job complexity, we return to the method of correlated vectors. We argue that a job is analogous to a mental test. Different jobs require different IQs (i.e., levels of g) to perform successfully. A novel test of Spearman’s hypothesis is to use the IQ of various jobs as predictors of the racial composition of their employees. If Spearman’s hypothesis is correct, as job IQ increases, the percentage of Black workers holding the job should decrease (but the percentage of White and Asian workers should increase). The opposite is predicted as job IQ decreases.

I would definitely recommend reading:
Gordon, R. A. (1997). Everyday life as an intelligence test: Effects of intelligence and intelligence context. Intelligence, 24(1), 203-320.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.372.8122&rep=rep1&type=pdf

Great paper. You are doing much the same. You treat the different jobs at items in a test, with their own g loadings and then do a test of Spearman's hypothesis on these.

By the way, this kind of study has been done for Academic majors (students):
http://www.sciencedirect.com/science/article/pii/S0191886902002155

It has also been done for gender, although this is more dubious due to the lack of agreement on whether there actually is a mean IQ difference or not.

We coded all but percent-Hispanic, as the BLS considers this group to be an ethnicity versus a race. That is, people in the BLS’ Hispanic category could be either White, Black or neither race. We also coded the identical data set for the year 2012 (BLS, 2013). We did this to cross-validate results found with the 2014 data, and to test whether the demographic content of different jobs is stable across years.

You say, “we”. Did you have two coders (there are two authors)? If so, where there any disagreements about the coding? How were these resolved?

Although the effect is not large, the mean IQ of all jobs, 104.5, is higher than the population IQ mean of 100 for individuals. We speculate this is likely because there are not many jobs with considerably low IQ requirements. That is, few jobs exist where the minimum IQ is in the 10th or 15th percentile. Consistent with this is the fact that the lower one’s IQ, the less likely he or she is to be in the labor force (see, e.g., Gottfredson, 2003; Herrnstein & Murray, 1994). At any rate, the mean value of 104.5 represents the population parameter for all jobs listed in the Wonderlic (2002) test manual.

This corresponds to the low correlation found between being out of a job and IQ at the individual level. You could back-estimate this correlation using this mean. Just a minor check.

As a cross check on validity, we correlated these same variables, but used the 2012 BLS data. The correlations were .46, -.57, and .20 respectively. Also, the 2012 and 2014 BLS data showed very strong stability in the demographic makeup of jobs over a two-year period. The correlations were .88, .87 and .87 for percent White, Black, and Asian (paired as 2012 and 2014 values), respectfully.

It would be better to just average the BLS values across years, no?

I also recommend reading the following paper, which is similar to yours:
McDaniel, M. A., & Kepes, S. (2014). An Evaluation of Spearman's Hypothesis by Manipulating g Saturation. International Journal of Selection and Assessment, 22(4), 333-342.
http://emilkirkegaard.dk/en/wp-content/uploads/An-Evaluation-of-Spearmans-Hypothesis-by-Manipulating-g-Saturation.pdf

This study checked SH by creating different IQ tests out of a collection of subtests. As expected, the group difference goes up the higher the g-loading of the IQ test created.

We found consistent support for this hypothesis. As job IQ increased, the percent of Black workers decreased. Conversely, the percent of White and Asian workers increased. Although the correlation was small for Asians, it was non-trivial for Whites, and large for Blacks. Spearman’s hypothesis is therefore supported in a novel way.

I'd like the authors to add some ideas for why the Asian% X IQ is so low. Theory-wise, it is predicted to be larger due to Asians' mean IQ being further away from the mean IQ, I think.

My ideas: 1) Asians are very clustered in certain geographic areas where not all job types are found. 2) Asians have substantially different job interests (no idea if this is true), which therefore throws the correlations off.

Note that the reliability of Asian% was just as high as that for White%/Black%, so it cannot be a reliability issue.

I'd like to authors to use multiple regression to predict the racial%'s. Perhaps the Asian% finding is due to Asians having some large preference for not working with people or preference for working with things. Multiple regression may be able to shed some light on this. Does the DOT supply full RIASEC scores by job or only the three datapoints listed?

Consistent with past research (Gottfredson, 1986; 2003; Schmidt & Hunter, 2004), we found that job complexity and IQ are strongly related. The DOT element, Data, correlated very strongly with job IQs as reported by the Wonderlic (2002). The People (but not Things) element was also a non-trivial predictor of job IQ. Again, job complexity and IQ are intrinsically linked.

I would like the authors to provide the effect sizes of the prior research so readers can see whether the effect sizes are similar. This is needed for cumulative science.
http://www.phil.vt.edu/dmayo/personal_website/Schmidt_StatSigTesting.pdf

Sounds good.

I have converted the data to CSV format (program neutral) and uploaded the files to an OSF repository. If the authors make users on OSF, I will transfer control of the repository to them.

https://osf.io/pxmjc/files/

Asians, as a group, does have a genetic basis, but it would be complicated to use because one would then have to include Native Americans and Eskimos (who are sometimes, sometimes not considered part of the US "Asian" category). Furthermore, many migrants from Latin/South America would also be classified as majority Asian by genomic analysis. So, yeah, it would be complicated, but doable to some degree.

My problem is that Hispanic is a category not mutually exclusive with Black, White, or Asian. The later three categories, however, are mutually exclusive with each other.

I'm thus really not comfortable coding, reporting, and interpreting data from this fourth group, whose composition is an unknown blend of the other three (race) groups, and whose numbers are also represented as counts in other race categories.

Hi Brian,

My mind must be fuzzy. Maybe Emil can jump in and help explain.

As I noted, "Hispanics" are generally delineated that way. What is somewhat unusual is the delineation of the Black, White, and Asian groups, from which, in this case, Latin American origin people are NOT excluded. But, since you are using simple correlations, why would the atypical delineation of the latter groups matter?

Currently you are using:
e.g., White/(White+Asian + Black)
Which is:
(non-Hispanic White + Hispanic White)/(total population - "other races")

But ideally, if you had the data, you would use:
e.g., non-Hispanic White/(total population - "other races")

And I'm just asking for, additionally:
Hispanic/(White+ Asian + Black)
Which is:
Hispanic/(total population - "other races")

The denominator is the same!

He is not using ratios, he is using the plain percentages.

"White%" not for instance "White%/Black%".

Asians, as a group, does have a genetic basis

No.

"Asian", in this context, is a class (one which includes: South Asians and East Asians) in a socio"racial" classification, one which has the following other divisions: Blacks and "Whites" (meaning: West Eurasians). By what set of genes could one arrange (classify) almost all of our "Asians" into their census class as opposed to that of Blacks and West Eurasians?

(One would have to reduce the class k to 2: (a) Blacks and (b) (East + South) + West Eurasians (i.e., out of Africans)).

After internal discussion. I now agree with John. Indians go into the west Eurasian cluster and thus cannot be grouped with the other Asians genetically without violating the rules of cladistics.

Bryan,

Quotes are yours unless otherwise specified.

REPLY: We did not cross-check jobs, but instead relied on the Wonderlic’s ability and expertise at generating estimated IQs for different jobs. Also, we didn’t have older Wonderlic data, and suspect—as you point out—that matching jobs might be frustrating.

Older Wonderlic data is given by Gottfredson's classic 1997 paper. It was in my link.

Old Wonderlic data from Gottfredson.

REPLY: Would it be ok to include “(but, see, <your link>)” immediately after the word “version” in our quote above? If not, we can delete this entire section.

No. Please see the discussion about your previous submission. The BDS does not have twice the g-loading of FDS. Dalliard and myself compiled several studies to show this, yet you made the same exact claim in your next submission. I find that odd.

This (the slightly higher mean IQ among the occupations) corresponds to the low correlation found between being out of a job and IQ at the individual level. You could back-estimate this correlation using this mean. Just a minor check.

REPLY: Not sure how to do this. If desired for our paper, could you please advise?

One could use a simple cutoff based model, which assumes that everybody above a certain cutoff are in work and everybody below is not. One could also use a more gradual change, but I didn't have a ready made function for this.

These are aggregated data, so one should use the weighted mean IQ, weighted by occupation population. These are not given in your datafile, so I could not do this.

I tried with the given number, but this produced a correlation of .67, which is much too strong. So my proposed method doesn't work. Just ignore this.

(code for this is found in my R code)

It would be better to just average the BLS values across years, no?

REPLY: Correlations seem more intuitive? The rank ordering of jobs by ethnic composition is pretty much the same across a two year span.

You misunderstood. I proposed that you use the average of the BLS 2014 and 2012 values to remove some of the slight 'measurement error'.

I did this in my replication.

REPLY: In addition to your hypotheses, we wonder about a potential floor effect on the percentage of Asians in each job. We’d prefer not to complicate the manuscript by addressing why the %Asian correlation was lower than might be expected (it was nonetheless non-trivial and significant). This might be an interesting question for an additional study, but the present study just predicted a significant/non-zero correlation between %Asian and IQ. Although the correlation was smallish, its existence was predicted by Spearman’s hypothesis (and testing this hypothesis was the purpose of our study). Also, since our data are just for the USA, wouldn’t a group six points above average IQ produce smaller over/under-representation relative to a group 15 points below average IQ?

1)
Not controlling for known confounds (interests in this case) which you already have the data for is not really defensible. After all, you are interested in the effect of cognitive ability itself, not whatever it is that it happens to be correlated with. If you use correlations, you will get a confounded estimate of the influence of cognitive ability itself.

2)
Merely looking for p < alpha vs. p > alpha results is not a good way of doing science. What matters is whether the numbers fit in size. Please see: http://www.phil.vt.edu/dmayo/personal_website/Schmidt_StatSigTesting.pdf

3)
By your reasoning, White% should produce the weakest correlation, yet it does not.

REPLY: Our correlational values are effect sizes, we think. Schmidt and Hunter’s paper was a meta-analysis and so reported effect sizes. Beyond reporting correlations, I’m not sure that Gottfredson calculated effect sizes. If I’m misinterpreting what you’re asking for here, please let me know.

Your correlations are effect sizes, yes. However, I asked for "the effect sizes of the prior research so readers can see whether the effect sizes are similar".

You present some new results. What the readers need to know is whether they fit in size with the previous results. For instance, if you find r = .20 and previous studies have found r = .95, something is wrong somewhere.

The data variable (complexity) was correlated with mean IQ at .86 in your study. You cite:

Gottfredson, L. S. (1986). Occupational aptitude patterns map: Development and implications for a theory of job aptitude requirements (Monograph). Journal of Vocational Behavior, 29, 254-291.

Gottfredson, L. S. (2003). g, jobs, and life. In H. Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 293-342). New York: Pergamon.

However, I could not find any complexity x mean IQ correlation in these papers. She does give job mean IQs and presents factor analysis results of job attributes, but does not appear to actually correlate them. Maybe I missed the number somewhere?

--

Data analysis
I replicated the authors' analysis and also did the required additional multiple regression analyses. I furthermore did some path models too, which allow for more precise causal modeling.

I used the average race% values from the BLS (2012 and 2014) to increase reliability. This results in the following correlations:


iq white black asian other data people things
iq 1.00 0.48 -0.61 0.23 -0.51 -0.86 -0.53 0.04
white 0.48 1.00 -0.85 -0.38 -0.49 -0.52 -0.37 -0.04
black -0.61 -0.85 1.00 -0.14 0.47 0.66 0.30 0.09
asian 0.23 -0.38 -0.14 1.00 -0.14 -0.21 0.13 -0.07
other -0.51 -0.49 0.47 -0.14 1.00 0.45 0.24 -0.04
data -0.86 -0.52 0.66 -0.21 0.45 1.00 0.50 -0.01
people -0.53 -0.37 0.30 0.13 0.24 0.50 1.00 -0.24
things 0.04 -0.04 0.09 -0.07 -0.04 -0.01 -0.24 1.00


Note that I have included "other", which is the residual group. This is some mix of Native Americans, Hispanics, mixed race persons and so on. The mean IQ of this group is usually found to be somewhat below the population mean (around 95), so one would expect a negative correlation, which is also found.

John prefers the rationized group proportions (for the lack of a better term). In this way the influence of the "other" group is removed. So, for Whites this is simply White% / (White% + Black% + Asian%). Same for the others. These correlations are:


iq white_ratio black_ratio asian_ratio data people things
iq 1.00 0.44 -0.61 0.22 -0.86 -0.53 0.04
white_ratio 0.44 1.00 -0.84 -0.42 -0.49 -0.35 -0.04
black_ratio -0.61 -0.84 1.00 -0.14 0.66 0.30 0.09
asian_ratio 0.22 -0.42 -0.14 1.00 -0.20 0.13 -0.07
data -0.86 -0.49 0.66 -0.20 1.00 0.50 -0.01
people -0.53 -0.35 0.30 0.13 0.50 1.00 -0.24
things 0.04 -0.04 0.09 -0.07 -0.01 -0.24 1.00


So this correction made little difference and seems unnecessary.

Of more interest are multiple regressions where we take the other job data into account. I used cognitive ability, people and things to predict each of the race%.

For Whites:


$coefs
Beta SE CI.lower CI.upper
iq 0.39 0.09 0.20 0.57
people -0.19 0.10 -0.37 0.00
things -0.09 0.08 -0.26 0.07

$meta
N R2 R2 adj. R2 10-fold cv
124.00 0.26 0.24 0.19


Blacks:


$coefs
Beta SE CI.lower CI.upper
iq -0.60 0.09 -0.77 -0.43
people 0.02 0.09 -0.16 0.19
things 0.12 0.07 -0.03 0.26

$meta
N R2 R2 adj. R2 10-fold cv
124.00 0.38 0.36 0.34


Asians:

$coefs
Beta SE CI.lower CI.upper
iq 0.41 0.10 0.21 0.61
people 0.34 0.10 0.14 0.55
things 0.00 0.09 -0.17 0.17

$meta
N R2 R2 adj. R2 10-fold cv
124.00 0.14 0.12 0.03

Notice the very low cross-validated R2. This is suspicious.


$coefs
Beta SE CI.lower CI.upper
iq -0.54 0.09 -0.72 -0.36
people -0.05 0.10 -0.24 0.14
things -0.03 0.08 -0.19 0.13

$meta
N R2 R2 adj. R2 10-fold cv
124.00 0.27 0.25 0.24


So, to recap, the standardized betas for the races with the other job predictors were:
White: .39
Black: -.60
Asian: .41
Other: -.54

Thus, the Asian result is not really an outlier once one takes into account their higher preference for working with people (people beta = .34). Note that White has a people beta at -.19, but it's not a reliable finding (cf. the confidence interval). Still, this means that Whites and Asians are pretty different on this preference, apparently.

Path models
Here one can stipulate the causal order of variables (kind of). I specified that the interests are caused by cognitive ability and that the race% are caused jointly by cognitive ability requirement and interests.

Here's the plots:

White:
[attachment=726]

Black:
[attachment=727]

Asian:
[attachment=728]

Other:
[attachment=729]

Looking at these we see that the results for the cognitive ability paths are consistent with the other results. Direct paths: .38, -.51, .23, -.51. For Asian and White there are opposite direction indirect paths thru the people interest as seen before.

--

Code for my replication is in the OSF folder "Emil_replication".

https://osf.io/pxmjc/files/

Bryan,

Quotes are yours unless otherwise specified.

REPLY: We did not cross-check jobs, but instead relied on the Wonderlic’s ability and expertise at generating estimated IQs for different jobs. Also, we didn’t have older Wonderlic data, and suspect—as you point out—that matching jobs might be frustrating.

Older Wonderlic data is given by Gottfredson's classic 1997 paper. It was in my link.

Old Wonderlic data from Gottfredson.

I cannot get back to revising this til the weekend (Chuck: I was waiting to write the formal revision until more reviews were in).

To clarify just the above, are you asking me to use my fingernail on the chart to estimate the mean IQs for each job in the figure, and then attempt matching them to my data set of 100+ jobs, just to see what the correlation might be?

I see also the figure says "reprinted by permission of the publisher".

Bryan,

Yes, that's the idea. It would be another validity check of your mean IQs by job. Of course, you may find that the jobs do not match very well and so this test is not possible. Gottfredson's list has a lot fewer jobs than your list, I think.

Due to the litigious nature of the US, many people ask for permission where none is needed. Reprinting a figure is clearly fair use.