Hello There, Guest!  

[OQSPS] Cognitive ability and political preferences in Denmark

#1
Journal:
Open Quantitative Sociology & Political Science

Authors:
Emil O. W. Kirkegaard
Julius D. Bjerrekær
Noah Carl

Title:
Cognitive ability and political preferences in Denmark

Abstract:
We investigated the relationships between cognitive ability and multi-dimensional political preferences in a recent Danish sample (n = 320). Respondents answered 10 questions pertaining to specific social issues, 10 questions pertaining to specific economic issues, as well as taking 4 cognitive items. They had previously taken a 5-item cognitive test (ICAR5), and been asked to rate themselves on social liberalism and economic liberalism. We documented a general factor of social liberalism across the questions on social issues, and a general factor of economic liberalism across the questions on economic issues. Self-assessed social liberalism had a small positive correlation with measured social liberalism (r = .12), while self-assessed economic liberalism had a moderate positive correlation with measured economic liberalism (r = .47). These findings were in line with our predictions. Contrary to our predictions, however, social liberalism and economic liberalism had a weak positive correlation (r = .10; 95% CI = [–.04, .22]) and cognitive ability was practically unrelated to both social liberalism (r = .06; 95% CI = [–.07, .19]) and economic liberalism (r = .05; 95% CI = [–.08, .18]).

Key words:
cognitive ability, intelligence, IQ, political preferences, political dimensions, Denmark, open data, preregistered, social liberalism, social conservatism, economic liberalism

Length:
~3800 words, 21 pages.

Files:
https://osf.io/xdpcq/
Note that the questionnaire is currently not available. It will be made available shortly (pending email reply from the survey handler).

External reviewers:
We will attempt to recruit an external reviewer who's familiar with this area. Perhaps Peter Kurrild-Klitgaard. Prof. Pol. Sci. @ Copenhagen Uni. http://forskning.ku.dk/find-en-forsker/?...sons/63119

Since this paper has two of the internal reviewers familiar with this topic, we probably need to recruit an additional temp. external reviewer or a new internal one.

Status
The results in the paper are not final because we are still collecting a little bit of data (perhaps 30 more cases). However, it is very unlikely that they will substantially change. For this reason, the numbers in the paper are temporary and will be updated.

The tables are temporary. Julius will typeset the paper in LaTeX, so there is no reason with bothering to make the tables in Word pretty. But they should be readable for the sake of review.
 Reply
#2
Grammar-type comments are in the attachment. More substantively:

1. It might improve the paper if the text explicitly indicated what research design elements were preregistered and what elements were not. Some of the text implies exploratory analyses (e.g., "we tried conditioning on..."), but ideally readers would not need to consult the preregistration plan to know what parts of the research design were preregistered.

2. The "Exclusions" section mentioned that about 20% of the sample failed at least one attention-check-type item, but the section did not mention whether these respondents were included in the analysis. It would be a good idea to explicitly indicate in this section whether respondents who failed a check were removed from the sample. I also think that the attention-check-type items might have been absent from the preregistration plan; if that's the case, then it might be worth noting this in the text.

3. The conclusion indicates that the results that differ from prior research might be attributable to the sample not being representative, but there appears to be no discussion about how representative the original sample or the follow-up study's sample might be of the population of interest. The paper might benefit from a discussion and/or analysis of how representative the follow-up sample is and a discussion of the extent to which the present study should be preferred or discounted relative to prior research that had a different result. It would also be worth quantifying in the conclusion how much the present study's results differ from results of prior research, such as by noting the cognitive ability/social liberalism correlations from Carl 2015b and Onraet et al. 2015. The 95% CI for the present research appears to be somewhat close to the 0.10 to 0.30 range quoted in the introduction.


Attached Files
.docx   COGNITIVE ABILITY AND POLITICAL PREFERENCES IN DENMARK.docx (Size: 161.24 KB / Downloads: 120)
 Reply
#3
Good criticism. We are working on revision.
 Reply
#4
Many thanks for the review.

(2016-Sep-25, 22:43:52)ljzigerell Wrote: Grammar-type comments are in the attachment.


Grammar errors have been changed.

(2016-Sep-25, 22:43:52)ljzigerell Wrote: 1. It might improve the paper if the text explicitly indicated what research design elements were preregistered and what elements were not. Some of the text implies exploratory analyses (e.g., "we tried conditioning on..."), but ideally readers would not need to consult the preregistration plan to know what parts of the research design were preregistered.


Analyses that were not pre-registered have been indicated as such in the text.

(2016-Sep-25, 22:43:52)ljzigerell Wrote: 2. The "Exclusions" section mentioned that about 20% of the sample failed at least one attention-check-type item, but the section did not mention whether these respondents were included in the analysis. It would be a good idea to explicitly indicate in this section whether respondents who failed a check were removed from the sample. I also think that the attention-check-type items might have been absent from the preregistration plan; if that's the case, then it might be worth noting this in the text.


It is now clearly stated in the text that all respondents who failed at least one of the control questions were excluded from the analyses.

(2016-Sep-25, 22:43:52)ljzigerell Wrote: 3. The conclusion indicates that the results that differ from prior research might be attributable to the sample not being representative, but there appears to be no discussion about how representative the original sample or the follow-up study's sample might be of the population of interest. The paper might benefit from a discussion and/or analysis of how representative the follow-up sample is and a discussion of the extent to which the present study should be preferred or discounted relative to prior research that had a different result. It would also be worth quantifying in the conclusion how much the present study's results differ from results of prior research, such as by noting the cognitive ability/social liberalism correlations from Carl 2015b and Onraet et al. 2015. The 95% CI for the present research appears to be somewhat close to the 0.10 to 0.30 range quoted in the introduction.


A comparison of the Danish adult population, the original sample and our sub-sample on age, sex and education has been included in the Discussion section. Onraet et al.'s (2015) meta-analytic effect size has been reported in the text. And it has been noted in the text that the upper bound of the confidence interval for the correlation between cognitive ability and social liberalism is equal to this effect size.
 Reply
#5
Two points:

1. All too often, the concepts of "social liberalism" and "economic liberalism" are disingenuous. Many people are liberal because they see liberalism as a means to weaken the existing system or the existing elite. Once these "liberals" gain power, they develop an interest in the state as a means to enforce their values, or as a supposed "temporary measure" to keep the other side from regaining power. It might be useful to question the subjects to find out whether they see liberalism as an end or as a means to an end. Do the subjects themselves accept your dichotomy of liberalism vs. non-liberalism?

2. Were there gender differences in the results? Usually, when the right or the left is on the upswing, women tend to lag behind men.
 Reply
#6
Peter,

1.
We can't ask the subjects more questions because the data collection is complete. As you can see in the paper, we are pretty clear about what our concepts are because we included all the 10 questions used to measure each in the paper. No verbal tricks. The subjects were not told about the purpose of the political questions, so the concept of liberalness never came up. In the previous survey where we asked about their self-rated stances, we did not use the word "liberal" either. We asked them about how much they prefer personal freedom and economic freedom with some examples of each (e.g. legalized cannabis, lower taxes).

In general, I prefer to stay away from contested political labels.

2.
As you can see in the regression tables, men were somewhat more freedom-preferring for both axes (controlling for other things). Standardized betas of .30 and .47. These, however, did not survive the LASSO regression. Possibly because we had to treat them as continuous variables for the purpose of that modeling. This deflates the results a bit, but I think we can be pretty certain that the sex difference is real. I did try to find a LASSO method that worked with GLM, so one can use a categorical variable, but did not get it to work properly.

I see that we forgot to include the simple bivariate results, i.e. correlations/d values for all the predictors with both political axes. We will revise to include that since it's important information.

http://rpubs.com/EmilOWK/208757
 Reply
#7
The pollster forgot to invite the 66-80 year olds for this study. He will do so now and this will probably increase the sample size by a bit, maybe 10-30 persons. It should also help with the age representation problem: currently mean age of 37.2 in this sample vs. 49.1 in 18-80 population Denmark.

It is unlikely to change the main findings because age does not seem to interact with CA for predicting political preferences.
 Reply
#8
I checked the revision and approve the submission.


For what it's worth, I think that "measure" should be "measured" in the passage "the correlation between self-assessed and measure social liberalism" (p. 14). Also, it's not a problem, but the figures use "personal freedom" and "economic freedom" for phenomena that the text refers to with "social liberalism" and "economic liberalism".
 Reply
#9
Good comment.

The reason for the slight inconsistencies in nomenclature is that I wrote the R code and thus the figure captions, and Noah wrote the paper. I will make it more consistent and fix the error you found.
 Reply
#10
In the abstract there is a general description of what was done, but no clear statement of objective. I think most readers would want to see something that clearly states something about the purpose of the study.



I am confused by this:
By going through a pollster, we administered 24 additional items to the sample:
After reading again, I assume that this simply means that when you used the pollster, you added the additional items. If so, the confusion I experienced was related to the word "by." It could be written to say "When the polling was repeated with a pollster, we added 24 additional items." ... Or something to that effect.


The final sample size was n = 327 (65% of the original n).
On first reading, it was unclear as to why the final sample size was smaller. I now assume that 327 was the number of people who responded to the pollsters, although it was not obvious to me that they contacted all of the first group or only part. Can this be worded differently?
Later (section 2.3) there is a comment about removing responses where random answering was indicated, but that resulted in a 20% reduction. Was the 20% reduction before or after the initial reduction? I assume that is what happened.



As Table indicates, responders were slightly younger, had slightly lower cognitive ability and were slightly less educated than non-responders. There was, therefore, some selection bias in responding to our survey.
Some readers will recall that the usual pattern of participation is skewed towards brighter people, due to their greater willingness to participate. This case went the other way. I assume this is random, but it might be worth a sentence or two to explain the likely cause of this somewhat unexpected response makeup.



Just a comment on the Likert scales: It seems to me that by dividing the scales into two ranges, an unnecessary distraction resulted. This meant lots of discussion about combining the results when there was nothing gained with respect to the paper. Using either one would have, in my opinion, been better and would not have resulted in any loss of value. Given that this was already done, there is no way to change it now.



Figures 3 and 4
Some of the item characteristic curves seem to have low slopes, making the inflection point indistinct. I understand that you can define the latent trait despite this, but it appears to me that those curves are not what you would want. The factor analysis shows 4 distributions that show reasonable looking shapes, but the others seem to be uninformative. There was little discussion of this, but there was a mention of too few easy items. Can more be said about the impact of this skew on the outcome? It leaves me wondering how one can justify using uneven item difficulty to indicate the relations that are discusses with respect to population preferences versus intelligence.



Surprisingly, there wasn’t much of a relationship in either case, not even a non-linear one: although both estimates are positive, their confidence intervals encompass zero.


and cognitive ability was practically unrelated to both social liberalism (r = .08; 95% CI = [–.04, .20]) and economic liberalism (r = .06; 95% CI = [–.06, .18]); although both of these correlations were in the expected direction, they were much smaller than we had anticipated.


Second, our measure of cognitive ability was somewhat crude, comprising just 9 items in total, and providing limited discriminability in the left-hand tail.


The above may be related to my comments concerning figures 3 and 4. The papers I have read on the subject of political orientation on left-right scale or liberal-conservative scales have shown that there is a correlation with intelligence (as you acknowledge near the end of the paper), although the results have been somewhat mixed. So, I too am surprised by this result and ask if it can be traced to the failure to use a population representative measurement of intelligence. I am a bit more confused in that the ICAR5 should have given an adequate measurement. I admit to confusion, which may be entirely mine and not a problem with the methodology.
 Reply
 
 
Forum Jump:

Users browsing this thread: 1 Guest(s)