Hello There, Guest!  

[OQSPS] Net Opposition to Immigrants of Different Nationalities

#11
Very nice analysis and write up. I'll offer a few suggestions:

1. I think that the manuscript should disclose that, for each of the 23 dataset countries, the YouGov immigration item had "Don't Know" responses for 22% to 29% of observations.

2. For the data analysis that I conducted, alternate specifications did not affect the inference about the correlation of net opposition and the log of arrest rates; the alternate specifications were a modified outcome variable (the more option, a four-option scale without Don't Knows, and a four-option scale with Don't Knows coded as "the same"); a revised coding of the country controls (Western Europe, as Western excluding Poland; West as Western including Romania, Russia, and Israel; including dichotomous controls for other regions, such as East Asia, Africa, and Latin America); and coding the percentage white for South Africa as 90% (the percentage of South Africans in the UK who are white, according to Wikipedia), instead of as the 8.2% of whites in South Africa.

It might be beneficial to the reader to note that the manuscript's main inferences do not change with particular alternate specifications, whether the aforementioned specifications or other specifications.

3. It might be valuable to note in the manuscript the countries coded as Western. The countries furthest below the regression line in Figure 1 are those most plausibly considered Western, but other plausibly Western countries are near or above the line, so it might be valuable to know which countries are coded Western.

4. Ideally, in Figure 1, the country labels would not be on top of one another, and the figures would include the correlation coefficients.

5. The abstract and conclusion provide two implications of the manuscript: accurate stereotypes, and opposition to immigration from certain groups being informed by rational beliefs about that group. Presumably, according to some perceptions, irrational reasons for opposition to immigration from a particular country are the race or ethnicity of the immigrants, so there might value in reporting standardized effects for the Western country variable, to see the extent to which immigrants from Western countries are favored, or, for that matter, to include regional controls to assess the extent to which particular regions are favored, although the small sample size and lack of relevant controls might not be conducive enough to this sort of analysis.

6. The final sentence of the manuscript notes the lack of data for characteristics of immigrant groups in the UK, but the percentage white and percentage English speakers controls used in Table 1 were based on the home countries and not on immigrants in the UK, so it's not clear why controls could not be included with data on characteristics from the home countries for, say, mean country IQ or international test scores.
 Reply
#12
Many thanks for the review.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: 1. I think that the manuscript should disclose that, for each of the 23 dataset countries, the YouGov immigration item had "Don't Know" responses for 22% to 29% of observations.


This has been noted in Section 2.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: It might be beneficial to the reader to note that the manuscript's main inferences do not change with particular alternate specifications, whether the aforementioned specifications or other specifications.


This has been noted in Section 2.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: 3. It might be valuable to note in the manuscript the countries coded as Western. The countries furthest below the regression line in Figure 1 are those most plausibly considered Western, but other plausibly Western countries are near or above the line, so it might be valuable to know which countries are coded Western.


This information has been included in a footnote in Section 2.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: 4. Ideally, in Figure 1, the country labels would not be on top of one another, and the figures would include the correlation coefficients.


The correlation coefficients have been included in the text, as well as in the Abstract, so I do not believe there is a strong rationale for also including them in the Figures. I would also claim that the country labels on the graphs are reasonably legible. I would therefore prefer not to change them.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: 5. The abstract and conclusion provide two implications of the manuscript: accurate stereotypes, and opposition to immigration from certain groups being informed by rational beliefs about that group. Presumably, according to some perceptions, irrational reasons for opposition to immigration from a particular country are the race or ethnicity of the immigrants, so there might value in reporting standardized effects for the Western country variable, to see the extent to which immigrants from Western countries are favored, or, for that matter, to include regional controls to assess the extent to which particular regions are favored, although the small sample size and lack of relevant controls might not be conducive enough to this sort of analysis.


Standardised coefficients for control variables are now displayed in the tables. I agree with the reviewer's intuition that, given the small sample size, adding regional controls would offer very little conceptual purchase.

(2016-Oct-18, 08:28:38)ljzigerell Wrote: 6. The final sentence of the manuscript notes the lack of data for characteristics of immigrant groups in the UK, but the percentage white and percentage English speakers controls used in Table 1 were based on the home countries and not on immigrants in the UK, so it's not clear why controls could not be included with data on characteristics from the home countries for, say, mean country IQ or international test scores.


The regression models now also control for national IQ.
 Reply
#13
Great job on the revision. I approve the submission.

Sorry for the delayed response: I did not receive (or at least notice) a message that there was a reply.

L.J
 Reply
#14
This is a very interesting study.
 
There are no page numbers. Add page numbers.
 
Abstract:
I do not understand this sentence: ”Moreover, the correlations are not accounted for by a”
You mean are not modified by considering/controlling for? Are stable?
 
Introduction: ”Indeed” should be replaced by something like ”This is backed by”.
 
In Figure 1 add for both figures in the heading the sources of data and the correlations, e.g.:
Scatterplots of the relationship of net opposition (YouGov poll) with log of immigrant arrest rates (r=.77) and log of immigrant arrest rates for violent crime (r=.77) (both Metropolitan Police).
 
Both r=.77?
 
Add a correlation table for all used variables.
 
Maybe a further good control: percentage of Muslims in each country of origin.
 Reply
#15
Many thanks for the review.

(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: There are no page numbers. Add page numbers.


Page numbers have been added.
 
(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: I do not understand this sentence: ”Moreover, the correlations are not accounted for by a”
You mean are not modified by considering/controlling for? Are stable?


Yes, that is what I mean. I would prefer not to change the phrasing here, as I believe it conveys the results of the regression analyses in a way that is easier for the non-specialist to understand. In my opinion, relatively straightforward language is desirable in the Abstract.

(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: Introduction: ”Indeed” should be replaced by something like ”This is backed by”.


Again, I would prefer not to change the phasing here, as I believe "Indeed" is more appropriate than "This is backed by". The reason being that the Sides and Citrin's (2008) result is not technically an example of stereotype accuracy.
 
(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: In Figure 1 add for both figures in the heading the sources of data and the correlations, e.g.:
Scatterplots of the relationship of net opposition (YouGov poll) with log of immigrant arrest rates (r=.77) and log of immigrant arrest rates for violent crime (r=.77) (both Metropolitan Police).


The title for Figure 1 has been changed accordingly.

(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: Both r=.77?


Yes, it appear so. I guess this isn't that surprising given that the correlation between the two measures of arrest rates is = .95.

(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: Add a correlation table for all used variables.


This has been added in Appendix A.

(2016-Nov-01, 21:21:44)HeinerRindermann Wrote: Maybe a further good control: percentage of Muslims in each country of origin.


Percentage Muslim has now been controlled for in the regression models.
 Reply
#16
Dear Noah,
Thanks.
Fine, but:
Why not add in subordinate clauses some paraphrases of your selected wording - so the readers would easier understand the meaning.
(for: ”Moreover, the correlations are not accounted for by a”, not modified by considering/controlling for? Are stable?; or similar)
(for: ”Indeed”, This is backed by; or similar)
I suggest acceptance.
Best,
Heiner
 Reply
#17
Published.

https://openpsych.net/paper/48
 Reply
#18
Following correspondence with Richard Lynn, I have added a post-publication supplement to the paper's OSF page.
 Reply
#19
It seems that a geographer has recently written a post in which he is heavily critical of this paper's methodology (link: http://www.healthgeomatics.com/award-win...-research/) These criticisms include the ecological design of the study ("it is not only observational, but does not actually measure anything about people, but rather, just aggregations of people"), and its use of a small, non-random sample of 23 countries while excluding others ("a small non-random sample is the holy grail of statistical badness"). Also criticized is Carl's failure to control for more variables that could be confounding, "like the economic wealth / productivity of the country of origin, historical tensions or media portrayals" and his conclusion that public beliefs about immigrants are generally accurate based simply on a study of one aspect of such beliefs (namely their involvement in crime). These concerns, I think, should be taken seriously and I'd like to see what Carl himself says about them.
 Reply
#20
Please find a point-by-point rebuttal below.

Quote:1. Ecological study design problem

The research design is ecological.  This means that the data are not individuals, but aggregates (groups) of individuals.  This is probably the weakest study design in the social sciences; it is not only observational, but does not actually measure anything about people, but rather, just aggregations of people.  One consequence of this is that these research designs tend to over-estimate model fit.  That is, any effects estimated tend to fit more poorly in the real world than they do in the model.  This is because these study designs under-estimate variability.  In this example, had the author used individual data on the perception of immigrants rather than averages to fit his models, he probably would have seen a weaker relationship than he observed.

Researchers interested in stereotype accuracy generally distinguish between consensual stereotypes on the one hand and personal stereotypes on the author (Jussim et al., 2015). Consensual stereotypes correspond to the average beliefs about some group of people, whereas personal stereotypes correspond to a particular individual’s beliefs about some group of people. (Note that this distinction was already mentioned in footnote 1 of my paper.) As Jussim et al. (2015) note, consensual steretypes are “usually assessed by sample means” [emphasis added]. 

Although my study was not concerned with the accuracy of consensual stereotypes per se, it was concerned with a related concept, namely average preferences for or against certain immigrant groups. And just as it is interesting to examine average beliefs about certain groups, it is also interesting to examine average preferences for or against those groups. Note that Ford (2011; published in a ‘mainstream’ journal) also examined average preferences for or against certain immigrant groups. 

Quote:2. Small sample problem

In addition to being a weak study design, the author relies on 23 observations to draw his conclusions.  Statistics can make up for small samples when study designs are strong and variables are measured without systematic error, but the small sample size used in this study is particularly troubling when combined with all the other problems with the study.  Small samples are a multiplier of all other problems.

Of course, it is always better to have more data, but my analysis was limited by what was available at the time. In particular, the 2016 YouGov poll that my study drew upon only asked bout 23 different immigrant nationalities. Hence, n = 23. 

However, note that the raw correlations were significant at the 0.1% level, and many of the associations in the multivariate models were significant at the 5% level or lower. Moreover, most of these associations were in the ‘expected’ direction (e.g., positive effects of percentage white, Western country and percentage English speakers; negative effect of percentage Muslim). These observations indicate that, as a matter of fact, there was no “small sample problem”. 

If there had been a “small sample problem”, the associations would have been only borderline significant, and the ones in the multivariate models would have been as likely to go in the ‘expected’ direction as in the opposite direction.

Quote:3. Bad sample

The researcher did not look at all immigrant data in the UK, but a small non-random sample of 23 countries.  There are a large number of Italian and Portuguese immigrants to the UK, but these data are not included in the study.  If they were included, the results may have looked different.  When the data we use are not exhaustive (complete) and not selected randomly, there is always the possibility that the selection of data used will affect out findings in a systematic way.  This is particularly problematic when the sample is small; a small non-random sample is the holy grail of statistical badness.

Once again, I looked at all data that were available at the time (n = 23 immigrant groups). It is true that the results might have been different if the number of immigrant groups had been larger. But note that in our unpublished study on immigration policy preferences in Denmark (which had a larger n = 32), we again observed a very strong association between level of opposition to immigrant groups from different origin countries and those groups net fiscal contributions in Denmark. 

Furthermore, although the aforementioned 2016 YouGov poll only asked about 23 different immigrant nationalities, it did encompass many of the nationalities with large immigrant populations living in the UK (e.g., Poland, Nigeria, France, Romania, Pakistan; see ONS, 2012). In addition, it included nationalities from nearly all the major world regions: Europe, the Middle East, South Asia, East Asia, Africa, North America, South America, Oceania.  

Quote:4. Missing variable problem

The author uses multiple regression model to control for the ‘confounding’ effect of things like whiteness, English speaking, being from a Western country, and religion on his observation that crime rates influence perception of immigrants.  He did not control for other potential confounding effects, however–like the economic wealth / productivity of the country of origin, historical tensions or media portrayals.  I added per capital GDP to his data set an observed that the log of per capita GDP correlates more strongly with perception of immigrants than the the log of crime rate.  It’s hard to know what variables to include in an analysis like this, but it matters, as the inclusion and exclusion of variables can change how data are interpreted.

It is not possible to anticipate every alternative model specifications that someone may prefer. The data were published online precisely in order to allow other researchers to run their own analyses. Moreover, there is disagreement about whether to control for a variable like GDP per capita because it is arguably endogenous (i.e., more of an ‘outcome’ measure than an ‘input’ measure). 

In addition, the multivariate models are arguably less interesting than the raw associations between crime rates and net opposition. As noted in the paper, YouGov asked the British public to say how important each of 14 characteristics should be when considering whether or not an economic migrant should be allowed into the UK. The two most important were ‘criminal record (major/violent)’ and ‘criminal record (minor/non-violent)’. Thus, even if respondents were using GDP per capita as a proxy, the stereotypes underling their immigration policy preferences can still be seen as ‘rational’. 

Quote:5. Non sequitur

Carl draws the conclusion that ‘public beliefs about immigrants are more accurate than often assumed’, but the bulk of his analysis does not meaningfully address this claim.  Carl has not defined what ‘accurate’ is, but no reasonable definition can be boiled down only to crime rate–that is, the negative contributions of immigrants.  If public opinions were really ‘accurate’, their perceptions would also correlate with the positive contributions of immigrants, and in fact there would be strong correlation between net utility of immigrants and the perception of immigrants.  But Carl focuses only on one possibly useful measure, and ignores the rest.  As such, even if the technical difficulties above were overlooked, his conclusion is a misdirection since he’s not really measuring accuracy of public opinion.

As mentioned above, accuracy was defined in exactly the same way as it is defined in the literature on stereotype accuracy, namely in terms of the correlation (or ‘correspondence’) between average beliefs/preferences and average criterion values (see Jussim et al., 2015). 

Given that Britons say an immigrant’s criminal history should be one of the most important characteristics when decided whether he should be admitted to the country, and there is a strong correlation between crime rates and net opposition, it seems reasonable to claim that their immigration policy preferences are informed––at least to some extent––by rational beliefs. 

Note that the paper acknowledged (in the Abstract, Introduction and Conclusion) that the British public systematically overestimates the percentage of immigrants in the population. It simply concluded that “public beliefs about immigrants are more accurate than is often assumed” [emphasis added]. Nowhere did it state that the public are entirely accurate about all aspects of immigration. 

References

Ford, R. (2011). Acceptable and unacceptable immigrants: How opposition to immigration in Britain is affected by migrants' region of origin. Journal of Ethnic and Migration Studies, 37, 1017–1037.

Jussim, L., Crawford, J.T., Rubinstein, R.S. (2015). Stereotype (in)accuracy in perceptions of groups and individuals. Current Directions in Psychological Science, 24, 490–497.

ONS. (2012). Population by country of birth and nationality. Office for National Statistics, published online.
 Reply
 
Forum Jump:

Users browsing this thread: 1 Guest(s)