[ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab

2014Nov02, 00:49:54,




[ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
Title:
Immigrant GPA in Danish primary school is predictable from countrylevel variables Authors: Emil O. W. Kirkegaard Abstract: I found immigrant GPA by mother's country of birth in a report from 2011. Correlation analyses shows that it is highly predictable from countrylevel variables: National IQ (.64), age heaping 1900 (.53), Islam prevalence (.75), average years of schooling (.74) and general socioeconomic factor (S) in both Denmark (.87) and internationally (.68). Files. 

2014Nov02, 03:13:04,




RE: [ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
Some errors below :
Quote:Correlation analyses shows that it is highly predictable from countrylevel variables: In general, area is a vague term to me. Country would have sufficed. Quote:which on page 13 lists the grade point average (GPA) of children by their mother's birth country/area. Finally, when you mention Malloy, you should probably cite a reference. Fuerst used L&V with Malloy's correction to L&V in one of his first papers for ODP. You can cite it. When you do correlation, I suggest you talk some more about normality on your data. You rarely do that. You can easily make histogram, stemandleaf, PP and QQ plot on R. And remember that outliers can be very influential when sample size is small. 

2014Nov02, 05:37:41,
(This post was last modified: 2014Nov02, 05:44:37 by Emil.)




Normality
I posted a scatterplot of GPA and IQ on twitter.
Code: > shapiro.test(unlist(GPA)) Shapiro test fails to reject the GPA as nonnormal. But histogram shows that it is not very normal looking.  The reason to use "area" or "region" is that two of the origin 'countries' are not countries, they are groups based on the remaining countries.  I have fixed the grammar error. I will wait with posting the updated version until you have clarified what you want me to write about the normality. 

2014Nov02, 19:21:44,




RE: [ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
You know that I always hated significance test. SW is useless because when N is large, the pvalue will always be lower than 0.05, let alone the problem of arbitrary cut off values for significance. I don't understand why you keep using it. I will never approve a paper that uses SW to examine normality, instead of histogram, PP and QQ plots. As I said, look at histogram, PP plot and QQ plot. That's all you need. If you don't want to display all the graphs but only one, I think you should probably select histogram.
By the way, If my memory is correct, the spatial transferability theory states that immigrant IQ will stay the same, and that the country of origin will predict test performance. Your paper says the result is consistent with ST theory, but it's just a correlational analysis, so it's only about the second prediction of the theory. 

2014Nov02, 19:32:02,
(This post was last modified: 2014Nov02, 21:00:42 by Emil.)




RE: [ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
(2014Nov02, 19:21:44)Meng Hu Wrote: You know that I always hated significance test. SW is useless because when N is large, the pvalue will always be lower than 0.05, let alone the problem of arbitrary cut off values for significance. It will only be so if the data is nonnormal. Of course, if N is very large (e.g. 10k), then even a very slight deviation from normality will cause p to be lower than .05 or .01. Instead of looking at the p value, you can look at the W value, which is a measure of normality. It is only .93 for these data. Usually, it is close to .99 when the data looks normal. Try: Code: x = rnorm(5000) You see that W is very close to 1, and p is high even though N=5000 (the function in R is limited to N=5k for some odd reason). Quote:I don't understand why you keep using it. I will never approve a paper that uses SW to examine normality, instead of histogram, PP and QQ plots. As I said, look at histogram, PP plot and QQ plot. That's all you need. If you don't want to display all the graphs but only one, I think you should probably select histogram. Because it gives a numerical estimate of the normality of the data. Eyeballing cannot do that. So e.g. if one wants to compare 100 samples automatically for which one is the most normal, one will need a test. Quote:By the way, If my memory is correct, the spatial transferability theory states that immigrant IQ will stay the same, and that the country of origin will predict test performance. Your paper says the result is consistent with ST theory, but it's just a correlational analysis, so it's only about the second prediction of the theory. If the IQs changed, but stayed in the same relative order, then correlation analysis will not detect it, that's right. ST hypothesis says they will generally stay the same, which also implies the order will stay generally the same, and for this reason the usual correlates of IQ will be found. 

2014Nov02, 20:15:44,
(This post was last modified: 2014Nov02, 20:30:26 by Meng Hu.)




RE: [ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
(2014Nov02, 19:32:02)Emil Wrote: It will only be so if the data is nonnormal. Of course not. I saw it many times my variables normally distributed when looking at histogram and PP plot, and yet SW says otherwise. Like i said: it's pvalue. Your argument is similar as saying that if p is significant, it can only be so if the effect size is large. Yet that is not true, and a lot of examples show that the effect size can be close to zero but p is significant. One example you have is from Pekkala Kerr et al. (2013) "School tracking and development of cognitive skills". They say that schooling reform shows no transfer effect, just because some tests are significantly improved, and others not. But when you calculate effect sizes, which they don't, the d gaps are between 0.00 and 0.03 (or 0.04). In my opinion, it's not different than to say the effect size is zero for each test. It's dangerous to use significance tests. I always said it, and I will always repeat it. And even if you trust W value, I don't trust cut off values. What is the .99 really means, given the operation to get W value, which is given here ? So, what does that mean when you have 0.95 or 0.96 instead of 0.99 ? How can you judge that ? If you think eyeballing is not nice, I will say cutoff values are not better. (2014Nov02, 19:32:02)Emil Wrote: If the IQs changed, but stayed in the same relatively order, then correlation analysis will not detect it, that's right. ST hypothesis says they will generally stay the same, which also implies the order will stay generally the same, and for this reason the usual correlates of IQ will be found. If IQ changes and rank order are not the same thing, and you see that rank order remains the same, you cannot conclude that IQ has not changed. This should be made clear.  I don't mind if you use both SW and histograms/plots. But make sure you don't rely on p values. 

2014Nov02, 22:39:48,




Shapiro test
Quote:Of course not. I saw it many times my variables normally distributed when looking at histogram and PP plot, and yet SW says otherwise. Like i said: it's pvalue. Your argument is similar as saying that if p is significant, it can only be so if the effect size is large. Yet that is not true, and a lot of examples show that the effect size can be close to zero but p is significant. One example you have is from Pekkala Kerr et al. (2013) "School tracking and development of cognitive skills". They say that schooling reform shows no transfer effect, just because some tests are significantly improved, and others not. But when you calculate effect sizes, which they don't, the d gaps are between 0.00 and 0.03 (or 0.04). In my opinion, it's not different than to say the effect size is zero for each test. The p value will only go down if the data are nonnormal. With large N's, even a slight deviation from normality will cause a low p value. However, merely using a large N is not sufficient, as I just showed with code above (N=5000). My argument is not one of the fallacies which is you discuss, where authors reason from "significant p value" to "realworld significance" or from "non significant pvalue" to "no effect". Quote:It's dangerous to use significance tests. I always said it, and I will always repeat it. W = 1 perfect normality. Any value below it indicates nonnormality. We can simulate datasets, so one can get a feel for the different values of W and how the data looks like. I did a little writeup here: http://emilkirkegaard.dk/en/?p=4452 As you can see from my simulations, W values do not change much as long as the distribution is reasonably normal. It was hard to get it below .95. Of course, when examining small datasets, such as the GPA one (N=19), sampling error can result in low W values even tho the population GPA distribution is normal (or normalish). In the case of GPA here, however, it is not very normal. I tried logtransforming it, but it is slightly less normal after that (Ws .9251 before, .9124 after). What do you want me to do with it? 

2014Nov03, 04:17:20,
(This post was last modified: 2014Nov03, 04:18:03 by Emil.)




Spearman
One idea is to use Spearman's rho instead.
The results however, are pretty much the same: Code: Variable rho with GPA P value N 

2014Nov03, 19:29:29,
(This post was last modified: 2014Nov03, 19:32:12 by Meng Hu.)




RE: [ODP] Immigrant GPA in Danish primary school is predictable from countrylevel variab
(2014Nov02, 22:39:48)Emil Wrote: The p value will only go down if the data are nonnormal. First, you say that p goes down only if the variable is not normally distributed. Then, you say it can go down if N is large (and you show it in your blog post). So, it's not "only if" the variabe is not normal. That's the problem I always pointed out. (2014Nov02, 22:39:48)Emil Wrote: In the case of GPA here, however, it is not very normal. I tried logtransforming it, but it is slightly less normal after that (Ws .9251 before, .9124 after). What do you want me to do with it? Sometimes, transformation does not work because the data follows a specific distribution (e.g., poisson). In such a case, use poisson regression. Sometimes, the data is truncated or censored. Then use truncated or tobit regression. You have used Spearman's rho, and it is a valid test here, given your histogram. But sometimes, even Spearman's test is not appropriate. Like I said, when you have censored, truncated or poisson distribution. Just because the data is not normal does not mean Spearman's test is the most appropriate solution. But of course, you can't see that with your SW test. That's why I prefer histogram. SW test cannot even tell anything about skewness and kurtosis. But histogram can do that. When you look only at SW test, you will come to the conclusion that the data is not normal, and that Pearson's test is not appropriate. Correct. But when it comes to choose the best method, SW fails badly. It cannot show you which method to apply. For your article anyway, I will only ask that you note the data is not normally distributed, as indicated by histogram (and SW test if you really want it) and then show that Spearman's correlation produces the same result. And I will approve. I don't think the analysis needs to be carried out even further. And I don't even have ideas. Pearson/Spearman is just sufficient. 

2014Nov03, 22:59:31,




Nonnormal
Quote: First, you say that p goes down only if the variable is not normally distributed. Then, you say it can go down if N is large (and you show it in your blog post). So, it's not "only if" the variabe is not normal. That's the problem I always pointed out. You have not read carefully enough. I said that it goes down when both of two conditions are met: 1) there is nonnormality in the data, 2) N is large. Condition (2) is not sufficient in itself, as I clearly showed by simulation. Quote:But of course, you can't see that with your SW test. That's why I prefer histogram. SW test cannot even tell anything about skewness and kurtosis. But histogram can do that. When you look only at SW test, you will come to the conclusion that the data is not normal, and that Pearson's test is not appropriate. Correct. But when it comes to choose the best method, SW fails badly. It cannot show you which method to apply. SW is not a tests of whether Spearman's method is the right. It is a test of normality. You wouldn't criticize IQ tests for not measuring height either. Quote:SW test cannot even tell anything about skewness and kurtosis. Same as above. Just use describe() for that. Skew is .3 and kurtosis is 1.39. Quote: For your article anyway, I will only ask that you note the data is not normally distributed, as indicated by histogram (and SW test if you really want it) and then show that Spearman's correlation produces the same result. And I will approve. I don't think the analysis needs to be carried out even further. And I don't even have ideas. Pearson/Spearman is just sufficient. I will update it with a histogram + Spearman's results later today. :) 

« Next Oldest  Next Newest »

Users browsing this thread: 2 Guest(s)