Meng Hu has stated that upping the reviewer count to 4 was a mistake, and Chuck seemed to agree with him. More required reviewers presumably increases the review quality, but it also increases the review time which.
We can change it back if there is general agreement about it. I'm not particularly vested in the number 3 or 4.
Back to [Archive] Meta
To be precise, 4 is not a problem per se. It's just too early, considering that you don't have enough reviewers now, and even if you have, most of them may be too busy. We have an illustration today; the review process is slow.
Before deciding to increase the count, you must ensure that the time for reviewing a given article is stable. For now, it's somewhat unstable. By stability, I meant something like "For an article of a given length, is there a lot of variation in the number of days required to get the necessary approvals ?" (you can create such a variable by, e.g., using a ratio of text length (i.e., number of pages) to time of reviews (i.e., number of days before it gets published)).
At the start of OP, everyone was "available" and the review process was fast, very fast. But now, it's very slow. When you reach a good stability, and thus a better predictability, you could have more confidence about whether raising the count will (or will not) make the reviews too difficult. If it does not increase the burden's share of everyone here, you increase the count. To know whether you have reached a good stability, you need to wait a little bit more, until the number of days a given article gets approved is predictable. One way to look at it is through standard deviation of this "number_of_days" variable, after controlling for text length. This can be done easily by way of regression analyses.
But one also needs to take into account the rate at which new papers are submitted. There might be some periods in which there are too many papers submitted at the same time; these "accidents" however should not be considered even further. So, when a regression analysis is done, these studies published in very busy days should be removed from the data under study.
To answer the question that is asked here, I would say it's preferable to try to get more reviewers now. If you can get more reviewers very soon, let's wait and see if the time of review has changed or not. But if really someone here wants to revert back to a count of 3, I do not disagree with that (and I can be in favor of it), because I think the reviewers here are quite knowledgeable and leave generally good comments.
Before deciding to increase the count, you must ensure that the time for reviewing a given article is stable. For now, it's somewhat unstable. By stability, I meant something like "For an article of a given length, is there a lot of variation in the number of days required to get the necessary approvals ?" (you can create such a variable by, e.g., using a ratio of text length (i.e., number of pages) to time of reviews (i.e., number of days before it gets published)).
At the start of OP, everyone was "available" and the review process was fast, very fast. But now, it's very slow. When you reach a good stability, and thus a better predictability, you could have more confidence about whether raising the count will (or will not) make the reviews too difficult. If it does not increase the burden's share of everyone here, you increase the count. To know whether you have reached a good stability, you need to wait a little bit more, until the number of days a given article gets approved is predictable. One way to look at it is through standard deviation of this "number_of_days" variable, after controlling for text length. This can be done easily by way of regression analyses.
But one also needs to take into account the rate at which new papers are submitted. There might be some periods in which there are too many papers submitted at the same time; these "accidents" however should not be considered even further. So, when a regression analysis is done, these studies published in very busy days should be removed from the data under study.
To answer the question that is asked here, I would say it's preferable to try to get more reviewers now. If you can get more reviewers very soon, let's wait and see if the time of review has changed or not. But if really someone here wants to revert back to a count of 3, I do not disagree with that (and I can be in favor of it), because I think the reviewers here are quite knowledgeable and leave generally good comments.
The review for Lynn and Bakhiet's papers is slow because the authors are very slow to respond, and only do so over email.
It is pretty hard to keep review times stable. They are not stable for Elsevier journals either. Try looking up the stats I collected. https://docs.google.com/spreadsheets/d/1x0rdMMjGIvY0iL4qJv3_KhyYiRWZXSWYaUbQWNmJ5co/edit#gid=395003811
SD is 133 and 68 for those two samples. 33 and 11 for OP journals. This value will increase when the current papers in review finally get published. Review has been very slow over the holiday and winter season, also due to exams on my part.
I have 3 papers in review currently which would more easily be published, so I rather not take part in a vote of the review process (potential conflict of interest).
When Rindermann gets approved, I can send the reviewers an email and ask them to state their opinion in this thread. Then we will go with the majority.
It is pretty hard to keep review times stable. They are not stable for Elsevier journals either. Try looking up the stats I collected. https://docs.google.com/spreadsheets/d/1x0rdMMjGIvY0iL4qJv3_KhyYiRWZXSWYaUbQWNmJ5co/edit#gid=395003811
SD is 133 and 68 for those two samples. 33 and 11 for OP journals. This value will increase when the current papers in review finally get published. Review has been very slow over the holiday and winter season, also due to exams on my part.
I have 3 papers in review currently which would more easily be published, so I rather not take part in a vote of the review process (potential conflict of interest).
When Rindermann gets approved, I can send the reviewers an email and ask them to state their opinion in this thread. Then we will go with the majority.
It's obvious I'm not expecting something perfectly stable. I think someone should look at the SD in "number_of_days" variable (after controlling for "number_of_pages" variable) and that the variable of "days" (the effect of number_of_pages being partialled out) should be plotted against the "date" variable. Given this plot, if you see that the time required to get through the approvals diminishes over time, and the SD being not high, you can probably think about raising the count.
© Copyright 2024 OpenPsych - All Rights Reserved