Suppose an author submits a paper. Reviewer A approves it as is. Reviewer B then criticizes some aspect of it and requires a new analysis. The author then submits a revision with the new analysis. Reviewer B approves.
Right now, how many approvals does the submission have? 1 or 2? It depends whether an approval for an earlier edition still counts, or whether approval of any edition counts as an approval for the final edition.
Perhaps the simplest policy is to have approval for any edition as counting for the latest edition. This however makes it possible for authors to introduce new material into later editions and smuggle it around the reviewing process.
So, what is the solution to this problem? Some options:
1. Let it be as is and hope that authors do not cheat.
2. Require new approval for any new revision.
3. Require new approval for any major revision.
4. Upon 3 approvals, the editor asks reviewers whether they approve the latest version (if relevant).
Personally, I favor (3). (1) is the simplest policy and the least work for reviewers. (2) is quite a lot of work for reviewers.
If (3) is chosen, then we need some policy regarding what counts as a 'major revision'. In my opinion, a major revision is any revision that includes changes that aren't just cosmetic: language, text on figures and the like. If there is a new section with new analyses, it is a major revision. If there is a new dataset introduced, it is a major revision.
Thoughts?
Back to [Archive] Meta
When someone is testing the null vs alternative hypothesis, there are only answers : yes or no. It's a dichotomy. There is no gradation. And so, it does not make a lot of sense. You have the same problem here. When you say big and small, you have to think about what happens when the revision is not big, not small, but in between. My suggestion is that the reviewer should still read it if it's more than just small changes.
At the same time, what's considered important for the author may not be seen as very important for some reviewers. They can say that it's irrelevant, does not add anything substantial to the initial text, etc., and thus may not feel the need to read it again.
Personally, I may not be motivated in reading the whole piece again. If the author does not wish to tell me exactly what has changed (e.g., pages, sections, etc.), I may not bother to read it again or I may be slowing. The best thing, in my opinion, is that the author copy-paste the added text in the forum, or put color or underline or else on the important and newly added text.
At the same time, what's considered important for the author may not be seen as very important for some reviewers. They can say that it's irrelevant, does not add anything substantial to the initial text, etc., and thus may not feel the need to read it again.
Personally, I may not be motivated in reading the whole piece again. If the author does not wish to tell me exactly what has changed (e.g., pages, sections, etc.), I may not bother to read it again or I may be slowing. The best thing, in my opinion, is that the author copy-paste the added text in the forum, or put color or underline or else on the important and newly added text.
Suppose an author submits a paper. Reviewer A approves it as is. Reviewer B then criticizes some aspect of it and requires a new analysis. The author then submits a revision with the new analysis. Reviewer B approves.
Right now, how many approvals does the submission have? 1 or 2? It depends whether an approval for an earlier edition still counts, or whether approval of any edition counts as an approval for the final edition.
Perhaps the simplest policy is to have approval for any edition as counting for the latest edition. This however makes it possible for authors to introduce new material into later editions and smuggle it around the reviewing process.
So, what is the solution to this problem? Some options:
1. Let it be as is and hope that authors do not cheat.
2. Require new approval for any new revision.
3. Require new approval for any major revision.
Personally, I favor (3). (1) is the simplest policy and the least work for reviewers. (2) is quite a lot of work for reviewers.
If (3) is chosen, then we need some policy regarding what counts as a 'major revision'. In my opinion, a major revision is any revision that includes changes that aren't just cosmetic: language, text on figures and the like. If there is a new section with new analyses, it is a major revision. If there is a new dataset introduced, it is a major revision.
Thoughts?
When you reach the number of approvals (3 for OBG), if there have been revisions since the approval, the editor simply asks the reviewers if they approve the final version for publication. Easy peasy :)
So, what is the solution to this problem? Some options:
1. Let it be as is and hope that authors do not cheat.
2. Require new approval for any new revision.
3. Require new approval for any major revision.
Personally, I favor (3). (1) is the simplest policy and the least work for reviewers. (2) is quite a lot of work for reviewers.
I would say (1) in the case of outside reviewers. Otherwise what Piffer said.
I don't see any graduation with this. Cosmetic changes → not major revision. Can someone (i.e. Hu?) mention an example of a revision that is not clearly minor or major by my criteria above?
Of course, it is easier if authors mark the changed content. We do this in most cases. For latex, it is not very easy to do, but one can note in the post on the forum what the changes are, so reviewers don't waste time re-reading an entire paper.
In case of Piffer's suggestion (4), this will introduce a slow-down since the editor has to wait for reviewers to respond. In some cases, this can take a while.
I'm fine with (1) or (3) or (4).
Of course, it is easier if authors mark the changed content. We do this in most cases. For latex, it is not very easy to do, but one can note in the post on the forum what the changes are, so reviewers don't waste time re-reading an entire paper.
In case of Piffer's suggestion (4), this will introduce a slow-down since the editor has to wait for reviewers to respond. In some cases, this can take a while.
I'm fine with (1) or (3) or (4).
I don't see any graduation with this. Cosmetic changes → not major revision. Can someone (i.e. Hu?) mention an example of a revision that is not clearly minor or major by my criteria above?
Of course, it is easier if authors mark the changed content. We do this in most cases. For latex, it is not very easy to do, but one can note in the post on the forum what the changes are, so reviewers don't waste time re-reading an entire paper.
In case of Piffer's suggestion (4), this will introduce a slow-down since the editor has to wait for reviewers to respond. In some cases, this can take a while.
I'm fine with (1) or (3) or (4).
(1) is prone to cheating. I think (3) is cumbersome as it requires a subjective judgment on what a minor or major revision is and they require reviewers to track the paper and re-read it several times until final publication. I think (4) is better as it is not based on subjective definition of minor/major and does not require the reviewers to re-read the paper several times until final approval.
Please give an unclear example regarding my criteria above.
(4) introduces as much slowdown as (2-3) since it is in effect the same just with the addition of the editor, so I don't see why you say it doesn't (???).
(4) introduces as much slowdown as (2-3) since it is in effect the same just with the addition of the editor, so I don't see why you say it doesn't (???).
Please give an unclear example regarding my criteria above.
(4) introduces as much slowdown as (2-3) since it is in effect the same just with the addition of the editor, so I don't see why you say it doesn't (???).
I say it doesn't because (4) requires only 1 approval before publication, therefore packing together all previous revisions into one. 2-3 splits the revisions, requiring reviewers to read again the paper at any major or minor revision. The problem of defining what is a major or minor revision is also huge and would make the review process open to attacks that it's not fair, since an author could argue that revision is minor hence not requiring more approvals...it's a really bad method in my opinion!
Please give an unclear example regarding my criteria above.
(4) introduces as much slowdown as (2-3) since it is in effect the same just with the addition of the editor, so I don't see why you say it doesn't (???).
I say it doesn't because (4) requires only 1 approval before publication, therefore packing together all previous revisions into one. 2-3 splits the revisions, requiring reviewers to read again the paper at any major or minor revision. The problem of defining what is a major or minor revision is also huge and would make the review process open to attacks that it's not fair, since an author could argue that revision is minor hence not requiring more approvals...it's a really bad method in my opinion!
You have misunderstood. There is no requirement that reviewers must review every major revision. They must however approve the last major revision for it to count for the final revision in proposal (3) above.
You state again problems in defining major/minor. I did so explicitly in my first post. I state again, please give an example of a situation where it is unclear whether it is a major or minor revision. And then please state how common you think such a grey zone revision is? I think not very common. One idea for solving this is studying previous submissions and their revisions. How strong is the inter-rater agreement on classifying revisions as minor/major?
---
In practice, solution (4) will require more than 1 approval for reviewers. The only way for a reviewer to approve only once is for 3 other reviewers to approve a submission, the editor sends the last revision to the 4th reviewer, who then approves it and only once. This is not very realistic.
Please give an unclear example regarding my criteria above.
(4) introduces as much slowdown as (2-3) since it is in effect the same just with the addition of the editor, so I don't see why you say it doesn't (???).
I say it doesn't because (4) requires only 1 approval before publication, therefore packing together all previous revisions into one. 2-3 splits the revisions, requiring reviewers to read again the paper at any major or minor revision. The problem of defining what is a major or minor revision is also huge and would make the review process open to attacks that it's not fair, since an author could argue that revision is minor hence not requiring more approvals...it's a really bad method in my opinion!
You have misunderstood. There is no requirement that reviewers must review every major revision. They must however approve the last major revision for it to count for the final revision in proposal (3) above.
You state again problems in defining major/minor. I did so explicitly in my first post. I state again, please give an example of a situation where it is unclear whether it is a major or minor revision. And then please state how common you think such a grey zone revision is? I think not very common. One idea for solving this is studying previous submissions and their revisions. How strong is the inter-rater agreement on classifying revisions as minor/major?
---
In practice, solution (4) will require more than 1 approval for reviewers. The only way for a reviewer to approve only once is for 3 other reviewers to approve a submission, the editor sends the last revision to the 4th reviewer, who then approves it and only once. This is not very realistic.
I see now that you've modified your initial post to include proposal (4), which I ideated. You should state this explicitly otherwise people will think that you had initially included my option, but you introduced it later only after I proposed it! It's good practice to specify which edits you make to your previous posts otherwise none is accountable for what they've said and it's easier to cheat.
I still do not like proposals 2 and 3. There should be no room for discussion on whether it's a major or minor revision. This will likely turn OP into a court and time will be wasted to hear cases of appeals for it being a minor/major revision.Other time will be wasted for the dubious study of inter-rater agreement which will solve nothing. I think proposal 4 is easier and clearer.
© Copyright 2024 OpenPsych - All Rights Reserved