I invite users to remember that the privilege of posting direct, unfiltered comments is very unusual in academic journals. They are usually screened by an editor who has full executive power and can decide whether to publish any complaints or not.

Also, apparently it needs to be clarified that the commitment to openness strictly applies to journal reviewers (registered and invited); thus the journal has "open peer review", not review that is necessarily open to anyone who shows up. In the spirit of openness, we allow comments from non-reviewers so long as they are on topic, are not disruptive and are constructive. And we allow non-reviewers to apply to be reviewers so long as they have some background in the relevant fields.

I invite users to remember that the privilege of posting direct, unfiltered comments is very unusual in academic journals. They are usually screened by an editor who has full executive power and can decide whether to publish any complaints or not.

Also, apparently it needs to be clarified that the commitment to openness strictly applies to journal reviewers (registered and invited); thus the journal has "open peer review", not review that is necessarily open to anyone who shows up. In the spirit of openness, we allow comments from non-reviewers so long as they are on topic, are not disruptive and are constructive. And we allow non-reviewers to apply to be reviewers so long as they have some background in the relevant fields.

Yes, reviewers need to be approved by the review team in order to review a paper. Potentially disruptive comments from outsiders are best reserved to other threads, such as this and this freedom we give of posting without editorial screening is not allowed in other journals. For example, PLOS ONE (which is the most "liberal"), allows users to post comments only AFTER articles have been published (http://journals.plos.org/plosone/s/comments). And then again, PLOS ONE explicitly FORBIDS users to freely make public allegations of misconduct without first raising the issue with the editors:
"Questions about experimental data are appropriate, but need to be phrased in a way that does not imply any misconduct on the part of the authors. If a reader is concerned about potential misconduct, such concerns must always be raised with senior editorial staff at PLOS. PLOS is the final arbiter of the suitability of content for inclusion in PLOS web sites."

Note: non-scientific discussion of the dataset is moved to this thread. Peer review threads are for just that, actual scientific peer review. Yes, that does mean that your discussion posts about that topic go into that thread.

It appears that "non-scientific" thread that is providing peer-review of the research methodology has been deleted.

While the link you have in "this thread" above doesn't go anywhere, I do now see the "ethical discussions" thread. Was that temporarily deleted?

More to the point, why do you not consider peer-review of the methodology used in this paper appropriate for this "open" peer-review process and forum?

Go and try that on other respectable and putatively open journals. Elsevier will not allow users to post any comments whatsoever. The "open" PLOS ONE allows users to post comments AFTER articles have been peer reviewed and published but even then, allegations of scientific misconduct cannot be posted without consulting with the editors first. So this discussion would not be tolerated by other journals. See:

"Questions about experimental data are appropriate, but need to be phrased in a way that does not imply any misconduct on the part of the authors. If a reader is concerned about potential misconduct, such concerns must always be raised with senior editorial staff at PLOS."

Since this thread has died out, let me revive it.

This thread started when I stated my personal opinion and asked a lot of questions concerning the ethics and the legality of the data collection. These comments might have been construed as an indictment of the authors and thus not allowed on PLOS, but this is not PLOS, and I have not seen anywhere here a similar policy (let me know if I missed it).

MichaelZimmer joined me in asking similar questions, although his were lost in the heavy-handed moderation done by the Editor-in-Chief/Lead Author/Forum Administrator.

User Chuck, who I surmise to be a potential reviewer, must have realised there was some validity with the concerns. On three separate occasions, he asked a bunch of questions that he felt would help clear up things. In the latest, he said:

Personal data means identifiable data. So the downloading and organizing of the data with the identifiers may have indeed been a violation. (Point 2.) Can someone clarify the matter?

I stepped in, and outlined and documented my theory of which laws or regulations the authors might actually have broken.

As for Duxide's suggestion:

As co-founder of this journal and editor of a sub-journal (OBG), I can act as moderator of this thread and the OKC paper thread if both parties (Emil) and pdehaye agree. I am not a lawyer or a legal expert on ethics so I cannot provide guidance but merely the role of neutral moderator.

• I am appreciative of the offer;
• I observe that Emil has not approved of it;
• I also observe that the administrator of this forum has disabled registration of new users on the site.

The last point contradicts the last few posts in this thread. Indeed, commenting is not open to all in this forum at this moment. I would think it also undermines the offer of Duxide, as it seems to be his goal (and is certainly mine) to encourage here a civilised discussion open to all.

This thread started when I stated my personal opinion and asked a lot of questions concerning the ethics and the legality of the data collection. These comments might have been construed as an indictment of the authors and thus not allowed on PLOS, but this is not PLOS, and I have not seen anywhere here a similar policy (let me know if I missed it).

We never had to deal with similar issues before. We didn't adopt the more conservative PLOS approach of denying users the right to publicly discuss ethical matters, but we reached a compromise by allowing you to flag the publication using this thread but removing it from the peer review thread.

• I am appreciative of the offer;
• I observe that Emil has not approved of it;
• I also observe that the administrator of this forum has disabled registration of new users on the site.

The last point contradicts the last few posts in this thread. Indeed, commenting is not open to all in this forum at this moment. I would think it also undermines the offer of Duxide, as it seems to be his goal (and is certainly mine) to encourage here a civilised discussion open to all.

I had hoped that a civilized discussion could take place. However, this requires consent of both parties. At this point, I'd say we should wait for OKC legal office's reaction to your complaint letter. If they dismiss it or do not consider this a strong violation of their rules, then we can consider this issue resolved and close this ticket. If they take action, we'll have to keep this ticket open.

We never had to deal with similar issues before. We didn't adopt the more conservative PLOS approach of denying users the right to publicly discuss ethical matters, but we reached a compromise by allowing you to flag the publication using this thread but removing it from the peer review thread.

That's not entirely true. There are at least two instances where it has been clear on this forum that Kirkegaard's actions were quite iffy. Once his ethics were directly questioned, and his answer was, in my judgement, unsatisfactory. Unfortunately no one in the forum acted on it.

At this point, I'd say we should wait for OKC legal office's reaction to your complaint letter. If they dismiss it or do not consider this a strong violation of their rules, then we can consider this issue resolved and close this ticket. If they take action, we'll have to keep this ticket open.

The ethical concerns exist independently of the legal concerns. I suggest you read up, for instance, on the Facebook Emotion Study. There the legal questions are much more favorable to the researchers (although some law professors still think some of their actions illegal in local laws, such as Maryland). Nevertheless, there was sufficient ethical concerns that this led the editorial board of PNAS to issue a formal "Editorial expression of concern" after the article's publication. It is even conceivable in my mind that some academic disciplines would have no trouble finding clearly illegal actions entirely ethical (reverse engineering internet platforms, for instance, is often against Terms of Service and is more and more likely to be a needed effort in science; note that the OkCupid dataset is not such an effort).

Let's now focus on the legal situation.

Since my complaint is against OkCupid, not Kirkegaard, OkCupid does not have the option to dismiss it or not consider it a strong violation of their rules. Of course, they could ignore my request for pre-arbitration dispute resolution, but then it would be up to a judge to decide on whether there was harm, negligence by OkCupid, or harm resulting from negligence. That judge would not be empowered to decide if Kirkegaard had violated Terms of Service, or at least it would not be his/her focus (and Kirkegaard would not be able to defend themselves there). On the other hand, OKCupid obviously has the option of turning against Kirkegaard and co-authors directly, or even U.S.-based co-conspirators such as Jayman and HBDchick (I am under the impression many of you would know them). They sort of already have started that process through DMCA takedowns on the data. We may never hear the result of this publicly however, and the editors might need to ask the authors directly for information in this direction.

Even then, the legal situation is not limited to the U.S. The Danish data protection authority might also investigate, and in fact they are considering it. They might need additional encouragement, so I wrote to them a letter (just mailed today) with hints that it might be wise to enlarge their investigation to Kirkegaard and co-authors' actions in Danish schools (i.e. previous papers which appeared here). I guess I will hear a response to this, but not editors to this journal unless they ask Kikegaard and co-authors.

While I applaud the editorial team's decision to proceed carefully on the matter, I do also think that it should proactively seek more information from the author as part of the open peer review process, and that these actions should be informed by ongoing legal procedures but also be independent. In any case, they are definitely part of the open peer review process.

The ethical concerns exist independently of the legal concerns. I suggest you read up, for instance, on the Facebook Emotion Study. There the legal questions are much more favorable to the researchers (although some law professors still think some of their actions illegal in local laws, such as Maryland). Nevertheless, there was sufficient ethical concerns that this led the editorial board of PNAS to issue a formal "Editorial expression of concern" after the article's publication.

The distinction between legal and ethical goes back at least to Socrates and then Kant, in modern times. But as history of philosophy shows this distinction, so history makes us aware that ethics has changed throughout time and what is ethical in one place or in a historical period is not necessarily ethical in a different place or historical period.
Whilst I understand that the authors fall under the Danish jurisdiction (and to the extent that they may or may not have violated OKC rules, under US law), it is difficult to determine which ethics code they should abide to.
They are not researchers employed by the university, hence in my view they do not submit to the regulations of their university's ethics committee. However, it's true that they published declaring their university as affiliation. I personally consider this a mistake but I think this can be solved by removing such affiliation from future versions of the paper.
Regarding the Facebook study, I consider PNAS editor's reaction very mild. They did not remove the paper (as you seem to request here) nor resign from the editorial team. Instead, they wrote a very balanced letter of concern, which amounts to a non-decision.

Let's now focus on the legal situation.

Since my complaint is against OkCupid, not Kirkegaard, OkCupid does not have the option to dismiss it or not consider it a strong violation of their rules. Of course, they could ignore my request for pre-arbitration dispute resolution, but then it would be up to a judge to decide on whether there was harm, negligence by OkCupid, or harm resulting from negligence. That judge would not be empowered to decide if Kirkegaard had violated Terms of Service, or at least it would not be his/her focus (and Kirkegaard would not be able to defend themselves there). On the other hand, OKCupid obviously has the option of turning against Kirkegaard and co-authors directly, or even U.S.-based co-conspirators such as Jayman and HBDchick (I am under the impression many of you would know them). They sort of already have started that process through DMCA takedowns on the data. We may never hear the result of this publicly however, and the editors might need to ask the authors directly for information in this direction.

Even then, the legal situation is not limited to the U.S. The Danish data protection authority might also investigate, and in fact they are considering it. They might need additional encouragement, so I wrote to them a letter (just mailed today) with hints that it might be wise to enlarge their investigation to Kirkegaard and co-authors' actions in Danish schools (i.e. previous papers which appeared here). I guess I will hear a response to this, but not editors to this journal unless they ask Kikegaard and co-authors.

While I applaud the editorial team's decision to proceed carefully on the matter, I do also think that it should proactively seek more information from the author as part of the open peer review process, and that these actions should be informed by ongoing legal procedures but also be independent. In any case, they are definitely part of the open peer review process.

I agree that any decisions should be informed by ongoing legal procedures. And we are waiting for feedback from OKC's legal office and now apparently also the Danish data protection authority. Autocratic and uninformed decisions are not in the spirit of this journal, so legal feedback will guide our future decisions on whether to withdraw the paper or re-publish it in a different format.

We never had to deal with similar issues before. We didn't adopt the more conservative PLOS approach of denying users the right to publicly discuss ethical matters, but we reached a compromise by allowing you to flag the publication using this thread but removing it from the peer review thread.

That's not entirely true. There are at least two instances where it has been clear on this forum that Kirkegaard's actions were quite iffy. Once his ethics were directly questioned, and his answer was, in my judgement, unsatisfactory. Unfortunately no one in the forum acted on it.

Actually I had specifically created a thread in May 2015 to deal with allegations of scientific misconduct, although these mostly concerned plagiarism and not the ethical issues that you raised. Apparently both you and Emil forgot it exists so this thread should not have been created and the comments should have been posted there: http://openpsych.net/forum/showthread.php?tid=208