Academics are growing wary of the peer review process amid mounting evidence that it is compromised by ideological biases and that it does not effectively or reliably ensure the quality of published research. Further, it is increasingly evident that the academic publishing industry’s primary reason for being is not to disseminate new knowledge to the public, but rather to serve as a bureaucratic metric for professional advancement. While the failure of the traditional peer review process might seem to justify pessimism on the part of scholars, the good news is that some academics are reimagining how academic publishing can work.
Recently, I had a chance to talk to two professors who have developed an alternative to the corrupted process of peer review. Dr. Harry Crane (Professor of Statistics at Rutgers) and Dr. Ryan Martin (Professor of Statistics at North Carolina State University) are the founders of Researchers.One, an online platform that allows scholars to publish new research and facilitates a critical dialogue to refine new scholarship. Our conversation addressed some of the shortcomings of the peer review process as it currently exists, and we discussed how the Researchers.One platform not only restores the most important functions of academic publishing, but also how it returns the ownership of intellectual property to the scholars themselves.
Video of our conversation can be viewed here, and an edited and abbreviated transcript is available below.
Researchers.One invites scholars of all disciplines to use its platform to publish their scholarship without the middleman.
Adam Ellwanger (AE): There’s been some discussion across a variety of fields (and at the James G. Martin Center recently) about the shortcomings of the peer review process. What are some of those limitations in your field of statistics and beyond?
Ryan Martin (RM): It helps to step back and think about the origin of the peer review process. The initial idea was that it was difficult to communicate these scientific developments, and so some kind of centralized type of organization was necessary to disseminate these ideas. But recently, a positive judgment from peer reviewers is thought of as an objective indicator of quality. And that’s not necessarily true.
Today, the bestowal of this mark of quality has become the primary role of the peer review process. But this creates a disincentive, especially for junior folks, to try to branch out and develop genuinely new ideas. So, it’s kind of like these mob movies like Goodfellas and Casino: the strategy to be successful in a world like that is to keep your head down and stay in line. That also happens in the academic world.
Harry Crane (HC): Yes, academia operates like the mafia, as anyone on the inside knows. The key is to think about how peer-reviewed scholarship is viewed outside academia, and how it actually operates on the inside. Peer-reviewed research is referenced all the time as evidence that a published claim is true. If something was peer reviewed, then that means it’s been vetted, so people assume it must be true. But that’s not actually what peer review does. In reality, peer-review is a purely administrative process that allows people to rise up the academic ladder. Whatever scholarly purpose it serves is secondary. That’s why we started Researchers.One.
AE: Tell us the story, then, of how you two developed the idea for Researchers.One and the function that you hoped it would serve.
RM: Well, we were both dissatisfied with certain experiences that we had, and it got to a point where we thought, well, what could we possibly do about this? Obviously, at first it was very scattered. But we wanted people to be able to disseminate their research with fewer obstacles. And to ensure that they could still have access to feedback from other scholars. Our aim was to separate the publication process from the peer review process.
HC: As I said, the peer-review process today has been co-opted by bureaucrats to serve administrative ends, and most likely will continue to serve this role for some time. But we don’t need to cater to that corrupted version of scholarship. Our platform advances an alternative which focuses solely on the scholarly aspects of peer review and research. We started with the low expectation that very few people would adopt this at first, not because there’s anything wrong with it, but because of career pressures. We were right, but recently we’ve been contacted by more and more people from all different fields who want to work with us.
AE: So, tell us a little bit about how Researchers.One works. If a scholar wants to submit some new research to your platform, what’s the process?
HC: In a lot of ways, it’s much simpler than the way that you would go about submitting to a traditional journal. And that’s by design: it shouldn’t be hard or prohibitive, and anyone should be allowed to do it. How do you do it? You go to the website, you set up an account, log in, fill out your name, title, abstract, and upload. The article is published immediately, and every article on the site can be reviewed publicly: anybody can comment on it, but no anonymous comments are allowed.
Our idea was that if something is going to get a stamp of approval, if someone is going to say “this is good research,” that person should put their name behind that judgment. In the traditional review process, reviewers can stay anonymous and avoid repercussions for shoddy or misleading work. Similarly, if somebody’s going to say a piece of research on Researchers.One is bad, they have to put their name to it. Their criticism is publicly posted on the site and everybody can see it. Then, the authors can respond if they want, and the process continues.
AE: Let me clarify for readers just how revolutionary this is. Essentially, you guys have lifted any gatekeeping mechanism for publication of new academic research. But you have retained a way to provide critical feedback that might improve the research—just after publication. Is that right?
HC: We haven’t retained it. We’ve restored it.
RM: Yeah, that’s right. There are no barriers. So right away, you can send it out on social media to get feedback. At its core, Researchers.One is about the dissemination of your work and the ability to get broad, critical feedback, which is essential. It’s not just about having a paper published: it’s about producing quality work and advancing knowledge.
AE: Let me jump to the obvious question that most academics would ask of your model. How do you ensure the quality of the scholarship without the process of peer review? I think that they might say, “how do we know that this is reliable research?”
RM: The short answer is that we don’t think that it’s our job to be ensuring quality. The typical style of peer-review relies on the subjective assessment of two or three anonymous reviewers. From our perspective, that judgment is better left to a broader community of scholars that help one another refine and test the work that’s been done. That’s not something any individual viewer or an editorial board can do. What’s required is time and visibility and transparency to determine which work is impactful.
I think about Newton and Darwin. There was nobody there saying, “okay, you need to include this kind of example” or “revise and resubmit.” This was not part of any sort of peer review process. And yet over time, it was determined that their ideas were groundbreaking and impactful. That older tradition is what we want to build on. So, it’s not really the two of us making judgments, or a few anonymous peer reviewers. It’s the scientific community, or the scholarly community, more generally.
AE: For the scholars who are concerned about the qualitative, gatekeeping role of peer review, I would also add that we’ve recently learned that peer review is not performing that gatekeeping quality very well anyway. This is what things like the replication crisis in psychology and the so-called “grievance studies” affair by James Lindsay, Helen Pluckrose, and Peter Boghossian have shown us.
HC: Well, in regard to the failure of the peer-review process to perform this gatekeeping function, the response seems to be that if something’s going wrong, you have to control it even more—that what we need is an even more arduous peer-review process. And that’s an approach that has shown itself to fail over and over again in a lot of different domains—peer review is just one of them.
In fact, Ryan and I have argued that removing the gatekeeper is critical to restoring and ensuring the quality of scientific research. Why is that? Well, it’s not just about removing the gatekeeper. It’s about removing the veneer of quality that comes with publication. So, if I publish something in Science or Nature right now, it has a very strong veneer of quality, and anyone can cite it and say “it must be good because it’s in Science.” And many will say that without having read a word of it. And that’s what happens in the media when their reports reference The New England Journal of Medicine or The Lancet.
These are journals that have famously published false information over the past year with little or no consequences. There’s nothing wrong with publishing things that are false in the pursuit of truth. The problem is when you publish false things under the premise that they’re correct. We’re absolutely not doing that at Researchers.One.
The other part is on the author’s side. If I could publish something mediocre (or even false) in Nature or Science, then I’m better off—a publication in a “top” journal, a line on my CV, and all the academic fanfare that comes with it. Compare that to Researchers.One, where the transparent lack of gatekeeping creates a much lower bar to publication and, in return, a much higher bar for the work to be widely accepted and recognized. So, there’s no incentive to publishing junk on Researchers.One, but a lot of incentive for publishing junk in Science, Nature, Lancet. That explains a lot of what’s happened with peer review and academia over the last 20-30 years.
AE: I know you’re both professors of statistics. Does Researchers.One publish work from any discipline, or just statistics and the sciences?
RM: There’s absolutely no restrictions. We’re happy to publish work in any field, English, History, anything. The great thing about talking with you is that you come from a very different disciplinary background than we do. What we want to do is bring scholarly communities together.
AE: It took us a while to schedule this chat because you guys have been working on some big expansions to the platform. Could you tell us a little bit about that?
HC: We’re in the process of adding two things. One is a platform to host virtual conferences in any field of study. We’ve had about four events so far, with more scheduled in the fall—some of which are the flagship events of certain professional societies. We also are going to be hosting journals starting very soon. Our first journal is in statistics. But we’ve also lined up a medical journal, a social psychology journal. And I think you may be starting a journal.
AE: That’s correct. I’ll be starting a journal of humanities and qualitative social sciences research called The Peerless Review. I think the title says it all. So, I encourage people who have inquiries about that to contact me. In the meantime, if somebody wants to check out Researchers.One, how can they access the site?
RM: That’s the URL: Researchers.One. If you don’t already have an account, you can easily create one for free, and then you’ll be ready to start submitting articles or reviewing other people’s research.
HC: We have a Twitter account, which is @researchersone.
AE: We forgot to mention one other thing: that authors retain all the rights to their work that appears on the site. So, if an author decides tomorrow that they don’t want to have it published on that platform anymore, they can take it down at their discretion.
RM: That’s a good point. There are some copyright terms, but the terms are meant to be as flexible as possible. You don’t give up any copyright privileges when you post research on Researchers.One. Authors are free to publish the paper on Researchers.One and submit to other journals.
HC: Why would you want to restrict your work to one place? Why wouldn’t you want to publish it as far and as wide as you possibly can? That’s our point of view. We’re just a platform. We’re not taking your ideas. We’re helping you to get your ideas out there and to build a community and to engage with other people. That’s what other journals and communities and platforms should be doing, too.
AE: You’re doing great work to restore some of the dignity to the process of academic publication. I wanted to bring this to the audience of the James G. Martin Center because I know that there are faculty among them who have wonderful, quality research that is sitting around unpublished due to the vicissitudes of the peer review process. Researchers.One would be a great home for that work, and we encourage you to submit your work.
Adam Ellwanger is an associate professor who studies rhetoric, writing, and politics. He is a member of Heterodox Academy, an organization that works to increase viewpoint diversity in American universities. You can reach him at adamellwanger@gmail.com.