We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already.
The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions.
The current academic enterprise relies heavily on third-party gatekeeping. We rely on others to do the vetting for us. The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
Any gatekeeper will naturally tend towards charging for access over time: It's a captive market, the economics demands it. Unless we eliminate that dependency, we cannot change the system.
Check out the "Collective action problem" described in this article. It describes why "Just publish on arXiV" isn't a practical solution. It doesn't lead to the problem being fixed, because of inertia against any individual breaking out of the system.
I've long wished that "journals" and academic societies would transition from a publishing model to a cultivation model. If everything is available on arXiv, that's great, but it also means the best of the best is mixed in with all the rest.
Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articles are on X topic". The paper's already there on arXiv, you could already read it before. But having a group of editors that cultivate a list of good articles (as well as the peer review process that can, in an ideal world, serve to improve a paper) can serve to make sifting through arXiv less overwhelming, and draw attention to papers in particular subfields, subject matter, or whatever other criteria might be relevant.
I don't see any reason why we shouldn't make this transitive. working professionals track the literature. if there were a standard way to publish a "I think this paper is interesting" signal, then we could roll that all up. there are certainly practitioners that I really do trust to be in the game for the right reasons, if they think a paper represents a contribution, then that's a strong signal for me.
This is quite similar to how eLife does publishing. You still have to submit to them but they basically just add reviewer comments and an “eLife Assessment” that serves as the quality/curation signal rather than a binary publish/reject
In the publishing world, there is this thing called the slush pile: the collection of unsolicited submissions, essentially the only way a person without an agent can break into the field. And you can find quite a few editors' experiences with the slush pile in various blog posts or articles online. And the general reaction goes from naïve wonder at the idea of finding the diamond-in-the-rough to frustration with the quality of the submissions and a realization that the actual game is to figure out how to reject submissions with as little reading as possible (because they don't have the time to do any reading!). This is before LLMs came about, which have made the slush pile problem much worse because they don't improve the quality of the submissions but the increase the amount of reading that needs to be done to reject them.
Academia has the same fundamental problem. We don't actually have the time to read every possible paper someone has for us, because keeping up with literature takes time that we don't have. And while relying on the quality of the journal or conference as a metric for "is this paper worth reading?" has issues, to be honest, it is more effective than other proposed solutions. When I have done the literature searches that delved into the unknown, low-quality tiers of journals... no, those results were not worth the time I spent reading them.
Why isn't a citation just a citation. It's a pointer to a source, that's all. If it implies some standards have been applied or editorial or scientific review has been done, then that's going to have to be paid for by someone. TFA implies that doesn't happen: [and then] we stop doing all that stuff and then the cash just pours out. So a citation to an article in Nature isn't any better than one on arXiV.
> So a citation to an article in Nature isn't any better than one on arXiV.
The real problem is that nobody can grade and compare article in different topics, so there are proxies like number of articles in "serious" journals (whatever that means[1]) and number of citations in "serious" journals (whatever that means[1]).
Do we count also citations in X/Tweeter, FaceBook, WordPress [2], StackOverflow, ... ?
If links in HN also count as citations, there are 3 additional citations for my last paper:
Unfortunately I think charging money is a necessary signal that this particular gatekeeper is doing a good job. We should recognise that money is a necessary part of this process, else there is no gate to keep. But we shpuld reverse the economics by having people pay to get their stuff peer reviewed. Imagine if reviewing research papers was something you could get paid to do, the incentive then isnt to rubber stamp things, actually your rating as a reviewer would come down to quality of reviews
Because there isn't such a relation. It's a thing people believe when they don't have actual experience with peer review. If anything, predatory journals and low-quality pubs can charge more, since publication is more guaranteed (and researchers reaching for these pay-to-publish journals are more desperate).
It's a reputation economy. Like review sites. They start off truthful, and then as time goes on incentives shift to bad actors to subvert it. Or they just sell out their reputation.
Yelp, TripAdvisor, wire cutter, hell even Google results themselves.
Once you start poisoning that well, it's difficult if not impossible to claw it back.
I tend to agree, but keep in mind that most likely you just don't even bother reading the shittiest of the shittiest papers just based on title and abstract. And for every good article there are like 10 unindexed shitty ones.
> But we shpuld reverse the economics by having people pay to get their stuff peer reviewed.
Not really. There would be perverse incentives where the publisher benefits from accepting more articles. For good journals that would be a conflict of interests at best where they would optimise the marketing-to-acceptance ratio. I can’t believe I am writing something good about scientific publisher, but at least when the reader pays they are incentivise to publish things that have an audience. Otherwise, they are going to cut corners, and I mean more than they currently do. And it’s not hypothetical, there are already terrible publishers doing this.
The problem is that this becomes a race to the bottom of actual quality and turns into advertising.
Sponsored reviews of products are basically this. If you are paying a reviewer for a stamp of approval and the reviewer sets the bar too high, why would you want to pay that reviewer? On the other end of the reviewer, it's easy to get more money by providing that stamp of approval to more people--not fewer--so they're incentivized to make it fairly easy to achieve.
I don't think there's really such a thing as a credible citation source. The citation just has to point to a source that hosts the genuine article; the credibility of the article comes from the articles that point to it, and the independent verifiability of the article's claims, not the location that it is hosted.
We have a gatekeeper already in the funding source - they do the work of vetting researchers prior to funding the work.
Piggy back this system so that the funding source publishes the papers itself, and researchers can only publish their papers that are directly funded.
This system requires the cooperation of an organization to build the publishing infrastructure, but this could be a lowest capable bidder, and less drag on the system overall.
> The first thing an academic does is check where a paper is published, before even reading it. It's a crutch
IMO, academics that do this are not very competent, because we have plenty of research suggesting that higher-profile journals are in fact less trustworthy in many ways, or that there is no correlation at all between reputation and quality (see my other post here in this thread).
Yes, some trash journals publish all trash, but, beyond that, competent researchers scan the abstract, look at sample sizes and basic stats, and if those check out, you skip to the methods and look for red flags there. Also, most early publications will be on an arXiv-like place anyway so you can't look to reputation yet.
Likewise, serious analytic reviews like meta-analyses don't factor in e.g. impact factor or paper citations, since that would be nonsense. They focus on methodology and stats.
I really think we ought to shame academics that are filtering papers based on journal alone, it is almost always the wrong way to make a quick judgement.
Do you not notice the circularity of your reasoning here?
Also I didn't say incompetent, I said "not very". More competent researchers make journal rep only a very small factor, and it is not via the "high rep = more trustworthy" direction (which is the bad heuristic), it is "pay-to-publish journals = not trustworthy" (better heuristic).
Once you have ruled out a publication being in a trash journal, reputation is only a very minor factor in consideration, and methodological and substantive issues are what matter.
It all depends on whether the paper fits the journal. Minor journals serve a useful service as a repository for minor results. And minor results are still worth publishing because they might provide a detail or technique later needed for a major result. The thing to be wary of is when you see a stunning result that should really be in _Nature_ or _Science_ in some minor journal. Why isn't it? Was it submitted there first and rejected? It would be nice if the history of a manuscript (and its peer review) stayed with a manuscript so you could see if the authors really corrected problems brought up by peer review or were just spamming journals with a flawed manuscript until they found one that published it.
Agree with all this. Once you've filtered / made decisions of quality based on the more substantive criteria, journal reputation can provide useful additional information / context. The case you mentioned is a good example.
There is no reasoning. I gave you a statement of fact:
I have personally seen highly talented and successful researchers use the heuristic of journal quality when looking at the state of their field. These people are highly competent by any standard. If you want to play word games with negation you could say they are not not very competent.
Ah, look, another smug sneer that ignores the evidence I presented, and makes another circular argument (i.e. that because academics look at rep, this is justified, even though I provided evidence disputing this).
I know what journals are better / not. But reputation only is helpful in letting you ignore trash journals, once you are out of trash land, rep is just not a very meaningful factor, and you have to focus on methodology and substance.
I literally said it was posted in this thread, and a quick Ctrl+F of my username on this page would have found you it in a half second: https://news.ycombinator.com/item?id=47249236
>The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
This is actually what ruined my respect for Academia.
My Science PhD buddy looked at the journal title and the claim, then said: "Its true!"
I look at him with horror. Who cares about the journal, I want to know data and methodology.
I've basically never forgiven Academia since this. I see even Ivys put out bad research and journals will publish bad research (Replication crisis and the ivy fake psychology studies)
For outsiders, there is a prestige to being a PhD or working as a professor. Now that I'm mid career and lived through the previous events I mentioned + seeing who stuck with academia... These are your C grade performers. They didnt get hired by industry, so they stayed in school. They are so protective of their artificial rank because they cannot compete in Industry. Its like being the cool person on the tennis team. They are locally cool, but not globally cool.
> This is actually what ruined my respect for Academia.
Spoken like someone who never went through grad school at a competitive R1 program
It was already a grueling 60-80 hour grind every week with frequent all nighters, high-pressure deadlines, absolute minimal pay, thankless duties, and plenty of politics. It's about the same for professors too.
We already paid our dues by helping peer review (for free) a half dozen papers for each one we submitted. Why should we be expected to review random papers on arxiv too...?
I went to an R1 university. Most students did not have a 60-80 hour grind. If they did, it was because of an overbearing advisor. Years later, those students are not ahead of those who had a more relaxed advisor.
And chances are: Those overbearing advisors are very invested in the current system.
>Why should we be expected to review random papers on arxiv too...?
The GP is not saying to review each paper you read or cite. They're complaining that a colleague accepted a claim after just reading the title and where the paper was published. Between that and doing a full review there's surely a world of options.
The problem is not that he was not willing to review it. It was that he was willing to conclude it was true. If he had said "that is interesting" or "that is plausible" or whatever, that is fine. It is concluding it is true that is the problem.
I don’t think folks in academia have come to terms with how much the above attitude has completely and nearly entirely undermined the credibility of the entire scientific and academic community in the eyes of the general public.
You don’t need a degree to understand how much utter junk science is being published by those who think they are superior to you. Just read a few actual papers end to end and look at the data vs conclusions and it becomes totally obvious very rapidly that you cannot “trust the science” since it’s rarely actual science being done any longer.
The academic community has utterly failed at understanding they needed to cull this behavior early and mercilessly. They did not, and it will be generations at best to rebuild the trust they once had. If they ever figure out they need to.
Things are going to get much worse before they get better. You can’t take any published paper at face value any longer without going direct to primary sources and bouncing it off an expert in the space you still trust to give you the actual truth.
I fear you are right here, and that the problem is far more dire than much of academia realizes. I know enough highly intelligent people (some even with family / spouses in academia, surprisingly) that are otherwise very e.g. left / liberal / progressive and open, that are still basically saying academia needs to be gutted / burned down.
I've no idea what the actual stats are on faith in academia overall today, but I don't think it is looking good.
On the whole you should rarely read papers, you want to read a whole literature in an area. Academics embedded in the field can do this easily. Academics outside of an area know to do this, and to bounce things off an expert to make sure you have the context and aren't over-indexing on a flashy result. Everybody learns the painful lesson in grad school to not just read a paper and believe everything will work as it says.
Somehow the general public and policymakers got the idea that if a paper gets published in any non-fake journal, this is an official endorsement that it's 100% correct, everything in it can be read in isolation, and it's safe to use all claims in the paper to direct policy immediately.
I think academia is partially to blame for encouraging people to believe this rather than insisting on explaining the nuances of how to interpret published research. On the other hand, nobody wants to hear a message that things are nuanced, and they will have to do costly hard work to get at the truth.
I think a world where "you can take any published paper at face value...without going direct to primary sources and bouncing it off an expert in the space" would be great, but it never existed, and it's just fundamentally impossible.
I wouldn't be surprised if the parent's complaint about his academic buddy who didn't read the paper's methods yet declared their findings as true, had misunderstood why his friend did so... which could have well been due to their additional knowledge about similar past findings/studies.
Go read /r/LawyerTalk and enjoy the horror of the dawning realization that this is a lot of professionals. I think it's an issue that stems from getting too deep into the minutiae of the technical and cultural matters of one's field; you become a really good scientist, or lawyer, or SWE (by the standards of scientists and lawyers and SWEs), and end up coming to conclusions that everyone outside the bubble looks at and says, "That's absolutely asinine." Well, laymen just don't understand the details, you know? (Even though the whole point of these professions is to provide services to laymen, fix problems laymen come to them with, and guide laymen to make practical and logical decisions when a $500/hr appointment isn't called for.)
These people take themselves too seriously, and other people only take them seriously when there are material ramifications for not doing so. Otherwise, they're viewed as pompous busy-bodies and don't do themselves any favors by playing to the role.
>It was already a grueling 60-80 hour grind every week with frequent all nighters, high-pressure deadlines, absolute minimal pay, thankless duties, and plenty of politics.
You know what else works really hard? A washing machine. Hard work alone doesnt create value. I could give you a spoon and tell you to dig a hole, or I can teach you how to use a Digger.
Some things are hard because you overcomplicate them. Some things are hard by their very nature.
Unless you are a Claude Shannon type, adding fundamental new knowledge to humanity's corpus is generally actually hard - at least in science & engineering. If you feel differently, I look forward to reading your groundbreaking papers!
Laundromats are the best business there is and are extremely profitable and seldom to never go out of business - you should look this up, it is fairly fascinating.
The laundromats around here closed down years ago. Not a characteristic of extremely profitable businesses.
There are a couple of strip malls nearby that have vacant rentables, vacant for years and years. Nobody has thought to put an extremely profitable laundromat in any of them.
Complete hogwash of a comment, based almost entirely on your limited experiences, to denigrate academic scientists.
If you even knew these people, you'd know that most that remain in academia never considered industry in the first place. These people were not rejected by industry. In fact, it is the other way around. *They rejected industry*. They did so, despite knowing they'd make more money, but chose to remain in academia because they wanted to spend their life pursuing research topics that interested them with independence. Sometimes they feel the fool when money is tight and the hours are relentlessly long, but never have I seen it happen because they were rejected by industry.
There's also a middle ground, i.e., renowned publishers who aren't free but still publish everything as OA. One example is Dagstuhl Publishing for CS research papers.
> The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything
I do some due diligence work from time to time. Uploading to arXiV is becoming a favorite tactic from companies trying to look impressive for investors. I’ve read a lot of “papers” submitted by startup founders that are obviously ChatGPT written slop uploaded to arXiV. They then go to investor and show their record of “published research”. Smart investors are catching on but there are a lot of investors who associate journals with quality and filtering and assume having a paper on there means something.
The filtering and curation problem is real. It seems like academic pettiness or laziness from the outside, until you see the volume of bad “papers” that everyone is trying to publish to chase the incentives.
I have had so many "why don't you just" conversations with academics about this. I know the "why don't you just" guy is such an annoying person to talk to, but I still don't really understand why they don't just.
This article pointed to a few cases where people tried to do the thing, i.e. the pledge taken by individual researchers, and the requirements placed by certain funding channels, and those sound like a solid attempt to do the thing. This shows that people care and are somewhat willing to organise about it.
But the thing I don't understand is why this can't happen at the department level? If you're an influential figure at a top-5 department in your field, you're friends with your counterparts at the other 4. You see them in person every year. You all hate $journal. Why don't you club together and say "why don't we all have moratorium on publishing in $journal for our departments?"
No temptation for individual research groups to violate the pledge. No dependence on individual funding channels to influence the policy. Just, suddenly, $journal isn't the top publication in that field any more?
I'm sure there are lots of varied reasons why this is difficult but fundamentally it seems like the obvious approach?
> If you're an influential figure at a top-5 department in your field ... you all hate $journal.
That's the problem, they don't hate these journals, they love them. Generally speaking they're old people who became influential by publishing in these journals. Their reputation and influence was built on a pile of Science and Nature papers. Their presentations all include prominent text indicating which figures came from luxury journals. If Science and Nature lose their prestige so do they (or at least that's what they think)
This was very apparent when eLife changed their publishing model. Their was a big outpouring of rage from older scientists who had published in eLife when it was a more standard "high impact" journal. Lots of "you're ruining your reputation and therefore mine".
Maybe I am underestimating the gap in status between the "influential figures" I imagine and the people I actually know.
I see: my friend has 10-15 years of experience in their field, they have enjoyed success and basically got the equivalent of a steady stream of promotions.
I map this onto my big tech/startup experience. I mentally model them as: they are "on top of the pile" of people that still do technical work. Everyone who still has the ability to boss them around, is a manager/institutional politician type figure who wouldn't interfere in such decisions as which journal to publish in.
But probably this mapping is wrong.
Also, I probably have a poor model of what agency and independence looks like in academia. In my big tech world, I have a pretty detailed model in my head of what things I can and can't influence. I don't have this model for academia which is gonna inevitably lead to a lot of "why don't you just".
Same thing happens to me when I moan about work to my friends. They say "I thought you were the tech lead, can't you just decree a change?" and I kinda mumble "er yeah but it doesn't really work like that". So here I'm probably doing that in reverse.
For example, spearheaded by Knuth, the community effectively abandoned the Journal of Algorithms and replaced with with ACM Transactions on Algorithms.
however it's difficult. a big factor is that professors feel obligated towards their students, who need to get jobs. even if the subfield can shift to everybody publishing in a new journal, non-specialists making hiring decisions may not update for a few years which hurts students in the job market.
I think the call for top-down policy makes sense b/c otherwise this is like every other tragedy of the commons situation. Each of those top-level researchers also has to think, "my department has junior faculty trying to build their publications list for tenure, we have post-docs and grad-students trying to get a high-impact publication to help them land a faculty job, we have research program X which is kind of in a race with a program at that other school lower down in the top 20. If we close off opportunities with the top journals, we put all of those at a competitive disadvantage."
For the grad students especially, there’d be a career advancement incentive to still publish in the top journals. The professors might still want to publish in them just out of familiarity (with a little career incentive as well, although less pronounced than the grad students).
I think it’d be a big ask from someone whose role doesn’t typically cover that sort of decision.
There are hundreds of reputable research universities around the world. Top-5 departments can't meaningfully change the culture of a field on their own. Top-100 perhaps could, but the coordination problem is much bigger on that level.
Grant funding reporting requirements. It would be easy to say self publish for free via the institutional library. But the NIH would not like that use of their money.
> So the solution here is straightforward: every government grant should stipulate that the research it supports can’t be published in a for-profit journal. That’s it! If the public paid for it, it shouldn’t be paywalled.
The article then acknowledges this isn't a magic solution to all the problems discussed, but it's so simple and makes so much sense as a first step.
I'm no expert here and there are probably unintended consequences or other ways to game that system for profit, but even if so wouldn't that still be a better starting point?
I think that's also a good proposal, and I don't think it conflicts with the "prestigious departments stop publishing in $journal" idea at all. Probably we want both.
Only difference is that the author is writing for a wide audience and his best angle to change the world is probably to influence the thinking of future policymakers. While I am just an annoying "why don't you just" guy, my "audience" is just the friends I happen to have in prestigious research groups.
Adam M also probably has lots of friends in prestigious research groups (IIUC although he complains a lot about academia he was quite successful within it, at least on its own terms). And the fact that he instead chooses to advocate government policy changes instead of what I'm proposing, is probably a good indication that he knows something I don't about the motivatioms of influential academics.
Imagine being a scientist and reading “if you take this grant, you cannot publish your results in any of the most prominent journals in your field.” Sounds good?
But IIUC there are entire fields where basically the whole US ecosystem is funded by federal grants. So if this policy gets enacted those journals are no longer prominent.
(Maybe you'd need an exception for fields where the centre of mass for funding is well outside of the US, though).
I explain here (https://news.ycombinator.com/item?id=47250811) but tl;dr it's because Universities need this system to get money and to give money. Nobody has yet proposed a solution which solves the money/prestige problem. With no money there's no research.
Worth pointing out a success story: all ACM publications have gone open access starting this year[1]. Papers are now going to be CC licensed, with either the very open CC-BY[2] license or the pretty restrictive (but still better than nothing!) CC-BY-NC-ND[3] license.
Computer science as a discipline has always been relatively open and has had its own norms on publication that are different from most other fields (the top venues are almost always conferences rather than journals, and turn-around times on publications are relatively short), so it isn't a surprise that CS is one of the first areas to embrace open access.
Still, having a single example of how this approach works and how grass-roots efforts by CS researchers led to change in the community is useful to demonstrate that this idea is viable, and to motivate other research communities to follow suit.
That works nicely if your institution participates in ACM Open (no such institution in my country, and no, my country is not in the list of lower-middle income countries).
The combination of 'publish or perish' with 'pay for publication' and 'miserly grant money' is deadly.
While in theory the idea is nice, in practice this is a problem (maybe not in most rich countries, but here definitely).
Nowadays, you could always get the article you are interested in, even if it is beyond a paywall. Hence, perversely, the old model (which I hate, for reasons well explained in the original post) worked better for me. :-(
> Robert Maxwell, one of the architects of the for-profit scientific publishing scheme. When he later went into debt, he plundered hundreds of millions of pounds from his employees’ pension funds. You may be familiar with his daughter and lieutenant Ghislaine Maxwell, who went on to have a successful career in child trafficking.
Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.
> So the solution here is straightforward: every government grant should ...
People who write such sentences have no idea what they are talking about or are being intentionally naive for whatever reason.
Just because your one-sentence solution reads simple doesn't make the actual solution simple. Because such a solution involves changes to laws, changes to entrenched interests, changes to distribution of money involved in the whole system, and changes to balance of powers between stakeholders. Unless the push for such changes is significant enough to overcome the current state of affairs (due to public opinion, redistribution of power or money, etc.), nothing will happen.
A solution to a problem that doesn't change the current state of affairs, which by your definition makes it a simple solution, is not an actual solution.
There are plenty of simple solutions to real problems whose only blocker is upsetting the status quo. "We have no housing...let's build more housing" is, in fact, a very simple solution. That it doesn't happen has nothing to do with it the solution itself.
This comment seems to confuse _straightforward_ with _easy_. On the merits, this proposal is well argued and has good points, and his solution—essentially extend the Biden approach with more strict requirements—makes sense.
Everything you mention will also have to happen, which means that doing this will definitely not be _easy_. That said, it is still a very _straightforward_ solution.
So maybe consider that when you give up on obvious things that are good based on some conspiracy theory that the "man" is trying to keep you down, what you're actually doing is being part of the system and endorsing it. Changes like this do happen, they just happen despite you.
Why would you claim that my statement is defeatist? It is just acknowledging the reality of how things work nowadays without saying anything about giving up. If anything, it cautions people that change will not come easily and to be prepared for it. I vote, give money to political/public causes, to go meetings of my federal representatives, post my opinions on HN, ... I am just being more realistic about my expectations.
You sound like my parents. As I get older, I drift less into this mentality, and more into "I am tired of this defeatist bullshit, and accepting corruption and stagnation". I'm going to leave the world a better place, and never give in to this. I will vote for and donate to candidates who also want to fight, and run myself.
> I will vote for and donate to candidates who also want to fight ...
Fight for what? I bet to change one or more of the things that I have mentioned above. I have said "nothing will change" unless these things change; I didn't say you/we shouldn't do anything.
In astrophysics we now have Open Journal of Astrophysics that is basically an overlay on top of arxiv. https://astro.theoj.org/ It is now catching on with ~ 200 papers published last year, after some of the astro journals started to charge the Golden Open Access publishing fees. I think now people realize how crazy it is to pay for hosting a PDF and for sending your draft to a few referees without paying anything for their work.
The article's discussion of how the open access mandate works is wrong. Federally funded research, when published (even in a closed-access journal) must be deposited in https://pmc.ncbi.nlm.nih.gov/ or a similar repository.
Edit: OA advocates have won pretty much everything we wanted, there's not much left to be outraged over.
But that's what the article says? The (correct) criticism of that system is that the publishers just replaced the subscription fees with article processing charges for OA publication, and still profit from public funding the same as before.
> Now people barely bring it up at all. It’s like a lion has escaped the zoo and it’s gulping down schoolchildren, but when people suggest zoo improvements, all the agenda items are like, “We should add another Dippin’ Dots kiosk”. If you bring up the loose tiger, everyone gets annoyed at you, like “Of course, no one likes the tiger”.
Is the switch from lion to tiger in that paragraph intentional? If only the quote from "everyone" was switched I might think it's intended to convey people trying to derisively dismiss the issue or something but it does specifically reference the tiger rather than the lion being brought up, so I was confused when I got to this part of the article.
This is not how computer science publishing works, however. Post it on arxiv, submit to a conference, get 3 peer reviews, accepted, “published”. 99% of papers are effectively open access for free.
> Academia is so cutthroat that anyone who righteously gives up an advantage will be outcompeted by someone who has fewer scruples. What we have here is a collective action problem.
And what, pray tell, is this advantage? If there is no utility to anyone in publishing in Science or Nature then how can it be an advantage.
I suspect it’s simply that these guys are a curation service. They separate the cranks from the science. They can be imperfect at this so long as important people separate the cranks from the science.
This kind of winnowing is pretty useful in general. Many universities are pretty much that and people pay to attend them.
It makes sense that a credentialing service would charge for the credential. It doesn’t make that much sense to say “no credentials allowed; you and timecube guy must be considered the same”. I want you to show your credentials. We all do because science is an empirical field and empiricism depends on facts. I cannot process your paper with pure reason.
If you looked under the microscope and saw light I don’t know that you didn’t. At scale I need someone to figure out “this wasn’t a Photoshop situation; that’s totally fluorescence”. Arguing that we should remove these organizations is similar to saying you should remove diplomas and so on. Anyone can continue to attend universities. They just can’t hand out diplomas. So no credentials. Only learning. Simple thing. Or perhaps not so.
SciHub is an amazing resource. I have read so many papers from varied topics out of my personal interest. That would not be possible for me without SciHub. The hold on new papers has stopped me from keeping current. If I were able to also publish papers that others could review that opens "science" to everybody. Then the only benefit of research institutions would be a concentration of big brains. That completely changes the landscape for scientific progress.
SciHub has shown us a new way to spread knowledge to all that are interested. I don't have the rigor for publishing but other individual experimenters might. It would be great if they could contribute to building human knowledge.
I think the only real solution is a distributed federated publishing and review platform. A node would be a library of papers for the host's interests. Just like physical journal collections, bigger institution would host more topics. Anybody can participate in the publication and review process. SciHub nailed storage and retrieval. Review is the hard part. Any rating system can be gamed. It would be very hard to convince people it is trustworthy.
There shouldn't be any prestige in publishing a paper. The prestige comes from being proven correct, from building our knowledge.
Arxiv isn't the solution. But i think computer science conferences are. These have the same scientific rigour and standards in the review process as journals in other scientific fields, but don't price gouge. Yes, conferences are also a bit expensive, but you get a lot for your money, and they usally aren't out to make a big profit.
The peer review process in CS is pretty bad, especially in conferences with a single round of reviews. When you combine this with high rejection rates, peer review becomes more about finding excuses to reject a submission than about trying to improve it.
Conferences also don't work that well as publication venues, as they often require that one of the authors must attend the conference physically. And it's not as much about money than about visa policies and travel restrictions. Even in the 2000s and 2010s, when international travel was easy, people from non-Western countries could often not get visas to attend conferences. And today the situation is much worse.
I've been to three international conferences in the past year. One was held in Europe. People from Russia and Israel had to present remotely, the former due to an ongoing war and the latter due to an unexpected war. Another was in the US, and there were fewer Europeans than usual, as many were not willing to take the risk. And the third one was in Japan. People from China could not attend due to increased tensions. People from Israel were there, but they were worrying if they would make it home before the next war. (They made it, and the war started a day or two later.)
Conferences can be truly wonderful, but not a universal replacement for publishing.
If you think journals are expensive, try sending your whole lab to a conference in another country. That may not let you in. Where some of the attendees have to fill out paperwork before talking to a foreign national. (does that ever make for awkward small talk...)
For all their many faults, journals provide access to a really wide audience, and- in theory- make it possible to form connections who wouldn't be able to meet directly.
Acknowledging, I am not a expert in this stuff, here is an idea: getting momentum for these sorts of things is so important, what is the journal that would be easiest to make a big example of, so that everyone understands that it is possible? Just completely mercilessly drive them out of business, and then hound their executives when they try to get other jobs. It appeals to peoples base instincts, but the last 10 years have shown those are pretty powerful. Then the movement which has formed around that can take down progressively bigger journals. Probably want a different organization building the alternative; the people with the personality to fight at the Vanguard of the revolution don’t tend to be great at building in the long-term.
Journals are not about providing access to science, much less public access.
Journals are an academic-career-advancement service. It therefore makes sense that they do not pay academics. You don't pay your customers.
That means they need to generate a secondary customer base elsewhere, who will pay. Those secondary customers happen to be the employers of the academics who are the primary customers. That socializes the cost of providing the service, since academics individually wouldn't be willing and able to pay.
Once journals have established a reputation, their policies and paywalls and fees are the result of trying to signal exclusivity and set an optimum market price.
Until the supply side of the research market largely agrees on a way to use open-access repositories like arXiv as a primary career-advancement signal, complaining about closed-access journals is tilting at windmills.
Changing the law to prevent journals from being able to copyright anything could potentially force the research industry to rapidly develop a new solution, but at the cost of short-term chaos and career instability for new academics.
Science has never been free, and it isn't mostly progressive; like the bulk of the population, it is hyper-conservative without admitting it. So, the first flaw lies in the very social structure of those who practice science.
The second problem, however, is a modern one: the pure, naked, and raw commercialization of science through "publish or perish", whereby the researcher is a Ford-style assembly line worker to be managed and who must be replaceable.
Without a MENTAL paradigm shift, even before a material one, we will only be able to plug small leaks on a ship with a torn hull.
This post misunderstands why the journal system exists, and why they work this way. It is not because there is an evil corporation controlling everyone else.
Universities do the research. They voluntarily choose to pay for and publish in journals. They could just decide not to do this. Literally, just don't publish in a journal, anymore, ever. Upload your study paper to a Dell Inspiron sitting in a closet in the university faculty lounge, connect it to the internet. Done. Why don't they do this?
#1. No guarantee anyone will look at it. You just spent years of time and money to come up with a research conclusion. Will anyone read it? Comment on it? Learn from it? Is it any good? Will anyone review it? It's after all just a paper sitting on a server. Without some kind of process to vet it independently, and publish it in a place where people can find the latest vetted papers, it's too much hassle for most people to ever find, much less trust.
#2. The reputation feedback loop. Universities give research grants to "well respected" researchers. You become a "well respected" researcher by having academic achievements. You get academic achievements by... doing research that gets published in a journal. Universities depend entirely on the prestige of the researcher and journal to decide who gets a grant. Because...
#3. Money is hard to get. In order to convince someone (government, private donor) to give your university money, you have to show them it's worth it. And the way they show that is.... the prestige of the researcher, being published in the prestigious journal. Look, we have cool peeps, publishing in cool journals! Give us more money!!
Therefore, the reason journals still exist, is Universities desperately need them. They don't want to pay an insane amount of money to a journal. But they don't really have a choice.
Could Universities replace journals with something else? Well, they could work hard to replace the "prestige machine" with other processes (which must enable them to get money, by showing their researchers are good, with vetted papers, published somewhere people will see them). They could replace the journal system with their own intra-university system. But it turns out, that costs a considerable amount of money, time, and resources... which is entirely what the "evil journal publishers" do. Universities would have to spin out their own entire corporation to do all that work, which would be a journal publisher. They know this is expensive, difficult, time-consuming, and they also know the existing system benefits them.
"Let's just throw papers on arXiv" does nothing to solve the money and prestige problem. So the world continues to turn as it has.
i am very glad to see others (presumably non-scientists) in this thread dunking on the false paradigm that "peer review = true". anyone who peddles this notion is naive or a moron.
while the author is correct that the for-profit publishing is definitely a negative externality, i can't help but feel they are missing the forest for the trees when it comes to all the other worse issues in academia.
a full explanation of which would be much too onerous for a hn comment, but in no particular order: rampant scientific fraud, waste of tax payer dollars, wage suppression via "students" and visa-dependent laborers (J1 visa abuse), publish or perish evaluation criteria, lack of management training, blatant and rampant racism, etc. etc. etc.
the whole system needs to burn down and be rebuilt from the ground up.
My experience with grad school is that they are shockingly stuck in their ways when it comes to organizational practices. They make even large tech companies look nimble.
Though at least in my field part of that is budgets are so tight it seems like most of the effort is needed to just keep the lights on. I don't see anyone who has bandwidth to help burn things down or rebuild in my department as much of the staff are already working unpaid overtime (and good luck getting funding for hiring many more).
I don’t understand why people care so much about the cost of journal subscriptions. If we add up all the revenue from all major scientific journal publishers, is that a big number in the context of the national economy? Or even compared to one major tech company?
I feel like this is one of those classic local minima where a community starving for resources fights vociferously amongst itself because they have internalized that they can’t win externally. From where I sit outside academia the problem with science seems obvious: there is not nearly enough money going into it.
I doubt bringing the heads of for-profit journals would change that under current national conditions in the U.S.
I think the solution is to launder all research papers through LLMs so the papers are no longer copyrightable, and let the rich journal owners fight with the LLM owners.
Like some other posters here I think that a paid service is probably a necessary evil for long term quality regulation (although currently it skews too much into evil)
Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it.
It doesn't. Not even close.
Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does.
Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1900's to deal with the fallout from another mistake ("publish or perish") that meant people had to try to publish even if they had nothing to say.
> Darwin's work wasn't peer reviewed. Nor Einstein's
Except it was…? This is absurdly ahistorical and the fact that you cross disciplines in trying to make an incorrect argument questions whether you are in science at all.
The structure of peer review in Darwin’s time was different, where experts wrote monographs and gave lectures at symposia that then led to letters among their peers. Which is what happens now, if you take a step back.
The volume of new work these days is incompatible with the older informal system, and is in some ways our new paradigm is superior as there is a formal period in which new works are reviewed.
Ah, but the naive public still broadly believes in peer review, and that high profile journals do good review. And the prominence and reputation that comes from these journals arguably then relies on this (increasingly false) public perception.
Would scientists feel the same if the public was more educated about how bad journals and peer review are? Not so easy to disentangle IMO.
The naive public does not believe anything in particular about peer review. They think new scientific results are significant when they read about them in the popular media, that’s it.
People who do need to work professionally with peer review, do understand what it actually does and its limitations.
You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
> The naive public does not believe anything in particular about peer review
You'd need to provide evidence or an argument for this. The media reports on things in part based on journal prestige, and likely when questioned, people will say they can trust such things because good scientists have looked at the work and say it is good. This would be an implicit belief that peer review is generally working well, even if they don't use the term "peer review".
> You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
Extremely presumptuous, as I work in this system, and have provided plenty of evidence for my claims. You've provided only sneers.
You've provided evidence that prominent journals experience retractions, fraudulent results, etc. All true. But it is not the job of peer reviewers to decide what gets published.
You've provided evidence that peer-reviewed science often turns out to be incomplete, inaccurate, wrong, fraudulent etc. All true. But it is not the job of peer reviewers to assure completeness, accuracy, or freedom from fraud.
A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.
The check on published scientific results is the scientific process itself, not the publishing process. Prominent results attract further investigation, which confirms or disproves the reality of the underlying phenomena. Again: that's not the job of peer review.
Do some people ascribe too much authority to peer review? Yes, for sure. IMO your comments in this thread are exacerbating that problem, not addressing it.
> A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.
Um, what? I have done all these things in reviews, and know other academics that have done these things as well. More confusingly though, if you are saying most reviewers don't do these things (which I agree with), this would only strengthen my point?
I'll let readers decide if it is my comments that exacerbate the problem, or if, perhaps, it is apologism for journalistic peer review that might be causing bigger issues in the present day.
Would be interesting if you would be willing to share a paper you reviewed and detail your review process of it. I don't see how one could check primary data or interrogate scientists in a blind review process, for example.
This is IMO just bad faith sealioning, you can look at the whole replication crisis in psychology and social science (esp. the work of people like Nick Brown and the GRIM test, or Uri Simonsohn), or sites like Retraction Watch, and see clear evidence of everything I am saying. There are endless papers in ML research going into issues with test datasets and data duplication, etc. In plenty of cases all data and code is made open, so it is trivial to check data issues and methods.
Also, review is back and forth, and has rounds: you almost always interrogate the scientists of the paper you are reviewing, this almost like the definition of peer review. I don't think you have any idea of what you are talking about at all.
Don't know why you are being downvoted, you are largely correct. I've provided plenty of evidence in another post in this thread showing that journal-based peer review is highly farcical.
EDIT: I still want review from a community of scientific peers. I just don't want this review to be in the hands of a tiny number of gatekeepers entangled with journals that largely just slow things down.
Because a lot of people are deeply invested in the present system perhaps? As the article pointed out, there's a lot of money involved, and there are a lot of people who've built their lives around flourishing in the existing system, cut-throat as it may be.
There should be a journal where it only publishes studies that have been replicated. Too much research slop is being generated for journals and we already know we have a severe reproducibility problem in science right now.
The problem with communists always seem to be their math is ridiculously bad. 1 billion in profits to the journals seems like a rounding error in an industry with almost 1 trillion in annual spending.
This is a fine example of where someone's understanding of the problem runs ahead of their understanding of the solution.
A few scattered thoughts:
1. There is a difference between pre and post publication peer review. These discussions almost invariably conflate the two, but part of the runaway success of spam journals is that the benefits of pre greatly outweigh the risks of post. Historically, there was some link: if an article had problems, you would open the table of contents n months later and (might) see a letter or further discussion. Now, the table of contents is google, and many readers have weaker links to the same venue over time for followup. At the metrics level, the reputational hit of bad articles is weaker. (studies have shown that retractions are often cited with the original intent years after a correction was published)
3. Outsourcing trust to someone who isn't the current evil... will only get you so far. No matter who takes over publishing, scientists are going to need to evolve new ways of evaluating work and each other, as the field grows far beyond what a small network can handle. Journals are a bad metric, but how does your dean evaluate 50 people hired to be the world's leading experts on (new and emerging field)? I've read plenty of these publisher=bad screeds, and most stop there. PubPeer exists for some, Twitter walkthroughs of papers were a great thing for a while, or there's also talk of overlay journals that decouple the act of publication (as a preprint) from the review-and-prestige piece.
4. The current system does two things: (a) provides a record of work done by students, who may labor under graduation requirements to publish something, whether their project is successful or not, (b) a shared record of current state of human knowledge, be it from researchers at a small college, or google, or pharma. Goal (a) puts a lot of pressure on peer review in "low tier" journals that even the reviewers don't like to cite, and I've had my doubts as to whether this is the best yield for effort.
The other factor preventing a fix is that people with no actual serious experience of academic publishing and peer review will defend these journals, because they still think that (journal-based) peer review acts like some kind of meaningful quality filter. But, it really doesn't.
Because someone is surely going to try to defend journals via peer review in this thread, I want to provide a counter to the arguments that journal peer review does much good. Also, since everyone knows that if you just go to a poor enough journal, you can be published, I am going to focus on the (IMO mostly false) claim that higher-profile journals are still doing a good thing here.
There are numerous studies showing that higher-profile journals in general have more retractions and research misconduct [1-2], lower research quality [3], in fact weaker statistical power and reliability [4], and that statistical reliability even in high prestige journals is still extremely poor overall [5]. Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
In 2014, 49.5% of the papers accepted by the first committee were rejected by the second (with a fairly wide confidence interval as the experiment included only 116 papers). This year, this number was 50.6%. We can also look at the probability that a randomly chosen rejected paper would have been accepted if it were re-reviewed. This number was 14.9% this year, compared to 17.5% in 2014. [7]
We should just move to arXiv-like approaches and allow the scientific community to broadly judge relevance and quality. Journals just slow things down and burn funding for very little gain or benefit to anyone other than the journal owners.
> higher-profile journals in general have more retractions and research misconduct [1-2]
Given that reviews are not a mechanism to check for truth but soundness, the higher profile the thing I would imagine there would be more misconduct. I mean would one risk prison to steal 10$ or to steal 1 million $?
> lower research quality [3]
To cite exactly from your link "the evidence is mixed about whether they are strongly correlated with indicators of research quality.". I find saying "lower" a bit too strong given the original quote.
> in fact weaker statistical power and reliability [4]
For a specific field "cognitive neuroscience and psychology papers published recently"!
> statistical reliability even in high prestige journals is still extremely poor overall [5]
> Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
It's a coin toss if paper could get accepted at all, and that's less than ideal but what the system should do (at least) is reject obvious crap, not ensure that something gets clearly accepted. The danger is False Positive (accepted even if it's crap) rather than False Negative (rejected even if it might be something useful).
Overall note: the review system is not ideal and should be improved. But it's a hard, complex and delicate problem.
Oh, I agree this is all super complex and delicate. If I had more time, I'd love to write a more nuanced, many-thousands-of-words blog post going into which journals and fields actually have good peer review and can be more / less trusted.
I just wanted to make a strong rhetorical case by highlighting some things that might be surprising to people making more naive defenses of journals via peer-review-based arguments.
I am sympathetic to the argument you wish to make, that peer review is no panacea, but the actual evidence you offer has nothing to do with this claim.
You are trying to say that high profile journals have more retractions, which is well known as you share.
How does that have anything to do with peer review? Are you saying that there is more review or less review in some cases and that influences retraction rate? In what evidence? In what world does the arxiv system moderate this discrepancy?
> How does that have anything to do with peer review?
I already addressed this. People know peer review can be bad, but some think "good journals" still do good peer review. This is not so clear.
> In what world does the arxiv system moderate this discrepancy?
Open systems allow the scientific community to figure out ways to properly assess research quality and value more cheaply, and without passing through (often arbitrary and random) small numbers of gatekeepers that don't even do a reliable or good job gatekeeping in the first place.
Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review.
I understand that we want arxiv to exist, and it does, and it’s growing. That doesn’t mean we don’t want Nature or Science to triage the most compelling stories.
Importantly, we can already begin the search for these ‘cheaper’ review strategies while not losing the helpful information filter we get by seeing where things are presented/published
> Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review.
No, it doesn't. The argument is that peer review is incompetent gatekeeping in general, and so slows things down and makes thing expensive. Also, I am countering the argument "we need journals because journals do peer review" by arguing "peer review by journals isn't clearly actually good", I am not saying "peer review in general is unneeded", as I support review by the entire scientific community, rather than journal gatekeepers.
> you fail to show how doing any peer review is strictly worse than doing no peer review
I wasn't trying to show that. I have provided plenty of arguments to show why killing journal-based peer review could definitely speed things up and so potentially make things better. I want actual organic review by the community, not by tiny groups of gatekeepers.
I do work in science, I am claiming that pre-publication / journalistic peer review is limiting (and biasing) the amount of post-publication / non-journalistic peer review that can happen, and it is not limiting this in a very reliable or even IMO particularly desirable way.
There is definitely a problem with the over-production of junk science, and we definitely need a way to filter this out somehow. I am just claiming journalistic / pre-publication peer review does not do this effectively or reliably at all anymore (if it ever did).
Everybody hates capitalism, but no one understands that they do. What you promote as a solution is simply a pebble on the path of the people who want to capitalize.
There are 3 realistic scenarios for your proposed solution:
- it will not pass
- it will be reformed later, or, if successful
- it will just make the capitalists appear at another point in the supply chain
The capitalists design the business models (profit making schemes) and legalize them. This is not an organic development of an industry.
What you hate is capitalism and capitalism will do this to any industry wherever it can attain steady profits.
If you're going to write an article titled "The one science reform we can all agree on, but we're too cowardly to do" and that one thing isn't explicitly stated in the first paragraph, I'm out.
Stop with the meandering nonsense and make your argument.
We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already.
The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions.
The current academic enterprise relies heavily on third-party gatekeeping. We rely on others to do the vetting for us. The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
Any gatekeeper will naturally tend towards charging for access over time: It's a captive market, the economics demands it. Unless we eliminate that dependency, we cannot change the system.
Check out the "Collective action problem" described in this article. It describes why "Just publish on arXiV" isn't a practical solution. It doesn't lead to the problem being fixed, because of inertia against any individual breaking out of the system.
I've long wished that "journals" and academic societies would transition from a publishing model to a cultivation model. If everything is available on arXiv, that's great, but it also means the best of the best is mixed in with all the rest.
Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articles are on X topic". The paper's already there on arXiv, you could already read it before. But having a group of editors that cultivate a list of good articles (as well as the peer review process that can, in an ideal world, serve to improve a paper) can serve to make sifting through arXiv less overwhelming, and draw attention to papers in particular subfields, subject matter, or whatever other criteria might be relevant.
I don't see any reason why we shouldn't make this transitive. working professionals track the literature. if there were a standard way to publish a "I think this paper is interesting" signal, then we could roll that all up. there are certainly practitioners that I really do trust to be in the game for the right reasons, if they think a paper represents a contribution, then that's a strong signal for me.
This is quite similar to how eLife does publishing. You still have to submit to them but they basically just add reviewer comments and an “eLife Assessment” that serves as the quality/curation signal rather than a binary publish/reject
In the publishing world, there is this thing called the slush pile: the collection of unsolicited submissions, essentially the only way a person without an agent can break into the field. And you can find quite a few editors' experiences with the slush pile in various blog posts or articles online. And the general reaction goes from naïve wonder at the idea of finding the diamond-in-the-rough to frustration with the quality of the submissions and a realization that the actual game is to figure out how to reject submissions with as little reading as possible (because they don't have the time to do any reading!). This is before LLMs came about, which have made the slush pile problem much worse because they don't improve the quality of the submissions but the increase the amount of reading that needs to be done to reject them.
Academia has the same fundamental problem. We don't actually have the time to read every possible paper someone has for us, because keeping up with literature takes time that we don't have. And while relying on the quality of the journal or conference as a metric for "is this paper worth reading?" has issues, to be honest, it is more effective than other proposed solutions. When I have done the literature searches that delved into the unknown, low-quality tiers of journals... no, those results were not worth the time I spent reading them.
Why isn't a citation just a citation. It's a pointer to a source, that's all. If it implies some standards have been applied or editorial or scientific review has been done, then that's going to have to be paid for by someone. TFA implies that doesn't happen: [and then] we stop doing all that stuff and then the cash just pours out. So a citation to an article in Nature isn't any better than one on arXiV.
> So a citation to an article in Nature isn't any better than one on arXiV.
The real problem is that nobody can grade and compare article in different topics, so there are proxies like number of articles in "serious" journals (whatever that means[1]) and number of citations in "serious" journals (whatever that means[1]).
Do we count also citations in X/Tweeter, FaceBook, WordPress [2], StackOverflow, ... ?
If links in HN also count as citations, there are 3 additional citations for my last paper:
http://www.example.com/gus_massa/very_good_paper_2026.pdf
http://www.example.com/gus_massa/very_good_paper_2026.pdf
http://www.example.com/gus_massa/very_good_paper_2026.pdf
[1] Which journals are serious and which are paper mills? In the extremes the difference is clear, but there in the middle there is a gray zone.
[2] A citation in Tao's blog in WordPress should be worth at least half official citation, or perhaps a whole point.
Unfortunately I think charging money is a necessary signal that this particular gatekeeper is doing a good job. We should recognise that money is a necessary part of this process, else there is no gate to keep. But we shpuld reverse the economics by having people pay to get their stuff peer reviewed. Imagine if reviewing research papers was something you could get paid to do, the incentive then isnt to rubber stamp things, actually your rating as a reviewer would come down to quality of reviews
> I think charging money is a necessary signal that this particular gatekeeper is doing a good job.
I’ve never seen the slightest relationship between the charge to read a paper and the quality of review.
Because there isn't such a relation. It's a thing people believe when they don't have actual experience with peer review. If anything, predatory journals and low-quality pubs can charge more, since publication is more guaranteed (and researchers reaching for these pay-to-publish journals are more desperate).
It's a reputation economy. Like review sites. They start off truthful, and then as time goes on incentives shift to bad actors to subvert it. Or they just sell out their reputation.
Yelp, TripAdvisor, wire cutter, hell even Google results themselves.
Once you start poisoning that well, it's difficult if not impossible to claw it back.
I tend to agree, but keep in mind that most likely you just don't even bother reading the shittiest of the shittiest papers just based on title and abstract. And for every good article there are like 10 unindexed shitty ones.
Yeah review takes time and time is money. This needs to be priced in somehow. Bonus side effect: Frauds get discovered and filtered out (in theory).
But who watches the watchers? I guess review fraud will need to be considered as well.
Scientific publishers do not pay for peer review. Reviews are done by researchers as part of their jobs which are paid for by their research grants.
> But we shpuld reverse the economics by having people pay to get their stuff peer reviewed.
Not really. There would be perverse incentives where the publisher benefits from accepting more articles. For good journals that would be a conflict of interests at best where they would optimise the marketing-to-acceptance ratio. I can’t believe I am writing something good about scientific publisher, but at least when the reader pays they are incentivise to publish things that have an audience. Otherwise, they are going to cut corners, and I mean more than they currently do. And it’s not hypothetical, there are already terrible publishers doing this.
The problem is that this becomes a race to the bottom of actual quality and turns into advertising.
Sponsored reviews of products are basically this. If you are paying a reviewer for a stamp of approval and the reviewer sets the bar too high, why would you want to pay that reviewer? On the other end of the reviewer, it's easy to get more money by providing that stamp of approval to more people--not fewer--so they're incentivized to make it fairly easy to achieve.
I don't think there's really such a thing as a credible citation source. The citation just has to point to a source that hosts the genuine article; the credibility of the article comes from the articles that point to it, and the independent verifiability of the article's claims, not the location that it is hosted.
We have a gatekeeper already in the funding source - they do the work of vetting researchers prior to funding the work.
Piggy back this system so that the funding source publishes the papers itself, and researchers can only publish their papers that are directly funded.
This system requires the cooperation of an organization to build the publishing infrastructure, but this could be a lowest capable bidder, and less drag on the system overall.
Exactly. The solution already exists. However another problem is that the arxiv is creeping towards the old model ...
> The first thing an academic does is check where a paper is published, before even reading it. It's a crutch
IMO, academics that do this are not very competent, because we have plenty of research suggesting that higher-profile journals are in fact less trustworthy in many ways, or that there is no correlation at all between reputation and quality (see my other post here in this thread).
Yes, some trash journals publish all trash, but, beyond that, competent researchers scan the abstract, look at sample sizes and basic stats, and if those check out, you skip to the methods and look for red flags there. Also, most early publications will be on an arXiv-like place anyway so you can't look to reputation yet.
Likewise, serious analytic reviews like meta-analyses don't factor in e.g. impact factor or paper citations, since that would be nonsense. They focus on methodology and stats.
I really think we ought to shame academics that are filtering papers based on journal alone, it is almost always the wrong way to make a quick judgement.
I have seen more than one PI at an R1 universities with multiple Nature publications use this heuristic. I would not call them incompetent.
Do you not notice the circularity of your reasoning here?
Also I didn't say incompetent, I said "not very". More competent researchers make journal rep only a very small factor, and it is not via the "high rep = more trustworthy" direction (which is the bad heuristic), it is "pay-to-publish journals = not trustworthy" (better heuristic).
Once you have ruled out a publication being in a trash journal, reputation is only a very minor factor in consideration, and methodological and substantive issues are what matter.
It all depends on whether the paper fits the journal. Minor journals serve a useful service as a repository for minor results. And minor results are still worth publishing because they might provide a detail or technique later needed for a major result. The thing to be wary of is when you see a stunning result that should really be in _Nature_ or _Science_ in some minor journal. Why isn't it? Was it submitted there first and rejected? It would be nice if the history of a manuscript (and its peer review) stayed with a manuscript so you could see if the authors really corrected problems brought up by peer review or were just spamming journals with a flawed manuscript until they found one that published it.
Agree with all this. Once you've filtered / made decisions of quality based on the more substantive criteria, journal reputation can provide useful additional information / context. The case you mentioned is a good example.
There is no reasoning. I gave you a statement of fact:
I have personally seen highly talented and successful researchers use the heuristic of journal quality when looking at the state of their field. These people are highly competent by any standard. If you want to play word games with negation you could say they are not not very competent.
> IMO, academics that do this are not very competent, because ...
Where's the cry-laugh emoji when I need it.
Of course academics check where stuff is published. Please...
There are still real journals put there, although you might not know which is which.
Ah, look, another smug sneer that ignores the evidence I presented, and makes another circular argument (i.e. that because academics look at rep, this is justified, even though I provided evidence disputing this).
I know what journals are better / not. But reputation only is helpful in letting you ignore trash journals, once you are out of trash land, rep is just not a very meaningful factor, and you have to focus on methodology and substance.
Where's the evidence you presented?
What are some higher-profile journals that are in fact less trustworthy in many ways?
I literally said it was posted in this thread, and a quick Ctrl+F of my username on this page would have found you it in a half second: https://news.ycombinator.com/item?id=47249236
>The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
This is actually what ruined my respect for Academia.
My Science PhD buddy looked at the journal title and the claim, then said: "Its true!"
I look at him with horror. Who cares about the journal, I want to know data and methodology.
I've basically never forgiven Academia since this. I see even Ivys put out bad research and journals will publish bad research (Replication crisis and the ivy fake psychology studies)
For outsiders, there is a prestige to being a PhD or working as a professor. Now that I'm mid career and lived through the previous events I mentioned + seeing who stuck with academia... These are your C grade performers. They didnt get hired by industry, so they stayed in school. They are so protective of their artificial rank because they cannot compete in Industry. Its like being the cool person on the tennis team. They are locally cool, but not globally cool.
> This is actually what ruined my respect for Academia.
Spoken like someone who never went through grad school at a competitive R1 program
It was already a grueling 60-80 hour grind every week with frequent all nighters, high-pressure deadlines, absolute minimal pay, thankless duties, and plenty of politics. It's about the same for professors too.
We already paid our dues by helping peer review (for free) a half dozen papers for each one we submitted. Why should we be expected to review random papers on arxiv too...?
I went to an R1 university. Most students did not have a 60-80 hour grind. If they did, it was because of an overbearing advisor. Years later, those students are not ahead of those who had a more relaxed advisor.
And chances are: Those overbearing advisors are very invested in the current system.
it varies enormously by field.
in CS you will have intense grind weeks around conference deadlines and a more manageable but challenging pace of life otherwise.
in wet lab science you live by the schedule set by your experiments, which often involves intense hours.
>Why should we be expected to review random papers on arxiv too...?
The GP is not saying to review each paper you read or cite. They're complaining that a colleague accepted a claim after just reading the title and where the paper was published. Between that and doing a full review there's surely a world of options.
The problem is not that he was not willing to review it. It was that he was willing to conclude it was true. If he had said "that is interesting" or "that is plausible" or whatever, that is fine. It is concluding it is true that is the problem.
I don’t think folks in academia have come to terms with how much the above attitude has completely and nearly entirely undermined the credibility of the entire scientific and academic community in the eyes of the general public.
You don’t need a degree to understand how much utter junk science is being published by those who think they are superior to you. Just read a few actual papers end to end and look at the data vs conclusions and it becomes totally obvious very rapidly that you cannot “trust the science” since it’s rarely actual science being done any longer.
The academic community has utterly failed at understanding they needed to cull this behavior early and mercilessly. They did not, and it will be generations at best to rebuild the trust they once had. If they ever figure out they need to.
Things are going to get much worse before they get better. You can’t take any published paper at face value any longer without going direct to primary sources and bouncing it off an expert in the space you still trust to give you the actual truth.
I fear you are right here, and that the problem is far more dire than much of academia realizes. I know enough highly intelligent people (some even with family / spouses in academia, surprisingly) that are otherwise very e.g. left / liberal / progressive and open, that are still basically saying academia needs to be gutted / burned down.
I've no idea what the actual stats are on faith in academia overall today, but I don't think it is looking good.
On the whole you should rarely read papers, you want to read a whole literature in an area. Academics embedded in the field can do this easily. Academics outside of an area know to do this, and to bounce things off an expert to make sure you have the context and aren't over-indexing on a flashy result. Everybody learns the painful lesson in grad school to not just read a paper and believe everything will work as it says.
Somehow the general public and policymakers got the idea that if a paper gets published in any non-fake journal, this is an official endorsement that it's 100% correct, everything in it can be read in isolation, and it's safe to use all claims in the paper to direct policy immediately.
I think academia is partially to blame for encouraging people to believe this rather than insisting on explaining the nuances of how to interpret published research. On the other hand, nobody wants to hear a message that things are nuanced, and they will have to do costly hard work to get at the truth.
I think a world where "you can take any published paper at face value...without going direct to primary sources and bouncing it off an expert in the space" would be great, but it never existed, and it's just fundamentally impossible.
I wouldn't be surprised if the parent's complaint about his academic buddy who didn't read the paper's methods yet declared their findings as true, had misunderstood why his friend did so... which could have well been due to their additional knowledge about similar past findings/studies.
Go read /r/LawyerTalk and enjoy the horror of the dawning realization that this is a lot of professionals. I think it's an issue that stems from getting too deep into the minutiae of the technical and cultural matters of one's field; you become a really good scientist, or lawyer, or SWE (by the standards of scientists and lawyers and SWEs), and end up coming to conclusions that everyone outside the bubble looks at and says, "That's absolutely asinine." Well, laymen just don't understand the details, you know? (Even though the whole point of these professions is to provide services to laymen, fix problems laymen come to them with, and guide laymen to make practical and logical decisions when a $500/hr appointment isn't called for.)
These people take themselves too seriously, and other people only take them seriously when there are material ramifications for not doing so. Otherwise, they're viewed as pompous busy-bodies and don't do themselves any favors by playing to the role.
>It was already a grueling 60-80 hour grind every week with frequent all nighters, high-pressure deadlines, absolute minimal pay, thankless duties, and plenty of politics.
You know what else works really hard? A washing machine. Hard work alone doesnt create value. I could give you a spoon and tell you to dig a hole, or I can teach you how to use a Digger.
Some things are hard because you overcomplicate them. Some things are hard by their very nature.
Unless you are a Claude Shannon type, adding fundamental new knowledge to humanity's corpus is generally actually hard - at least in science & engineering. If you feel differently, I look forward to reading your groundbreaking papers!
Weirdly, I do have my contributions to science. I run a pretty popular blog, 250k-1M users per year.
Academia will refer to my stuff. Various levels of the US government use my data.
To be honest, I think I got lucky + I was a (hardcore) Stoic for a decade + my hobby was scientific.
> You know what else works really hard? A washing machine. Hard work alone doesnt create value.
My washing machine creates a lot of value for me. The time it saves me is incredibly valuable.
Most machines that work really hard are valuable because they free up time.
This wasn’t the clever burn you thought it was.
Its a line from National Lampoon's Xmas Vacation.
Value is what you're willing to pay for something.
Laundromats aren't particularly profitable businesses.
Laundromats are the best business there is and are extremely profitable and seldom to never go out of business - you should look this up, it is fairly fascinating.
The laundromats around here closed down years ago. Not a characteristic of extremely profitable businesses.
There are a couple of strip malls nearby that have vacant rentables, vacant for years and years. Nobody has thought to put an extremely profitable laundromat in any of them.
This is anecdotal - look this up outside of your strip malls. It has the some of the lowest failure rates of just about any business and 20-35% ROI
I've heard that laundromats are used for money laundering. This may explain their profitability, but it's a front, not a business.
Complete hogwash of a comment, based almost entirely on your limited experiences, to denigrate academic scientists.
If you even knew these people, you'd know that most that remain in academia never considered industry in the first place. These people were not rejected by industry. In fact, it is the other way around. *They rejected industry*. They did so, despite knowing they'd make more money, but chose to remain in academia because they wanted to spend their life pursuing research topics that interested them with independence. Sometimes they feel the fool when money is tight and the hours are relentlessly long, but never have I seen it happen because they were rejected by industry.
I just checked in case it had changed, but Arxiv is nowhere near as free-for-all as you imply.
Any crank who learned to use LaTeX is not allowed to post articles willy-nilly. You need endorsements in the field.
There's also a middle ground, i.e., renowned publishers who aren't free but still publish everything as OA. One example is Dagstuhl Publishing for CS research papers.
Maybe studies could be dual published in open access publications and private.
Then you get the private branded badge social proof and access can continue.
Also, til anyone can publish to arxiv.org?
> The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything
I do some due diligence work from time to time. Uploading to arXiV is becoming a favorite tactic from companies trying to look impressive for investors. I’ve read a lot of “papers” submitted by startup founders that are obviously ChatGPT written slop uploaded to arXiV. They then go to investor and show their record of “published research”. Smart investors are catching on but there are a lot of investors who associate journals with quality and filtering and assume having a paper on there means something.
The filtering and curation problem is real. It seems like academic pettiness or laziness from the outside, until you see the volume of bad “papers” that everyone is trying to publish to chase the incentives.
Just putting it on arXiv does not automatically make it OA. It needs a permissive license.
I think people in this post are using arXiv as sort of metonymy / stand-in for OA here, but, yes.
I have had so many "why don't you just" conversations with academics about this. I know the "why don't you just" guy is such an annoying person to talk to, but I still don't really understand why they don't just.
This article pointed to a few cases where people tried to do the thing, i.e. the pledge taken by individual researchers, and the requirements placed by certain funding channels, and those sound like a solid attempt to do the thing. This shows that people care and are somewhat willing to organise about it.
But the thing I don't understand is why this can't happen at the department level? If you're an influential figure at a top-5 department in your field, you're friends with your counterparts at the other 4. You see them in person every year. You all hate $journal. Why don't you club together and say "why don't we all have moratorium on publishing in $journal for our departments?"
No temptation for individual research groups to violate the pledge. No dependence on individual funding channels to influence the policy. Just, suddenly, $journal isn't the top publication in that field any more?
I'm sure there are lots of varied reasons why this is difficult but fundamentally it seems like the obvious approach?
> If you're an influential figure at a top-5 department in your field ... you all hate $journal.
That's the problem, they don't hate these journals, they love them. Generally speaking they're old people who became influential by publishing in these journals. Their reputation and influence was built on a pile of Science and Nature papers. Their presentations all include prominent text indicating which figures came from luxury journals. If Science and Nature lose their prestige so do they (or at least that's what they think)
This was very apparent when eLife changed their publishing model. Their was a big outpouring of rage from older scientists who had published in eLife when it was a more standard "high impact" journal. Lots of "you're ruining your reputation and therefore mine".
Maybe I am underestimating the gap in status between the "influential figures" I imagine and the people I actually know.
I see: my friend has 10-15 years of experience in their field, they have enjoyed success and basically got the equivalent of a steady stream of promotions.
I map this onto my big tech/startup experience. I mentally model them as: they are "on top of the pile" of people that still do technical work. Everyone who still has the ability to boss them around, is a manager/institutional politician type figure who wouldn't interfere in such decisions as which journal to publish in.
But probably this mapping is wrong.
Also, I probably have a poor model of what agency and independence looks like in academia. In my big tech world, I have a pretty detailed model in my head of what things I can and can't influence. I don't have this model for academia which is gonna inevitably lead to a lot of "why don't you just".
Same thing happens to me when I moan about work to my friends. They say "I thought you were the tech lead, can't you just decree a change?" and I kinda mumble "er yeah but it doesn't really work like that". So here I'm probably doing that in reverse.
it has been known to happen.
For example, spearheaded by Knuth, the community effectively abandoned the Journal of Algorithms and replaced with with ACM Transactions on Algorithms.
however it's difficult. a big factor is that professors feel obligated towards their students, who need to get jobs. even if the subfield can shift to everybody publishing in a new journal, non-specialists making hiring decisions may not update for a few years which hurts students in the job market.
I think the call for top-down policy makes sense b/c otherwise this is like every other tragedy of the commons situation. Each of those top-level researchers also has to think, "my department has junior faculty trying to build their publications list for tenure, we have post-docs and grad-students trying to get a high-impact publication to help them land a faculty job, we have research program X which is kind of in a race with a program at that other school lower down in the top 20. If we close off opportunities with the top journals, we put all of those at a competitive disadvantage."
For the grad students especially, there’d be a career advancement incentive to still publish in the top journals. The professors might still want to publish in them just out of familiarity (with a little career incentive as well, although less pronounced than the grad students).
I think it’d be a big ask from someone whose role doesn’t typically cover that sort of decision.
There are hundreds of reputable research universities around the world. Top-5 departments can't meaningfully change the culture of a field on their own. Top-100 perhaps could, but the coordination problem is much bigger on that level.
Grant funding reporting requirements. It would be easy to say self publish for free via the institutional library. But the NIH would not like that use of their money.
I like the author's idea:
> So the solution here is straightforward: every government grant should stipulate that the research it supports can’t be published in a for-profit journal. That’s it! If the public paid for it, it shouldn’t be paywalled.
The article then acknowledges this isn't a magic solution to all the problems discussed, but it's so simple and makes so much sense as a first step.
I'm no expert here and there are probably unintended consequences or other ways to game that system for profit, but even if so wouldn't that still be a better starting point?
I think that's also a good proposal, and I don't think it conflicts with the "prestigious departments stop publishing in $journal" idea at all. Probably we want both.
Only difference is that the author is writing for a wide audience and his best angle to change the world is probably to influence the thinking of future policymakers. While I am just an annoying "why don't you just" guy, my "audience" is just the friends I happen to have in prestigious research groups.
Adam M also probably has lots of friends in prestigious research groups (IIUC although he complains a lot about academia he was quite successful within it, at least on its own terms). And the fact that he instead chooses to advocate government policy changes instead of what I'm proposing, is probably a good indication that he knows something I don't about the motivatioms of influential academics.
Imagine being a scientist and reading “if you take this grant, you cannot publish your results in any of the most prominent journals in your field.” Sounds good?
But IIUC there are entire fields where basically the whole US ecosystem is funded by federal grants. So if this policy gets enacted those journals are no longer prominent.
(Maybe you'd need an exception for fields where the centre of mass for funding is well outside of the US, though).
The result is that open access journals would very rapidly, perhaps instantly, become prominent.
I explain here (https://news.ycombinator.com/item?id=47250811) but tl;dr it's because Universities need this system to get money and to give money. Nobody has yet proposed a solution which solves the money/prestige problem. With no money there's no research.
Worth pointing out a success story: all ACM publications have gone open access starting this year[1]. Papers are now going to be CC licensed, with either the very open CC-BY[2] license or the pretty restrictive (but still better than nothing!) CC-BY-NC-ND[3] license.
Computer science as a discipline has always been relatively open and has had its own norms on publication that are different from most other fields (the top venues are almost always conferences rather than journals, and turn-around times on publications are relatively short), so it isn't a surprise that CS is one of the first areas to embrace open access.
Still, having a single example of how this approach works and how grass-roots efforts by CS researchers led to change in the community is useful to demonstrate that this idea is viable, and to motivate other research communities to follow suit.
[1]: https://authors.acm.org/open-access/acm-open-for-authors-hom...
[2]: https://creativecommons.org/licenses/by/4.0/
[3]: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
That works nicely if your institution participates in ACM Open (no such institution in my country, and no, my country is not in the list of lower-middle income countries).
The combination of 'publish or perish' with 'pay for publication' and 'miserly grant money' is deadly.
While in theory the idea is nice, in practice this is a problem (maybe not in most rich countries, but here definitely).
Nowadays, you could always get the article you are interested in, even if it is beyond a paywall. Hence, perversely, the old model (which I hate, for reasons well explained in the original post) worked better for me. :-(
> Robert Maxwell, one of the architects of the for-profit scientific publishing scheme. When he later went into debt, he plundered hundreds of millions of pounds from his employees’ pension funds. You may be familiar with his daughter and lieutenant Ghislaine Maxwell, who went on to have a successful career in child trafficking.
Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.
A good biography of Maxwell - "Fall" by John Preston.
And he apparently stole £763m...
That reminds me of the recent term "Epstein class". It seems more and more fitting.
> So the solution here is straightforward: every government grant should ...
People who write such sentences have no idea what they are talking about or are being intentionally naive for whatever reason.
Just because your one-sentence solution reads simple doesn't make the actual solution simple. Because such a solution involves changes to laws, changes to entrenched interests, changes to distribution of money involved in the whole system, and changes to balance of powers between stakeholders. Unless the push for such changes is significant enough to overcome the current state of affairs (due to public opinion, redistribution of power or money, etc.), nothing will happen.
A solution to a problem that doesn't change the current state of affairs, which by your definition makes it a simple solution, is not an actual solution.
There are plenty of simple solutions to real problems whose only blocker is upsetting the status quo. "We have no housing...let's build more housing" is, in fact, a very simple solution. That it doesn't happen has nothing to do with it the solution itself.
This comment seems to confuse _straightforward_ with _easy_. On the merits, this proposal is well argued and has good points, and his solution—essentially extend the Biden approach with more strict requirements—makes sense.
Everything you mention will also have to happen, which means that doing this will definitely not be _easy_. That said, it is still a very _straightforward_ solution.
This defeatist attitude is why we can't have nice things anymore.
Fun fact, all of those things happened and this is already government policy for any NSF grant: https://www.nsf.gov/policies/document/faq-public-access
So maybe consider that when you give up on obvious things that are good based on some conspiracy theory that the "man" is trying to keep you down, what you're actually doing is being part of the system and endorsing it. Changes like this do happen, they just happen despite you.
Why would you claim that my statement is defeatist? It is just acknowledging the reality of how things work nowadays without saying anything about giving up. If anything, it cautions people that change will not come easily and to be prepared for it. I vote, give money to political/public causes, to go meetings of my federal representatives, post my opinions on HN, ... I am just being more realistic about my expectations.
You sound like my parents. As I get older, I drift less into this mentality, and more into "I am tired of this defeatist bullshit, and accepting corruption and stagnation". I'm going to leave the world a better place, and never give in to this. I will vote for and donate to candidates who also want to fight, and run myself.
> I will vote for and donate to candidates who also want to fight ...
Fight for what? I bet to change one or more of the things that I have mentioned above. I have said "nothing will change" unless these things change; I didn't say you/we shouldn't do anything.
In astrophysics we now have Open Journal of Astrophysics that is basically an overlay on top of arxiv. https://astro.theoj.org/ It is now catching on with ~ 200 papers published last year, after some of the astro journals started to charge the Golden Open Access publishing fees. I think now people realize how crazy it is to pay for hosting a PDF and for sending your draft to a few referees without paying anything for their work.
Is the goal to get rid of the journals or ensure open access? Because the US already has open access mandates for federally funded research. Immediate and without embargo. https://www.lib.iastate.edu/news/upcoming-public-access-requ...
it is open access, but often 'golden' or 'diamond' OA, where the researcher has to pay few K$ for this.
This is discussed in the article.
The article's discussion of how the open access mandate works is wrong. Federally funded research, when published (even in a closed-access journal) must be deposited in https://pmc.ncbi.nlm.nih.gov/ or a similar repository.
Edit: OA advocates have won pretty much everything we wanted, there's not much left to be outraged over.
But that's what the article says? The (correct) criticism of that system is that the publishers just replaced the subscription fees with article processing charges for OA publication, and still profit from public funding the same as before.
> Now people barely bring it up at all. It’s like a lion has escaped the zoo and it’s gulping down schoolchildren, but when people suggest zoo improvements, all the agenda items are like, “We should add another Dippin’ Dots kiosk”. If you bring up the loose tiger, everyone gets annoyed at you, like “Of course, no one likes the tiger”.
Is the switch from lion to tiger in that paragraph intentional? If only the quote from "everyone" was switched I might think it's intended to convey people trying to derisively dismiss the issue or something but it does specifically reference the tiger rather than the lion being brought up, so I was confused when I got to this part of the article.
This is not how computer science publishing works, however. Post it on arxiv, submit to a conference, get 3 peer reviews, accepted, “published”. 99% of papers are effectively open access for free.
The title of the article of "science" not "computer science".
Yes, and it opens by talking about STEM fields. I consider CS part of both STEM and science generally.
> Academia is so cutthroat that anyone who righteously gives up an advantage will be outcompeted by someone who has fewer scruples. What we have here is a collective action problem.
And what, pray tell, is this advantage? If there is no utility to anyone in publishing in Science or Nature then how can it be an advantage.
I suspect it’s simply that these guys are a curation service. They separate the cranks from the science. They can be imperfect at this so long as important people separate the cranks from the science.
This kind of winnowing is pretty useful in general. Many universities are pretty much that and people pay to attend them.
It makes sense that a credentialing service would charge for the credential. It doesn’t make that much sense to say “no credentials allowed; you and timecube guy must be considered the same”. I want you to show your credentials. We all do because science is an empirical field and empiricism depends on facts. I cannot process your paper with pure reason.
If you looked under the microscope and saw light I don’t know that you didn’t. At scale I need someone to figure out “this wasn’t a Photoshop situation; that’s totally fluorescence”. Arguing that we should remove these organizations is similar to saying you should remove diplomas and so on. Anyone can continue to attend universities. They just can’t hand out diplomas. So no credentials. Only learning. Simple thing. Or perhaps not so.
The credential is the useful thing.
SciHub is an amazing resource. I have read so many papers from varied topics out of my personal interest. That would not be possible for me without SciHub. The hold on new papers has stopped me from keeping current. If I were able to also publish papers that others could review that opens "science" to everybody. Then the only benefit of research institutions would be a concentration of big brains. That completely changes the landscape for scientific progress.
SciHub has shown us a new way to spread knowledge to all that are interested. I don't have the rigor for publishing but other individual experimenters might. It would be great if they could contribute to building human knowledge.
I think the only real solution is a distributed federated publishing and review platform. A node would be a library of papers for the host's interests. Just like physical journal collections, bigger institution would host more topics. Anybody can participate in the publication and review process. SciHub nailed storage and retrieval. Review is the hard part. Any rating system can be gamed. It would be very hard to convince people it is trustworthy.
There shouldn't be any prestige in publishing a paper. The prestige comes from being proven correct, from building our knowledge.
Arxiv isn't the solution. But i think computer science conferences are. These have the same scientific rigour and standards in the review process as journals in other scientific fields, but don't price gouge. Yes, conferences are also a bit expensive, but you get a lot for your money, and they usally aren't out to make a big profit.
The peer review process in CS is pretty bad, especially in conferences with a single round of reviews. When you combine this with high rejection rates, peer review becomes more about finding excuses to reject a submission than about trying to improve it.
Conferences also don't work that well as publication venues, as they often require that one of the authors must attend the conference physically. And it's not as much about money than about visa policies and travel restrictions. Even in the 2000s and 2010s, when international travel was easy, people from non-Western countries could often not get visas to attend conferences. And today the situation is much worse.
I've been to three international conferences in the past year. One was held in Europe. People from Russia and Israel had to present remotely, the former due to an ongoing war and the latter due to an unexpected war. Another was in the US, and there were fewer Europeans than usual, as many were not willing to take the risk. And the third one was in Japan. People from China could not attend due to increased tensions. People from Israel were there, but they were worrying if they would make it home before the next war. (They made it, and the war started a day or two later.)
Conferences can be truly wonderful, but not a universal replacement for publishing.
If you think journals are expensive, try sending your whole lab to a conference in another country. That may not let you in. Where some of the attendees have to fill out paperwork before talking to a foreign national. (does that ever make for awkward small talk...)
For all their many faults, journals provide access to a really wide audience, and- in theory- make it possible to form connections who wouldn't be able to meet directly.
Acknowledging, I am not a expert in this stuff, here is an idea: getting momentum for these sorts of things is so important, what is the journal that would be easiest to make a big example of, so that everyone understands that it is possible? Just completely mercilessly drive them out of business, and then hound their executives when they try to get other jobs. It appeals to peoples base instincts, but the last 10 years have shown those are pretty powerful. Then the movement which has formed around that can take down progressively bigger journals. Probably want a different organization building the alternative; the people with the personality to fight at the Vanguard of the revolution don’t tend to be great at building in the long-term.
Journals are not about providing access to science, much less public access.
Journals are an academic-career-advancement service. It therefore makes sense that they do not pay academics. You don't pay your customers.
That means they need to generate a secondary customer base elsewhere, who will pay. Those secondary customers happen to be the employers of the academics who are the primary customers. That socializes the cost of providing the service, since academics individually wouldn't be willing and able to pay.
Once journals have established a reputation, their policies and paywalls and fees are the result of trying to signal exclusivity and set an optimum market price.
Until the supply side of the research market largely agrees on a way to use open-access repositories like arXiv as a primary career-advancement signal, complaining about closed-access journals is tilting at windmills.
Changing the law to prevent journals from being able to copyright anything could potentially force the research industry to rapidly develop a new solution, but at the cost of short-term chaos and career instability for new academics.
Science has never been free, and it isn't mostly progressive; like the bulk of the population, it is hyper-conservative without admitting it. So, the first flaw lies in the very social structure of those who practice science.
The second problem, however, is a modern one: the pure, naked, and raw commercialization of science through "publish or perish", whereby the researcher is a Ford-style assembly line worker to be managed and who must be replaceable.
Without a MENTAL paradigm shift, even before a material one, we will only be able to plug small leaks on a ship with a torn hull.
This post misunderstands why the journal system exists, and why they work this way. It is not because there is an evil corporation controlling everyone else.
Universities do the research. They voluntarily choose to pay for and publish in journals. They could just decide not to do this. Literally, just don't publish in a journal, anymore, ever. Upload your study paper to a Dell Inspiron sitting in a closet in the university faculty lounge, connect it to the internet. Done. Why don't they do this?
#1. No guarantee anyone will look at it. You just spent years of time and money to come up with a research conclusion. Will anyone read it? Comment on it? Learn from it? Is it any good? Will anyone review it? It's after all just a paper sitting on a server. Without some kind of process to vet it independently, and publish it in a place where people can find the latest vetted papers, it's too much hassle for most people to ever find, much less trust.
#2. The reputation feedback loop. Universities give research grants to "well respected" researchers. You become a "well respected" researcher by having academic achievements. You get academic achievements by... doing research that gets published in a journal. Universities depend entirely on the prestige of the researcher and journal to decide who gets a grant. Because...
#3. Money is hard to get. In order to convince someone (government, private donor) to give your university money, you have to show them it's worth it. And the way they show that is.... the prestige of the researcher, being published in the prestigious journal. Look, we have cool peeps, publishing in cool journals! Give us more money!!
Therefore, the reason journals still exist, is Universities desperately need them. They don't want to pay an insane amount of money to a journal. But they don't really have a choice.
Could Universities replace journals with something else? Well, they could work hard to replace the "prestige machine" with other processes (which must enable them to get money, by showing their researchers are good, with vetted papers, published somewhere people will see them). They could replace the journal system with their own intra-university system. But it turns out, that costs a considerable amount of money, time, and resources... which is entirely what the "evil journal publishers" do. Universities would have to spin out their own entire corporation to do all that work, which would be a journal publisher. They know this is expensive, difficult, time-consuming, and they also know the existing system benefits them.
"Let's just throw papers on arXiv" does nothing to solve the money and prestige problem. So the world continues to turn as it has.
i am very glad to see others (presumably non-scientists) in this thread dunking on the false paradigm that "peer review = true". anyone who peddles this notion is naive or a moron.
while the author is correct that the for-profit publishing is definitely a negative externality, i can't help but feel they are missing the forest for the trees when it comes to all the other worse issues in academia.
a full explanation of which would be much too onerous for a hn comment, but in no particular order: rampant scientific fraud, waste of tax payer dollars, wage suppression via "students" and visa-dependent laborers (J1 visa abuse), publish or perish evaluation criteria, lack of management training, blatant and rampant racism, etc. etc. etc.
the whole system needs to burn down and be rebuilt from the ground up.
My experience with grad school is that they are shockingly stuck in their ways when it comes to organizational practices. They make even large tech companies look nimble.
Though at least in my field part of that is budgets are so tight it seems like most of the effort is needed to just keep the lights on. I don't see anyone who has bandwidth to help burn things down or rebuild in my department as much of the staff are already working unpaid overtime (and good luck getting funding for hiring many more).
I don’t understand why people care so much about the cost of journal subscriptions. If we add up all the revenue from all major scientific journal publishers, is that a big number in the context of the national economy? Or even compared to one major tech company?
I feel like this is one of those classic local minima where a community starving for resources fights vociferously amongst itself because they have internalized that they can’t win externally. From where I sit outside academia the problem with science seems obvious: there is not nearly enough money going into it.
I doubt bringing the heads of for-profit journals would change that under current national conditions in the U.S.
The big pain point is being between jobs where you lose your institutional access to the scientific literature.
I think the solution is to launder all research papers through LLMs so the papers are no longer copyrightable, and let the rich journal owners fight with the LLM owners.
we should
1) pay reviewers. 2) you can't publish unless a reviewer replicates your work.
yes. It can be done.
https://www.orgsyn.org/
Like some other posters here I think that a paid service is probably a necessary evil for long term quality regulation (although currently it skews too much into evil)
And a non-profit can provide such a service.
Tax payer momey should always go to creation of open source software for everything
Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it.
It doesn't. Not even close.
Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does.
Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1900's to deal with the fallout from another mistake ("publish or perish") that meant people had to try to publish even if they had nothing to say.
> Darwin's work wasn't peer reviewed. Nor Einstein's
Except it was…? This is absurdly ahistorical and the fact that you cross disciplines in trying to make an incorrect argument questions whether you are in science at all.
The structure of peer review in Darwin’s time was different, where experts wrote monographs and gave lectures at symposia that then led to letters among their peers. Which is what happens now, if you take a step back.
The volume of new work these days is incompatible with the older informal system, and is in some ways our new paradigm is superior as there is a formal period in which new works are reviewed.
Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. I thought that was obvious.
What you're calling "peer review" is what I would call "discussed" or "debated" which it certainly was.
I dispute your claim that the new paradigm is superior.
> Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used.
Accepted. But now there is Arxiv and Biorxiv and even Medrxiv—so we're back to where things were, it seems.
> Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it.
No actual working scientist thinks this.
“Glitchc” has it right elsewhere in this thread: the motivating force behind journals is prominence and reputation, not truth.
Ah, but the naive public still broadly believes in peer review, and that high profile journals do good review. And the prominence and reputation that comes from these journals arguably then relies on this (increasingly false) public perception.
Would scientists feel the same if the public was more educated about how bad journals and peer review are? Not so easy to disentangle IMO.
The naive public does not believe anything in particular about peer review. They think new scientific results are significant when they read about them in the popular media, that’s it.
People who do need to work professionally with peer review, do understand what it actually does and its limitations.
You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
> The naive public does not believe anything in particular about peer review
You'd need to provide evidence or an argument for this. The media reports on things in part based on journal prestige, and likely when questioned, people will say they can trust such things because good scientists have looked at the work and say it is good. This would be an implicit belief that peer review is generally working well, even if they don't use the term "peer review".
> You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
Extremely presumptuous, as I work in this system, and have provided plenty of evidence for my claims. You've provided only sneers.
You've provided evidence that prominent journals experience retractions, fraudulent results, etc. All true. But it is not the job of peer reviewers to decide what gets published.
You've provided evidence that peer-reviewed science often turns out to be incomplete, inaccurate, wrong, fraudulent etc. All true. But it is not the job of peer reviewers to assure completeness, accuracy, or freedom from fraud.
A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.
The check on published scientific results is the scientific process itself, not the publishing process. Prominent results attract further investigation, which confirms or disproves the reality of the underlying phenomena. Again: that's not the job of peer review.
Do some people ascribe too much authority to peer review? Yes, for sure. IMO your comments in this thread are exacerbating that problem, not addressing it.
> A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.
Um, what? I have done all these things in reviews, and know other academics that have done these things as well. More confusingly though, if you are saying most reviewers don't do these things (which I agree with), this would only strengthen my point?
I'll let readers decide if it is my comments that exacerbate the problem, or if, perhaps, it is apologism for journalistic peer review that might be causing bigger issues in the present day.
Would be interesting if you would be willing to share a paper you reviewed and detail your review process of it. I don't see how one could check primary data or interrogate scientists in a blind review process, for example.
This is IMO just bad faith sealioning, you can look at the whole replication crisis in psychology and social science (esp. the work of people like Nick Brown and the GRIM test, or Uri Simonsohn), or sites like Retraction Watch, and see clear evidence of everything I am saying. There are endless papers in ML research going into issues with test datasets and data duplication, etc. In plenty of cases all data and code is made open, so it is trivial to check data issues and methods.
Also, review is back and forth, and has rounds: you almost always interrogate the scientists of the paper you are reviewing, this almost like the definition of peer review. I don't think you have any idea of what you are talking about at all.
EDIT: Heck, just hop on over to https://openreview.net/ and take a look at the whole review process for some random paper (e.g. https://openreview.net/forum?id=cp5PvcI6w8_)
Don't know why you are being downvoted, you are largely correct. I've provided plenty of evidence in another post in this thread showing that journal-based peer review is highly farcical.
EDIT: I still want review from a community of scientific peers. I just don't want this review to be in the hands of a tiny number of gatekeepers entangled with journals that largely just slow things down.
Because a lot of people are deeply invested in the present system perhaps? As the article pointed out, there's a lot of money involved, and there are a lot of people who've built their lives around flourishing in the existing system, cut-throat as it may be.
> Because a lot of people are deeply invested in the present system perhaps?
I mean, right, yes, of course. Much of the downvotes are cognitive dissonance, obviously. I suppose I meant the question rhetorically.
The OP is exactly right, in my view: the current charade of paywalled-journal "peer review" is broken beyond repair.
See also from the same author:
* https://www.experimental-history.com/p/the-rise-and-fall-of-...
* https://www.experimental-history.com/p/the-dance-of-the-nake...
* https://www.experimental-history.com/p/lets-build-a-fleet-an...
There should be a journal where it only publishes studies that have been replicated. Too much research slop is being generated for journals and we already know we have a severe reproducibility problem in science right now.
Based on title, I was figured it was referring to USA moving to the metric system
U.S. scientists already did, at least at work. They still drive home at imperial speeds though.
I thought it was USA custom metrics? Not less ridiculous, I guess.
The problem with communists always seem to be their math is ridiculously bad. 1 billion in profits to the journals seems like a rounding error in an industry with almost 1 trillion in annual spending.
This is a fine example of where someone's understanding of the problem runs ahead of their understanding of the solution.
A few scattered thoughts:
1. There is a difference between pre and post publication peer review. These discussions almost invariably conflate the two, but part of the runaway success of spam journals is that the benefits of pre greatly outweigh the risks of post. Historically, there was some link: if an article had problems, you would open the table of contents n months later and (might) see a letter or further discussion. Now, the table of contents is google, and many readers have weaker links to the same venue over time for followup. At the metrics level, the reputational hit of bad articles is weaker. (studies have shown that retractions are often cited with the original intent years after a correction was published)
2. The phrase "for profit" is doing a lot of work in this article. Some mega publishers, like ACS, are technically non profit member societies stapled to a mega-publisher, and have been strongly opposed to OA policies in the past. [1] https://en.wikipedia.org/wiki/American_Chemical_Society#Cont... [2] https://www.acs.org/content/dam/acsorg/about/aboutacs/financ...
3. Outsourcing trust to someone who isn't the current evil... will only get you so far. No matter who takes over publishing, scientists are going to need to evolve new ways of evaluating work and each other, as the field grows far beyond what a small network can handle. Journals are a bad metric, but how does your dean evaluate 50 people hired to be the world's leading experts on (new and emerging field)? I've read plenty of these publisher=bad screeds, and most stop there. PubPeer exists for some, Twitter walkthroughs of papers were a great thing for a while, or there's also talk of overlay journals that decouple the act of publication (as a preprint) from the review-and-prestige piece.
4. The current system does two things: (a) provides a record of work done by students, who may labor under graduation requirements to publish something, whether their project is successful or not, (b) a shared record of current state of human knowledge, be it from researchers at a small college, or google, or pharma. Goal (a) puts a lot of pressure on peer review in "low tier" journals that even the reviewers don't like to cite, and I've had my doubts as to whether this is the best yield for effort.
The other factor preventing a fix is that people with no actual serious experience of academic publishing and peer review will defend these journals, because they still think that (journal-based) peer review acts like some kind of meaningful quality filter. But, it really doesn't.
Because someone is surely going to try to defend journals via peer review in this thread, I want to provide a counter to the arguments that journal peer review does much good. Also, since everyone knows that if you just go to a poor enough journal, you can be published, I am going to focus on the (IMO mostly false) claim that higher-profile journals are still doing a good thing here.
There are numerous studies showing that higher-profile journals in general have more retractions and research misconduct [1-2], lower research quality [3], in fact weaker statistical power and reliability [4], and that statistical reliability even in high prestige journals is still extremely poor overall [5]. Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
We should just move to arXiv-like approaches and allow the scientific community to broadly judge relevance and quality. Journals just slow things down and burn funding for very little gain or benefit to anyone other than the journal owners.[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC3187237/
[2] https://www.pnas.org/doi/10.1073/pnas.1212247109
[3] https://pmc.ncbi.nlm.nih.gov/articles/PMC9382220/
[4] https://journals.plos.org/plosbiology/article?id=10.1371%2Fj...
[5] https://www.frontiersin.org/journals/human-neuroscience/arti...
[6] https://journals.plos.org/plosone/article?id=10.1371%2Fjourn...
[7] https://blog.neurips.cc/2021/12/08/the-neurips-2021-consiste...
> higher-profile journals in general have more retractions and research misconduct [1-2]
Given that reviews are not a mechanism to check for truth but soundness, the higher profile the thing I would imagine there would be more misconduct. I mean would one risk prison to steal 10$ or to steal 1 million $?
> lower research quality [3]
To cite exactly from your link "the evidence is mixed about whether they are strongly correlated with indicators of research quality.". I find saying "lower" a bit too strong given the original quote.
> in fact weaker statistical power and reliability [4]
For a specific field "cognitive neuroscience and psychology papers published recently"!
> statistical reliability even in high prestige journals is still extremely poor overall [5]
According to https://www.frontiersin.org/journals/human-neuroscience/arti... they kind of targeted bio/medical/psychology field for the analysis. Which seems to me very focused to be able to draw general conclusions.
> Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
It's a coin toss if paper could get accepted at all, and that's less than ideal but what the system should do (at least) is reject obvious crap, not ensure that something gets clearly accepted. The danger is False Positive (accepted even if it's crap) rather than False Negative (rejected even if it might be something useful).
Overall note: the review system is not ideal and should be improved. But it's a hard, complex and delicate problem.
Oh, I agree this is all super complex and delicate. If I had more time, I'd love to write a more nuanced, many-thousands-of-words blog post going into which journals and fields actually have good peer review and can be more / less trusted.
I just wanted to make a strong rhetorical case by highlighting some things that might be surprising to people making more naive defenses of journals via peer-review-based arguments.
I am sympathetic to the argument you wish to make, that peer review is no panacea, but the actual evidence you offer has nothing to do with this claim.
You are trying to say that high profile journals have more retractions, which is well known as you share.
How does that have anything to do with peer review? Are you saying that there is more review or less review in some cases and that influences retraction rate? In what evidence? In what world does the arxiv system moderate this discrepancy?
> How does that have anything to do with peer review?
I already addressed this. People know peer review can be bad, but some think "good journals" still do good peer review. This is not so clear.
> In what world does the arxiv system moderate this discrepancy?
Open systems allow the scientific community to figure out ways to properly assess research quality and value more cheaply, and without passing through (often arbitrary and random) small numbers of gatekeepers that don't even do a reliable or good job gatekeeping in the first place.
Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review.
I understand that we want arxiv to exist, and it does, and it’s growing. That doesn’t mean we don’t want Nature or Science to triage the most compelling stories.
Importantly, we can already begin the search for these ‘cheaper’ review strategies while not losing the helpful information filter we get by seeing where things are presented/published
> Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review.
No, it doesn't. The argument is that peer review is incompetent gatekeeping in general, and so slows things down and makes thing expensive. Also, I am countering the argument "we need journals because journals do peer review" by arguing "peer review by journals isn't clearly actually good", I am not saying "peer review in general is unneeded", as I support review by the entire scientific community, rather than journal gatekeepers.
> you fail to show how doing any peer review is strictly worse than doing no peer review
I wasn't trying to show that. I have provided plenty of arguments to show why killing journal-based peer review could definitely speed things up and so potentially make things better. I want actual organic review by the community, not by tiny groups of gatekeepers.
> I want actual organic review by the community, not by tiny groups of gatekeepers.
But this happens—and good work is cited and talked about. I can't tell if you work in science, but this latter part is obvious.
I do work in science, I am claiming that pre-publication / journalistic peer review is limiting (and biasing) the amount of post-publication / non-journalistic peer review that can happen, and it is not limiting this in a very reliable or even IMO particularly desirable way.
There is definitely a problem with the over-production of junk science, and we definitely need a way to filter this out somehow. I am just claiming journalistic / pre-publication peer review does not do this effectively or reliably at all anymore (if it ever did).
Everybody hates capitalism, but no one understands that they do. What you promote as a solution is simply a pebble on the path of the people who want to capitalize.
There are 3 realistic scenarios for your proposed solution:
The capitalists design the business models (profit making schemes) and legalize them. This is not an organic development of an industry.What you hate is capitalism and capitalism will do this to any industry wherever it can attain steady profits.
I absolutely hate this style.
If you're going to write an article titled "The one science reform we can all agree on, but we're too cowardly to do" and that one thing isn't explicitly stated in the first paragraph, I'm out.
Stop with the meandering nonsense and make your argument.
[dead]
[flagged]