Publishing is the trivial part of academic publishing. Preprints already solved the problem of disseminating research results cost-effectively to everyone interested decades ago. The hard part is assigning merit.

Academics need merits to get jobs, promotions, grants, and prizes. The people assigned to evaluate the merits almost never have enough time and/or expertise to actually evaluate the quality of research. If they already know the person they are evaluating by reputation, they evaluate the reputation. Otherwise they use things like publication venues, citation counts, academic pedigrees, and earlier grants and prizes as proxies. Anything that tries to replace prestigious for-profit journals needs to provide non-expert evaluators a way of determining which published papers are likely to be of higher quality than the average.

ML, and other preprint-heavy fields like TCS, mathematics, and physics, all have prestigious venues coexisting with preprint servers. Sometimes these venues are even ArXiV overlays. I don’t see the correlation between open access and merit assignment.

I think you're missing the part where academics still submit to these journals and conferences just like everyone else. They just also put it on ArXiV or theri own website at the same time or sometimes before publishing in the bigger journals.

The majority of journal copyright agreements allow authors to post the article publicly if they choose.

I have a friend in economy, which is also a very clossed field with respect to publishing research, and he said that doing so might result in legal action taken against you, or pressure at the least.

That said, I'm in physics and everybody publishes on the ArXiv, either before or after submiting to the journal. From what I see (thanks to SciHub) the information on either of them is the same, except when there is an update it usually is only submited to the ArXiv.

The humanities are weird, people write and defend their PhD and then they can keep their PhD thesis confidential for years!?

The theory seems to be that the thesis doesn't count as a publication so you must keep it secret while they turn it into papers/book??

There are similar things in other fields, but I think formal peer review still supercedes them, at least as it's perceived. Things are changing though.

I do think unless there are some significant changes to the system there will be some tipping point where journals will start being ignored but I'm not sure how that will occur.

The journal —which you'll want to publish in if you want to make career progress— will then say they won't publish your work because it's not "novel".

So yes, perverse incentives all around.