First, the need to publish is due to the publish-or-perish strategy, that forces authors to put out 1-2 papers per year in order to stay afloat. If you work in research, you realize at some point that advancing the state of the art in any field is hard. It takes intuition, dedication, some luck, and most importantly a LOT of time. That is, /years/ of work where nothing comes out. The world-class papers you see in Nature are works of years, not months.
Ignore bad research for a moment: imagine you had only great researchers, which is what we'd all like to sustain. This is /fundamentally/ incompatible with the current model. You have two strategies as a researcher: put out crappy work to sustain a larger project, or publish bite-sized pieces of your larger work over time, which leads to incredibly specific papers that taken by themselves seem useless. If you take the second path, it's obvious that you'll also take advantage of the weak spots of the system, and cite yourself on each subsequent paper you publish. This is common knowledge. Because you need whatever you have to stay afloat.
The first strategy (keep the bigger task going) is almost never attainable. In fact, if you need infrastructure, such as a lab of any kind, it's impossible to pull off. This is something that only a tenured researcher working in a very successful team or with good connections might be able to do. Aside from your vision, and any grant you might have, you need mindshare in your fellows and your supervisors to simply let you do it. This is much harder than it needs to be, with too many factors out of your control.
As for the "unread" output, I'm not worried. I assume there's a good 30% of papers which are just turmoil due to the above system. As said, this very system has pushed authors to publish even more, increasing artificially the number of publications one would have written. This is also placing a burden on new authors that need to do literature research. Finding all relevant articles about any subject has become a massive task, even ignoring the problem of getting the articles themselves.
But how can you tell if the published works are worth it or not? It's impossible. Consider the evolution of all the odd fields of math, most of which would have seemed just ludicrous at the time of publication for anybody but the author. Stuff like origami to fold an antenna on a space telescope. Again, if you considered all past works and projected forward, you'd realize there's no way an honest reviewer could tell "this is useless".
First, the need to publish is due to the publish-or-perish strategy, that forces authors to put out 1-2 papers per year in order to stay afloat. If you work in research, you realize at some point that advancing the state of the art in any field is hard. It takes intuition, dedication, some luck, and most importantly a LOT of time. That is, /years/ of work where nothing comes out. The world-class papers you see in Nature are works of years, not months.
Ignore bad research for a moment: imagine you had only great researchers, which is what we'd all like to sustain. This is /fundamentally/ incompatible with the current model. You have two strategies as a researcher: put out crappy work to sustain a larger project, or publish bite-sized pieces of your larger work over time, which leads to incredibly specific papers that taken by themselves seem useless. If you take the second path, it's obvious that you'll also take advantage of the weak spots of the system, and cite yourself on each subsequent paper you publish. This is common knowledge. Because you need whatever you have to stay afloat.
The first strategy (keep the bigger task going) is almost never attainable. In fact, if you need infrastructure, such as a lab of any kind, it's impossible to pull off. This is something that only a tenured researcher working in a very successful team or with good connections might be able to do. Aside from your vision, and any grant you might have, you need mindshare in your fellows and your supervisors to simply let you do it. This is much harder than it needs to be, with too many factors out of your control.
As for the "unread" output, I'm not worried. I assume there's a good 30% of papers which are just turmoil due to the above system. As said, this very system has pushed authors to publish even more, increasing artificially the number of publications one would have written. This is also placing a burden on new authors that need to do literature research. Finding all relevant articles about any subject has become a massive task, even ignoring the problem of getting the articles themselves.
But how can you tell if the published works are worth it or not? It's impossible. Consider the evolution of all the odd fields of math, most of which would have seemed just ludicrous at the time of publication for anybody but the author. Stuff like origami to fold an antenna on a space telescope. Again, if you considered all past works and projected forward, you'd realize there's no way an honest reviewer could tell "this is useless".