it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments.
maybe as an intermediate step we could make available all the recordings to the peer reviewers and only offer the raw experimental data bundled in the paper publicly? maybe in the future we can have 1TB studies without breaking a sweat? maybe all the money we give to publishers can be spent on servers to archive all the primary data so at least we aren't simply filling the pockets of MBAs?
> maybe as an intermediate step we could make available all the recordings to the peer reviewers
The issue is clearly not the amount of data available to peer reviewers considering it's already easy to detect major flaws in a quarter of published peer reviewed research. The issue is that peer reviewers do a shoddy job which should surprise no one having ever published peer reviewed research.
And to be fair why should they do better? It's generally unpaid, it's poorly paid when it is paid and it's not particularly well considered.
If we discover that we can´t trust researchers then what else are we left with? Doctor-patient privacy works if the doctor is truthful in their reporting
It's generally permitted to share de-identified patient data. As long as you're not sharing patients names, medical record numbers, birthdays, and a couple of other fields, you should be fine.
Maybe we could do a double blind (including scientists) study where everyone waived their rights & are recorded then in another "typical" conditions study do none of that and compare the two and see which one seems to have the best, most accurate results.
>it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments
This is a great approach IMO. Additionally skeptics (for example anti-vaxxers), should be physically present at the trials.
What is preventing someone from having 200 participants, but saying they only had 100 participants, and then only providing evidence for 100 participants?
The researchers will have to have gone through some kind of third party agency to get the partecipants. This agency should be queried to see the number they report
How does this agency determine who can meaningfully participate in the study? Are they going to have the expertise to make that determination for _every_ study that could possibly be conducted?
What is the difference (to a layperson) between cherry-picking participants and rejecting participants because they do not meet your study's criteria?
Who funds this agency?
Do the members of the reviewing agency have their own biases, and might those biases tarnish the reputation of a study that is actually well-conducted? (hint: this already happens in journals)
Pfizer had a trial with 21823 people in the Expirement group and 21823 people in the Placebo group. In the results they excluded data for 1790 from the Expirement group and 1585 Placebo group. However, _crucially_ Pfizer never claimed there were only 100 people in the study after starting with 200; you know pfizer excluded 3375 people because _Pfizer told you_.
Every clinical trial paper I've read has a discussion of inclusion and exclusion criteria. I think for the trial to be registered, it has to include this information.
"unrelated reasons" should not be an acceptable excuse though. either state the reason or it goes into the trash. and if they were private reasons you can still contact them to confirm they left on their own volition and/or they didn't finish the trial without getting into specifics.
you only need one lie to suspect the whole thing
maybe as an intermediate step we could make available all the recordings to the peer reviewers and only offer the raw experimental data bundled in the paper publicly? maybe in the future we can have 1TB studies without breaking a sweat? maybe all the money we give to publishers can be spent on servers to archive all the primary data so at least we aren't simply filling the pockets of MBAs?