> In Moderna's trial, 15,000 study participants were given a placebo, which is a shot of saline that has no effect. Over several months, 90 of them developed Covid-19, with 11 developing severe forms of the disease.
> Another 15,000 participants were given the vaccine, and only five of them developed Covid-19. None of the five became severely ill.
I was thinking about this since I read the article. The number you state seems to be 0.5% of total subjects. To my understanding, this is not how "p < 0.005" is employed.
If we take the control group, we have P(Corona)=90/15000.
The likelyhood of getting as an extreme result in the vaccine group is then P(0 cases) + ... + P(5 cases) = [math and statistics] = the actual p-value.
How did you determine the statistical significance in your post?
What seems to be vague... They didn't contract Covid, but we're they exposed to it in a lab environment, or just set off on their way to be evaluated x months later?
It is fine for it to be vague. It is implied because medical ethics and international humanitarian law preclude exposing subjects to a deadly virus, let alone for testing purposes, let alone in a blind study.
Additionally, you can apply common sense that if 15,000 people were exposed to the virus with no protection, more than 90 of them would become infected.
>They didn't contract Covid, but we're they exposed to it in a lab environment, or just set off on their way to be evaluated x months later?
This is what is referred to as a 'challenge trial'[0]. It is something the UK is apparently working on but afaik, neither of the current vaccine candidates have used any form of challenge trial. This is one of the difficulties with vaccine testing...if I were to vaccinate everyone in the world against small pox, how do I prove it's effectiveness?
Edit: for some fun, I figured I should link to the 'COVID challenge trial volunteers advocacy organization'. Yes, an advocacy group for people who want to be infected with COVID. https://1daysooner.org/
Challenge trials also don't provide as much useful information.
Nobody is getting covid on purpose, in a lab. Being protective or unprotective against a lab dose isn't the same as being protective or unprotective against a real-world dose.
I don't think anyone has really proposed this. Challenge trials are much more about getting quick answers to efficacy of treatment -- ethically, it's at least somewhat defensible to expose a volunteer if you also have medicine/vaccine for them, even if you're not sure how well it will work. It would be wildly unethical to give the virus to folks just to see how much virus it takes to get sick.
You might also get useful information about how people's immune systems react, when you give them too little of the virus to get infected. It's not only about figuring out the dose for the main trial. It also could have been done 6 months ago...
"It would be wildly unethical to give the virus to folks just to see how much virus it takes to get sick."
Depends. If there are volunteers for it, (which I can imagine to be the case) - and the volunteers are fully aware of the risk - then it might be ethical to let them proceed and save millions of other people.
1. You have to have a control group, so half of test subjects would be exposed to the virus without having received the vaccine.
2. The highest-priority recipients for a vaccine are members of high-risk groups; it'd be pointless to run a challenge trial full of low-risk individuals.
So a challenge trial wouldn't just mean giving the virus to a few thousand vaccinated macho 20-year-olds. It'd mean giving the virus to unvaccinated 80-year-old care home residents.
> it'd be pointless to run a challenge trial full of low-risk individuals.
This is binary all-or-nothing thinking. It considers 90% the same as 0%, since both are not 100%.
Testing on healthy young people you learn how a normal immune system reacts to a vaccine candidate. High risk groups mostly have similar immune systems.
Even if somehow you could only vaccinate everyone under 60, that would do enormous good stopping the spread.
Another way of making the point: we did COVID challenge trials on monkeys first, because their immune systems are a decent model for that 80-year-old human's. Well, a 30-year-old human is an even better model of an 80-year-old human. A challenge trial on young people wouldn't prove everything, but it would give a high-information signal quickly.
> This is binary all-or-nothing thinking. It considers 90% the same as 0%, since both are not 100%.
Thinking is not just "binary or not"; there are degrees. If exposing that group of people would be 90% pointless, it's not much less out of the question than if it were 100% pointless.
In all phase3 trials large groups of people, as similar to each other in all variables as possible, will be either given a placebo or the actual vaccine and then go back to their normal lives.
The trials have a set "Covid-19 cases" mark where the stop to evaluate. The Pfizer vaccine has 164 contractions of Covid-19 as their mark to conclude the research. This research seems to aim for 151.
The Moderna vaccine, which is based on similar mRNA technology as BioNTech’s, is expected to be assessed by the FDA on a final analysis of 151 Covid cases among trial participants who will be followed on average for more than two months.
If both groups are similar enough you can then say how effective the vaccine is in preventing it.
This, by the way, is why most phase 3 trials take place in the US and/or Brazil (among other places): the more Covid-19 is around the faster you can get to the set number of contractions and conclude the phase 3.
I don't think they have started yet, but such trials are planned in the UK.
There's no easy blanket "unethical" ruling from on high. Obviously this isn't a risk-free thing to volunteer for, but it isn't risk free for nurses to go to work either, and thankfully, they still do.
Also, you'd possibly be giving an unrealistic dose in the lab. If you give someone 1000x the viral load they'd ever get in the real world, the vaccine might not work. Or might work but send the immune system into a crazy overdrive cytokine storm.
Well, given that 90/15000 = 0.6% in the control group developed the disease, you can consider the vaccine group as a Bernouili trial of n = 15000 with probability p = 0.6%. Then the probability of observing 5 or fewer cases is 3.4*10^-32, from the tail probability of the binomial distribution.
Of course, that's assuming that five guys from the vaccine group didn't get infected at the same after-ski party, or any funny business that violates statistical independence ...
Naive question: If the null hypothesis is that there is no difference, wouldn't that imply 95/30,000, p = 0.0031, putting the probability of observing less than 5 cases at a much more reasonable 1*10^(-14).
I think Fisher's exact test [1] is most commonly used in these types of trials. But the P-value roland (parent comment) provided also make sense to me. For the Fisher's test R says...
Covid NoCovid
_____ _______
Vaccine 5 1495
NoVaccine 90 1410
9.0e-22 one tailed
4.5e-22 two tailed
No, I think you must estimate p only using the cohort where people were not treated, otherwise you will underestimate the population fraction. In order to test if the null hypothesis is true, we can't assume that's its true when constructing the test.
> In Moderna's trial, 15,000 study participants were given a placebo, which is a shot of saline that has no effect. Over several months, 90 of them developed Covid-19, with 11 developing severe forms of the disease.
> Another 15,000 participants were given the vaccine, and only five of them developed Covid-19. None of the five became severely ill.
https://edition.cnn.com/2020/11/16/health/moderna-vaccine-re...