COVID Shot Blues
Sunset In Grand Staircase Escalante National Monument
[Important Updates Below]
I got my COVID shot today – the Johnson & Johnson 1 shot version – – in southern Utah. Good to go. No adverse reaction.
The Utah health center people were very helpful and I was able to capitalize on a cancellation and walk in with no appointment and get the J&J 1 shot version, which avoids a lot of logistical problems as I go down the road. It would have been a problem if I had to hang around for 28 days to get the second shot.
As I suspected, I read the fact sheet they give you and there is a significant difference between the usual FDA approval process and the Emergency Use Authorization.
That’s an example of what I constantly disparage as “streamlined regulatory review” and privatization.
There’s a significant tradeoff under the “emergency” justification. Bottom line: the FDA’s methodological, data, scientific, and legal & regulatory standards were relaxed.
The “ongoing clinical trial” sample size was just 21,895 people.
Half got placebo, so that’s only 10,948 people who got the vaccine. If you then figure that most of these people very likely were wearing masks and taking other preventive measures, that’s a very thin scientific basis and statistical sample size to support a conclusion of safety and efficacy, no?
[Update: 4/7/21 – I was very surprised that the fact sheet distributed by the healthcare facility with the vaccine did not make any claims about efficacy. Why is that?
According to CDC: (emphasis mine)
The J&J/Janssen vaccine was 66.3% effective in clinical trials (efficacy) at preventing laboratory-confirmed COVID-19 illness in people who had no evidence of prior infection 2 weeks after receiving the vaccine. People had the most protection 2 weeks after getting vaccinated.
How much of that 66.3% is a result of masks and other preventative behaviors? Does the clinical trial examine that question?
The reason I ask this is because it is likely that the people who receive the vaccine may assume they are immune and engage in risky behavior with no mask.
On top of that, according to CDC, the protection does not kick in until 2 weeks after the shot:
laboratory-confirmed COVID-19 was 66.3% (95% confidence interval [CI] = 59.9%–71.8%) ≥14 days after vaccination and 65.5% (95% CI = 57.2%–72.4%) ≥28 days after vaccination.
According to the fact sheet I received, the clinical trial size was 21,895. But according to the CDC clinical trial fact sheet, it was far larger:
The body of evidence for the Janssen COVID-19 vaccine was primarily informed by one international Phase III clinical trial initiated in September 2020 that enrolled approximately 40,000 participants aged 18–100 years.
What explains this large inconsistency? This does not inspire confidence.
According to the CDC link to clinical trials:
To guide its deliberations regarding the vaccine, ACIP used the Evidence to Recommendations (EtR) framework,† following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. The ACIP recommendation for the use of the Janssen COVID-19 vaccine under an EUA is interim and will be updated as additional information becomes available.
I would need to do a lot more research to understand what these frameworks are and how they compare to the standard FDA review process.
But according to CDC, here’s the data, factors and criteria they evaluated:
Within the EtR framework for the Janssen COVID-19 vaccine, ACIP considered the importance of COVID-19 as a public health problem, as well as resource use, benefits and harms, patients’ values and preferences, acceptability, feasibility, and equity. After a systematic review of available data, the work group used the GRADE approach to assess the certainty of evidence for outcomes related to the vaccine, rated on a scale of 1 (high certainty) to 4 (very low certainty) (4). Work group conclusions regarding certainty of evidence for the Janssen COVID-19 vaccine were discussed at public ACIP meetings (3).
Here are the CDC’s findings with respect to those methods:
From the GRADE evidence assessment, the level of certainty for the benefits of the Janssen COVID-19 vaccine was type 2 (moderate certainty) for the prevention of symptomatic COVID-19. Evidence was also type 2 (moderate certainty) for the estimate of prevention of COVID-19–associated hospitalization and death. Evidence was type 3 (low certainty) for the estimates of prevention of SARS-CoV-2 seroconversion. Regarding certainty of evidence for possible harms after vaccination, evidence was type 1 (high certainty) for reactogenicity and type 2 (moderate certainty) for serious adverse events. Data reviewed within the EtR framework supported the use of the Janssen COVID-19 vaccine. ACIP determined that COVID-19 is a major public health problem and that use of the Janssen COVID-19 vaccine is a reasonable and efficient allocation of resources. Although there was variability in how populations value receipt of a COVID-19 vaccine, it was determined that for most populations, the desirable effects outweigh the undesirable effects, making the Janssen COVID-19 vaccine acceptable to implementation stakeholders. The Janssen COVID-19 vaccine is feasible to implement, requiring only a single dose and refrigerator temperatures (36°F–46°F [2°C–8°C]) for transportation and storage. These characteristics will allow for expanded availability of the Janssen COVID-19 vaccine in most community settings and mobile sites when this vaccine becomes more widely available. In addition, persons who want to complete their vaccination schedule quickly or who might have difficulty returning for a second dose might prefer a single-dose vaccine. The feasibility of administering the Janssen COVID-19 vaccine in a wider variety of settings provides an opportunity to improve equitable access to an effective COVID-19 vaccine. However, advancing health equity, particularly in populations who experience disproportionate COVID-19 morbidity and mortality, requires engagement with community leaders to identify and remove barriers to COVID-19 vaccination, including those related to vaccine access and vaccine confidence. Community engagement and education will be important as new COVID-19 vaccines are authorized for use. The GRADE evidence profile and supporting evidence for the EtR framework are available at https://www.cdc.gov/vaccines/acip/recs/grade/covid-19-janssen-vaccine.html and https://www.cdc.gov/vaccines/acip/recs/grade/covid-19-janssen-etr.html.
These conclusions sounds very subjective, ambiguous, non-transparent, and uncertain to me. It will take a lot more work to review the GRADE & EtR frameworks.
One thing is clear, however, and the public is not being informed about it. The FDA is making “policy” decisions, not purely “science” decisions.
The GRADE framework makes that clear:
The policy question was, “Should vaccination with Janssen COVID-19 vaccine be recommended for persons 18 years of age and older during an Emergency Use Authorization?” The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group were prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (critical), all-cause death (important), SARS-CoV-2 seroconversion (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (critical) and reactogenicity grade ≥3 (important).
Here’s how CDC describes their scientific and policy judgments.
For the critical outcomes, the certainty of evidence was moderate for prevention of symptomatic COVID-19 and for prevention of hospitalizations due to COVID-19, and moderate for serious adverse events. For important outcomes, the certainty of evidence was moderate for prevention of all-cause death, low for prevention of seroconversion, and high for reactogenicity.
This is the most important CDC document, yet it took me over an hour chasing links to find it!
I wish someone would do a side by side comparison of the FDA standard review process with the Emergency Use Authorization so people could understand the trade-offs and how FDA weighed multiple factors.
Chasing links and digging deeper into the science, we find an Appendix 1 titled: “Studies Included in the Review Of Evidence”
The Appendix raises questions. It looks like efficacy was tested with 43,783 people (just one study, conducted by Jansen, the manufacturer), but serious adverse effects were only 840 people.
What explains that?
CDC did include this caveat:
Post-marketing surveillance will be critical to detect any rare serious adverse events which were not identified in the clinical trial.
I guess this is the bottom line – FDA is relatively confident of the science, with caveats:
GRADE Summary
The initial GRADE evidence level was type 1 (high) for each outcome because the body of evidence was from randomized controlled trials (Table 4). In terms of critical benefits, the available data indicated that the vaccine was effective for preventing symptomatic COVID-19 and for preventing hospitalization due to COVID-19; these were each downgraded one point for serious concern of indirectness related to the median two months follow-up (type 2, moderate). The certainty in the effect estimate for the important outcome of all-cause death was downgraded one point for serious concern of indirectness related to the median two months follow-up (type 2, moderate). The certainty in the estimate for seroconversion was downgraded two points due to very serious concern of indirectness because the efficacy against seroconversion based on day 71 serology may not be a direct measure of efficacy over a relevant period of time for an emergency use authorization and the subset of 6.8% with serology data likely differed substantially from the total population of interest (type 3, low). The certainty in the estimate of the effect for serious adverse events was downgraded one point due to serious concern of indirectness related to the median two months follow-up (type 2, moderate). No serious concerns impacted the certainty in the estimate of reactogenicity (type 1, high) (Table 4).
I encourage readers with expertise to weigh in and drop me an email.
But, one thing is clear: the data are not complete and this is an ongoing experiment, i.e. “EUA is interim” and “will be updated as additional information becomes available”. ~~~ End Update]
Of course, there are virtually no press reports about these scientific and regulatory issues.
I’m no anti-vaxxer, but I’m not confident in this either.
[Update – 5/7/21 – Clarification. I heard 2 experts on NPR 1A show last night. A listener asked the same question I have about the differences between EUA and traditional FDA regulatory review (see below). I need to clarify that based on what I heard. One expert said the EUA does not change the statistical basis, compared to the traditional FDA review process. Both involve the same sample size for a Phase 3 Clinical trial. Apparently the main difference is the length of monitoring for long term adverse effects (safety criterion). But any viral vaccine adverse effects are expected to occur in 6-8 weeks after shot, so this should not be a problem. As such, this is not as big a concern as I initially assumed. ~~~ end]
[Update – 4/13/21 – In what might be the understatement of the year, I concluded this long complex post with this:
I’m no anti-vaxxer, but I’m not confident in this either.
YIKES! I’ve had a slight headache and abdominal and kidney pain, similar in location to but not nearly as severe as kidney stones. If I stroke out, sue the bastards!
[Update – 4/10/21 – Jeff St. Claire nails it again:
+ In a survey of 77 epidemiologists from 28 countries, two-thirds thought that we had a year or less before the coronavirus mutates to the extent that the majority of first-generation vaccines are rendered ineffective and new or modified vaccines are required. Of those surveyed, nearly a third gave a timeframe of nine months or less. Fewer than one in eight said they believed that mutations would never render the current vaccines ineffective. ~~~ end update]