How Good are Flu Shots? Their Impact on Mortality

I have been hesitant about posting anything related to influenza vaccines. This is in part because the data changes from season to season, and the amount of literature that comes out is staggering (seriously, take a look at pubmed and you will see what I mean). Further, there seems to be a lot of misinformation with regards to the benefits of the influenza vaccine (though, given the state of media today, I think you can say that about anything. Anyone remember Plandemic?). I also think many physicians/NPs/PAs do not actually know the data behind the efficacy of flu vaccines, though to be fair I was in that camp prior to writing about this particular topic. With this in mind, I figured I’ll take a look at one particular aspect of influenza vaccine that I particularly find interesting and that is its role in mortality rates, specifically in the elderly.

There are a few things to keep in mind when looking at the data. First of all, the influenza-related mortality rate has been decreasing with each subsequent season (1). 

This trend began early in the 20th century (with the exception being the Spanish influenza pandemic), well before the 1940s when the influenza vaccine was introduced (1). 

One other issue with evaluating mortality rates also comes from how these are recorded. For instance, the CDC “influenza associated underlying influenza and pneumonia mortality” describes deaths caused by influenza and pneumonia. This may overestimate the true mortality rate of influenza (or under-estimate it). For instance, the recorded influenza mortality rates are actually lower than the CDC influenza-associated mortality estimates (1)

Finally, the quality of the data out there is variable. This is due to the nature of the topic. We are never going to get a randomized-controlled trial looking at the mortality rates and as a result, the vast majority of these studies are observational cohort studies which tend to have biases that cannot be accounted for.  For instance, the receipt of vaccination may be influenced by health-care utilization. In other words, those who have more comorbidities and visit their physician are more likely to get vaccinated at the request of the physician. The opposite can also be true, where healthier patients are more likely to get the vaccine, which can bias the results towards the vaccine groups. Propensity score matching and logistic regression can be used to attempt to match these groups as best as you can, but you cannot control for all the biases. Endpoints tend to vary as well, ranging from “confirmed influenza” to “influenza-like illnesses” or admission with pneumonia. You can see the issue here, as influenza-like illnesses may include non-flu viruses, and many patients who get influenza may never even show up to get tested in the first place. One author noted in a comment to BMJ how a meta-analysis found no effect of vaccination in elderly against influenza, but had a large effect on all cause mortality (3). He argues this could be attributed to inherent differences in the groups who get a vaccine and those who do not. I mention this because there is still a lot of controversy regarding the topic. This comes mostly from policy-decisions, in terms of mass vaccination campaigns and whether they are worth pursuing or if we should allow vaccinations to go to the highest risk patients. I will not go into that topic as it is out of my league, but I wanted to mention the caveats of the data prior to actually diving into it as well as mention the limitations on the topic. 

Cohort Studies

One early cohort study evaluated influenza vaccines in 3 large health systems in patients over 65yrs over 2 influenza seasons in the late 90s (4). After adjustments, influenza vaccinations were associated with decreased hospitalizations for pneumonia and influenza as well as all-cause death among all sites combined:

Notably, however, influenza-related hospitalizations did not reach statistical significance when looking at each season/health system individually, but mortality was still significantly lower across all sites:

A Swedish study of 3 influenza seasons also evaluated all-cause mortality in those aged >65yrs (5). After controlling for age and comorbidities, crude death rates per 1000 person-months in vaccinated vs unvaccinated subjects were 2.82 vs 5.68 in 1998/1999, 3.03 vs 5.63 in 1999/2000, and 2.94 vs 5.09 in 2000/2001. This corresponded to a vaccine effectiveness of 44%, 40%, and 37% after adjustment, per season. Following adjustments for hazard ratios during influenza seasons and following offseasons, the protective effects decreased to 14%, 19%, and 1% per season:

A larger retrospective study of 18 cohorts evaluated elder,ly patients across 10 seasons (6). Using propensity score matching and comparing risk of hospitalization with non-influenza seasons, influenza vaccines were associated with decreased hospitalizations (aOR 0.73, 95% CI 0.68-0.77, vaccine effectiveness 27%) and death (aOR 0.52, 95% CI 0.50-0.55, vaccine effectiveness 48%). This held true in the 2 seasons where there was a poor match between vaccine and virus strain (aOR 0.63, 95% CI 0.57-0.69, 37% vaccine effectiveness. The authors also evaluated the risk of death with to account for confounders, and the risk of hospitalization and death was still lower, though effectiveness decreased: 

An observational study of 9 influenza seasons found that all-cause mortality was decreased in patients >65 who got an influenza vaccine, driven by cardiovascular or respiratory deaths (7). This effect was diminished the older the patient was. The authors conclude that excess mortality in this population would have been 9.8% if none had been vaccinated, suggesting a 47% reduction in excess mortality attributed to vaccination.

It should be noted that it is difficult to differentiate if the excess mortality was related to influenza, or influenza-related complications or some other factors (i.e. those who did not get the flu shot were already much sicker to begin with). Indeed, this is another topic of discussion, as mentioned previously there may be difficulty in accounting for unmeasured differences between groups. Several methods have been implemented to circumvent this issue. For instance, this retrospective study of 10 Canadian provinces from 1997 to 2004 used a pre-/post-intervention study design to evaluate the impact of vaccine on influenza-associated mortality (8). Ontario adopted a large-scale universal influenza immunization program for those >6mo of age. Multivariate regression models were used to generate an expected baseline of events during non-influenza weeks and compare this to observed events during influenza seasons. During this period, Ontario implemented their program in 2000, although all other provinces also saw an increase in vaccinations (Ontario increase 20% vs 11% in the other provinces). Despite this, influenza associated mortality decreased in 74% in Ontario (RR 0.26, 95% CI 0.20-0.34) compared to 57% in other provinces (RR 0.43, 95% CI 0.37-0.55, ratio of RRs = 0.61, p-value 0.002). The decrease was more noticeable in patients older than 75yo. There was also a decrease in influenza-associated health care use across all groups. 

Another method use is to compare mortality rates between influenza and non-influenza periods. 

A Dutch retrospective analysis evaluated with influenza seasons in patients >65yo (9).. They assessed mortality in comparison to eight consecutive summer periods where influenza is usually absent and used propensity score regression analysis and multivariate regression analysis to adjust for confounders. Influenza vaccination was associated with decreased mortality after using 3 models to adjust for confounders when compared to mortality in the summer months. 

This also held true after adjusting for unmeasured confounders. 

A similar study from the Netherlands from 2003-2015 (10)  found that a mean of 13% (7.2-19) were averted per year, with 35% of deaths being averted by vaccination. A simulation going into 2025 found similar results, after accounting for attack rate, vaccine match, and loss of immunity. 

I tend to prefer studies that evaluate several influenza seasons, as these tend to give a better picture and averages out the annual vaccination rates, attack rates, as well as vaccine/circulating strain mismatch. Despite this, some single season cohorts have also had similar findings. A single-center retrospective study (11) evaluated the need for ICU, mechanical ventilation, and death in the 2014-2015 influenza season and used multivariable logistic regression analysis to adjust for different variables. Multivariate analysis found that patients who were vaccinated had lower odds of death (OR 0.29, 95% CI 0.09-0.99), mechanical ventilation (OR 0.3.5, 95% CI 0.14-0.83), and serious outcome (ICU admission/death, OR 0.19-0.87). Notably, subgroups within the mortality outcome did not reach statistical significance (including age 50-64, >65yo, cardiovascular or lung disease). This may have been due to mismatch between predominant circulating virus strain and vaccine. It should be noted the result here was barely statistical significant, and as I mentioned earlier, it includes all cause mortality rather than influenza-related mortality, though there were lower odds of mechanical ventilation, which is good. A larger cohort that included 52 healthcare facilities estimated that vaccine effectiveness during this time period against any influenza A or B virus was 38% (95% CI 31-43%), with roughly 8054 deaths being prevented during this season. This was mostly driven by those >65yo (12).

The authors also estimated a NNT of 1223 overall (95% CI 578-3438) to prevent one influenza hospitalization, with those >65yo having a NNT of 462.

Reviews and Meta-analysis

Several Cochrane reviews have been done to evaluate the efficacy of influenza vaccines. There are obvious issues with this, including the different types of patients, settings, outcomes, and vaccines (monovalent, trivalent, quadrivalent). An early Cochrane review (13) of 48 studies, including over 66,000 patients found that vaccines were 30% effective (95% CI 27 to 41%) against influenza-like illness if the vaccine content matched WHO recommendations and circulating strain, which decreased to 12% (95% CI 0 to 28%) when there was a mismatch. They were found to be 80% effective against confirmed influenza (95% CI 56% to 91%), but decreased to 50% when there was a mismatch (95% CI 27 to 65%). This Cochrane review did not evaluate mortality. A more recent Cochrane review (14) evaluated 8 RCTs conducted in elderly patients, amounting to over 5000 participants. The risk of influenza over a season decreased from 6% in the unvaccinated group to 2.4% in the vaccinated group (efficacy 58%, RR 0.42, 95% CI 0.27 to 0.66), though one of the things the authors mention is the heterogeneous nature of vaccines tested. Vaccines were found to reduce the risk of influenza-like illness from 6% to 3.5% (efficacy 41%, RR 0.59, 95% CI 0.47 to 0.73). There was no difference in mortality from the studies cited, likely due to the fact the data was underpowered to show any differences. 

A different review from one of the authors of the Cochrane analysis actually had interesting findings (15). Vaccines were not found to be effective against influenza in community-dwelling elerly patients (RR 0·19, 95% CI 0·02–2·01). This was the same for influenza-like illness (RR 1·05, 95% CI 0·58–1·89) and  pneumonia (RR 0·88, 95% CI 0·64–1·20).  When the vaccines matched the circulating strain, they prevented admissions for both pneumonia and influenza (effectiveness 26%, 95% CI 12-38), and all-cause mortality (effectiveness 42%, 95% CI 24-55). Similar findings for elderly patients in group homes were reported, with good efficacy for influenza-like illness (effectiveness 23%, 95% CI 6-36), though this was not significant for influenza (RR 1.04, 95% CI 0.43-2.51). Well matched vaccines prevented hospital admissions (effectiveness 45%, 95% CI 16-64) and all-cause mortality (effectiveness 60%, 95% CI 23-79). Again, it seems difficult to marry the findings of lack of efficacy for influenza but improved all cause mortality. Whether this is due to patient selection or other confounding variables, it is unclear to me. 

Conflicting Studies

Not all data supports the mortality benefit of influenza vaccine. For instance, an observational study (16) evaluated 3 decades of influenza-seasons and vaccine effectiveness and used cyclical regression to estimate excess mortality. However they found no differences in pneumonia and influenza hospitalization or excess mortality when compared to summer months. A well-publicized British study published in the Annals also found similar results (17). They used regression discontinuity design to evaluate the efficacy of influenza vaccination in patients aged 65. In short, during the 2000 to 2001 influenza season, there was a significant increase in the vaccination rates in those aged 65yo. They used this design to compare patients just over 65 and those just under 65 to see if the increase in vaccination rates at around age 65 was associated with decreased mortality/hospitalization. Comparing an age range of 63.5 to 66.5yo, there were no difference in hospitalization rates or mortality at age 65yo:

They compared their estimates to those of other studies and found that estimates of hospitalization rates from cohort and case-control studies were outside of the 95% CI of the estimates from this study. This was the same for all-cause mortality. In other words, their estimates were different from the estimates of other studies, and the authors suggest this could be due to less bias/confounding, the difference in the use of UK data vs US data, or protection from herd immunity in the population studied. While a large trial, it is still an observational study so there is risk of bias here, however they do have several seasons to draw data upon. Despite this, several reviews and other observational studies still suggest benefit. 

Take-Home Points:

I think the debate with regards to the efficacy of influenza vaccination will continue with each passing season. While it is difficult to tease out how data suggest modest benefit of influenza vaccines on flu-like illnesses or with contracting influenza, it seems the major benefit comes from preventing complications such as pneumonia, hospitalizations, and death. Whether all-cause mortality cited in these studies is related to other comorbidities or are related to influenza complications, the data for vaccinations in the elderly is compelling enough to continue to recommend it. Another issue to think about is herd immunity; i.e. if we vaccinate a large proportion of the entire population, will we see similar or even better results in terms of all cause mortality, even for the high risk population? This is an interesting question, and there is some literature on the benefits of vaccinating healthcare workers and reduced mortality for their patents (18, 19). This would suggest there may be a benefit mass immunization, how it would work in the real world remains to be seen. 

References:

1. Doshi P. Trends in recorded influenza mortality: United States, 1900-2004. Am J Public Health. 2008 May;98(5):939-45. doi: 10.2105/AJPH.2007.119933. Epub 2008 Apr 1. Erratum in: Am J Public Health. 2009 Aug;99(8):1353-4. PMID: 18381993; PMCID: PMC2374803.
2. Thompson WW, Shay DK, Weintraub E, Brammer L, Cox N, Anderson LJ, Fukuda K. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA. 2003 Jan 8;289(2):179-86. doi: 10.1001/jama.289.2.179. PMID: 12517228.
3. Jefferson Tom. Influenza vaccination: policy versus evidence BMJ 2006; 333 :912 doi: https://doi.org/10.1136/bmj.38995.531701.80
4. Nordin J, Mullooly J, Poblete S, Strikas R, Petrucci R, Wei F, Rush B, Safirstein B, Wheeler D, Nichol KL. Influenza vaccine effectiveness in preventing hospitalizations and deaths in persons 65 years or older in Minnesota, New York, and Oregon: data from 3 health plans. J Infect Dis. 2001 Sep 15;184(6):665-70. doi: 10.1086/323085. Epub 2001 Aug 9. PMID: 11517426.
5. Ortqvist A, Granath F, Askling J, Hedlund J. Influenza vaccination and mortality: prospective cohort study of the elderly in a large geographical area. Eur Respir J. 2007 Sep;30(3):414-22. doi: 10.1183/09031936.00135306. Epub 2007 May 30. PMID: 17537767.
6. Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med. 2007 Oct 4;357(14):1373-81. doi: 10.1056/NEJMoa070844. PMID: 17914038.
7. Fireman B, Lee J, Lewis N, Bembom O, van der Laan M, Baxter R. Influenza vaccination and mortality: differentiating vaccine effects from bias. Am J Epidemiol. 2009 Sep 1;170(5):650-6. doi: 10.1093/aje/kwp173. Epub 2009 Jul 22. PMID: 19625341; PMCID: PMC2728831.
8. Kwong JC, Stukel TA, Lim J, McGeer AJ, Upshur RE, Johansen H, Sambell C, Thompson WW, Thiruchelvam D, Marra F, Svenson LW, Manuel DG. The effect of universal influenza immunization on mortality and health care use. PLoS Med. 2008 Oct 28;5(10):e211. doi: 10.1371/journal.pmed.0050211. PMID: 18959473; PMCID: PMC2573914.
9. Groenwold RH, Hoes AW, Hak E. Impact of influenza vaccination on mortality risk among the elderly. Eur Respir J. 2009 Jul;34(1):56-62. doi: 10.1183/09031936.00190008. Epub 2009 Feb 12. PMID: 19213779.
10. Backer JA, Wallinga J, Meijer A, Donker GA, van der Hoek W, van Boven M. The impact of influenza vaccination on infection, hospitalisation and mortality in the Netherlands between 2003 and 2015. Epidemics. 2019 Mar;26:77-85. doi: 10.1016/j.epidem.2018.10.001. Epub 2018 Oct 11. PMID: 30344024.
11. Kaselitz, Timothy B. MD, MPH∗; Martin, Emily T. PhD, MPH†; Power, Laura E. MD, MPH†,‡; Cinti, Sandro MD‡,§ Impact of Vaccination on Morbidity and Mortality in Adults Hospitalized With Influenza A, 2014–2015, Infectious Diseases in Clinical Practice: November 2019 – Volume 27 – Issue 6 – p 328-333 doi: 10.1097/IPC.0000000000000777
12. Rolfes MA, Flannery B, Chung JR, O’Halloran A, Garg S, Belongia EA, Gaglani M, Zimmerman RK, Jackson ML, Monto AS, Alden NB, Anderson E, Bennett NM, Billing L, Eckel S, Kirley PD, Lynfield R, Monroe ML, Spencer M, Spina N, Talbot HK, Thomas A, Torres SM, Yousey-Hindes K, Singleton JA, Patel M, Reed C, Fry AM; US Influenza Vaccine Effectiveness (Flu VE) Network, the Influenza Hospitalization Surveillance Network, and the Assessment Branch, Immunization Services Division, Centers for Disease Control and Prevention. Effects of Influenza Vaccination in the United States During the 2017-2018 Influenza Season. Clin Infect Dis. 2019 Nov 13;69(11):1845-1853. doi: 10.1093/cid/ciz075. PMID: 30715278; PMCID: PMC7188082.
13. Demicheli  V, Di Pietrantonj  C, Jefferson  T, Rivetti  A, Rivetti  D. Vaccines for preventing influenza in healthy adults. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No.: CD001269. DOI: 10.1002/14651858.CD001269.pub3. Accessed 31 December 2020.
14. Demicheli V, Jefferson T, Di Pietrantonj C, Ferroni E, Thorning S, Thomas RE, Rivetti A. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev. 2018 Feb 1;2(2):CD004876. doi: 10.1002/14651858.CD004876.pub4. PMID: 29388197; PMCID: PMC6491101.
15. Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet. 2005 Oct 1;366(9492):1165-74. doi: 10.1016/S0140-6736(05)67339-4. Epub 2005 Sep 22. Erratum in: Lancet. 2006 Mar 25;367(9515):986. PMID: 16198765.
16. Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005 Feb 14;165(3):265-72. doi: 10.1001/archinte.165.3.265. PMID: 15710788.
17. Anderson ML, Dobkin C, Gorry D. The Effect of Influenza Vaccination for the Elderly on Hospitalization and Mortality: An Observational Study With a Regression Discontinuity Design. Ann Intern Med. 2020 Apr 7;172(7):445-452. doi: 10.7326/M19-3075. Epub 2020 Mar 3. PMID: 32120383.
18. Kheok SW, Chong CY, McCarthy G, Lim WY, Goh KT, Razak L, Tee NW, Tambyah PA. The efficacy of influenza vaccination in healthcare workers in a tropical setting: a prospective investigator blinded observational study. Ann Acad Med Singap. 2008 Jun;37(6):465-9. PMID: 18618057.
19. Ahmed F, Lindley MC, Allred N, Weinbaum CM, Grohskopf L. Effect of influenza vaccination of healthcare personnel on morbidity and mortality among patients: systematic review and grading of evidence. Clin Infect Dis. 2014 Jan;58(1):50-7. doi: 10.1093/cid/cit580. Epub 2013 Sep 17. PMID: 24046301.