Editor’s Note: This article covers research conducted in part by CIDRAP researchers. Please note that CIDRAP News operates independently in relation to CIDRAP's research and policy programs.
See also: Public health groups say flu vaccine is best tool, despite limitations
Oct 25, 2011 (CIDRAP News) – A rigorous new analysis of 44 years' worth of studies is raising questions about the evidence for the effectiveness of influenza vaccines in elderly people and, for certain types of vaccines, in children and younger adults as well.
Applying very strict criteria to filter out potential bias and confounding, a US research team sifted more than 5,000 studies and found only 31 that they felt provided reliable evidence about the efficacy and effectiveness of flu vaccines. The findings were published online today by Lancet Infectious Diseases.
Overall, the researchers found that existing flu vaccines can provide moderate protection from laboratory-confirmed flu, but protection is much lower in some groups and during some seasons. Pooled evidence from randomized controlled trials (RCTs) showed that standard trivalent inactivated vaccines (TIVs) had an efficacy of 59% in young adults (18 to 65), but the authors found no RCTs demonstrating TIV efficacy in adults 65 and older or children aged 2 to 17. (In the United States, because of the recommendation that seniors should be vaccinated, RCTs of flu vaccination in those over 65 have been considered unethical since the 1960s.)
For the live attenuated flu vaccine (LAIV)—the nasal spray version—evidence from 10 RCTs showed 83% efficacy in children 6 months to 7 years old, but no RCTs supported its efficacy in older children or adults. The authors' main conclusion is that use of existing flu vaccines should continue, but better ones are needed.
Because of flu vaccines' track record of safety and "moderate efficacy," the authors state, "We believe the current influenza vaccines will continue to have a role in reduction of influenza morbidity until more interventions are available. However, evidence for consistent high-level protection is elusive for the present generation of vaccines, especially in individuals at risk of medical complications or those aged 65 years or older."
The authors of the analysis are Michael T. Osterholm, PhD, MPH, of the University of Minnesota's Center for Infectious Disease Research and Policy (CIDRAP), publisher of CIDRAP News; Nicholas S. Kelley, PhD, also of the CIDRAP staff; Alfred Sommer, MD, MHS, of the Johns Hopkins Bloomberg School of Public Health in Baltimore; and Edward A. Belongia, MD, of the Marshfield Clinic Research Foundation in Marshfield, Wis. The study was funded by the Alfred P. Sloan Foundation.
The report comes a year after federal health officials began recommending seasonal flu vaccination for everyone over the age of 6 months, and amid the annual fall vaccination campaign. It raises the prospect of sowing doubts in the public's mind about the value of flu vaccines, a risk that the authors acknowledge.
The new analysis is by no means the first to raise questions about the effectiveness of flu vaccines. As the authors observe, several research groups have shown that some previous observational studies overestimated the mortality benefits of flu vaccination in the elderly. The studies were shown to be unreliable because of the "healthy user effect": reasonably healthy elderly people were more likely to be vaccinated, and those in poorer health were less likely to be vaccinated and more likely to die for any reason.
Moreover, it is well known that people over 65 years mount a weaker immune response to flu vaccines than younger adults do, as measured by antibody titers. Because of this, last year Sanofi Pasteur rolled out a flu vaccine for the elderly containing four times as much antigen as the standard injectable vaccine.
Stringent criteria
The authors used stringent criteria to limit their analysis to studies likely to be most reliable. They defined flu vaccine efficacy as the reduction in flu risk after vaccination as established by an RCT. And they defined flu vaccine effectiveness as the relative reduction in flu risk in vaccinated persons as shown in observational studies that used medically attended, lab-confirmed flu as the primary outcome. For observational studies, they sought those that reported vaccine effectiveness for individual seasons and in which control subjects tested negative for flu by polymerase chain reaction (PCR) or viral culture.
They assert that previous studies used diagnostic findings or end points that were not very good indicators of influenza infection. In particular, they say questions have been raised about the use of serology, or antibody levels, since the 1940s and '50s. Antibody levels rise in response to vaccination with TIV. If a vaccinated person later becomes infected, the authors say, the infection may not produce the fourfold rise in antibody levels that is generally seen as confirmation of an infection. This can lead to overestimation of the effect of TIV.
The authors searched the literature for studies addressing flu vaccine efficacy or effectiveness from 1967 through early this year. They came up with 5,707 studies, but only 31—17 RCTs and 14 observational studies—met their strict criteria. These trials had about 54,000 participants from 23 countries and covered 24 flu seasons. The researchers excluded studies that used nonspecific outcomes, such as mortality or influenza-like illness.
There were 10 RCTs that assessed TIV efficacy over 12 flu seasons, none of which involved only people 65 and older or only children aged 2 to 17. Eight of the 12 seasonal analyses showed significant efficacy, with a pooled efficacy finding of 59% (95% confidence interval [CI], 51%-67%) for adults aged 18 to 64. The median efficacy was 62%, with a range of 16% to 76%.
The efficacy of LAIV was also assessed in 10 RCTs that covered 12 flu seasons. Six studies focused on children from 6 months to 7 years old and revealed that the vaccine was efficacious in all eight flu seasons involved, with a pooled efficacy of 83% (95% CI, 69%-91%) and median efficacy of 78% (range, 57%-93%). But of three RCTs in adults aged 18 to 49 years, none showed significant efficacy.
Another RCT of the spray vaccine involved people 60 years and older and found an efficacy level of 42% (95% CI, 21%-57%), but protection appeared to be lower in those in their 60s (31%) and higher in those 70 and older (57%).
For observational studies, the authors included nine that dealt with seasonal flu vaccines, mainly TIV, and five that focused on the pandemic 2009 H1N1 vaccine. The seasonal vaccine studies included 17 seasonal or cohort analyses. Six (35%) of these 17 analyses showed significant effectiveness against lab-confirmed flu. For children aged 6 to 59 months, vaccine effectiveness was found in 3 of 8 seasons. Of two studies focusing on adults 65 and older, one showed vaccine effectiveness of 59%.
The five pandemic vaccine studies were conducted in Europe and Canada, and most of the vaccinated subjects received an adjuvanted vaccine. Four of the studies found a median vaccine effectiveness of 69% (range, 60%-93%) for preventing medically attended flu, but there were relatively few flu cases in people 65 and older. The fifth study showed the vaccine was 90% effective (95% CI, 48%-100%) for preventing hospital admission with reverse transcriptase (RT)-PCR–confirmed flu.
Summing up their findings, the authors write that they found "substantial gaps in the evidence base for some age-groups with regard to efficacy for TIV and LAIV, including the lack of RCT showing TIV to be protective in 2- to 17-year-olds and people over 65 and an absence of RCTs supporting LAIV efficacy in the age range of 8 through 59 years.
"The evidence from these trials and observational studies suggests that presently available influenza vaccines can provide moderate overall protection against infection and illness, with LAIV providing a consistently higher level of protection in children aged 7 years or younger," the report states. It adds that, aside from LAIV in young children, the studies "also show substantial variability by season and age-group that cannot be attributed to differences in study design or outcome measures."
The report notes that the Advisory Committee on Immunization Practices, which frames immunization policies for the Centers for Disease Control and Prevention (CDC), has not recommended LAIV preferentially over TIV in children under age 8, despite the much more consistent evidence of efficacy for LAIV.
The authors also comment that the observational studies of the 2009 pandemic vaccine may offer the best yardstick of vaccine effectiveness because the vaccine closely matched the circulating H1N1 strain. The finding of a median 69% effectiveness in these studies "is not adequate for a pandemic setting where the antigenic match is ideal and antigenic drift has not occurred," they state.
In defense of strict inclusion criteria
In an accompanying commentary, Heath Kelly and Marta Valenciano write that Osterholm and colleagues used more restrictive criteria than those used by the Cochrane collaboration in recent analyses of flu vaccine benefits. The Cochrane group defined efficacy as the relative reduction in risk of symptomatic, lab-confirmed flu and defined effectiveness as protection against flu-like illness, a less specific outcome. Kelly is with the Victoria Infectious Diseases Reference Laboratory in Melbourne, Australia, and Valenciano works at EpiConcept in Paris.
Also, the Cochrane analysis included studies that relied on a serologic definition of influenza, unlike the new meta-analysis, Kelly and Valenciano write. They say this exclusion is the main reason so few studies were included in the new analysis, but it has merit, since serology is less sensitive in people who have received TIV.
The end points used by Osterholm and colleagues—influenza confirmed by RT-PCR or viral culture—are "100% specific, but sensitivity (especially for culture) might be lower," say Kelly and Valenciano. They note that the latest Cochrane review estimated TIV efficacy in healthy adults at 73% (95% CI, 54%-84%) in seasons when circulating and vaccine strains were well matched and 44% in seasons when they were not.
The two commentators second the Lancet authors' call for improved flu vaccines and for routine effectiveness studies of licensed vaccines, using RT-PCR–confirmed infections, "because culture will miss cases and serology alone will overestimate vaccine efficacy and effectiveness." Kelly and Valenciano also suggest reexamining some policy options, without specifying which ones.
Osterholm and colleagues conclude their report with an appeal to develop "a new generation of more highly effective and cross-protective vaccines that can be manufactured rapidly," something that will require partnerships between industry and government. "In the meantime, we should maintain public support for present vaccines that are the best intervention available for seasonal influenza," they say.
Still important protection
In an interview yesterday, Osterholm amplified on these themes.
"Our data strongly support that there can be a moderate level of protection from influenza vaccines," he said. "As an intervention goes, that's still an important level of protection."
As for the sparse evidence for benefits in people over 65 in particular, he observed that a study sponsored by Kaiser Permanente showed an 8% reduction in hospitalization in older people who were vaccinated, adding, "That would seem to be a good return on investment."
Kelley, his coauthor, commented, "For seniors, there's a reduction [in flu risk with vaccination] and it's greater than the reduction from a lot of other interventions in that age-group."
When asked about the possibility of anti-vaccine groups misusing his group's findings to undermine support for flu vaccination, Osterholm commented, "The clear message we've sent here is to continue to use this vaccine. We can't be caught in the mindset that it works all the time or it doesn't work at all. It would be really unfortunate if we threw the baby out with the bath water."
He was also asked about the possibility of using adjuvants to improve existing flu vaccines, since the pandemic vaccine studies focused on adjuvanted vaccines and showed a somewhat higher effectiveness (69%) than the other studies. No adjuvanted flu vaccines have been licensed in the United States.
He pointed out that the pandemic vaccine was closely matched to the circulating virus strain and the studies involved a mostly young population. "Adjuvants by themselves are a long way from being shown as a savior," he said, adding that public worries about the safety of adjuvants could be a barrier to public acceptance.
While affirming that the public health community should continue to support the use of existing flu vaccines, Osterholm said the perception that the vaccines are good enough is a barrier to developing better ones.
"We've heard that a major barrier is the perception in the industry that we're up against a vaccine that's been effective and cheap," he said. He added that many experts are very optimistic about the scientific prospects for much better flu vaccines, but public will and acceptance are needed to move forward.
Osterholm MT, Kelley NS, Sommer A, et al. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis 2011 (published online Oct 25) [Abstract]
Kelly H, Valenciano M. Estimating the effect of influenza vaccines. (Commentary) Lancet Infect Dis 2011 (published online Oct 25) [Extract]
See also:
Oct 25 CIDRAP News story "Public health groups say flu vaccine is best tool, despite limitations"
