Mar 11, 2011 (CIDRAP News) – Researchers from the University of Michigan say one measure of flu vaccine efficacy that has been used in a number of past controlled trials is not very accurate, and that this may have led to a degree of overselling of the protection the vaccines provide.
Writing in the Journal of Infectious Diseases, Joshua G. Petrie and colleagues describe the effects of using different "end points," or measures for detecting infection, in a 4-year flu vaccine efficacy trial they conducted. They found that using serologic measures—an increase in flu antibodies—in vaccinated individuals as evidence of infection leads to an overestimate of vaccine efficacy.
They also concluded that virus isolation—growing the flu virus in cell culture—was not very accurate, as the viruses are hard to culture, resulting in missed cases. The most accurate tool, they found, was real-time polymerase chain reaction (RT-PCR).
In their trial, the use of RT-PCR showed that the protective efficacy of the vaccine was about 70%. "That may suggest that we should lower the usual description of vaccine efficacy from 70%-90% in healthy adults to closer to 70%; however, further confirmation by other studies is desirable," the report says.
"Basically what we're saying is that in some studies, depending on the design, there may have been an overestimate of vaccine efficacy based on serologic outcome," Arnold S. Monto, MD, senior author of the study, told CIDRAP News. "It requires reevaluation of some of these studies."
The report says that trials conducted in the US military decades ago established that inactivated flu vaccines were 70% to 90% effective in healthy adults. PCR was not available then, and the researchers typically used serology to identify infections in the vaccinated participants versus the placebo recipients.
The article explains that some have suggested that the use of this serologic criterion may lead to an overestimation of vaccine efficacy because of an "antibody ceiling." The idea is that once antibody titers increase in response to a vaccine, they can rise no higher in response to actual infection, and infections in vaccinated people would not be identified. Despite this concern, the use of serologic outcomes in vaccine efficacy trials has continued to some extent, the authors say.
4-year trial shows overestimation
The Michigan researchers conducted a vaccine efficacy trial that spanned the four flu seasons from 2004-05 through 2007-08. Healthy adults between the ages of 18 and 49 were randomly assigned to receive either the trivalent inactivated (injected) vaccine (TIV), the live attenuated (nasal spray) vaccine (LAIV), or a placebo, with only 1 of every 6 volunteers getting the placebo. The interventions were used each year except 2006-07, when the duration of protection provided by vaccination the previous season was studied.
The team used all three methods—RT-PCR, viral culturing, and serology—to test for flu in those who had flu-like illnesses. For the serology, the participants' antibody levels were assessed through hemagglutination inhibition (HAI) titer before and after vaccination and at the end of the flu season. A fourfold increase in antibody titer after vaccination was regarded as evidence of response to the vaccine, and a further fourfold increase after the season was defined as evidence of infection.
The team recruited 1,247, 2,058, and 1,952 volunteers in the three intervention seasons. As confirmed by RT-PCR, a total of 166 participants contracted influenza A/H3N2 over the three seasons, and 29 contracted influenza B. Because only two cases of H1N1 flu were detected, that subtype was left out of the analysis.
All the H3N2 and B cases that were confirmed by cell culture were also confirmed by RT-PCR, but only 69% of the cases confirmed by RT-PCR were also confirmed by cell culture.
The serologic outcomes differed between the TIV and LAIV groups.
Most (79%) of the volunteers who contracted PCR-confirmed H3N2 flu after receiving TIV showed a serologic response to the vaccine. However, only 23% of this group showed the subsequent further increase in antibodies signaling serologic evidence of infection.
Among the LAIV recipients, only 22% of those who contracted H3N2 showed a serologic response to the vaccine. But 87% of this group showed serologic evidence of infection.
Among the 29 type B infections in vaccinated participants, 63% of TIV recipients and 22% of LAIV recipients showed a serologic response to the vaccine. Most of these participants showed serologic evidence of infection, but because appropriate blood samples were not available for all, the percentages were not reported.
The researchers also assessed the severity of flu cases in vaccinated and placebo recipients. They found that for H3N2 cases, duration of illness was only slightly less in the vaccinated people, but for type B cases, the difference was greater and nearly significant (3.7 days versus 9.4 days; P=.055). Most of the people in both groups described their cases as severe, but the vaccinated participants were less likely to seek medical care. Also, reduced viral shedding was observed among the vaccinated people, as estimated from the RT-PCR values.
Overall, the authors say their findings, including previous reports on the separate years of their trial, "have demonstrated that, using only a serologic outcome, estimates will be biased in favor of overestimating inactivated vaccine efficacy." They add that the use of RT-PCR for detecting flu "has now become a validated standard."
They acknowledge that serology looked better as evidence of infection in the type B cases and the LAIV group. For the type B cases, the difference might be explained by the small number of cases or the circulation of different type B lineages. In this study, the live vaccine did not seem to trigger strong antibody responses by itself, so an increase in antibody after subsequent infection was easily seen, the report says.
If, as indicated by the Michigan findings, flu vaccines are about 70% effective in young, healthy adults, and if about 50% of the population is vaccinated, then about a quarter of all flu cases may occur among vaccinated persons, the authors suggest.
Monto said that proportion is likely to be even higher, given that flu vaccines are less protective in elderly people. But he noted that there is some evidence that even when vaccination doesn't prevent flu, it reduces the risk of complications and hospitalization.
He said physicians need to remember that if someone who has been vaccinated has a flu-like illness, they shouldn't assume it can't be flu. "This is not really news to most people, but for some reason they have to be reminded about this," he commented.
Other experts concur
The Michigan findings drew favorable comments from other experts consulted by CIDRAP News.
Robert B. Couch, MD, a respiratory-disease and vaccine expert at Baylor College of Medicine in Houston, said he concurs with the idea that RT-PCR should be the standard for confirming infection in flu vaccine trials.
"I agree with [the authors'] conclusion but primarily because RT-PCR is a great 'equalizer' for all sites, since RT-PCR should carry about the same sensitivity and specificity at each site, provided the expertise is there to prevent false-positives," Couch commented by e-mail. "This will expand the number of sites that can do good sensitivity testing for influenza infections and vaccine evaluations. Most labs doing such testing lack expertise in flu methods, as most only do HAI tests for serology and isolation tests in a general lab or other non-flu-experienced location. For these, RT-PCR testing is superior."
Couch also said it's his understanding that the Food and Drug Administration asks for virus isolation rather than PCR data in flu vaccine studies. "If that's still the case, this will help in modifying that requirement," he said.
As for how to describe the efficacy of flu vaccines, Couch said the 70% figure is OK for him, but so is 70% to 90%, provided the vaccine matches the circulating strains. "I will support 'about 70%' because Arnold [Monto] uses this to say vaccines need to be improved, which I very strongly support," he added.
Lone Simonsen, PhD, a research professor and research director in the Department of Global Health at George Washington University, said she was interested to learn that the high end of the 70% to 90% efficacy estimate came from 1960s trials in which seroconversion was used as an end point.
She said she agrees with the authors that the 90% efficacy measured in those studies is "almost certainly an overestimate due to the 'antibody ceiling' bias—so that those vaccinated were far less likely to seroconvert and be identified as seroconversion-confirmed cases if they were truly infected, than unvaccinated individuals. So the authors make a good case for the true efficacy of TIV being in the range of 70% for young adults."
Simonsen also said it's important to remember that the revised 70% efficacy estimate pertains only to young healthy adults and only when the vaccine matches up well with circulating strains. "Also, the number of cases of influenza in the study was small, and [the authors] did not study H1N1 infections," she observed.
The findings drew praise from Nick Kelley, MSPH, who has conducted an in-depth analysis of flu vaccine effectiveness studies in his capacity as assistant director of the CIDRAP Comprehensive Influenza Vaccine Initiative, a project funded by the Alfred P. Sloan Foundation. Kelley is a research assistant at the University of Minnesota Center for Infectious Disease Research and Policy, publisher of CIDRAP News.
Kelley said that doubts about the validity of using serologic measures have existed since the earliest flu vaccine efficacy studies in the 1940s and 1950s. The study "puts the final nail in the coffin as far as using serology as an end point for vaccine efficacy studies," he said. "It basically reminds us of what we knew back in the 1940s and 1950s but forgot."
He commented that serologic end points have generally led to much higher estimates of efficacy than has lab confirmation.
The bottom line for the Michigan researchers is that better vaccines are needed. "These findings reinforce the need for improved influenza vaccines, perhaps even for young adults, the group in which the vaccine is thought to work best," they write.
Petrie JG, Ohmit SE, Johnson E, et al. Efficacy studies of influenza vaccines: effect of end points used and characteristics of vaccine failures. J Infect Dis 2011; early online publication Mar 4 [Full text]
