Jan 19, 2012 (CIDRAP News) – A lengthy new analysis of unpublished clinical trial data is renewing questions about the effectiveness of the influenza drug oseltamivir (Tamiflu), saying that although the drug shortens flu symptoms by about a day, there is no evidence that it reduces hospital admissions.
The report by Tom Jefferson and colleagues, members of the Cochrane Collaboration, also says that, despite sifting thousands of pages of documents, the team couldn't gather enough data to determine whether oseltamivir reduces flu complications and transmission. The 200-page report was published yesterday by The Cochrane Library.
The report also raises some questions about the safety of oseltamivir, saying some reports of adverse events possibly related to the drug were not included in published reports of clinical trials.
Roche, manufacturer of Tamiflu, said numerous clinical trials and medical experience have shown that oseltamivir is effective and usually well-tolerated. The company said it supplied 3,200 pages of data to Jefferson's team, though the researchers say in the report that the company withheld some data they requested.
European and US health officials said they had not had time to study the report, but they noted that it ignores a significant body of observational data supporting the effectiveness of oseltamivir against both seasonal flu and the 2009 pandemic virus.
The analysis was sponsored by the UK National Institute for Health Research Health Technology Assessment program. The Cochrane Collaboration is an international network of researchers who study the quality of evidence for healthcare interventions. Jefferson is an independent epidemiologist based in Rome.
Targeting publication bias
The report is the latest of several reviews of the neuraminidase inhibitors (NIs, including oseltamivir and zanamivir [Relenza]) from Cochrane researchers. The authors say their aim was to eliminate publication bias—the fact that studies with positive findings about an intervention are more likely to be published than those with negative or inconclusive findings.
During the 2009 pandemic, the Australian and British governments asked for an update of the previous Cochrane review of NIs, according to the report. That update ended up being inconclusive, because the researchers could not verify the data behind manufacturer and government claims about oseltamivir. The authors point in particular to a meta-analysis of 10 clinical trials, eight of which have never been published.
"We identified that a large number of studies, including data from 60% of the people who have been involved in randomised, placebo-controlled phase III treatment trials of oseltamivir have never been published," Jefferson said in a press release from Wiley-Blackwell, publisher of the Cochrane Library. "We are concerned that these data remain unavailable for scrutiny by the scientific community."
Accordingly, the researchers decided to try "to get to the bottom of the issue of the effects of NIs by appraising evidence from unpublished clinical study reports . . . and regulatory documents containing comments and reviews," the report says. The authors eventually collected 16,000 pages of data from the drug manufacturers and US, European, and Japanese regulatory agencies, according to the press release.
The team identified 67 clinical trials of oseltamivir and zanamivir, most of them in adults, but 42 of these were unusable because of insufficient information or unresolved discrepancies in the data. Of the 25 trials included in the analysis, 15 were of oseltamivir and 10 of zanamivir, and all were sponsored by the drug manufacturers.
The analysis focused mainly on oseltamivir because it is much more widely used and stockpiled and is on the World Health Organization list of essential drugs. In the end, the researchers decided to postpone its analysis of zanamivir, awaiting receipt of individual patient data promised by GlaxoSmithKline, its manufacturer.
After sifting all the oseltamivir data, the researchers concluded that the drug shortens the time to alleviation of symptoms by an average of about 21 hours, from the average of 160 hours of symptoms in placebo-group patients. But from seven studies that examined the question, they found no effect on the rate of hospitalizations.
The report adds, "Due to limitations in the design, conduct and reporting of the trial programme, the data available to us lacked sufficient detail to credibly assess a possible effect of oseltamivir on complications and viral transmission."
The researchers say they found some inconsistencies between published trial reports and the unpublished trial records. The press release says some of these had to do with oseltamivir safety. The unpublished reports mentioned "serious adverse events," some of them possibly related to the drug, but one of the two most cited publications does not mention those events, while the other says there were "no drug-related serious adverse events," according to the release.
Another finding is that subjects randomly assigned to receive oseltamivir were less likely to be diagnosed as having flu than were control subjects (odds ratio based on eight studies, 0.83; 95% confidence interval, 0.73 to 0.94). The authors assert that oseltamivir probably interfered with the antibody response and that therefore the placebo and treatment arms of the flu-infected subgroup in trials—defined in part by a rise in antibody titers—are not comparable.
They further suggest that this raises doubts about the proposed effects of oseltamivir. One possibility, they say, is that by interfering with antibody production, oseltamivir "has the effect of selecting the strongest antibody responders," who are probably the healthiest people and those least likely to experience flu complications.
Jefferson and colleagues conclude with a prediction that the full clinical study reports from the trials could clarify the outstanding questions about oseltamivir. "These full clinical study reports are at present unavailable to us," they write.
ECDC expert weighs in
Dr Angus Nicoll, head of the influenza program at the European Centre for Disease Prevention and Control (ECDC) and honorary professor at the London School of Hygiene and Tropical Medicine, said it was difficult to comment definitively on a 200-page document that has only just gone public, but it did not seem to be saying much that the Jefferson group had not said before.
"What is important now is that independent scientists comment on the document and ask questions through the 'Submit Comments'field at the manuscript," he said (see link below). "As I understand it, these comments will be responded to—and that is the Cochrane process for reviews like this so they now undergo open peer review."
However, Nicoll felt that a fundamental design problem of the review is that it excludes all observational data, commenting, "Jefferson has a very pure approach not considering observational data." He said the disadvantage of this approach is that it lacks power to detect uncommon but important effects—either good results like the prevention of severe flu complications or rare adverse effects of drugs.
"Our reading of the observational data, like that published in the BMJ by China CDC recently from the pandemic [see link below], is that early oseltamivir use is associated with less severe complications," he said. "The observational data are weak, but they are mostly pointing towards a protective effect. That particularly came out in the pandemic, but is also the case with seasonal flu. Just relying on trial data can lead to the old trap of thinking that absence of evidence of effectiveness (protection against complications) equals evidence of no effect."
Nicoll also said that there are two or three evidence-based reviews of observational data coming and they should be awaited so that the trial and observational data can both be considered.
He also took issue with the Cochrane team's conclusion that there is no clinical trial evidence that NIs reduce severe outcomes in flu patients. He said they discounted analyses by Miguel Hernan and Marc Lipsitch of Harvard University.
In an earlier report, according to Nicoll, the Jefferson group examined a meta-analysis of clinical trials of oseltamivir by Kaiser and others. (The new report says the meta-analysis included 10 manufacturer-funded trials, eight of which have never been published.) When Jefferson and colleagues tried to reanalyze the data in the Kaiser paper they couldn't reproduce the findings, Nicoll said.
But subsequently Hernan and Lipsitch asked for more primary data from the 10 trials and reanalyzed them along with additional data, producing results consistent with oseltamivir reducing flu complications. Their report was published in Clinical Infectious Diseases last year (see link below).
"Lipsitch and Hernan are a very well respected independent group, so when we saw their findings we realized there was evidence of reduced complications from trials in seasonal flu," Nicoll said.
Finally, Nicoll commented that the bottom line of the Jefferson reviews is always that there should be more large, randomized trials, independent of industry and now, even independent of governments. Clearly those could not include people at higher risk from flu, such as older people, young children, and those with other conditions, he said, noting that the sample sizes needed for such trials would be impossibly large.
"There are some questions that you have to answer without relying on trials alone," he concluded.
CDC official notes observational data
Tim Uyeki, MD, MPH, a flu expert at the US Centers for Disease Control and Prevention (CDC), said he had not read the report, but, like Nicoll, he commented that the Cochrane review did not consider the observational data supporting oseltamivir. Uyeki is deputy chief for science in the Epidemiology and Prevention Branch of the CDC's Influenza Division.
"There is a substantial amount of data from observational studies of oseltamivir treatment of seasonal flu and pandemic flu to suggest clinical benefit when treatment is initiated early," and even some benefit when treatment is begun more than 2 days (but less than 5 day) after illness onset," Uyeki said. "There are some limited studies that have also suggested a reduction in risk of admission to ICU or death."
He added that the observational studies all point in the direction of clinical benefits. "When the ACIP [Advisory Committee on Immunization Practices] and CDC and other groups are making clinical recommendations and recommendations for public health, it's important to look at all of the evidence that's available, including evidence from randomized placebo-controlled trials as well as observational data," he added.
In a statement released yesterday, Roche officials defended oseltamivir's track record and the company's openness in supplying data to Jefferson's team.
"Numerous clinical trials and real-life medical experience show Tamiflu is effective in reducing the severity and duration of influenza symptoms as well as specified secondary complications," the company said. "Analyses by health authorities and independent research groups show that, during the 2009 H1N1 pandemic, Tamiflu saved lives and reduced hospitalization. Similar benefits have also been reported for seasonal influenza."
The company said the drug has been used to treat and prevent influenza in 90 million people, including nearly 20 million children, in 80 countries. In addition, "clinical studies show that Tamiflu is well tolerated in the overwhelming majority of cases. Like all medicines, Tamiflu can have side effects and these are listed in the Summary of Product Characteristics."
Roche also said it provided the Cochrane group "3,200 pages of very detailed information, enabling their questions to be answered."
"All completed Roche sponsored clinical studies on the safety and efficacy of Tamiflu are available as peer-reviewed publications or in summary form on www.roche-trials.com," the statement said. "More detailed clinical trial reports are available for use by investigators on a password-protected site, enabling researchers to verify the findings of these studies and publications relating to them."
Jefferson T, Jones MA, Doshi P, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Library 2012 January [Abstract]
Accompanying Cochrane Library editorial
Wiley-Blackwell press release via EurekAlert
BMJ report on effectiveness of oseltamivir in China
Clinical Infectious Diseases abstract of meta-analysis by Hernan and Lipsitch