GAO questions anthrax detection methods

Apr 18, 2005 (CIDRAP News) The Government Accountability Office (GAO) says federal agencies may not be able to reliably rule out the presence of anthrax contamination in a building because their sampling and detection methods have not been adequately tested.

The GAO's conclusion is based on an examination of the approaches used by three agenciesthe US Postal Service (USPS), the Environmental Protection Agency (EPA), and the Centers for Disease Control and Prevention (CDC)to hunt for anthrax in 286 postal facilities after the anthrax mailings in 2001.

"None of the agencies' activities to detect anthrax contamination in the postal facilities were validated," the GAO said in prepared congressional testimony based on the report.

"The lack of validation of agencies' activities, coupled with limitations associated with their targeted sampling strategy, means that negative results may not be reliable," the report says. It recommends that the secretary of homeland security take on the task of ensuring that pathogen detection methods are validated and coordinating environmental testing for pathogens by different agencies.

The GAOCongress's investigative agencyprepared its report for the House Government Reform Subcommittee on National Security, Emerging Threats, and International Relations. GAO staff member Keith A. Rhodes gave a 19-page summary of the 119-page report in testimony prepared for delivery to the committee on Apr 5.

For its evaluation, the GAO broke the agencies' sampling activities down into five steps: sampling strategy (where and how many samples were gathered), sample collection methods, sample transportation, sample extraction in the lab, and sample analysis.

The agencies mainly used a targeted sampling strategy, collecting samples mostly from areas they judged likeliest to be contaminated. The GAO takes issue with this approach, saying the agencies should have done probability sampling to achieve "wide-area coverage" and provide statistical confidence in negative results.

The report also finds fault with the methods used to gather samples. In most cases the agencies used dry swabs on surfaces, though they also used some moistened swabs, wet wipes, HEPA (high-efficiency particulate air filter) vacuuming, and air samples. None of the collection methods were tested in advance, so the agencies "had no information available for reliably choosing one method over another and no information on the limits of detection to use when evaluating negative results," the report states.

In shipping their samples, the agencies followed regulations designed mainly to prevent leaks and protect workers. The GAO complains that the regulations did not address the matter of protecting the samples from extreme temperatures or other factors that could compromise their biological integrity and lead to false-negative test results.

The agencies mostly used processing solutions to extract samples from dry swabs and get them onto plates for culturing, though in some cases they brought dry swabs into direct contact with plates. Either way, the GAO says, "definitive scientific information regarding extraction efficiency is lacking," casting additional doubt on the reliability of negative results.

The agencies used any of four different preliminary tests and three confirmatory tests to identify anthrax in extracted samples. The number of different tests used, in combination with differences at other stages of the sampling process, increases the level of uncertainty about the results, the GAO contends.

Of the 286 postal facilities tested, 23 tested positive. For two of the 23in West Palm Beach, Fla., and Wallingford, Conn.the first tests were negative but later tests turned positive. The Wallingford facility didn't test positive until the fourth round of testing. In another building, in West Trenton, N.J., no anthrax was found in three rounds of tests, even though a worker had contracted cutaneous anthrax.

The GAO puts the bottom line thus: "Because the agencies did not use an empirical process to validate their testing methods, the agencies had limited information available for reliably choosing one method over another and no information on the detection limit to use when evaluating negative results."

"The sampling strategy used by the agencies could not provide any statistical confidence with regard to the basic question: Is this building contaminated?" the report says.

The agencies have made some changes in their procedures on the basis of lessons learned from the 2001 attacks and have funded some new research. But these efforts, though important, "do not address the issue of validating all activities related to sampling," in the GAO's view.

The GAO recommends that the secretary of the Department of Homeland Security (DHS) lead an effort to develop a definition of validation and to ensure that the whole set of sampling activities is validated. That should include studies to "develop probability-based sampling strategies that take into account the complexities of indoor environments." Also, the DHS chief should coordinate the activities of agencies with expertise in environmental testing.

When they read a draft of the report, the CDC, USPS, and DHS all agreed that the methods for detecting anthrax were not validated and that a systematic validation effort is needed, the report states.

However, DHS officials took exception to the role the GAO recommends for their department. DHS maintained that the EPA has "the primary responsibility of establishing the strategies, guidelines and plans for the recovery from a biological attack, while HHS [the Department of Health and Human Services] has the lead role for any related public health response and guidelines," the final report states. DHS promised to "coordinate with EPA to ensure appropriate investments are made to explore improved sampling."