May 16, 2007 (CIDRAP News) – At first glance, the Web page looks like an overhead shot of a fantastic game board: a map—identifiably Los Angeles—sprinkled with faceted roundels in a half-dozen colors.
But the data graphically displayed at WhoIsSick.org are from the real world. The roundels represent reports of symptoms volunteered by site users: runny nose, cough, fever, headache, muscle aches, and digestive trouble. The site's founder, a California tech entrepreneur named PT Lee, drew on the new Web technology of Google Maps and the Web trend toward user participation to create a 21st-century service that a 19th-century epidemiologist would recognize: geographic surveillance of illness trends.
WhoIsSick, which went live 2 months ago after a year of planning, grew out of Lee's frustration over his wife's holiday bout with a gastrointestinal bug.
"We sat in the emergency room for 4 hours, just to find 5 minutes into seeing the doctor that there was a horrible stomach flu going around," Lee said. "Afterward, I thought: How does someone find out if something is going around? Traditionally they ask their friends, or maybe their doctor—but that's inefficient. Maybe the Web could provide a way to tap the wisdom of crowds."
The site has registered about 200,000 visitors so far. Its roughly 20,000 illness reports have come from the United States, Canada, the United Kingdom, Germany, and Australia. On May 11, there were 846 illnesses reported in San Francisco, 155 in Atlanta, 154 in Minneapolis, and 657 in New York City. Parents love the site because it alerts them their kids might bring something home, Lee said, though some observers have called it "a hypochondriac's dream."
Like other user-generated Web sites such as Wikipedia and the FluWiki, WhoIsSick takes data from anyone who is interested and has an Internet connection that can handle its data-rich pages. But whether the data it gathers prove useful or merely amusing, the site confirms something that long predates the Internet: the intuitive appeal of symptom information collected from many people—otherwise known as syndromic surveillance—as a tool for monitoring public health.
Enthusiasm wanes
WhoIsSick's citizen-generated syndromic surveillance has emerged just as official syndromic surveillance, conducted by federal, state, and local health authorities, is facing significant reexamination and challenge.
Small-scale syndromic surveillance—pencil-and-paper monitoring of school absenteeism or cruise-ship stomach bugs—has a long history. But the practice received a huge boost after the 2001 anthrax letter attacks. Planners fixed on electronic syndromic surveillance as the best hope for providing an early warning of the next bioterrorist attack, contending that rapid monitoring and analysis of symptom clusters, emergency department complaints, over-the-counter drug sales, and other data sources could alert them to health anomalies faster than traditional paper-based reports of diagnoses.
The spike in interest was followed by a vast increase in funding. Congress has appropriated more than $1 billion for state and local health departments to improve their preparedness, and given $230 million to the largest syndromic surveillance program, the Centers for Disease Control and Prevention's BioSense, intended to connect the CDC to hospitals nationwide. About 30 syndromic surveillance programs, some of them predating BioSense, now operate around the United States; potentially thousands could be created if every public health jurisdiction constructed one.
But 5 years after the initial burst of enthusiasm, syndromic surveillance is losing adherents. In a Senate hearing a year ago, two prominent public health scientists, Dr. Tara O'Toole of the Center for Biosecurity of the University of Pittsburgh Medical Center and Dr. Nicole Lurie of the RAND Corp., recommended that Congress cease funding the BioSense program until fundamental information-sharing issues are worked out. And last December, a number of speakers at a 2-day meeting convened by the Institute of Medicine (IOM) expressed skepticism over how well syndromic surveillance is working.
"The expectations for syndromic surveillance were unrealistic, initially; there was the expectation it would magically give us, immediately, all kinds of advantages," said Stephen S. Morse, PhD, a member of the IOM Forum on Microbial Threats, which called the meeting, and director of the Center for Public Health Preparedness at Columbia University. "We still don't know how to use it most effectively, what kinds of systems work best and what the limitations are. Syndromic surveillance is very promising, but obviously it is still a work in progress."
The reexamination of syndromic surveillance comes as both US and global public health agencies turn increasingly toward it. The Pandemic and All-Hazards Preparedness Act, passed Dec 7, 2006, requires the Department of Health and Human Services to use information technology to create a national surveillance network. And the World Health Organization's new International Health Regulations, which take effect Jun 15, require countries to monitor not just specific diseases, but any apparent outbreak that could constitute a "public health emergency of international concern."
Too many false alarms
The new skepticism over syndromic surveillance is directed at its claimed usefulness for early detection of a bioterror attack. That claim is hard to test, since—as far as anyone can tell—there has been no bioterrorism to warn of.
"It is not fair to say that no attack has ever been discovered by syndromic surveillance," Morse pointed out, "because how many attacks have we had? There have been two in the United States, the Rajneeshees and the anthrax letters" in 1984 and 2001—both before current syndromic surveillance systems were built.
But those concerned with syndromic surveillance's failings point not to the systems' failure to warn, but to how often they warn of outbreaks that do not exist.
The New York City Department of Health and Mental Hygiene has more experience with syndromic surveillance than any other public health jurisdiction: It created its first system in 1994 and in the ensuing decade tested, and often rejected, a number of means of monitoring health data. It now runs six separate detection systems that electronically monitor patient complaints in emergency departments, dispatch codes for 911 calls, prescription and over-the-counter drug sales, reports from school nurses, and some outpatient information. Much of its early experience involved checking out false alarms.
"We were spending a lot of time investigating things that we quickly came to realize were not of public health significance," said Assistant Commissioner Dr. Marci Layton.
Every false positive, she said, required a decision whether to mobilize health department resources to track down a possible victim, take a history, and obtain specimens for testing. "We were wasting the public health infrastructure, our communicable-disease folks, having them go in and investigate things that in the end didn't mean something, while at the same time they were not working on other issues," Layton said.
Because bioterror events are so rare, the probability of any alarm being false is high, according to Michael Stoto, PhD, and colleagues at Georgetown University. They estimate that if 1,000 US jurisdictions created syndromic surveillance systems, they would collectively experience more than one false alarm per day.
But making a system less sensitive, so that it rings fewer false alarms, risks making it less timely—thereby undermining the early-warning mission for which it exists.
Many health departments that receive a syndromic surveillance alarm will wait 1 or 2 days and look for corresponding blips in other types of data before launching an investigation, said Stoto, a professor of health systems administration and population health at Georgetown. Simulations by Stoto and his collaborators and by two other research teams have found that, with the detection algorithms that syndromic surveillance systems use to process data, they would take days to spot a large bioterror attack. The signal of smaller outbreaks would not rise through the surrounding noise—though they might be spotted by an alert clinician first.
Harvesting data on natural outbreaks
But while syndromic surveillance is being questioned as an early-warning system for new outbreaks, it is gaining backers for a different use: to deliver richer information on existing, naturally occurring outbreaks.
New York City, for example, has used the same surveillance systems that returned unsatisfying early-warning data to gather very detailed information about outbreaks of norovirus and the start and progress of influenza season. The automated system that analyzes patients' chief complaints in emergency departments delivers sensitive and specific reports of flu-like illness.
"This is not something we could do with previous systems in anything close to the same level of detail," said Rick Heffernan, director of the data unit in the city's Bureau of Communicable Disease. "In a severe year, we would be able to document whether severity and the ages affected were atypical. In a pandemic there would be a lot of interest in as much data as possible, so there is a real value and role there."
The Clark County Health District in Las Vegas gathered enough data from its syndromic surveillance system to detect the start of flu season 2 weeks earlier than usual, according to a presentation published by the National Association of City and County Health Officials. The extra time let local health officials push test kits out to sentinel physicians, allowing for early identification of the circulating strain; it also gave them enough lead time to communicate with the local media to encourage flu vaccination before the local epidemic.
And the city government of Lubbock, Tex., and the Texas Department of State Health Services reported at last year's National Syndromic Surveillance Conference that a private-label system called SYRIS (not part of the BioSense program) allowed officials to assess illnesses among Hurricane Katrina refugees evacuated to the area and determine that the outbreaks were limited and would not spread.
In each case, syndromic surveillance systems did not deliver warnings within hours of the presence of unusual pathogens. Instead, they returned detailed information within several days about outbreaks of common organisms—information that local jurisdictions used to make decisions about alerting residents and deploying resources.
"Syndromic surveillance is potentially a very valuable technique," Stoto said, "but we have been concentrating on one potential use that is probably not the most promising one."
Some question funding
Given the evidence that syndromic surveillance may not work as it was intended to, users in the public health system are beginning to ask whether funding should still be directed chiefly toward obtaining early warnings.
At the 2006 Senate hearing that questioned the BioSense program, Dr. Lisa Kaplowitz, deputy commissioner of the Virginia Department of Health, urged lawmakers to allocate funds for staff as well as technology. "I just want to put in a plea that, while we build our electronic IT systems, we make sure that we have the people who can analyze the data and who can generate the rapid response," she said.
Layton, of New York City, said the same.
"It's a question of whether you invest in the smoke alarm, or the firemen and the trucks to respond to the alarm," she said. "To me, syndromic surveillance is just a smoke alarm. If we do not equally invest in the ability to respond when there is a concerning alarm, that would be naive."
See also:
WhoIsSick.org
http://whoissick.org/sickness/
Report by Lubbock and Texas state health officials on use of SYRIS system to detect illnesses among Katrina victims
http://thci.org/_documents/temp/Syndromic Surveillance Conf2006.doc
