Figure 2

Figure 2. Weekly alerts from community event–based surveillance for Ebola virus disease, by district, Sierra Leone, February 27–September 30, 2015.

During February 27–September 30, 2015, a total of 12,126 alerts were reported through CEBS in 9 Sierra Leone districts (Figure 2). Tonkolili was the first district to report on a consistent basis, beginning on February 27, followed by Moyamba (March 6), Pujehun (March 14), Kenema (March 31), Kambia (April 13), Bo (April 18), Kono (April 23), Bombali (April 27), and Kailahun (June 14). From June 14 onward, the districts were collectively reporting an average of 79 alerts per day.

Of the 12,126 alerts reported, 86% (10,421) were for deaths, 14% (1,646) for illnesses, and <1% for unsafe burials (7) or other (52) (Table 1). The alert rate per 100,000 persons per day differed by district and alert type. Of note, Moyamba and Kambia generated the highest rates of death alerts (3.81 and 2.19, respectively), and Kailahun and Bombali reported the highest rates of sick alerts (0.67 and 0.32, respectively) (Table 2). The CEBS death reporting rates were substantially lower than the expected crude death rate of 4.66 deaths/100,000 persons/day used by the World Health Organization (11).

In terms of the 6 defined trigger events, the most commonly cited was >2 sick or dead household members (n = 205, 2%). Sickness or death of a traveler was the second most cited (n = 191, 2%). In total, the 6 defined trigger events accounted for <5% of the alerts (Table 1). The seventh trigger event category (i.e., other) accounted for the most alerts (n = 11,558, 95%); a total of 10,042 (87%) of the alerts categorized as other were for deaths in the community (Table 1). Surveillance supervisors and community health officers escalated 93% of the alerts to the District Ebola Response Centers for follow-up.

A total of 287 (2%) of all persons who triggered alerts met the suspected, probable, or confirmed case definitions. Of these 287 persons, 215 (75%) were detected after death, and 271 (94%) tested negative for EVD. During the study period, 16 confirmed EVD cases were reported by CEBS in Kambia (n = 13) and Tonkolili (n = 3); half of the infected persons were detected while alive. During this same period, the Ministry of Health and Sanitation documented 53 confirmed cases in the CEBS districts. Overall, the sensitivity for confirmed case detection by CEBS was 30% (16/53 confirmed cases), and the PPV was 6% (16/287 suspected, probable, or confirmed cases). Sensitivity was 27% (13/49 confirmed cases) in Kambia and 75% (3/4 confirmed cases) in Tonkolili; PPV was 7% (13/175 suspected, probable, or confirmed cases) in Kambia and 9% (3/33 suspected, probable, or confirmed cases) in Tonkolili.

During the 6-week subanalysis in Kambia, the Ministry of Health and Sanitation database identified 13 confirmed EVD cases in the district. CEBS staff reported 8 of these patients, of whom 7 were alive at the time of the alert. Upon further investigation, we found that 3 of the cases were reported by community health monitors who also served as contact tracers through the contact tracing reporting system. Therefore, CEBS was not the main reporting source for these 3 cases, and the cases were not recorded in the CEBS database or counted as cases detected by CEBS. For the remaining 5 cases, 3 were classified as other trigger events, 1 was in a sick contact of a confirmed case-patient, and 1 was a sick member of a household.

Six of the 13 confirmed case-patients in Kambia had no epidemiologic links when they were detected; CEBS staff identified 4 of these case-patients. The time from symptom onset to detection ranged from 1 to 3 days for the 4 cases identified by CEBS and was 5 and 7 days, respectively, for the 2 cases detected by other components of the national surveillance system (Table 3). For the latter 2 cases, 1 of the ill persons lived in a village that was not covered by a community health monitor at the time of detection, and the case was identified after the person had died; the other person resided in a CEBS-covered village, but the case was not detected until the ill person was admitted to a local hospital.

During the subanalysis in Kambia, surveillance staff from other districts indicated that the CEBS network had detected additional outbreaks that were not caused by Ebola. Community health monitors identified 2 measles clusters in Kono and 1 measles cluster in Bombali, leading to the initiation of investigations and implementation of control measures, including isolation of ill persons and vaccination of susceptible children. Community health monitors in Bombali and Kono also reported suspected chickenpox clusters.

Our evaluation indicates that, during its period of operation, CEBS effectively generated alerts for and detected nearly one third of all EVD cases found in its districts. Although this would rightly be considered a low sensitivity for an independent surveillance system, CEBS was designed to supplement a larger, established system. The low PPV of CEBS also was expected because of the tendency of event-based systems to provide higher sensitivity while generating a large number of false alerts (12) and because there were few true EVD cases. Ruling out EVD in times of low transmission requires investigation of all alerts and isolation and testing of all suspected case-patients (13), most of whom will be determined to be uninfected.

Our data from the subanalysis in Kambia are too few to draw a meaningful conclusion, but they suggest that CEBS may be capable of quickly finding cases with no identified epidemiologic links. If this is true, the system could be used to detect the early stages of new infectious disease outbreaks or to rapidly identify the spread of disease to new geographic areas during ongoing outbreaks or epidemics. Nevertheless, even within this small sample of cases, CEBS failed to detect 2 cases with no known epidemiologic links, which highlights the need for adequate coverage of villages by community health monitors, development of stronger links between communities and health monitors, and vigilance by the monitors.

One unexpected finding was that CEBS detected a large number of deaths in the community. Although not intended to serve as a reporting system for community deaths, CEBS did contribute to death reporting, which was a major initiative of the national government and social mobilization programs. By detecting dead bodies that were then tested and found to be negative for Ebola infection, CEBS helped to confirm the lack of virus transmission, thereby providing some evidence that the epidemic had, in fact, ended in a given district. However, death reports are a late indicator of infection and, thus, do not enable isolation of patients early in the disease course, a control measure that could result in reduced transmission (14–16). The death reporting rates also were considerably lower in most CEBS districts than would be expected based on estimated death rates (11); consequently, CEBS reporting rates were not a substitute for death surveillance or registration.

Another unexpected finding was the detection of 3 measles outbreaks. Given extensive undervaccination and undertreatment of other communicable diseases during the epidemic, it was expected that a large number of disease outbreaks would go undetected (17,18), but we did not anticipate that CEBS would detect a few of them. CEBS staff might have detected these clusters because some of the trigger events, such as >2 sick or dead persons in a household, were not specific to EVD. The staff also might have identified the clusters because they were looking for signs of illness in their communities, irrespective of the cause or signs or symptoms. This finding provides some indication that community-based surveillance could be used to provide early warning of a variety of diseases of public health concern.

Our evaluation also revealed several critical weaknesses in CEBS, some of which may be due to the rapid implementation of the system. First, community health monitors primarily relied on the trigger category other to classify community deaths and alerts, rather than the defined trigger events that they were taught to seek out. This lack of use of defined trigger events could imply that some of the triggers were not sensitive enough to capture Ebola virus transmission. It is also possible that the staff miscategorized the alerts and that many alerts did, in fact, fit a trigger event category. However, most alerts categorized as other were not reported with sufficient information to assess whether they fit a defined trigger event category. Before future systems are widely implemented, the validity of triggers should be more rigorously tested, and refresher training of staff should be regularly provided to reinforce trigger event recall. Rapid implementation at scale is difficult to achieve while also providing comprehensive training and developing strong links between the community and the surveillance team. Ideally, community-based surveillance should be developed and implemented when a large outbreak is not underway. Such a system would then be in place and available for adaptation if a public health crisis arises.

Another weakness of CEBS is that community health monitors reported relatively few illnesses, which is concerning for a system that aimed to detect illness quickly to reduce opportunities for virus transmission. The low proportion of illness alerts may indicate that the intended meaning of illness was unclear or, more likely, that community health monitors were concerned about negative consequences from the community for reporting an event, particularly if the affected person was not infected with Ebola virus.

A final weakness of note is that CEBS detected few unsafe burials. This lack of reporting could reflect the general challenge faced by EVD surveillance in exposing a cultural tradition that communities intentionally guard closely. However, by February 2015, when awareness of the Ebola virus transmission risks of traditional burials were more fully understood, community members may have bypassed their community health monitors to relay information about unsafe burials directly to the preexisting burial management alert system (19).

Our evaluation has several limitations. First, we conducted the evaluation during a period of low Ebola virus transmission; therefore, we cannot draw conclusions about how CEBS would perform in a high-transmission environment. Second, the subanalysis in Kambia lasted only 6 weeks and involved only a few cases. Consequently, the results regarding the ability of CEBS to find cases with no identified epidemiologic links at the time of detection cannot be considered conclusive. Additional implementation and evaluation of CEBS in future EVD outbreaks would provide data to assess the relative merits of this approach. Third, we could not analyze the sensitivity of CEBS to detect other disease outbreaks because no reference standard exists to inform the denominator, and no reporting mechanism exists within CEBS to inform the numerator. Fourth, CEBS was implemented primarily in rural settings, so we do not know how the system would perform in a densely populated urban setting, such as a capital city. Last, an anthropologic understanding of the lack of illness and burial reporting would inform a more comprehensive interpretation of these results.

The Sierra Leone Ministry of Health and Sanitation plans to use community-based surveillance as part of its Integrated Disease Surveillance and Response system, which is tasked with detecting and responding to several priority diseases, conditions, and events (20). Community-based surveillance could extend disease surveillance beyond district health clinics to the village level and provide an early warning function. This would contribute to meeting the core capacity requirements of the International Health Regulations to detect and report disease at the community level to facilitate the immediate implementation of control measures before an outbreak expands further (21,22). Nonetheless, our evaluation reveals several challenges that should be addressed. Detailed assessments should be undertaken to determine how community health monitors recognize and categorize symptomatic illness and the barriers to their ability and willingness to report illness. The assessment results should then be applied to refine trigger definitions and processes. Given that valid disease measures are the basis of an effective surveillance system, these issues are the most pressing ones that need to be studied and addressed to strengthen future iterations of community-based surveillance. In addition, trigger definitions should remain simple to ensure that community health monitors can understand and correctly apply them, which may mean that a few salient, event-based triggers would be more effective than several case-based, specific triggers. Alternatively, more extensive, regularly repeated training of community health monitors might be needed to ensure adequate recall and reporting of more complicated triggers. Furthermore, to sustain efficacy and performance, community-based surveillance must be fully integrated into the overall surveillance system and adequately supported to ensure response capacity.