\documentclass{article}
\usepackage[affil-it]{authblk}
\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\usepackage{natbib}
\usepackage{url}
\usepackage{hyperref}
\hypersetup{colorlinks=false,pdfborder={0 0 0}}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
\errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}
\usepackage[ngerman,english]{babel}
\usepackage[utf8]{inputenc} % for input encoding
\usepackage{authblk} % for authors multi affiliations
\begin{document}
\title{Measles Outbreak Response Decision-making Under Uncertainty: A retrospective analysis}
Christopher J. Fonnesbeck$^{1}$, Katriona Shea$^{2,3}$, Spencer Carran$^{3}$, Jose Cassio de Moraes$^{4}$, Christopher Gregory$^{5}$, James L. Goodson$^{6}$, and Matthew Ferrari$^{2,3}$
$^{1}$Department of Biostatistics, Vanderbilt University School of Medicine, Eleventh Floor, Suite 11000, 2525 West End Avenue, Nashville, TN, USA \\
$^{2}$Department of Biology and Intercollege Graduate Degree Program in Ecology, 208 Mueller Laboratory, The Pennsylvania State University, University Park, Pennsylvania, USA \\
$^{3}$Center for Infectious Disease Dynamics, Department of Biology, Eberly College of Science, The Pennsylvania State University, University Park, Pennsylvania, USA \\
$^{4}$Faculdade de Ciencias Medicas da Sanda Casa de Sao Paulo, Sao Paulo, Brazil \\
$^{5}$Arboviral Diseases Branch, Division of Vector-Borne Diseases, National Center for Enteric and Zoonotic Infectious Diseases, US Centers for Disease Control and Prevention, Fort Collins, Colorado, USA \\
$^{6}$Accelerated Disease Control and Vaccine Preventable Disease Surveillance Branch, Global Immunization Division, Center for Global Health, US Centers for Disease Control and Prevention, Atlanta, Georgia, USA \\
Journal for submission: Journal of the Royal Society Interface\\
Abstract Word Count: 193 (Limit: 200)\\
Text Word Count: 6953 (Limit: 8000), note this is inclusive of references and figure legends\\
Table count: 0 (Limit: NA)\\
Figure count: 8 (Limit: NA)\\
References: 43 (Limit: NA)
\date{\today}
\maketitle
\selectlanguage{english}
\begin{abstract}
Resurgent outbreaks of vaccine-preventable diseases that have previously been controlled or eliminated have been observed in many settings. Reactive vaccination campaigns may successfully control outbreaks but must necessarily be implemented in the face of considerable uncertainty. Real-time surveillance may provide critical information about at-risk population and optimal vaccination targets, but may itself be limited by the specificity of disease confirmation. We propose an integrated modeling approach that synthesizes historical demographic and vaccination data with real-time outbreak surveillance via a dynamic transmission model and an age-specific disease confirmation model. We apply this framework to data from the 1996-1997 measles outbreak in S\selectlanguage{ngerman}ão Paulo, Brazil. To simulate the information available to decision-makers, we truncated the surveillance data to what would have been available at 1 or 2 months prior to the realized interventions. We use the model, fitted to real-time observations, to evaluate the likelihood that candidate age-targeted interventions could control the outbreak. Using only data available prior to the interventions, we estimate that a significant excess of susceptible adults would prevent child-targeted campaigns from controlling the outbreak and that failing to account for age-specific confirmation rates would under-estimate the importance of adult targeted vaccination.
Keywords: measles epidemic, Bayesian disease model, outbreak control, vaccine intervention, decision analysis%
\end{abstract}\selectlanguage{ngerman}%
\section{Introduction}\label{introduction}
Reports of resurgent outbreaks of vaccine-preventable diseases following long periods of relative absence are increasingly common \hyperref[csl:1]{[1]}; \hyperref[csl:2]{[2]}; \hyperref[csl:3]{[3]}; \hyperref[csl:4]{[4]}. Several factors may contribute to the occurrence of such outbreaks. McLean and Anderson \hyperref[csl:5]{[5]} predicted that such outbreaks should be expected because of the ``honeymoon'' phenomenon following the introduction of vaccination, whereby post-vaccination cohorts no longer experience high rates of natural immunization to supplement population immunity following vaccination activities \hyperref[csl:6]{[6]}. Further, population-level vaccination rates may decline over time as immigration from areas of low vaccination coverage lead to a buildup of susceptible individuals, or the reduction in individual infection risk \hyperref[csl:7]{[7]} leads to apathy about vaccination within the population. Also, local stochastic extinction may result in temporary breakdown of local transmission, even though populations remain susceptible to subsequent outbreaks upon the reintroduction of infection \hyperref[csl:8]{[8]}.
Increasingly, in the event of a measles outbreak, outbreak response immunization (ORI) is recommended as an intervention. The goals of these ORIs are two-fold: 1) to protect high risk groups (\emph{i.e.} young children) and 2) to attenuate the current outbreak \hyperref[csl:9]{[9]}; \hyperref[csl:10]{[10]}. To achieve the former goal, ORI campaigns routinely target children 6-59 months of age \hyperref[csl:9]{[9]}. To achieve the latter goal, the campaign must reach some target level of immunization (\(P_c\)) --- \emph{i.e.} a percentage reduction of the susceptible population --- such that effective reproductive number, \(R_e\), will be below 1 and the outbreak will end. From the standard susceptible-infected-removed (SIR) model, this level of immunization is \(P_c = 1-1/R_e\) \hyperref[csl:11]{[11]}. To identify this target coverage and appropriately plan a campaign, one must estimate both the value of \(R_e\) itself, which determines the necessary reduction of the susceptible population required to end the outbreak, and the age distribution of the susceptible population, which allows us to identify the critical age classes to be targeted in a campaign. For example, if \(R_e=1.5\) one must then reduce the susceptible fraction by 33\%; if 80\% immunization of susceptibles in the target age groups can be achieved (considering both campaign coverage and vaccine efficacy), then at a minimum, the intervention should target the first $a$ age groups whose summed proportion $\frac{\sum_{i=1}^a s_i}{S}$ of the susceptible population $S$ exceeds $0.33/0.8 = 0.41$.
Experience with past outbreaks can provide guidance about likely values of \(R_{e}\) \hyperref[csl:12]{[12]} and the likely distribution of the susceptible population \hyperref[csl:13]{[13]}; \hyperref[csl:14]{[14]}. However, in the case of resurgent outbreaks, which follow periods of relatively low measles incidence, there may be insufficient data on which to base estimates of \(R_e\). In these settings, it is the current outbreak itself which may provide the most relevant empirical information \hyperref[csl:12]{[12]}; \hyperref[csl:15]{[15]}; \hyperref[csl:16]{[16]}. However, clinical confirmation of measles cases through case-based surveillance systems has relatively low specificity, particularly in settings of low prevalence \hyperref[csl:17]{[17]}; \hyperref[csl:18]{[18]}; \hyperref[csl:19]{[19]}; \hyperref[csl:20]{[20]} when other illnesses that result in fever and rash (e.g. rubella, Dengue) may be misdiagnosed as measles \hyperref[csl:18]{[18]}. Given that the various etiologies of febrile illness may disproportionately affect different age classes, reliance on a clinical definition alone may result in a biased assessment of the age classes at risk \hyperref[csl:17]{[17]}; \hyperref[csl:12]{[12]}. Further, misdiagnosis of cases may bias the assessment of the rate of increase of total cases and lead to a biased estimate of \(R_{e}\) and hence the vaccination coverage necessary to limit the outbreak. Though serological confirmation of measles cases is preferred to clinical confirmation, resources often limit the proportion of clinical cases that can be confirmed by serology in outbreak settings in time to be of use to decision-makers.
A variety of methods are available to estimate \(R_{e}\) from early surveillance data \hyperref[csl:12]{[12]}; \hyperref[csl:21]{[21]}; \hyperref[csl:22]{[22]}; \hyperref[csl:23]{[23]}. The use of mathematical modeling to estimate the age distribution of the susceptible population during an outbreak is rare; decisions about age targeting have been classically based on prior experience or early evaluation of cases confirmed earlier in the outbreak \hyperref[csl:24]{[24]}. Here we present an epidemic model that uses serological confirmation on a subset of cases to estimate age-specific confirmation, and use these estimates to correct the observed number of reported cases before being used to estimate epidemic parameters.
The age distribution of the susceptible population is usually unknown at the start of a measles outbreak. The mean and variance of the age distribution of susceptible individuals is expected to increase as the prevalence of infection declines \hyperref[csl:13]{[13]}; \hyperref[csl:25]{[25]} and during periods of measles absence \hyperref[csl:12]{[12]}. As a result, historical surveillance data may be misleading with respect to the current distribution of susceptibles. Initial estimates of the susceptible population can be reconstructed from demographic rates, historical records of routine and supplemental vaccination coverage, and measles incidence\hyperref[csl:26]{[26]}; however, uncertainty in these rates and historical incidence mean that \emph{a priori} estimates of the susceptible population may be significantly biased. Further, lack of data on migration, heterogeneity of vaccination coverage, and clustering of susceptibles means that these estimates will be highly uncertain in the best of circumstances. The 1997 measles outbreak in Sao Paulo, Brazil, presented in detail below had many more adolescent and adult cases than was expected based on historical rates \hyperref[csl:27]{[27]}. This pattern of unexpectedly wide age distributions of cases has been recently seen in outbreaks in Malawi \hyperref[csl:28]{[28]} and Mongolia (ref: http://www.wpro.who.int/mongolia/mediacentre/releases/20160505-measles-outbreak-faqs/en/).
This work comprises a retrospective analysis of a resurgent measles outbreak in Sao Paulo, Brazil in 1996-1997. This outbreak followed several years of relatively low measles incidence as a result of both routine and supplemental measles vaccination. Brazil began its national immunization plan in 1973, with a single dose of measles vaccine. In 1992, Brazil introduced a nation-wide recommendation of a second routine dose of measles vaccine, while adopting a goal of measles elimination by the year 2000.
Also beginning in 1992, Brazil began conducting national supplemental measles vaccination campaigns. Notably, Sao Paulo state conducted a restricted campaign in 1992, targeting children between the ages of 9m-10y (compared to a 9m-14y target elsewhere in the country) and did not participate in a 1995 national campaign targeting children between the ages of 1-3y. The outbreak itself resulted in over 30,000 confirmed cases in Sao Paulo State, with an unexpectedly high proportion of cases in adults; 60\% were in individuals greater than 20 years of age. Several limited vaccination campaigns, targeting children under 4 years of age, health workers, and some adults were implemented between June and August, prior to a widespread campaign targeting all children between 6 months and 4 years of age that was conducted in August of 1997. As a consequence of the broad age distribution of susceptibles, typical childhood-based ORIs may not have resulted in sufficient immunity to limit the outbreak. Following the outbreak, Brazil implemented a Supplementary Emergency Measles Action Plan in 1999 with significant increases in surveillance, case investigation, and rapid response \hyperref[csl:29]{[29]}. Between 2001 and 2013 all cases that occurred in Brazil were attributed to importations. During 2013-2014 there was an outbreak of measles in Ceara, Brazil \hyperref[csl:30]{[30]}. This outbreak was confirmed to have ended after the last case was registered in July 2015 and in September 2016 the International Expert Committee for Documenting and Verifying Measles, Rubella, and Congenital Rubella Syndrome Elimination in the Americas declared measles eliminated in the whole of the Americas.
We created a novel statistical model that combines an \emph{a priori} model of the susceptible population based on available immunization coverage information, a time series model of the progression of the outbreak, and an age-specific model of IgM serological confirmation of suspected cases to estimate \(R_{e}\)and the age distribution of the susceptible population. To illustrate how real-time surveillance could be applied to inform the design of ORI age targets, we generated estimates of \(R_{e}\) and the resulting ORI target recommendations at both 1 and 2 months prior to the vaccination campaign that was conducted on 15 August. We argue that a flexible approach to ORIs can better incorporate the information gained in the early stages of an
outbreak to identify campaign age targets and that unforeseen biases in clinical diagnosis can be mitigated through the incorporation of serological confirmation and high quality surveillance data.
\section{\texorpdfstring{\textbf{Methods}}{Methods}}\label{methods}
\subsection{\texorpdfstring{\emph{Data}}{Data}}\label{data}
Case-based records of individuals presenting with clinical measles symptoms (fever, rash, and at least one of the following symptoms: cough, conjunctivitis, or coryza \hyperref[csl:17]{[17]}) during the calendar year of 1996 were provided by the \selectlanguage{ngerman}\textit{Ministério da Saúde} (Ministry of Health) in Brazil. Fields for each record included the county in which the case presented, reporting date, age in months or years, and the results (positive, negative, or inconclusive) of a serological test for measles specific IgM, if one was conducted. After limiting the data to cases presenting in urban S\~{a}o Paulo and discarding those with incomplete records, there were 10,810 cases presenting with clinical symptoms only
and 23,699 cases with serological tests, of which 1067 were discarded due to lack of reagent or improper collection and thus treated as clinical cases.
The age distribution of the population in Sao Paulo was extrapolated from the decadal census. Historical rates of routine vaccination coverage were taken from (http://apps.who.int/gho/data/node.main.A826).
\subsection{\texorpdfstring{\emph{Confirmation Bias
Model}}{Confirmation Bias Model}}\label{confirmation-bias-model}
We specified a structured case confirmation submodel to retrospectively
determine the age group-specific probabilities of lab confirmation (\emph{i.e.}
lab positive for measles specific IgM) for measles in Sao Paulo,
conditional on clinical diagnosis. Individual lab confirmation events
\(c_i\) were modeled as Bernoulli random variables,
with the probability of confirmation being allowed to vary by age group:
\begin{equation}
c_{i} \sim Bernoulli(p_{a[i]})
\end{equation}
where \(a[i]\) denotes the age group for the individual
indexed by \emph{i}. There were 16 age groups, the first 15 of which
were 5-year age intervals \emph{{[}0,5), {[}5, 10), \ldots{} , {[}70,
75)}, with the last interval including all individuals 75 years and
older. Since our choices of age group boundaries were arbitrary, we
allowed the probabilities of adjacent groups to be correlated with one
another. To this end, the transformed set of probabilities was
modeled as a multivariate normal random variable:
\begin{equation}
\beta^{(conf)} \sim N(\mu, \Sigma)
\end{equation}
where the transformation is the logit (\(\text{logit}(p) = \log[p / (1-p)] \)), which serves to convert probabilities (defined on the [0,1] interval) to the real line:
\begin{equation}
\text{logit}(p_a) = \beta_a^{(conf)}
\end{equation}
where the corresponding covariance matrix was tridiagonal, incorporating a
correlation term \(\rho\) that was assumed, for simplicity, constant among groups. This was to allow confirmation rates to be correlated between neighboring age groups, since the age boundaries were arbitrarily defined.
\begin{equation}
\Sigma = \left[{
\begin{array}{cccccc}
{\sigma^2} & {\sigma^2 \rho} & 0& \ldots & {0} & {0} \\
{\sigma^2 \rho} & {\sigma^2} & \sigma^2 \rho & \ldots & {0} & {0} \\
{0} & \sigma^2 \rho & {\sigma^2} & \ldots & {0} & {0} \\
\vdots & \vdots & \vdots & & \vdots & \vdots\\
{0} & {0} & 0 & \ldots & {\sigma^2} & \sigma^2 \rho \\
{0} & {0} & 0 & \ldots & \sigma^2 \rho & {\sigma^2}
\end{array}
}\right]
\end{equation}
To estimate the true (latent) number of cases for each age group \(I_a\), the estimated probabilities were used to correct the clinically-reported cases \(n_a^{(clinic)}\), as modeled by a binomial distribution, and the total number of cases then calculated as the sum of this estimated value \(x_a^{(clinic)}\) and the lab-confirmed cases \(x_a^{(lab)}\):
\begin{equation}
\begin{split}
x_a^{(clinic)} &\sim \text{Bin}(n_a^{(clinic)},p_a) \\
I_a &= x_a^{(clinic)}+x_a^{(lab)}
\end{split}
\end{equation}
The set of \(I = \{I_a\}\) was, in turn, used to inform the estimate of the latent age distribution of the infected class. A natural probability distribution to model age classes is the multinomial distribution, which is parameterized by a set of probabilities, here corresponding to the expected proportion in each age group; the corresponding prior for this vector of probabilities is the Dirichlet distribution. These are specified by:
\begin{equation}
\begin{split}
f^{(age)} &\sim \text{Dirichlet}(\mathbf{1}) \\
\{I_a\} &\sim \text{Multinomial}(I, f^{(age)})
\end{split}
\end{equation}
where \(f^{(age)}\)is a vector of age-specific proportions and \(\mathbf{1}\) a vector of ones, which is used as a non-informative prior for the Dirichlet.
\subsection{\texorpdfstring{\emph{Disease Dynamics
Model}}{Disease Dynamics Model}}\label{disease-dynamics-model}
\emph{Initial Susceptible Population}
We used the population age structure in 1996 and the history of vaccination (through routine immunization and campaigns) prior to 1997 to estimate the initial number of susceptible individuals in each age class at the beginning of the 1997 outbreak. We modeled the number of susceptible individuals in each annual age class at the beginning of 1997 \(S_{a,0}\) as a finite mixture of local susceptibles \(S_{a,0}^{(L)}\) and excess susceptibles \(S_{a,0}^{(E)}\), where excess susceptibles are defined as those that cannot be predicted from local demographic processes. We model local susceptibles, \(S_a^{(L)}\) as a binomial draw from the Sao Paulo population:
\begin{equation}
S_a^{(L)} \sim \text{Binomial}(N_a, p_s)
\end{equation}
where \(N_a\) is the population size in age class \(a\) and \(p_s\) the realized susceptibility. Uncertainty in this probability was specified with a beta distribution having expected value \(\mu_s\), which was calculated based on historical immunization activity in the city:
\begin{equation}
\begin{split}
\mu_s &= \pi^{(\text{vacc})} \pi^{(\text{SIA})}\pi^{(\text{nat})} \\
&= (1- 0.85 V_a) \prod_{j=1}^2 \left[1- \psi_j I_{a,j} \right] \left[\exp \sum_{y=0}^{a-1} \theta_{a-y} \right],
\end{split}
\end{equation}
where $\pi^{(\text{vacc})}$, $\pi^{(\text{SIA})}$, and $\pi^{(\text{nat})}$ are proportions of residual susceptibility following routine vaccination, supplemental immunization activities (SIA) and natural immunity, \(V_a\) is the routine vaccination coverage experienced by age class \(a\) when they were eligible for routine vaccination between 9 and 12 months of age. We assume that efficacy of the routine dose is 85\% \hyperref[csl:31]{[31]}\footnote{We assume efficacy is a function of age, hence 85\% at 9 months (largely due to interference with maternal immunity), with higher efficacy for older age groups.}, while SIAs conducted in 1987 and 1992 were conservatively assumed to result in 80\% reduction of susceptibles in their targeted age classes \hyperref[csl:32]{[32]}. Further, \(\psi_{j}\) is the coverage of the \(j\)\textsuperscript{th} SIA and \(I_{a,j}\) is an indicator function that is 1 if age class a was eligible for the \(j\)\textsuperscript{th} SIA and 0 otherwise, and \(\theta_i\) the force of infection in year i (\emph{i.e.}, the contribution of natural immunity). The force of infection is assumed to be \(1/A\) in the absence of vaccination, where \(A=\frac{L}{R_0}\) \hyperref[csl:11]{[11]} is the mean age of infection in the absence of vaccination, L is the life expectancy, and \(R_0\) is the basic reproduction number. The force of infection \(\theta_i\) in the presence of vaccination is then approximated as \((1-V_a)/A\) \hyperref[csl:11]{[11]}. The basic reproduction number \(R_0\) is estimated below. We present an analysis of sensitivity to assumptions about historical vaccination coverage in the supplement.
We modeled excess susceptibles using a data augmentation approach. A latent number of susceptibles, constrained to a value uniformly distributed between zero and 1 million, were added to the model. These excess susceptibles were assigned an age distribution, modeled as Gaussian with unknown mean \(\mu_{age}\) and standard deviation \(\sigma_{age}\) and added to the resident susceptibles for a total susceptible estimate in the S\~{a}o Paulo population: \(S_a = S_a^{(L)} + S_a^{(E)}\). Because the number and distribution of excess susceptibles were modeled as latent variables, the estimates of their values were determined by the data, rather than being specified using prior information. This provided a means of accounting for infections that may not have been provided by a local population of susceptibles, but potentially by a pool of non-resident susceptibles migrating from areas of low immunity. We also fit an alternative version of the model assuming no excess susceptibles (\emph{i.e.} the distribution of susceptibles is forced to be consistent with local demographics), to assess the effect of including excess susceptibles explicitly in our model.
We modeled the number of measles cases in each age class \(a\) and time step \(t\) as a Poisson random variable:
\begin{equation}
I_a \sim \text{Poisson}\left(S_{a,t-1} \frac{I_{t-1}B_a}{N} \right),
\end{equation}
where \(S_{a,t-1}\) is the number of susceptibles in age class a at time \(t-1\), \(I_{t-1}\) a row vector of the number of infected individuals in each age class at time \(t-1\), and \(B_a\) is the \(a\)th column of the Who Acquires Infection From Whom (WAIFW) matrix. We model the WAIFW matrix as an assortative matrix \(B\)
\begin{equation}
B = \left[{
\begin{array}{ccccc}
{\beta} & {\beta \delta} & \ldots & {\beta \delta^{k-2}} & {\beta \delta^{k-1}} \\
{\beta \delta} & {\beta} & \ldots & {\beta \delta^{k-3}} & {\beta \delta^{k-2}} \\
{\beta \delta^2} & {\beta \delta} & \ldots & {\beta \delta^{k-4}} & {\beta \delta^{k-3}} \\
\vdots & \vdots & \ddots & \vdots & \vdots \\
{\beta \delta^{k-1}} & {\beta \delta^{k-2}} & \ldots & {\beta \delta} & {\beta} \\
\end{array}
}\right]
\end{equation}
that assumes that interaction among age groups declines exponentially with difference in age. The basic reproduction number, R\textsubscript{0}, in the model of the initial population size is taken as the dominant eigenvalue of the matrix B.
Our assumed contact matrix structure is simple, reflecting the fact that there is little information in the available data to inform a more complex parameterization. Nevertheless, the decay parameter allows for some flexibility in transmission dynamics; if there is little difference in the contact rate among age groups, the estimated $\delta$ will be close to one, while lower (higher) contact among disparate groups will result in $\delta$ estimates less (greater) than one. We include results from an alternative structure that assumes no age-related decay in contact rate in the supplement.
\subsection{Model Fitting}\label{model-fitting}
Our model was fit using data truncated at two different dates during the outbreak, to simulate the information state at June 15 and July 15, 1997. These two dates, which include 3698 and 11,982 cases, respectively, represent two contrasting levels of available monitoring data on which to potentially base intervention decisions. Hence, excluding any potential lags in reporting, the model was fit only to the information that would have been available at those time points (this includes the estimation of age-specific confirmation bias). Hence, four different management scenarios were modeled:
\begin{itemize}
\item June 15 information state, adjusted for age-specific confirmation bias
\item June 15 information state, unadjusted for confirmation bias
\item July 15 information state, adjusted for age-specific confirmation bias
\item July 15 information state, unadjusted for confirmation bias
\end{itemize}
All models were fit using PyMC 2.3 \hyperref[csl:33]{[33]}, a software package for the Python programming language that fits
Bayesian statistical models using Markov chain Monte Carlo \hyperref[csl:34]{[34]} sampling. Each model was sampled for 50,000 iterations using a Metropolis-Hastings sampling algorithm, with the first 40,000 samples discarded conservatively as a warmup period, and the remaining sample was assessed for lack of convergence using the Geweke diagnostic \hyperref[csl:35]{[35]}. Hence, all inference was based on the final 10,000 samples from each model run.
\subsection{Evaluating Outbreak Response Campaigns}\label{ori}
The classic result states that a fraction \(p_c \gt 1-1/R_e\) of susceptibles must be immunized (i.e. vaccinated and seroconverted) in order to reduce \(R_e \lt 1\) \hyperref[csl:11]{[11]}. Young children are conventionally targeted in outbreak response vaccination campaigns because they are, on average, more likely to be susceptible and to experience severe complications because of measles infection. In populations with a long history of measles
control, the age distribution of susceptibles is frequently wider \hyperref[csl:25]{[25]} and wider age campaigns are often considered to reach larger proportion of susceptibles. As \(R_e\) , and thus the \(p_c\), increases, then one might target a wider age range to increase the proportion of susceptibles immunized for a given coverage. Here, both the estimate of \(R_e\) and the age distribution of susceptibles are conditional on both date of the estimate (June or July) and the model used (all clinical cases or age-corrected). For each model, we calculated the necessary vaccination threshold assuming a campaign that achieves 90\% coverage of the target population and 95\% efficacy. We then evaluated whether there would have been empirical support on 15 June or 15 July that outbreak response campaigns targeting individuals from 6m-5y, 6m-15y, 6m-30y, or a mixed strategy targeting children 6m-5y and adults 20-30y would have met this necessary target.
\section{\texorpdfstring{\textbf{Results}}{Results}}\label{results}
The 1997 outbreak in S\~{a}o Paulo began in December 1996 and spread throughout
S\~{a}o Paulo State, peaking in August 1997 (Figure 1a). The outbreak
resulted in 25,393 total suspected cases, of which 13,516 were confirmed
by serology (IgM). The age distribution of cases was strongly bimodal,
with 22\% of suspected and confirmed cases below 2 years of age and a
secondary mode at between 23-26 years of age (Figure 1b).\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.98\columnwidth]{figures/image07/image07}
\caption{{\textbf{Figure 1}. a) Weekly time series of suspected (blue) and lab
confirmed (orange) measles cases in municipal Sao Paulo during 1997. b)
Number of suspected (blue) and lab confirmed (orange) measles cases by
age in municipal Sao Paulo during 1997.%
}}
\end{center}
\end{figure}
The probability of a clinical case being confirmed by positive IgM serology was strongly age dependent (Figure 2). Clinical cases less than 20y were significantly more likely to be IgM negative (posterior mean between 27\%-42\%, Figure 2); individuals between age 20-40y were significantly more likely to be IgM positive (\textgreater{}50\%). The consequence of this bias is that the observed age distribution of clinical cases over-represents the number and proportion of childhood cases.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.98\columnwidth]{figures/image11/image11}
\caption{{\textbf{Figure 2} . The estimated proportion of suspected measles cases
in each 5 year age group that were IgM positive. Box plots indicate the central 50\% of the posterior density and the posterior mean. Whiskers indicate the 2.5th and the
97.5th percentiles.%
}}
\end{center}
\end{figure}
If we assume that the entire pool of susceptibles is from the local population (no excess susceptibles), the best fit model for 15 June data yields an estimate of \emph{R\textsubscript{0}} = 57 (95\% credible interval: 54-60). This estimate is not consistent with previously published estimates of \emph{R\textsubscript{0}} for measles \hyperref[csl:36]{[36]}; \hyperref[csl:37]{[37]}; thus we interpret this as a result of poor model fit and do not consider this model parameterization further (See supplement for details).
We fit the full epidemic model, including resident and excess susceptibles, using data through either 15 June or 15 July under the assumption that either all clinical cases were true measles cases or that true measles cases were a sub-set of the reported clinical cases (\emph{i.e.}, corrected for confirmation bias), where the confirmation rate was determined by the age-specific confirmation model. The age distribution estimate of total susceptibles (both local and excess) was similar at both observation points (Figure 3a and b). The model fit using the age confirmation model consistently estimated larger numbers of susceptibles in the {[}0,5) and {[}20,25{)} year age classes and fewer susceptibles in the {[}15,20) year age classes (Figure 3a and b). Estimates of \emph{R\textsubscript{0}} from the 15 July data was 12.4 (95\% CI 11.5-13.5) for the age confirmation model and 12.75 (95\% CI 12-13.5) for model fit to data using only clinical confirmation. The corresponding estimate using data truncated to 15 June was 11.75 (95\% CI 10-14) and 11.5 (95\% CI 10-12.5) for the clinical only and age confirmation models respectively.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.98\columnwidth]{figures/image08/image08}
\caption{{\textbf{Figure 3} A) The estimated total number of susceptible
individuals in each age class based on data observed on 15 July using
the age confirmation model (red) and the clinical only model (blue).
B) The estimated number of resident susceptibles - total susceptibles
minus excess susceptibles - in each age class based on data observed on
15 July using the age confirmation model (orange) and the clinical only
model (blue).%
}}
\end{center}
\end{figure}
The effective reproduction number $R_e$, which describes the rate at which the epidemic spread in 1997 in a partially immune population, was estimated to be clearly greater than 1 for on both 15 June and 15 July using the age confirmation model (Figure 4). Notably, while the posterior mean estimate for $R_e$ on 15 June was 1.125, the 95\% credible did not overlap the estimate that results from including the data through 15 July. Ignoring the age
confirmation model and treating all clinical cases as measles cases yielded a 15 June estimate of $R_e$ = 1.02 with a 95\% credible interval
(0.95-1.08) that includes 1. Using all data through 15 July, the estimate of the $R_e$ was comparable, irrespective of whether clinical cases were corrected using the age confirmation model or not.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.98\columnwidth]{figures/image13/Unknown-1}
\caption{{\textbf{Figure 4.} Estimated effective reproduction number, $R_e$, estimated by the age confirmation model and the clinical only model, using data available on 15 June and 15 July.%
}}
\end{center}
\end{figure}
These differences in $R_e$, with posterior means ranging from 1.02 - 1.25 depending on time (June or July) and observation model (clinical or age-corrected), though small in absolute value, reflect larger practical differences in the vaccination response required to stop the outbreak. If we use these estimates to calculate minimum vaccination thresholds (assuming campaigns that reach 90\% of the target population and achieves 95\% effectiveness), this establishes vaccination targets ranging from 2 to 20\% of susceptibles; conservatively using the 95th percentile of the posterior distribution of $R_e$ estimates, this shifts the target range to
8 - 24\% (Figure 5). From this, we define an ORI campaign as \emph{sufficient} if it is expected to reduce the susceptible population by at least the threshold amount. Based on the July estimates, neither confirmation model predicts a 5-and-under strategy to be sufficient for stopping the outbreak (Figure 5). Using the June estimates under the age confirmation model, there is the suggestion that targeting children under 5 years would be
sufficient, while the June estimate fit to the clinical cases only predicts all strategies would be comfortably above the threshold. In all cases, ORI strategies that target individuals up to 15 or 30 years of age, as well as a mixed strategy that targets children under 5 and adults between 20-30 years of age, are predicted to be sufficient to stop the outbreak.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/Unknown-2/Unknown-2}
\caption{{\textbf{Figure 5}: A comparison of the estimated immunization threshold that would be achieved by four vaccination strategies (each targeting different age classes) to the minimum vaccination threshold necessary to reduce the effective reproduction number below one. Estimates of the necessary threshold and 95\% credible interval are shown by black lines and grey regions, respectively, based on data available on 15 June and 15 July for the age-confirmation and clinical only models. Colored rectangles give the 95\% credible intervals for the estimated immunization achieved by each of the 4 vaccination strategies. If the estimated immunization achieved by a campaign is higher than the minimum vaccination threshold required, then the campaign is assumed to be sufficient to stop the outbreak.%
}}
\end{center}
\end{figure}
\section{Discussion}\label{discussion}
The 1997 measles outbreak in S\~{a}o Paulo was unexpected, having followed several years of high routine vaccination coverage, SIAs, and relatively low incidence. Further, the age distribution of cases, with a secondary mode among adults, had not been observed in previous outbreaks. Thus, while historical precedent often serves as a guide for outbreak response, in this case, such precedent would have greatly under-estimated outbreak risk and the vaccination targets necessary to control the outbreak. Here we have presented a novel approach for integrating real-time outbreak surveillance into the evaluation of an evolving outbreak in order to evaluate candidate response strategies. In doing so, we have developed a model for interpreting clinical measles surveillance that acknowledges that the correlation between clinical measles symptoms and lab confirmation of positive measles IgM serology is age-specific. Further, we have shown that relying on clinical confirmation alone can significantly bias inference about transmission rate ($R_e$) and the minimal vaccination targets required to stop an outbreak.
In the case of S\~{a}o Paulo in 1997, estimates of $R_e$ were similar as was the assessment of which campaign age targets were sufficient to reduce $R_e$ below 1, regardless of whether confirmation bias was corrected, when fit to data on 15 July. Thus, while our proposed model accounting for age-specific bias in serologic confirmation explicitly estimates the uncertainty in clinical diagnosis, it results in little practical difference in the interpretation of risk $R_e$ or candidate interventions on 15 July. However, using only the data available on 15 June, estimates based only on clinical confirmation data would have grossly under-estimated risk and over-estimated the benefit of a vaccination campaign targeting children below 5 years of age.
Although outbreak risk can be evaluated \emph{a priori}, outbreaks themselves are often the first indication of the build-up of susceptibles or gaps in immunity. In 1997, the age distribution of cases in Sao Paulo indicated a dangerous gap in immunity among individuals between 15-35 years of age. The SIA conducted in 1987, targeting children below 14 years of age, would be expected to have immunized individuals below 23 years of age in 1997; those older than 23 years would have been born prior to a national immunization system in Brazil and would be expected to have experienced natural infection during their childhood. We estimated that excess susceptibles between 15-35 years of age may have accounted for 63\% of all susceptibles during the 1997 outbreak. We term these as \emph{excess} susceptibles because they are excess relative to the expected age distribution of susceptibles based on historical rates of natural infection, routine vaccination, and SIAs. We are unable to positively identify the source of these excess susceptibles; they may have been the result of over-estimating the coverage of previous vaccination programs, or migrants from low coverage or low transmission risk areas that were unlikely to be exposed to vaccination or natural infection. While the former explanation is possible, insufficient vaccination coverage would be expected to result in more circulating infection, which would still likely result in exposure to natural infection, and thus immunity, by adulthood. The latter explanation requires that individuals were recent immigrants to Sao Paulo and had not been exposed to either vaccination or natural infection as children in the region that the emigrated from. Measles rarely persists endemically in small populations below some critical community size \hyperref[csl:38]{[38]}; \hyperref[csl:39]{[39]}; thus it is possible that recent migrants from small villages might have not been exposed to natural infection. A case-control trial conducted after the 1997 outbreak found that recent immigration to Sao Paulo was a significant risk factor for measles infection during the outbreak \hyperref[csl:27]{[27]} . Further, immigration rates into Sao Paulo in 1991 were highest among individuals between 15 and 30 years of age \hyperref[csl:40]{[40]}, which is consistent with the age distribution of the excess susceptibles estimated by our models. While this does not confirm that immigration or gaps in prior immunization were the source of the adult susceptibles during the 1997 outbreak, this analysis does suggest that these adult susceptibles may have played a significant role in the outbreak; absent the excess susceptibles, $R_e$ at the start of the outbreak would have been comfortably less than 1. Other recent measles outbreaks have exhibited this same age-profile, with an unexpectedly large number of adult cases (\emph{e.g.} Malawi \hyperref[csl:28]{[28]}, Mongolia [ref: website above], China \hyperref[csl:41]{[41]}). Thus, strategies for monitoring and targeting immunity gaps in adults (\emph{e.g} migrant susceptibles, in this case) may be useful in preventing future outbreaks. Moreover, outbreak response strategies should consider adult-targeted vaccination when surveillance indicates a large number of adult susceptibles.
Though our models account for the age distribution of susceptibles, we make very simplistic assumptions about age-specific transmission; namely that within age-class transmission is the same for all ages, and between age-class transmission decays exponentially with difference in ages. Important recent work has shown that age-specific mixing rates are likely to vary considerably and may be culturally specific \hyperref[csl:42]{[42]}. It is possible that higher contact (and thus, transmission) rates among adults means that adult susceptibles disproportionately contributed to this outbreak. Fitting an age-specific transmission matrix with differential age-specific rates would have greatly inflated the model complexity and is beyond the scope of the present work, but may be worthwhile in cases where age-specific contact network structure information is available \hyperref[csl:43]{[43]}. In any case, we did not feel it appropriate to substitute an empirical contact matrix based on a different location (\emph{e.g.} \hyperref[csl:42]{[42]}), given the likelihood of significant geographic and cultural variation in patterns of transmission. But, we evaluated the sensitivity to our assumed network structure by estimating the model without the decay rate parameter, instead assuming an even simpler model of invariant between-group contact rates. Model predictions proved to not be strongly sensitive to this change in assumed structure (see Appendix B).
There are several potential explanations for the age-specific serological confirmation bias for cases with clinical symptoms. There are many etiologies that may generate fever and rash symptoms in young children \hyperref[csl:17]{[17]}; \hyperref[csl:18]{[18]}; \hyperref[csl:19]{[19]}; \hyperref[csl:20]{[20]}, which would increase the rate of false positives in children based on clinical symptoms alone. Further, young children may be more likely to be brought to clinic shortly after the onset of symptoms, when IgM titers may not yet have reached detectable levels. Regardless of the cause of this bias, the result is that assessing the overall age distribution of cases may tend to underestimate the role of adults if based on clinical confirmation alone. Further, as the age distribution of cases presenting with symptoms may change over the course of an outbreak (in Sao Paulo, there were relatively more young children near the start and end of the outbreak, and more adults in the middle), clinical cases may reflect time-varying confirmation and bias estimates of transmission rates. If true, by correcting for age-specific confirmation bias, we also necessarily correct for temporal variation in false positives. We see the effect of this in the higher estimated $R_e$ using the confirmation bias corrected model with the 15 June data (Figure 4), relative to that of the clinical confirmation model; though the model predicts fewer true measles cases (\emph{i.e}. false positives are removed), the exponential increase from the start of the outbreak to 15 June is steeper and this estimate is consistent with the higher $R_e$ estimate from the longer time series available on 15 July.
While we have demonstrated the potential for integrating real-time monitoring information to inform and improve outbreak response, the ability to provide model-based predictions is just one component for an operational system for adaptive control of outbreaks. For example, our approach does not take into account the investment of time and resources needed to prepare for ORI, such as vaccine procurement, microplanning, training, or social mobilization. A complete decision support system would fully integrate costs and constraints into the planning process so that management objectives can be adequately met. However, because we implemented our model using an extensible, open-source Bayesian modeling framework (PyMC), one can readily integrate it into a larger decision support system, as appropriate.
This work highlights the value of an integrated approach to model-based inference and prediction in supporting decision-making for the control of measles outbreaks. We used a Bayesian hierarchical model structure to integrate three critical sources of uncertainty that are common in emerging and re-emerging outbreaks: the structure and parameter values of the underlying disease dynamics model, partial observability of epidemic process due to imperfect reporting or diagnostics, and the current level of risk due to uncertainty in historical levels of vaccine coverage and efficacy. The full model is comprised of component sub-models, which are linked explicitly via conditional probability statements; this results in parameter estimates and evaluation of candidate vaccination targets that fully incorporate the uncertainty in the dependent variables, ultimately yielding predictions that more fully account for the lack of complete information at hand for decision-making. For example, the likely presence of a pool of excess susceptibles in the population was not evident based on historical immunization records, but was made apparent only when this information was integrated with outbreak monitoring data, via a realistic model of measles dynamics. From this, the model predicted that only campaigns targeting adults, as well as children, were likely to be effective in stopping the outbreak, and this was evident from data available as early as July 15. In practice, while targeted efforts to vaccinate school children and adults were made in 1997, before and after the August campaign targeting children between 6m and 4 years, this analysis suggests, retrospectively, that there may have been evidence to support broader vaccination efforts targeting older age groups. Despite the apparent complexity of the integrated model, it was straightforward to implement and fit using readily-available, open-source software, which allowed us to iteratively evaluate key assumptions regarding model structure and prior information in terms of their influence on model estimates and predictions. We believe this approach holds promise as a structured decision-making tool for a variety of disease outbreak scenarios, by efficiently incorporating all available information and incorporating monitoring information to provide robust decision support under uncertainty.
\appendix
\section{Python model}
The full model used to obtain estimates for this manuscript is available as a Jupyter notebook:
\verb|https://goo.gl/EowM1u|
The notebook includes all data preparation steps, the full PyMC model, and diagnostic outputs.
\section{Sensitivity analyses}
\textbf{Removal of excess susceptible pool}
A major assumption of our model is the existence of a pool of excess susceptibles, perhaps the result of migrant influxes from other regions of Brazil that did not receive the same historical immunization regime. To assess the effect of this component on the fit of the model and the resulting estimates, we modularized the model to remove the availability of this population by setting a flag in the model code. This flag results in the infection process being generated entirely from the local susceptible population, rather than as a mixture of local and excess infections.
The estimated $R_0$ value under this model was 56 (95 \% BCI = [53, 59]), which is much higher than typical reproduction numbers for measles\hyperref[csl:12]{[12]}. Hence, assuming a large number of excess susceptible adult individuals appears to be important in the development of a model with reasonable fit.
\textbf{Uncertainty in population immunity}
Our model uses historical records of routine vaccination, supplemental immunization and likely natural exposure to estimate immunity in the population in 1996. Each of these sources of immunity are entered into the model without associated estimates of uncertainty in their values, and hence the product of these sources of immunity are determinstically multiplied together to yield a value for the probability of susceptibility that is then used in a binomial model to estimate the number of susceptibles in the population at the start of the outbreak. Thus, the only source of uncertainty in the size of the susceptible population in the model is that associated with the binomial sampling process.
To evaluate the effect of assuming known values for historical immunity, we ran a modified model that allocated epistemic uncertainty to the routine vaccination coverage and to supplemental vaccination campaigns. For the former, the historic data were used to specify shape and scale parameters of a beta distribution for each year (1980-1996), which were used as priors for the coverage during that year. The range of priors is shown in Figure A1.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.7\columnwidth]{figures/priors/Unknown}
\caption{{\textbf{Figure A1}: Beta prior distributions for annual vaccination coverage proportions from 1980 to 1995. Earlier years are light, later years dark.%
}}
\end{center}
\end{figure}
For supplemental immunization, rather than assuming a fixed 20\% residual susceptibility (i.e. 20\% of children susceptible to measles prior to the SIA remain susceptible after the SIA, due to both coverage an efficacy) during the years 1987 and 1992, we sampled the value from a uniform distribution between 0.2 and 0.325, reflecting the possibility that the residual susceptibility could be higher (but not lower) than the number used in the model.
The result of these added priors was to increase the uncertainty in estimates of population susceptibility for the lab confirmation model, but not the clinical confirmation model (Figure A2). With this uncertainty included, the lab confirmation and clinical confirmation models differ in the assessment of whether a campaign targeting children under 5 years would be sufficient to reduce $R_e < 1$ (Figure A2). As in the main text, the estimated vaccination threshold required to reduce $R_e < 1$ assumes that a campaign will achieve 90\% coverage of susceptibles in the target population and 95\% vaccine efficacy, using the formula $P = \frac{1-1/R_e}{(.90)(.95)}$.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.7\columnwidth]{figures/Unknown1/Unknown}
\caption{{\textbf{Figure A2}: A comparison of the estimated immunization threshold that would be achieved by four vaccination strategies (each targeting different age classes) to the minimum vaccination threshold necessary to reduce the effective reproduction number below one, \emph{when prior uncertainty in the estimates of population susceptibility for the lab confirmation model is used}. Estimates of the necessary threshold and 95\% credible interval are shown by black lines and grey regions, respectively, based on data available on 15 June and 15 July for the age-confirmation and clinical only models. Colored rectangles give the 95\% credible intervals for the estimated immunization achieved by each of the 4 vaccination strategies. If the estimated immunization achieved by a campaign is higher than the minimum vaccination threshold required, then the campaign is assumed to be sufficient to stop the outbreak.%
}}
\end{center}
\end{figure}
\textbf{Alternative contact matrix structure}
To evaluate the sensitivity of our results to the assumed structure in contact rates among age groups, we fit an alternative model using a simplified network. Rather than have contact decline exponentially with the difference in age between groups, we here assumed that between-group contact was constant, governed by a coefficient $\rho$ that is applied to all off-diagonal elements of the matrix:
\begin{equation}
B = \left[{
\begin{array}{cccc}
{\beta} & {\beta \rho} & \ldots & {\beta \rho} \\
{\beta \rho} & {\beta} & \ldots & {\beta \rho} \\
\vdots & \vdots & \ddots & \vdots \\
{\beta \rho} & {\beta \rho} & \ldots & {\beta} \\
\end{array}
}\right]
\end{equation}
The resulting estimates of vaccination thresholds based on this contact structure is shown in Figure A3. The estimates are qualitatively and quantitatively similar to the estimates based on a decaying inter-age contact assumption (Figure 5).\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/constant-corr/constant-corr}
\caption{{\textbf{Figure A3}: A comparison of the estimated immunization threshold that would be achieved by four vaccination strategies (each targeting different age classes) to the minimum vaccination threshold necessary to reduce the effective reproduction number below one, \emph{when constant between-age-group interaction is assumed (rather than interactions decaying with age)}. Estimates of the necessary threshold and 95\% credible interval are shown by black lines and grey regions, respectively, based on data available on 15 June and 15 July for the age-confirmation and clinical only models. Colored rectangles give the 95\% credible intervals for the estimated immunization achieved by each of the 4 vaccination strategies. If the estimated immunization achieved by a campaign is higher than the minimum vaccination threshold required, then the campaign is assumed to be sufficient to stop the outbreak.%
}}
\end{center}
\end{figure}
\selectlanguage{english}
\FloatBarrier
\section*{References}\sloppy
\phantomsection
\label{csl:1}1.Hersh BS, Fine PEM, Kent WK, Cochi SL, Kahn LH, Zell ER, Hays PL, Wood CL. 1991 {Mumps outbreak in a highly vaccinated population}. \textit{The Journal of Pediatrics} \textbf{119}, 187–193. (doi:10.1016/s0022-3476(05)80726-7)
\phantomsection
\label{csl:2}2.Cherry JD. 2012 {Epidemic Pertussis in 2012 {\textemdash} The Resurgence of a Vaccine-Preventable Disease}. \textit{New England Journal of Medicine} \textbf{367}, 785–787. (doi:10.1056/nejmp1209051)
\phantomsection
\label{csl:3}3.Celentano LP, Massari M, Paramatti D, Salmaso D, Tozzi AE. 2005 {Resurgence of Pertussis in Europe}. \textit{The Pediatric Infectious Disease Journal} \textbf{24}, 761–765. (doi:10.1097/01.inf.0000177282.53500.77)
\phantomsection
\label{csl:4}4.Shibeshi ME \textit{et al.}. 2014 {Measles resurgence in southern Africa: Challenges to measles elimination}. \textit{Vaccine} \textbf{32}, 1798–1807. (doi:10.1016/j.vaccine.2014.01.089)
\phantomsection
\label{csl:5}5.McLean AR, Anderson RM. 1988 {Measles in developing countries Part I. Epidemiological parameters and patterns}. \textit{Epidemiol. Infect.} \textbf{100}, 111. (doi:10.1017/s0950268800065614)
\phantomsection
\label{csl:6}6.Jansen VAA. 2003 {Measles Outbreaks in a Population with Declining Vaccine Uptake}. \textit{Science} \textbf{301}, 804–804. (doi:10.1126/science.1086726)
\phantomsection
\label{csl:7}7.Omer SB, Salmon DA, Orenstein WA, deHart MP, Halsey N. 2009 {Vaccine Refusal Mandatory Immunization, and the Risks of Vaccine-Preventable Diseases}. \textit{New England Journal of Medicine} \textbf{360}, 1981–1988. (doi:10.1056/nejmsa0806477)
\phantomsection
\label{csl:8}8.Ferrari MJ, Grais RF, Bharti N, Conlan AJK, Bj{\o}rnstad ON, Wolfson LJ, Guerin PJ, Djibo A, Grenfell BT. 2008 {The dynamics of measles in sub-Saharan Africa}. \textit{Nature} \textbf{451}, 679–684. (doi:10.1038/nature06509)
\phantomsection
\label{csl:9}9.Cairns KL, Perry RT, Ryman TK, Nandy RK, Grais RF. 2011 {Should Outbreak Response Immunization Be Recommended for Measles Outbreaks in Middle- and Low-Income Countries? An Update}. \textit{Journal of Infectious Diseases} \textbf{204}, S35–S46. (doi:10.1093/infdis/jir072)
\phantomsection
\label{csl:10}10.Grais RF, Strebel P, Mala P, Watson J, Nandy R, Gayer M. 2011 {Measles vaccination in humanitarian emergencies: a review of recent practice}. \textit{Conflict and Health} \textbf{5}, 21. (doi:10.1186/1752-1505-5-21)
\phantomsection
\label{csl:11}11.Anderson RM, May RM. 1981 {The Population Dynamics of Microparasites and Their Invertebrate Hosts}. \textit{Philosophical Transactions of the Royal Society B: Biological Sciences} \textbf{291}, 451–524. (doi:10.1098/rstb.1981.0005)
\phantomsection
\label{csl:12}12.Durrheim DN, Crowcroft NS, Strebel PM. 2014 {Measles {\textendash} The epidemiology of elimination}. \textit{Vaccine} \textbf{32}, 6880–6883. (doi:10.1016/j.vaccine.2014.10.061)
\phantomsection
\label{csl:13}13.Goodson JL, Masresha BG, Wannemuehler K, Uzicanin A, Cochi S. 2011 {Changing Epidemiology of Measles in Africa}. \textit{Journal of Infectious Diseases} \textbf{204}, S205–S214. (doi:10.1093/infdis/jir129)
\phantomsection
\label{csl:14}14.Takahashi K, Kanda H, Kim JY. 2014 {Reasons for Non-vaccination among Patients Who Acquired Measles: Lessons from the Local Measles Epidemics in Japan.}. \textit{West Indian Med J} \textbf{63}, 647–9.
\phantomsection
\label{csl:15}15.Merl D, Johnson LR, Gramacy RB, Mangel M. 2009 {A Statistical Framework for the Adaptive Management of Epidemiological Interventions}. \textit{PLoS ONE} \textbf{4}, e5807. (doi:10.1371/journal.pone.0005807)
\phantomsection
\label{csl:16}16.Shea K, Tildesley MJ, Runge MC, Fonnesbeck CJ, Ferrari MJ. 2014 {Adaptive Management and the Value of Information: Learning Via Intervention in Epidemiology}. \textit{PLoS Biology} \textbf{12}, e1001970. (doi:10.1371/journal.pbio.1001970)
\phantomsection
\label{csl:17}17.Hutchins SS \textit{et al.}. 2004 {Evaluation of the Measles Clinical Case Definition}. \textit{The Journal of Infectious Diseases} \textbf{189}, S153–S159. (doi:10.1086/379652)
\phantomsection
\label{csl:18}18.Ho HJ, Low C, Ang LW, Cutter JL, Tay J, Chan KP, Ooi PL, Thoon KC, Goh KT. 2014 {Progress towards measles elimination in Singapore}. \textit{Vaccine} \textbf{32}, 6927–6933. (doi:10.1016/j.vaccine.2014.10.046)
\phantomsection
\label{csl:19}19.Guy RJ, Andrews RM, Kelly HA, Leydon JA, Riddell MA, Lambert SB, Catton MG. 2004 {Mumps and rubella: a year of enhanced surveillance and laboratory testing}. \textit{Epidemiology and Infection} \textbf{132}, 391–398. (doi:10.1017/s0950268804001955)
\phantomsection
\label{csl:20}20.Dietz V, Rota J, Izurieta H, Carrasco P, Bellini W. 2004 {The laboratory confirmation of suspected measles cases in settings of low measles transmission: conclusions from the experience in the Americas}. \textit{Bulletin of the World Health Organization} \textbf{82}, 852–857.
\phantomsection
\label{csl:21}21.Chiew M, Gidding HF, Dey A, Wood J, Martin N, Davis S, McIntyre P. 2013 {Estimating the measles effective reproduction number in Australia from routine notification data}. \textit{Bulletin of the World Health Organization} \textbf{92}, 171–177. (doi:10.2471/blt.13.125724)
\phantomsection
\label{csl:22}22.Ferrari MJ, Bj{\o}rnstad ON, Dobson AP. 2005 {Estimation and inference of R0 of an infectious pathogen by a removal method}. \textit{Mathematical Biosciences} \textbf{198}, 14–26. (doi:10.1016/j.mbs.2005.08.002)
\phantomsection
\label{csl:23}23.Lipsitch M. 2003 {Transmission Dynamics and Control of Severe Acute Respiratory Syndrome}. \textit{Science} \textbf{300}, 1966–1970. (doi:10.1126/science.1086616)
\phantomsection
\label{csl:24}24.Minetti A, Hurtado N, Grais RF, Ferrari M. 2013 {Reaching Hard-to-Reach Individuals: Nonselective Versus Targeted Outbreak Response Vaccination for Measles}. \textit{American Journal of Epidemiology} \textbf{179}, 245–251. (doi:10.1093/aje/kwt236)
\phantomsection
\label{csl:25}25.Ferrari MJ, Grenfell BT, Strebel PM. 2013 {Think globally act locally: the role of local demographics and vaccination coverage in the dynamic response of measles infection to control}. \textit{Philosophical Transactions of the Royal Society B: Biological Sciences} \textbf{368}, 20120141–20120141. (doi:10.1098/rstb.2012.0141)
\phantomsection
\label{csl:26}26.Takahashi S, Metcalf CJE, Ferrari MJ, Moss WJ, Truelove SA, Tatem AJ, Grenfell BT, Lessler J. 2015 {Reduced vaccination and the risk of measles and other childhood infections post-Ebola}. \textit{Science} \textbf{347}, 1240–1242. (doi:10.1126/science.aaa3438)
\phantomsection
\label{csl:27}27.Camargo MCC, de Moraes KC, Souza VAUF, Matos MR, Pannuti CS. 2000 {Predictors related to the occurrence of a measles epidemic in the city of S{\~{a}}o Paulo in 1997}. \textit{Revista Panamericana de Salud P{\'{u}}blica} \textbf{7}. (doi:10.1590/s1020-49892000000600001)
\phantomsection
\label{csl:28}28.Minetti A \textit{et al.}. 2013 {Lessons and Challenges for Measles Control from Unexpected Large Outbreak Malawi}. \textit{Emerg. Infect. Dis.} \textbf{19}, 202–209. (doi:10.3201/eid1902.120301)
\phantomsection
\label{csl:29}29.2003 {Interruption of Measles Transmission in Brazil 2000{\textendash}2001}. \textit{The Journal of Infectious Diseases} \textbf{187}, S111–S120. (doi:10.1086/368030)
\phantomsection
\label{csl:30}30.Leite RD, Barreto JLTMS, Sousa AQ. 2015 {Measles Reemergence in Cear{\'{a}} Northeast Brazil 15 Years after Elimination}. \textit{Emerging Infectious Diseases} \textbf{21}, 1681–1683. (doi:10.3201/eid2109.150391)
\phantomsection
\label{csl:31}31.Uzicanin A, Zimmerman L. 2011 {Field Effectiveness of Live Attenuated Measles-Containing Vaccines: A Review of Published Literature}. \textit{Journal of Infectious Diseases} \textbf{204}, S133–S149. (doi:10.1093/infdis/jir102)
\phantomsection
\label{csl:32}32.Pannuti CS, Moraes JC, Souza VA, Camargo MC, Hidalgo NT. 1991 {Measles antibody prevalence after mass immunization in São Paulo, Brazil.}. \textit{Bull World Health Organ} \textbf{69}, 557–60.
\phantomsection
\label{csl:33}33.Patil A, Huard D, Fonnesbeck CJ. 2010 {PyMC: Bayesian Stochastic Modelling in Python}. \textit{Journal of Statistical Software} \textbf{35}. (doi:10.18637/jss.v035.i04)
\phantomsection
\label{csl:34}34.Geyer C. 2011 {Introduction to Markov Chain Monte Carlo}. In \textit{Chapman {\&} Hall/{CRC} Handbooks of Modern Statistical Methods}, Informa {UK} Limited. (doi:10.1201/b10905-2)
\phantomsection
\label{csl:35}35.Geweke J, Berger JO, Dawid AP. 1992 {Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments}. \textit{In Bayesian Statistics 4}
\phantomsection
\label{csl:36}36.Edmunds WJ, Gay NJ, Kretzschmar M, Pebody RG, Wachmann H. 2000 {The pre-vaccination epidemiology of measles mumps and rubella in Europe: implications for modelling studies}. \textit{Epidemiol. Infect.} \textbf{125}, 635–650. (doi:10.1017/s0950268800004672)
\phantomsection
\label{csl:37}37.Mossong J, Muller CP. 2000 {Estimation of the basic reproduction number of measles during an outbreak in a partially vaccinated population}. \textit{Epidemiol. Infect.} \textbf{124}, 273–278. (doi:10.1017/s0950268899003672)
\phantomsection
\label{csl:38}38.Conlan AJK, Grenfell BT. 2007 {Seasonality and the persistence and invasion of measles}. \textit{Proceedings of the Royal Society B: Biological Sciences} \textbf{274}, 1133–1141. (doi:10.1098/rspb.2006.0030)
\phantomsection
\label{csl:39}39.Keeling MJ, Grenfell BT. 1997 {Disease extinction and community size: modeling the persistence of measles.}. \textit{Science} \textbf{275}, 65–7.
\phantomsection
\label{csl:40}40.de Moraes JC, Camargo MCC, de Mello MLR, Hersh BS, Glasser JW. 2016 {The 1997 Measles Outbreak in Metropolitan S{\~{a}}o Paulo Brazil: Strategic Implications of Increasing Urbanization}. In \textit{Mathematical and Statistical Modeling for Emerging and Re-emerging Infectious Diseases}, pp. 269–289. Springer Nature. (doi:10.1007/978-3-319-40413-4_16)
\phantomsection
\label{csl:41}41.Zheng X \textit{et al.}. 2015 {Investigation of a Measles Outbreak in China to Identify Gaps in Vaccination Coverage Routes of Transmission, and Interventions}. \textit{PLOS ONE} \textbf{10}, e0133983. (doi:10.1371/journal.pone.0133983)
\phantomsection
\label{csl:42}42.Mossong J \textit{et al.}. 2008 {Social Contacts and Mixing Patterns Relevant to the Spread of Infectious Diseases}. \textit{PLoS Med} \textbf{5}, e74. (doi:10.1371/journal.pmed.0050074)
\phantomsection
\label{csl:43}43.Bhattacharyya S, Ferrari MJ. in press{Age-specific mixing generates transient outbreak risk following critical-level vaccination}. \textit{Epidemiology and Infection}
\end{document}