Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

Seong Hyeon Park; Young Hoon Lee

doi:10.1155/2019/6031789

Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

Park, Seong Hyeon;Lee, Young Hoon 2019-12-09 00:00:00 Hindawi Journal of Healthcare Engineering Volume 2019, Article ID 6031789, 14 pages https://doi.org/10.1155/2019/6031789 Research Article Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors Seong Hyeon Park and Young Hoon Lee Department of Industrial Engineering, Yonsei University, D1010, 50, Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea Correspondence should be addressed to Seong Hyeon Park; s.park10@yonsei.ac.kr Received 10 July 2019; Accepted 11 November 2019; Published 9 December 2019 Academic Editor: Ping Zhou Copyright © 2019 Seong Hyeon Park and Young Hoon Lee. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A two-tiered ambulance system, consisting of advanced and basic life support for emergency and nonemergency patient care, respectively, can provide a cost-eﬃcient emergency medical service. However, such a system requires accurate classiﬁcation of patient severity to avoid complications. *us, this study considers a two-tiered ambulance dispatch and redeployment problem in which the average patient severity classiﬁcation errors are known. *is study builds on previous research into the ambulance dispatch and redeployment problem by additionally considering multiple types of patients and ambulances, and patient clas- siﬁcation errors. We formulate this dynamic decision-making problem as a semi-Markov decision process and propose a mini- batch monotone-approximate dynamic programming (ADP) algorithm to solve the problem within a reasonable computation time. Computational experiments using realistic system dynamics based on historical data from Seoul reveal that the proposed approach and algorithm reduce the risk level index (RLI) for all patients by an average of 11.2% compared to the greedy policy. In this numerical study, we identify the inﬂuence of certain system parameters such as the percentage of advanced-life support units among all ambulances and patient classiﬁcation errors. A key ﬁnding is that an increase in undertriage rates has a greater negative eﬀect on patient RLI than an increase in overtriage rates. *e proposed algorithm delivers an eﬃcient two-tiered ambulance management strategy. Furthermore, our ﬁndings could provide useful guidelines for practitioners, enabling them to classify patient severity in order to minimize undertriage rates. Emergency care and transport of patients should be both 1. Introduction highly ﬂexible and rapid because small time delays might have Ambulance operating methods are highly important for the a negative impact on emergency patients. However, in an emergency medical service (EMS) system as they directly EMS system where patient numbers are highly uncertain, aﬀect the patient survival rate and medical service quality. preplanned scheduling or operation solutions may not op- Two types of decision are required during ambulance timally respond to ﬂuctuating situations. *erefore, real-time decision-making is required, which must consider system operations: (1) the dispatch decision, i.e., which ambulance to send to an emergency call, and (2) the redeployment dynamics such as time-varying demands (emergency calls), time-varying traﬃc, and the diﬀerent ﬁrst-aid times required decision, i.e., the waiting location to which the ambulance that has just completed a patient-transport service should by patients. Another important consideration in ambulance be sent. *e goal of ambulance operations is to provide operations is the diﬀerent severity of the transported patients. patients with appropriate emergency treatment within a *e majority of patients are nonemergency patients. *ey short time period and then transport the patient to the request an ambulance because of a lack of transportation, hospital for speciﬁc advanced treatment. *erefore, an inability to ambulate, domestic violence, or poor social sit- eﬃcient strategy is required for dispatching and rede- uations while a few of them can either walk or use public ploying ambulances. transport to reach a hospital [1, 2]. Transfer of nonemergency 2 Journal of Healthcare Engineering patients into types based on their severity [6, 17, 18]. patients by ambulance can be delayed due to the preferential transfer of emergency patients because their deterioration rate However, these studies all assumed that patient severity can be immediately and accurately determined when the call is of health may be much lower. However, as only limited in- formation is delivered during calls to the emergency operator, received. Furthermore, few studies have considered the it is risky to designate a patient’s severity as low and delay the possibility of errors when classifying patient severity during dispatch of an ambulance to the patient. *erefore, all ambulance operations. McLay and Mayorga [19] mathe- emergency calls must be responded to immediately regardless matically addressed patient classiﬁcation errors during of the classiﬁed severity of patients; in South Korea, it is ambulance operations. *ey classiﬁed patient priorities in regulated by law. the all-ALS system into three levels and optimized the ambulance operation policy by using the Markov decision Based on the criteria used in South Korea, ambulances are classiﬁed into two types based on the patients’ level of process (MDP) model. *ey then compared two cases, in which middle-priority patients were classiﬁed as high-risk urgency [3]. (1) An advanced life support (ALS) vehicle is suitable for emergency-patient transport. It must be ac- and low-risk patients. In this context, we propose an approximate dynamic companied by paramedics who can perform more special- ized medical care and is designed with more stringent programming (ADP) model that runs on a discrete event standards, including the minimum area for the patient in the simulation to optimize the dispatch-and-redeployment ambulance and the medical equipment to be installed inside. policy of a two-tiered ambulance system by considering (2) A basic life support (BLS) vehicle is suitable for non- errors in patient-severity classiﬁcation. *e computational emergency patient transport. It provides basic medical experiment environment was created based on actual his- services with relatively little medical equipment and is ac- torical data from Seoul by considering the probability dis- tribution of demand-and-service time, time-varying companied by emergency medical technicians (EMTs). *erefore, high-risk emergency patients transported by BLS demand, and traﬃc speed. *e computational experiments show that our proposed algorithm performs better than the units would be at risk because they may not receive adequate care during transport. *e corresponding ambulance sys- greedy policy. In addition, we identify the inﬂuence and correlation between classiﬁcation errors and the ratio of ALS tems are also classiﬁed into two types: an “all-ALS system” that operates all ambulances as ALS vehicles and a “two- units to BLS units based on patient risk level. *is can tiered ambulance system (tiered system)” that uses a com- provide insights into patient-classiﬁcation attitudes and bination of ALS and BLS units. Previous research has de- ambulance management strategies. bated the superiority of all-ALS or mixed-ALS/BLS ambulance management systems according to their relative 2. Problem Description risks, treatment times, and cost eﬀectiveness [4–9]. To operate a two-tiered ambulance system eﬃciently, an In this study, we use an ADP algorithm to optimize am- emergency center should attempt to classify the severity of bulance dispatch and redeployment decisions in order to the patients during the emergency call. However, the lack of reduce the risk level of patients through rapid trans- information obtained from the call inevitably leads to patient portation. *e approach assumes that the strategic level of severity classiﬁcation errors, which could have a devastating decision-making, such as the location of the emergency impact on the patient risk level. However, although previous center and hospital and the number of ambulances, is ﬁxed. research has attempted to optimize ambulance dispatch and In addition, real-time dispatch and redeployment decisions redeployment strategies, they have not considered the ex- are dealt with at the operational level. *e ambulance op- istence of these classiﬁcation errors. For example, Brotcorne erating environment is assumed to comprise a two-tiered et al. [10] and Jagtenberg et al. [11] revealed that the greedy ambulance, two types of patient classes with diﬀerent se- policy of allocating the nearest ambulance to patients does verities, and patient classiﬁcation errors. *ese consider- not always yield the best performance. Moreover, research ations are not only key factors inﬂuencing decision-making into optimizing decisions in real time has achieved more but are also close to that of an actual ambulance operating realistic results [12]. Maxwell et al. [13], Nasrollahzadeh et al. environment. [14], Maxwell et al. [15], and Schmid [16] all showed that the Patients calling the emergency services are classiﬁed into approximate dynamic programming (ADP) model works two groups: high-and low-risk patients with high and low well as a real-time ambulance model of operational policy severity levels, respectively. We denote the severity of pa- A A optimization. However, although the ADP produced a near- tients as H (L ) if the actual severity of the patient is high C C optimal solution in limited experiments, all of these studies (low) risk, and H (L ) if the classiﬁed severity of the patient assumed one type of ambulance and no classiﬁcation errors. is high (low) risk. High-risk patients are described as life- *us, more sophisticated two-tiered ambulance opera- threatened if they do not receive adequate treatment within a tions are required that consider the existence of classiﬁcation given response time threshold (RTT). Although low-risk errors. Furthermore, it is important to determine (1) how the patients are not life-threatened, it is preferable to treat them optimal operation policy changes according to the classiﬁ- quickly to increase the service satisfaction level and prevent cation errors and (2) what type of classiﬁcation decision their treatment from becoming complicated and turning should be taken for ambiguous patients to minimize patient them into high-risk patients. risk. Some studies have considered the classiﬁcation of *e operation process of the ambulance and the time patient severity in mixed ALS/BLS systems by categorizing spent during the process are shown in Figure 1. *e Journal of Healthcare Engineering 3 Redeployment/dispatch Dispatch decision decision Ambulance arrival at scene Time Patient call arrival Service time Transport Service time Redeployment with patient time at hospital time Time required for proper care with adequate type of ambulance Time required for proper care when BLS is transporting a high-risk patient Figure 1: Flowchart of the ambulance operation process and decision-making points. ambulances typically remain at the emergency center. When *e response time (RT), which is typically used as an a patient is reported, the decision maker decides which evaluation measure of the EMS system, denotes the time ambulance to send to the patient using information of the from the patient report being obtained at the emergency severity classiﬁcation. When an ambulance arrives at the center to the ambulance arriving at the scene. However, in patient location, the actual severity of the patient becomes this study, we use the time required for proper care known, and the patient receives a ﬁrst-aid service. *e (RT_PC), which is the time from the patient report being ambulance then transports the patient to the nearest hospital obtained at the emergency center to the patient beginning to receive appropriate treatment. *at is, unless the ambulance emergency room. After the ambulance arrives at the hos- pital, the patient is transferred to the hospital staﬀ. After is the correct type to handle the severity of the patient, the patient only begins receiving appropriate treatment once the delivering the patient to the hospital, the decision maker determines whether there are any patients waiting to be ambulance arrives at the hospital. For example, if an ALS allocated an ambulance. If such a patient exists, the am- transports a high- or low-risk patient, or if a BLS transports a bulance is allocated to the patient; if a patient does not exist, low-risk patient, the RT_PC does not diﬀer from the original the decision maker determines which emergency center the RT. However, when a BLS transports a high-risk patient, ambulance should be relocated to. When a patient is re- providing appropriate treatment quickly is complicated by ported and no ambulance is available, which is a rare oc- the lack of specialized medical resources, such as a respirator currence in reality and has thus far not been noted in any or emergency medical staﬀ [20]. *us, the end time for the previous experiments, the patient is placed in a virtual RT_PC is the time that the ambulance arrives at the hospital. *e criterion for measuring RT_PC is also expressed in queue. In this situation, when an ambulance is about to be placed into an idle state, a high-risk patient is allocated at a Figure 1. In this study, we propose a risk level index (RLI) that higher priority than low-risk patients, regardless of the report-arrival time. For patients within the same risk level, reﬂects the diﬀerent risk levels of patient groups with dif- an ambulance is allocated on a ﬁrst-come-ﬁrst-served basis. ferent severity as another performance measure of the EMS If an ambulance is idle when a patient is waiting, the am- system. RLI is the response time adjusted to the risk of the bulance must respond to the patient, regardless of the lo- patient. *e RLI function f(RT, S ) is a function of RT_PC cation of the patient; i.e., a delay in ambulance allocation is and actual patient severity (S ), as shown in equation (1) and not allowed. Figure 2: A A ⎧ ⎨ C · RT_PC + 1􏼈RT C> RTT􏼉 · Penalty, if S � H , H P f􏼐RT, S 􏼑 � (1) A A C · RT_PC, if S � L . *e RLI increases linearly with RT_PC but with diﬀerent *e evaluation index of ambulance operations in EMS slopes depending on the severity of the patient. When the systems usually includes the RT [16, 23], the survival rate, RT_PC of high-risk patients exceeds the RTT, a penalty of which is a continuous function of RT [24–28], and the constant value is added. *e value of these parameters can be coverage level, which is the proportion of reports covered set according to the decision of an EMS system manager if within a predeﬁned RTT [29, 30]. However, these have some C ≥ C ≥ 0. *e RTT is typically set to 8 min or 9 min limitations. First, it is diﬃcult to use the RT index to consider H L [21, 22]. the diﬀerence among each patient group with diﬀerent 4 Journal of Healthcare Engineering Table 1: Probability of classiﬁcation errors for patient severity. Classiﬁed severity Probability C C High risk (H ) Low risk (L ) High risk (H ) 1 − α α Actual severity Low risk (L ) Β 1 − β Penalty (1 − α)Pr Pr A C � , H H Time required for (1 − α)Pr A + β 1 − Pr A􏼁 H H Response time threshold proper care (2) of high-risk patients (1 − β) 1 − Pr A􏼁 Pr A C � . High-risk patient L |L α · Pr A + (1 − β) 1 − Pr A􏼁 H H Low-risk patient Figure 2: Risk level index function. 3. Model and Solution Algorithm severities and determine whether the report is covered within the RTT. Second, the quantitative measurement of *e process of the EMS system is modeled as a semi-MDP survival rate over RT is not easily medically validated due to model that runs on discrete event simulation. *e state tran- the diﬀerent status levels of each patient; thus, previous sition function depends partly on the controllable decisions of studies used diﬀerent survival-rate functions. In addition, dispatch and redeployment and partly on unmanageable sto- higher priority might be assigned to a patient whose survival chastic events, such as patient arrival and service completion. A rate is high but rapidly decreasing than to a patient whose decision is made at the time an event occurs that requires new survival rate is already low; this raises an ethical issue. Lastly, decision-making. In other words, the simulation time jumps to as the coverage level only checks whether RT is within RTT, the real time of the next event instead of adding a constant unit it does not evaluate the exact RT; this might cause the time of time. *us, multiple calls are never received simultaneously. th immediately before the RTT to be labeled as the RT, Here, we let τ denote the time when the t event occurs. neglecting the condition of “the sooner, the better” and In a semi-MDP environment, dynamic programming potentially ignoring patients already waiting for longer than (DP) can be used to obtain optimal policies. DP uses the RTT. Conversely, the RLI used in this study has advantages state S, action a, contribution function C(S, a), and of including all characteristics, such as the patient’s severity, transition probabilities. In this study, S represents the state RT, and coverage level. of the EMS system associated with the ambulance and the *e proposed RLI is not an entirely new concept as patient. *e state of ambulance i is denoted by vector a � several studies have used an objective function that either {a1, a2, . . . , a6}, and the state of patient j is denoted by considers the risk associated with matching ambulance type vector p � 􏼈p1, p2, . . . , p4􏼉. Attributes a1–a6 represent and patient severity [6, 14] or that considers a linearly in- the ambulance type (ALS or BLS), ambulance location, creasing risk over time with a penalty for exceeding the time ambulance status (idle, moving toward patient, service in threshold [31]. *e RLI function for high-risk patients can be patient’s location, moving toward hospital, service in viewed not only as the response time adjusted by the patient hospital, and moving back toward emergency center), risk but also as a weighted sum of multiple objectives, the RT patient ID if the ambulance is assigned to the patient, and the coverage level, whereas the RLI function for low-risk destination (speciﬁc patient/hospital/emergency center) if patients is a relatively low-weighted RT. the ambulance is in transit, and time remaining until When classifying patients as high or low risk, two types arrival at destination, respectively. Attributes p1–p4 of error may occur (Table 1). *e undertriage rate α is the represent the patient’s location, time when an incident is probability of classifying a high-risk patient (H ) as a low- reported, status (waiting/in service), and classiﬁed se- risk patient (L ), and the overtriage rate β is the probability verity, respectively. *e set of all ambulances is A, and the of classifying a low-risk patient (L ) as a high-risk patient set of all patients is P. *e state S of event t at time τ is t t (H ). *e purpose of this study is not to determine the exact represented as a vector 􏽮a , p 􏽯 . Action a decides i j i∈A,j∈P value of these errors but to investigate the inﬂuence of these which idle ambulance to send to which waiting patient, or errors; thus, the authors assumed that errors α and β are which emergency center to relocate an ambulance to that known in advance by using historical data. has just been labeled idle after completing its service to a Moreover, the ratio of actual high-risk patients to all hospital. patients (Pr A) is assumed to be known from the historical *e contribution function C(S , a ) returns the value of t t data. In this study, Pr A � 24.8%, according to a survey by the reward given when action a is performed in state S . In t t Vandeventer et al. [32]. *erefore, if α, β, and Pr A are this study, we deﬁne the expected RLI of the patient as the known, we can calculate the probability of a patient being reward value, and the contribution function is described in A C correctly classiﬁed as high risk (Pr ) and vice versa equation (3). As the exact RT is not known when performing H |H (Pr A C): action a at state S , the average RT for the distance is used: L |L t t Slope C Slope C Risk level index Journal of Healthcare Engineering 5 A A C ⎪ Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if ALS carry H patient, H H L H ⎪ | | ⎪ A A C Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if ALS carry L patient, ⎨ H L L L | | C S , a􏼁 � (3) t t A A C ⎪ Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if BLS carry H patient, H H L H ⎪ | | ⎩ A A C Pr A C · f RT, H + Pr A C · f RT, L , if BLS carry L patient. 􏼁 􏼁 H L L L | | X (S ) is a function that returns the action to be taken in *e ADP further uses the postdecision state and ag- state S , when policy π is used. *e greedy policy π mini- gregation techniques to increase the computation speed. π a mizes C(S , X (S )); that is, at every decision point, an Postdecision state S represents the state immediately after t t t action is taken by only considering the reward that can be the decision to take action a at time τ and before the ex- received at the current state. However, we aim to obtain a ternal information W is received. *us, after a decision is t+1 policy that considers the eﬀects of the current action on made to perform action a in state S , as shown in Figure 3, t t future situations. *us, ADP is used to ﬁnd a policy that the time does not elapse and the process goes into post- minimizes the expected value of the total discounted sum of decision state S deterministically. Next, external in- t π the patient’s RLI over a long period, c C(S , X (S )), formation is received between τ and τ , and the process t�0 t t t t+1 where c∈ [0, 1] is a discount factor expressing how much goes into state S . *e ADP at the current time τ estimates t+1 t future rewards are worth in the present. As the time interval the value of postdecision state S instead of state S by t t+1 between rewards is not constant and cannot be precisely using equation (7) instead of equation (5); thus, calculation predicted in advance, the discount rate is set to a constant for of the expectation value in equation (5) can be omitted. *e simple application. V(S ) denotes the value of being in state ADP has a large computational advantage for estimating the S under policy π; that is, the expected value of the total value of being in a postdecision state, as it can use the discounted sum of the RLI. *en, V(S ) can be recursively deterministic value of postdecision state at the decision point expressed using the Bellman equation form, as in equation instead of computing possibilities of reaching the next state (4). W denotes the external information related to the status S for all possible states: t t+1 change known between τ and τ ; for example, obtaining a n a,n t− 1 t v � min C S , a􏼁 + cV S 􏼁 􏼁. t t t (7) new patient call and the ambulance arrival time at the patient t location or hospital. Let the state transition function be S , Furthermore, V is now a value function that returns an then S (S , a , W ) represents the state at time τ when t t t+1 t+1 a,n approximate value of being in postdecision state S and is external information W is received after taking action a t+1 t updated using equation (8) instead of equation (6): in state S , which is S : t t+1 a,n n,t a,n n,t V S 􏼁 ⟵ 􏼐1 − δ 􏼑V S 􏼁 + δ 􏽢 v. (8) M t s t s V S􏼁 � min 􏼐C S , a􏼁 + cΕ􏽨V􏼐S S , a , W 􏼁 􏼑􏽩􏼑. t t t t t t+1 (4) In addition, aggregation is used to reduce computation and generalize the evaluation of the value function across *e size of the state space increases rapidly as the other similar states. Diﬀerent but similar states are aggre- problem size becomes larger; i.e., as the dimensions of state S gated only to approximate the value function at the decision- and external information W increases. *us, the calculation making point. After the decision, the states are disaggregated of every V(S ) in the reverse direction starting from V(S ) at t T and proceed to the next simulation event. In this study, terminal time τ within a reasonable time is almost im- temporal and spatial aggregations are used. *e temporal possible as V(S ) is evaluated for all states S ∈ S. *erefore, t t TA and spatial aggregation sets are, respectively, denoted as ϕ we used ADP, which is a type of reinforcement learning and SA TA SA and ϕ , where the levels are |ϕ | � 3 and |ϕ | � 9. a powerful tool for solving stochastic and dynamic problems Temporal aggregation is achieved by dividing the day into and making real-time decisions [33]. ADP approximates TA TA three time zones as 01 : 00–08 : 00 (ϕ ), 08 : 00–11 : 00 (ϕ ), V(S ) iteratively and in the forward direction. It makes a 1 2 TA and 11 : 00–01 : 00 (ϕ ), depending on the incidents; each of decision to minimize 􏽢 v at each iteration and decision point. these time zones has similar demands (calls). For the spatial In equation (5), 􏽢 v is a sample estimate of the value of being in aggregation, the space is divided into a grid divided into nine state S obtained in iteration n at time τ , and V is a value t t squares with three equal sections along both the horizontal function that returns an approximate value of being in a and vertical axes. *e state’s attributes used for the evalu- certain state obtained from all previous steps: ation are the number of idle or relocating ambulances and n M n 􏽢 v � min 􏼐C S , a􏼁 + cΕ􏽨V􏼐S S , a , W 􏼁 􏼑􏽩􏼑. the number of patients waiting to be allocated an ambulance. t t t+1 (5) t t Other attributes of the state are omitted. In other words, the aggregated state that stores the value v is used to update V(S ) to make it more accurate, as n,t of the value function is a vector of 19 dimensions consisting shown in equation (6), and δ is the step size in iteration n at n,t of the number of idle ambulances and pending patients in time τ , where 0≤ δ ≤ 1. t s each of nine square regions and the time zone. *e value of n n,t n n,t n V S 􏼁 ⟵ 􏼐1 − δ 􏼑V S 􏼁 + δ 􏽢 v . (6) the value function for all aggregated states is stored in a t s t s t 6 Journal of Healthcare Engineering r r V⟵Π s , z , V􏼁 . (10) S : postdecision 1 M W : exogenous a state information However, if theΠ operator is used at every decision- S M a making instant, the time required per iteration is greatly increased, although the number of iterations is reduced because the reference state is compared with all other states. *erefore, in this study, the Π operator is applied sto- chastically to take advantage of the computational time. At the end of each iteration, we probabilistically sample ten states for each time zone, with the probability being pro- portional to the number of visits to that state. *en, with only the sampled states as reference points, all other states are updated through theΠ operator. Using the stochastic monotonicity-preserving projection, the approximation of the value function can be eﬀectively updated by applying the Figure 3: State-transition diagram. operator in a much more time-eﬃcient manner. Here, we propose the mini-batch monotone-ADP algo- lookup table. Aggregation reduces the size of the table, and rithm, which stochastically uses the monotonicity-preserving the use of the postdecision state reduces the number of times projection to modify the monotone-ADP algorithm proposed a table is queried. Algorithm development process in this in the study by Powell [33]. *e detailed algorithm is shown in study so far builds on previous research into the ambulance Figure 4. *e initial value of V aﬀects the tradeoﬀ relationship dispatch and redeployment problem by additionally con- between exploration and exploitation. In this study, as the sidering multiple types of patients and ambulances, and minimization problem is considered, the initial value of V is patient classiﬁcation errors; so, we recommend to see set to 0 to explore as many action decisions as possible. [16, 33] for full details. However, as it is still a large table, we also use the 4. Numerical Experiments and Results monotonicity-preserving projection operatorΠ introduced by Jiang and Powell [34]. If the expected contribution between 4.1. Experimental Design. *is study used actual data ob- some states can be compared in advance, this operator can be tained on March 2015 for Songpa-gu, Seoul, Korea. Songpa- used to reduce the computation by eﬃciently approximating gu is a high-density neighborhood with a population of the value function. In this study, if state S has (1) a greater or approximately 680,000 and an area of approximately 90 km . equal number of idle ALS vehicles at each emergency center, Actual historical data on patient arrival rate and traﬃc was (2) a greater or equal number of idle ambulances at each obtained from the South Korean Open Data Portal (data.- emergency center, (3) a lesser or equal number of pending go.kr) and the Seoul Traﬃc Information Center, respectively. high-risk patients in each region (aggregated space), and (4) *ese data reveal an average of 127.9 calls per day, of which fewer patients who have not been assigned an ambulance in 24.8% are assumed to be high risk [32] *e area contains each region than state S , then being in state S would result in three hospitals with emergency rooms, six ambulances, and better contributions than being in state S . If S dominates S as six ﬁre stations that function as waiting locations (Figure 5). described, S≽ S . In this study, because the aim is to minimize Actual data on the time-varying demand and changes in RLI, V(S), the expected value of being in state S, should be less ambulance speed over time were also used in the model. ′ ′ than V(S ) if S≽ S . Patient calls were generated from a Poisson process with r r Let s ∈ S be a reference state, z ∈ R be a reference diﬀerent parameters for each district, and the arrival time of r r value, and (s , z ) be a reference point for comparison. *e the calls at each district was also generated using a Poisson value function is V∈ R , and the monotonicity-preserving process with a time-varying parameter. *e average number d d projection operation is deﬁned asΠ : S × R × R ⟶ R . of patients who arrived from the entire Songpa-gu area over *e component of the output vector of Π at state s is time is shown in Figure 6, and the average speed of an deﬁned as ambulance in traﬃc is shown in Figure 7. r r z , if s � s , It is assumed that up to two ambulances can be placed in a ⎧ ⎪ waiting location at one time. *e coeﬃcients of the RLI r r r ⎨ z ∧ V (s), if s ≼ s, s≠ s , r r function were set to C � 1, C � 0.25, Penalty � 30, and Π s , z , V􏼁 (s) � (9) H L M t r r r z ∨ V (s), if s ≽ s, s≠ s , t RTT � 7 min, respectively, based on basic interviews with EMS practitioners. As the RLI can be viewed as an adjusted RT, this V (s), otherwise. setting means that exceeding the RTT is equivalent to a 30-min In general, for every iteration of ADP, theΠ operator is delay. Moreover, 4 min for a high-risk patient is equal to 1 min applied every time after updating the value function with for a low-risk patient. However, diﬀerent values can be applied r r value z for current state s . Jiang and Powell [34] showed depending on the practitioners’ opinion. *e service time at the that the value function converges quickly with fewer iter- patient’s location was assumed to follow a gamma distribution ations because the monotonicity of the state set is always with a scale parameter θ � 3.57 and a shape parameter k � 6.2, maintained by using theΠ operator as follows: with an average of 22.12 min. *e service time at the hospitals M Journal of Healthcare Engineering 7 Step 0. Initialize V = 0 ∀t Set n = 1, iteration termination time τ , learning termination criteria N Step 1. Set τ = 0 Get initial state S aer a warm-up period Step 2. For every decision point τ , Calculate the sample estimate: Step 2a. n a,n v = min (C(S , a ) + γV(S )) t t t Step 2b. Update the value function: a,n n,t–1 a,n n,t–1 V (S ) (1 – δ )V(S ) + δ v t–1 s t–1 s n a,n Step 2c. Take action argmin (C(S ,a ) + γV(S )); go to the next decision point t t t τ ; t+1 a,n Step 2d. If τ ≥ τ , go to Step 3; else, update state S to S t+1 T t t+1 TA Step 3. If n = N, terminate.; else, for each time -zone ϕ , Step 3a. Sample 10 states with a probability of nn TA a,m TA a,m p = ∑ ∑ 1 / ∑ ∑ 1 for each state s ∈  s τ ∈ϕ m=1 {s=S } s,τ ∈ϕ m=1 {s=S } t i t t i t Step 3b. Perform monotonicity projection operator on all sampled reference state S : r r V Π (s ,V(s ),V) Step 4. Increase n by 1 and return to Step 1 Figure 4: Details of the mini-batch monotone-ADP algorithm. 0 6 12 18 24 Time (h) Figure 6: Average arrival time of all emergency calls received in one day. Waiting location Hospital Figure 5: Locations of emergency centers and hospitals in Songpa- gu, Seoul. 0 6 12 18 24 Time (h) was also assumed to follow a gamma distribution with a scale Figure 7: Average vehicle speeds over a day. parameter θ � 5.02 and a shape parameter k � 3.0 with an average of 15.05 min, as inferred by Maxwell et al. [15]. *e step size of the proposed ADP algorithm was set to (see the studies by Jiang and Powell [34] and Ryzhov et al. n,t n δ � 1/ 1 a,m , which is the reciprocal of the [35] for details). *e discount factor c was set to 0.9, which is 􏽐 􏽐 s t m�1 s�S { } number of visiting states S from the beginning to time τ at future-oriented and showed the best performance in simple iteration n. *is step size almost deﬁnitely assures conver- tests. *e algorithm was implemented in Python, and all gence as the number of iterations increases with a well- experiments were run on a computer with an i5-4460 CPU. known result in stochastic approximation because it satisﬁes We varied the following three factors to determine their ∞ n,t ∞ n,t 2 􏽐 δ �∞ and 􏽐 (δ ) <∞, with some regularity inﬂuence on the RLI: the ratio of ALS to the total number of n�0 s n�0 s assumptions regarding the underlying stochastic processes ambulances (hereafter the ALS ratio), the undertriage rate α, Avg. speed of vehicle (km/h) Avg. number of calls received 8 Journal of Healthcare Engineering and the overtriage rate β. In this experiment, the ALS ratios paired t-test for the ADP and greedy policies, the former with respect to the six ambulances were 0.0, 0.17, 0.33, 0.5, performed signiﬁcantly better in 97 of 100 combinations at the 95% conﬁdence level. In most experiments, the p value 0.67, 0.83, and 1.0. Each error α and β was divided into ﬁve increments of 0.1, beginning at 0. A complete factorial ex- was less than 0.001, indicating that the dominant perfor- periment was performed for each combination of factors. mance was very signiﬁcant. Table 3 shows the diﬀerence in *e learning phase of the proposed ADP algorithm, RLI between the ADP and the greedy policy based on the which approximates the optimal value function, was ter- ALS ratio, which had the greatest eﬀect on patient risk level. minated based on a two-h limit instead of the number of Overall, the patient RLI decreased by 0.486 when using the iterations. We drew each point in Figure 8 to represent the ADP policy, which was an improvement of 11.2% over the average value of the RLI for 100 iterations. As Figure 8 greedy policy. shows, the RLI gradually decreased and converged after an average of 5387.7 iterations. *e policy optimized by the 4.3. Factors Aﬀecting the Risk Level Index. *e results of proposed ADP algorithm (hereafter the ADP policy) was multiway ANOVA tests on the RLI in the ADP policy are tested 100 times for each experiment. Each iteration of the shown in Table 4 and Figure 10. *e ANOVA was conducted learning phase and each test of optimized policy were run for using SAS software. As expected, the RLI decreased with seven simulation days after a warm-up time of one simu- increasing ALS ratio and decreasing undertriage rate α or lation day, which is suﬃcient time to eliminate the inﬂuence overtriage rate β. *e main eﬀects on error α, error β, and of an arbitrary initial position of the ambulances. ALS ratio were signiﬁcant, as were the interaction eﬀects of the α × ALS ratio and β × ALS ratio, with a signiﬁcance level of 0.01 and a p value of less than 0.001. *e interaction eﬀect 4.2. Comparison with Greedy Policy. As mentioned in Sec- of α × β was signiﬁcant with a p value of less than 0.05, but tion 3, the greedy policy moves the ambulance in a way that the magnitude of the eﬀect was negligible; therefore, a de- minimizes C(S , X (S )); thus, it only considers the con- t t tailed analysis was not conducted. *e interaction eﬀect of tribution at the current state and does not consider the α × β × ALS ratio was not signiﬁcant. *e ALS ratio had the eﬀects of the current action on future situations. However, greatest impact on the RLI, followed by α, the α × ALS ratio because it still considers the patient severity classiﬁcation interaction, the β × ALS ratio interaction, and β. Figure 10 errors, ambulance type, and the expected response time of shows that each factor has a nonlinear eﬀect on RLI. the current state, it is a basic and reasonable policy that is *e main eﬀects of the diﬀerent factors are summarized expected to perform at least better than a myopic policy that in Figure 11 using the averages of the experimental values for allocates the nearest ambulance to the patient and relocates all levels of the factors. *e eﬀect of undertriage rate was the former to the nearest available waiting location. distinctly nonlinear; RLI increased rapidly as α increased It is diﬃcult to compare the various policies with a small from 0 to 0.1. Moreover, when error α increased, there was number of simulation experiments because the RLI has high an increased frequency of assigning a BLS to misclassiﬁed variability due to the inherent uncertain nature of patient actual high-risk patients, leading to a negative impact on the numbers in the EMS system. *erefore, we used the common patient’s risk level. Conversely, the RLI increased linearly random number (CRN), a variance reduction technique, to with increasing β; however, this eﬀect was not large because eﬃciently compare alternative policies with a small number of assigning ALS to a misclassiﬁed actual low-risk patient does simulations. *is was made possible because the CRN method not immediately and directly increase that patient’s risk synchronizes a random number stream for some variables to level, but rather indirectly aﬀects the ability of future high- generate the same random number in every alternative policy risk patients to cope. Another reason for the small eﬀect is when running the simulation. In this study, we compared the that the absolute number of high-risk patients is relatively greedy policy and the ADP policy under the condition that the small. As the ALS ratio decreased, the RLI increased more random number streams of the patients’ occurrence times, rapidly. Figures 12 and 13 show the interaction eﬀect be- locations, and actual and classiﬁed severities were synchronized. tween the undertriage rate α or overtriage rate β and ALS To ﬁnd the minimum number of ALS vehicles capable of ratio on the RLI. As the ALS ratio increased, the RLI was less eﬀectively transporting high-risk patients, the average RLI aﬀected by both classiﬁcation errors; however, when the ALS according to the ALS ratio was analyzed as shown in Fig- ratio was relatively low, α generated a greater diﬀerence in ure 9. Figure 9 shows that the RLI increased sharply when RLI than β. the ALS ratio decreased to less than 0.5. *is was a result of the frequent assignment of BLS to high-risk patients because there were insuﬃcient ALSs to treat them. In a further 4.4. Operational Properties of the Improved Ambulance Op- experiment that restricted BLS from transporting high-risk eration Policy. Although the ADP policy performs better patients, these patients continued to accumulate in the queue than the greedy policy, understanding how ambulance if the ALS ratio was less than 0.5. *is indicates that the ALS operations based on the ADP policy diﬀer from those of the in the EMS system had insuﬃcient capacity; therefore, greedy policy is complex. *us, to gain a greater un- derstanding of operational properties and more general and subsequent analyses of the experimental results will only evaluate situations in which the ALS ratio is above 0.5. intuitive insights into decision-making in the proposed optimized ambulance operation policy, we developed and Table 2 shows the results of the RLI of the two policies for each of the four ALS ratios and ﬁve levels of α and β. In the analyzed additional indices other than RLI. Journal of Healthcare Engineering 9 3.9 3.5 3.4 3.8 3.3 3.7 3.2 3.6 3.1 3.5 3.0 3.4 2.9 2.8 3.3 Iteration Iteration (a) (b) Figure 8: Risk level index for iterations of the learning phase: (a) α � β � 0, ALS Ratio � 0.83 and (b) α � β � 0.4, ALS Ratio � 0.83. average, 30% of patients classiﬁed as low risk were assigned an ambulance other than the nearest ambulance. *e second index measured was the present orientation for patients classiﬁed as low-risk index (PLI), which refers to the ratio achieved when the ALS nearest to a low-risk patient is allocated when the nearest ambulance is that speciﬁc ALS and other idle ambulances are present. As the PLI value increased when a low-risk patient was assigned to the nearest ALS, a larger PLI value can be considered a more short- sighted dispatch approach, whereas a smaller PLI value is a more forward-looking dispatch approach. As a result, the PLI was minimally aﬀected by the overtriage rate β (Table 6) but increased with increasing undertriage rate α. *us, when the undertriage rate α was low, a patient classiﬁed as low risk was relatively frequently allocated an ambulance that is farther away, even if there was a closer ALS, in order to 0.00 0.17 0.33 0.50 0.67 0.83 1.00 prepare for potential high-risk patients in future. On the ALS ratio contrary, when the undertriage rate α was high, the nearest ALS was frequently assigned to a patient even if they were Greedy ADP classiﬁed as low risk. Furthermore, PLI exhibited nonlinear characteristics with undertriage rate α. When α increased Figure 9: Comparison of the risk level index for greedy and ADP from 0, the PLI increased considerably; however, when α patient transport policies based on the ALS (advanced life support) exceeded 0.2, the increase in PLI was reduced. ratio. Transporting a low-risk patient via ALS instead of BLS is a relatively ineﬃcient way of using ambulance resources *e ﬁrst of these indices, the future orientation for because it is an oversupply of the medical service. *us, patients classiﬁed as high-risk index (FHI), refers to the ratio the third index measured was the ineﬃciency of the ALS achieved when the nearest ALS is not allocated, or the index (IAI), which refers to the ratio achieved by allo- cating an ALS to a patient classiﬁed as low risk. Table 7 nearest ambulance is not allocated to a patient classiﬁed as high risk when other idle ALSs are present. *e FHI was shows the IAI for error α and error β, which is the average value of all experiments except for an ALS ratio of 1.0. IAI close to 0 in almost all situations (Table 5). As the availability of an ALS increased as the ALS ratio increased, the FHI increased as α increased but was not signiﬁcantly aﬀected by β. In other words, as the undertriage rate increased, the increased slightly from 0.05 to 0.09, which was slightly fu- ture-oriented but still very low. *is means that almost all ineﬃcient use of ALS vehicles increased as more ALSs patients classiﬁed as high risk, regardless of the magnitude of were assigned to patients classiﬁed as low risk. IAI also the error and the ALS ratio, were assigned the nearest ALS or increased considerably and nonlinearly with α, similar to a BLS if it was closer. In other words, ambulances tried to PLI. However, if the undertriage rate exceeded 0.2, the respond as quickly as possible to patients classiﬁed as high increase in IAI began to decrease. Finally, the average time risk in any situation. On the contrary, dispatching ambu- required to relocate an ambulance was 2.61 min for the greedy policy and 3.84 min for the ADP policy. *is in- lances to patients classiﬁed as low risk was less aﬀected by the distance between the patient and the ambulance. On dicates that although the greedy policy tried to relocate Risk level index Risk level index Risk level index 4600 10 Journal of Healthcare Engineering Table 2: Risk level index of each ambulance operation policy, ALS ratio, undertriage rate α, and overtriage rate β. Overtriage rate β ALS ratio Undertriage rate α 0 0.1 0.2 0.3 0.4 Greedy ADP Greedy ADP Greedy ADP Greedy ADP Greedy ADP 0 5.12 3.99 5.26 4.21 5.48 4.32 5.57 4.61 5.68 4.89 ∗ ∗∗ 0.1 5.58 5.11 5.81 5.31 5.91 5.60 6.12 5.93 6.19 6.12 ∗ ∗ ∗∗ ∗∗ 0.5 0.2 5.98 5.75 6.34 5.97 6.30 6.12 6.49 6.45 6.62 6.67 ∗ ∗ 0.3 6.50 6.26 6.74 6.38 6.73 6.47 6.97 6.69 7.10 6.79 0.4 7.00 6.45 7.19 6.49 7.41 6.68 7.46 6.78 7.52 6.90 0 4.07 3.43 4.13 3.45 4.18 3.52 4.20 3.59 4.30 3.70 0.1 4.47 3.99 4.47 4.14 4.51 4.23 4.60 4.32 4.61 4.46 0.67 0.2 4.78 4.38 4.77 4.47 4.89 4.40 4.90 4.52 4.93 4.68 0.3 5.12 4.45 5.06 4.60 5.24 4.72 5.16 4.65 5.32 4.75 0.4 5.43 4.55 5.49 4.75 5.48 4.70 5.69 4.69 5.59 4.84 0 3.47 2.99 3.47 3.03 3.57 3.09 3.55 3.12 3.58 3.09 0.1 3.66 3.31 3.78 3.29 3.77 3.41 3.77 3.38 3.79 3.43 0.83 0.2 3.90 3.39 3.88 3.40 3.90 3.45 3.89 3.43 3.95 3.47 0.3 4.03 3.43 4.08 3.47 4.04 3.47 4.08 3.56 4.08 3.46 0.4 4.17 3.46 4.23 3.45 4.17 3.54 4.30 3.46 4.36 3.49 0 3.18 2.71 3.17 2.72 3.19 2.8 3.15 2.76 3.16 2.73 0.1 3.18 2.74 3.20 2.79 3.17 2.8 3.17 2.77 3.18 2.80 1 0.2 3.18 2.74 3.18 2.76 3.17 2.83 3.17 2.77 3.17 2.83 0.3 3.18 2.78 3.17 2.80 3.17 2.80 3.16 2.78 3.17 2.78 0.4 3.14 2.81 3.16 2.80 3.16 2.85 3.17 2.82 3.16 2.79 ∗ ∗∗ Note. p is less than 0.001 in all experiments except p< 0.05; not signiﬁcant. Table 3: Diﬀerence in the risk level index between the ADP and greedy policies. Risk level index (ADP-greedy) ALS ratio Average Maximum Minimum 0.5 − 0.485 (8.0%) − 1.160 (22.0%) 0.000 (0.0%) 0.67 − 0.535 (11.0%) − 0.996 (17.5%) − 0.156 (3.4%) 0.83 − 0.536 (13.6%) − 0.872 (20.0%) − 0.346 (9.5%) 1 − 0.389 (12.3%) − 0.467 (14.7%) − 0.314 (9.9%) Average − 0.486 (11.2%) Table 4: ANOVA results of the risk level index. Dependent variable: risk level index (RLI) Source DF Sum of squares Mean square F value Pr> F Model 99 16158.86 163.22 402.25 <.0001 Error 9900 4017.07 0.41 Corrected total 9999 20175.93 R-square Coeﬀ var Root MSE RLI mean 0.801 15.592 0.637 4.085 Source DF Anova SS Mean square F value Pr> F ALS ratio 3 13722.50 4574.17 11272.90 <.0001 α 4 1257.75 314.44 774.93 <.0001 β 4 107.24 26.81 66.07 <.0001 ALS ratio × α 12 933.14 77.76 191.64 <.0001 ALS ratio × β 12 112.73 9.39 23.15 <.0001 α × β 16 10.95 0.68 1.69 0.042 ALS ratio × α × β 48 14.56 0.30 0.75 0.9013 ambulances to make them idle as quickly as possible, the 5. Discussion ADP policy tried to relocate ambulances to positions in One of the major diﬃculties of an EMS system that which they could better respond to future patients, which led to improved performance. transports patients by emergency ambulances is that they Error α Journal of Healthcare Engineering 11 1.0 6.5 6.0 0.9 5.5 0.8 5.0 0.7 4.5 0.6 4.0 0.5 3.5 0.4 0.3 3.0 0.0 0.1 0.2 0.2 0.1 0.3 0.0 0.4 Figure 10: Risk level index plot for the ALS ratio, error α, and error β. 4.40 4.50 4.20 4.10 3.90 3.60 3.30 3.80 0 0.1 0.2 0.3 0.4 0 0.1 0.2 0.3 0.4 Error β Error α (a) (b) 6.00 5.70 5.40 5.10 4.80 4.50 4.20 3.90 3.60 3.30 3.00 2.70 2.40 0.50 0.67 0.83 1 ALS ratio (c) Figure 11: Eﬀect of (a) error α, (b) error β, and (c) ALS ratio on the patient risk level index. have to respond as quickly as possible despite limited am- patients were categorized into two groups: high risk bulance resources. As not all patients are actual emergency (emergency) and low risk (nonemergency), where the ma- patients, it is clear that using a mixed ALS/BLS system based jority fall into the latter category. A mixed ALS/BLS (two- tiered ambulance) system in which ALS and BLS vehicles are on the severity of the patient’s condition is a more eﬃcient management strategy that will enable ambulances to re- suitable for transporting high-risk and low-risk patients, spond to patients more rapidly. However, a key limitation of respectively, was also considered. Two types of classiﬁcation mixed ALS/BLS systems is the high risk of errors when errors were assumed. *e undertriage rate α was the classifying the severity of the patient’s conditions. probability of false classiﬁcations of actual high-risk patients, *erefore, we developed an ADP model to optimize the and the overtriage rate β was the probability of false clas- ambulance dispatch and redeployment policy whilst in- siﬁcations of actual low-risk patients. To develop a realistic cluding patient severity classiﬁcation errors, which has not model, system dynamics such as the time-varying traﬃc and been suﬃciently addressed by previous research. *e frequency of patient occurrence and ambulance service time Error β Risk level index Risk level index Risk level index ALS ratio Risk level index 12 Journal of Healthcare Engineering 7.5 7.5 6.5 6.5 5.5 5.5 4.5 4.5 3.5 3.5 2.5 2.5 0 0.1 0.2 0.3 0.4 0.5 0.67 0.83 1 Error α ALS ratio AR = 0.5 AR = 0.83 α = 0.0 α = 0.3 AR = 0.67 AR = 1.0 α = 0.1 α = 0.4 α = 0.2 (a) (b) Figure 12: Interaction eﬀect of ALS ratio and error α on the risk level index. 7.0 7.0 6.5 6.5 6.0 6.0 5.5 5.5 5.0 5.0 4.5 4.5 4.0 4.0 3.5 3.5 3.0 3.0 2.5 2.5 0 0.1 0.2 0.3 0.4 0.5 0.67 0.83 1 Error β ALS ratio AR = 0.5 AR = 0.83 β = 0.0 β = 0.3 AR = 0.67 AR = 1.0 β = 0.1 β = 0.4 β = 0.2 (a) (b) Figure 13: Interaction eﬀect of ALS ratio and error β on the risk level index. Table 5: Future orientation for patients classiﬁed as high-risk index Table 7: Ineﬃciency of the ALS index (IAI) for classiﬁcation errors. (FHI) according to the ALS ratio. Undertriage rate α 0 0.1 0.2 0.3 0.4 ALS ratio 0.50 0.67 0.83 1 IAI 0.40 0.70 0.82 0.86 0.88 FHI 0.05 0.07 0.08 0.09 Overtriage rate β 0 0.1 0.2 0.3 0.4 IAI 0.73 0.73 0.73 0.74 0.74 Table 6: Present orientation for patients classiﬁed as low-risk index were based on historical data. As a result, the proposed ADP (PLI) for classiﬁcation errors. model reduced the risk level index (RLI) for all patients by an Undertriage rate α 0 0.1 0.2 0.3 0.4 average of 11.2% compared to the greedy policy. PLI 0.50 0.72 0.80 0.83 0.84 We also analyzed the magnitude and correlation of the Overtriage rate β 0 0.1 0.2 0.3 0.4 eﬀects of α, β, and the ALS ratio on the patient RLI under PLI 0.74 0.74 0.74 0.74 0.74 optimized ambulance dispatch and relocation policies. *e Risk level index Risk level index Risk level index Risk level index Journal of Healthcare Engineering 13 patient RLI decreases when the ALS ratio increases or either Conflicts of Interest classiﬁcation error decreases. ALS ratio has the greatest *e authors declare that they have no conﬂicts of interest. impact on RLI, followed by α, α × ALS ratio interaction, β × ALS ratio interaction, and β. *e interaction eﬀects show that the patient RLI is less aﬀected by changes in both Acknowledgments classiﬁcation errors as ALS ratio increases. Furthermore, a key observation is that α is much more sensitive than β in *is work was supported by the National Research Foun- terms of the patient RLI. *erefore, it is desirable to classify dation of Korea (NRF) grant funded by the Korea gov- patient severity in order to minimize the undertriage rate, ernment (MSIT) (2017R1E1A1A03070757). even though it may increase the overtriage rate. For ex- ample, a patient whose condition is unclear or ambiguous References and cannot be classiﬁed accurately would be classiﬁed as high risk. Furthermore, we evaluated the characteristics of [1] Y. Choi, Need to Find Measures to Reduce Non-emergency or the optimized ambulance operation policy. Patients clas- Habitual 911 Users, Minan Newspaper, 2016, http://www. siﬁed as high risk were almost always assigned the nearest minan21c.com/ezview/article_main.html?no�9319. ALS regardless of the error level or ALS ratio. However, [2] S. Lee, Do not use 911 Ambulances for Non-Emergency Pa- patients classiﬁed as low risk were more likely to be al- tients Anymore, Yangcheon Newspaper, 2016, http://ycnews. kr/sub_read.html?uid�7891&section�sc8&section2�%B1% located the nearest ALS as the undertriage rate increased. E2%B0%ED. Moreover, the manner in which ambulances operated was [3] Ministry of Health and Welfare Republic of Korea, Guidelines not signiﬁcantly aﬀected by the overtriage rate. *ese for Registration and Management of Ambulance, Ministry of ﬁndings could serve as useful guidelines for optimizing Health and Welfare Republic of Korea, Seoul, South Korea, ambulance operations when patient severity classiﬁcation 3rd edition, 2015. errors exist. Although the experimental environment was [4] J. P. Ornato, E. M. Racht, J. J. Fitch, and J. F. Berry, “*e need limited to Seoul, we expect that these results would not be for ALS in urban and suburban EMS systems,” Annals of signiﬁcantly diﬀerent in other regions with characteristics Emergency Medicine, vol. 19, no. 12, pp. 1469-1470, 1990. similar to Seoul, e.g., urban areas with a similar density of [5] B. Wilson, M. C. Gratton, J. Overton, and W. A. Watson, ambulance, base, and demand. However, in order to ﬁnd “Unexpected ALS procedures on non-emergency ambulance out the impact of speciﬁc regional characteristics on the calls: the value of a single-tier system,” Prehospital and Di- saster Medicine, vol. 7, no. 4, pp. 380–382, 1992. ADP policy, further research is required, such as learning a [6] K. C. Chong, S. G. Henderson, and M. E. Lewis, “*e vehicle new policy in the area with diﬀerent characteristics, e.g., mix decision in emergency medical service systems,” rural areas with fewer ambulances and demands, or ap- Manufacturing & Service Operations Management, vol. 18, plying transfer learning using a policy that has completed no. 3, pp. 347–360, 2016. learning in a similar area. [7] O. Braun, R. McCallion, and J. Fazackerley, “Characteristics of *e main goal of this study was to develop an algorithm midsized urban EMS systems,” Annals of Emergency Medi- to determine the optimal ambulance operation policy using cine, vol. 19, no. 5, pp. 536–546, 1990. a realistic model that includes patient severity classiﬁcation [8] C. M. Slovis, T. B. Carruth, W. J. Seitz, C. M. *omas, and errors and then provide insights into classifying patient W. R. Elsea, “A priority dispatch system for emergency severity and ambulance operational strategy by identifying medical services,” Annals of Emergency Medicine, vol. 14, no. 11, pp. 1055–1060, 1985. the eﬀects of several factors and useful indices under the [9] J. Stout, P. E. Pepe, and V. N. Mosesso, “All advanced life optimized policy. *erefore, the goal was not to demonstrate support vs tiered response a mbulance systems,” Prehospital the superiority of an all-ALS system versus a mixed-ALS/ Emergency Care, vol. 4, no. 1, pp. 1–6, 2000. BLS system or to increase the accuracy of patient classiﬁ- [10] L. Brotcorne, G. Laporte, and F. Semet, “Ambulance location cation. Although the total number of ambulances is assumed and relocation models,” European Journal of Operational to be constant in this study, the number of ambulances Research, vol. 147, no. 3, pp. 451–463, 2003. available under one budget can vary due to diﬀerences in [11] C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei, “Dynamic operating and purchasing costs between ALS and BLS. In- ambulance dispatching: is the closest-idle policy always op- creasing the total number of ambulances with a high ratio of timal?,” Health Care Management Science, vol. 20, pp. 1–15, BLS may contribute positively to reducing the patient risk level. In addition, if it was possible to lower the undertriage [12] D. Degel, L. Wiesche, S. Rachuba, and B. Werners, “Time- dependent ambulance allocation considering data-driven and overtriage rates, EMS system managers could consider empirically required coverage,” Health Care Management controlling these errors and the conﬁguration of ambulances Science, vol. 18, no. 4, pp. 444–458, 2015. to enable eﬀective decision-making that could minimize the [13] M. S. Maxwell, M. Restrepo, S. G. Henderson, and patient risk level within a limited budget. *e ﬁndings of this H. Topaloglu, “Approximate dynamic programming for study could be useful for such future research. ambulance redeployment,” INFORMS Journal on Computing, vol. 22, no. 2, pp. 266–281, 2010. Data Availability [14] A. A. Nasrollahzadeh, A. Khademi, and M. E. Mayorga, “Real- time ambulance dispatching and relocation,” Manufacturing *e data used to support the ﬁndings of this study are & Service Operations Management, vol. 20, no. 3, pp. 389–600, available from the corresponding author upon request. 2018. 14 Journal of Healthcare Engineering [15] M. S. Maxwell, S. G. Henderson, and H. Topaloglu, “Tuning [32] S. Vandeventer, J. R. Studnek, J. S. Garrett, S. R. Ward, approximate dynamic programming policies for ambulance K. Staley, and T. Blackwell, “*e association between am- bulance hospital turnaround times and patient Acuity, des- redeployment via direct search,” Stochastic Systems, vol. 3, tination hospital, and time of day,” Prehospital Emergency no. 2, pp. 322–361, 2013. Care, vol. 15, no. 3, pp. 366–370, 2011. [16] V. Schmid, “Solving the dynamic ambulance relocation and [33] W. B. Powell, Approximate Dynamic Programming for Op- dispatching problem using approximate dynamic pro- erations Research: Solving the Curses of Dimensionality, John gramming,” European Journal of Operational Research, Wiley & Sons, New York, NY, USA, 2nd edition, 2011. vol. 219, no. 3, pp. 611–621, 2012. [34] D. R. Jiang and W. B. Powell, “An approximate dynamic [17] D. Bandara, M. E. Mayorga, and L. A. McLay, “Priority programming algorithm for monotone value functions,” dispatching strategies for EMS systems,” Journal of the Op- Operations Research, vol. 63, no. 6, pp. 1489–1511, 2015. erational Research Society, vol. 65, no. 4, pp. 572–587, 2013. [35] I. O. Ryzhov, P. I. Frazier, and W. B. Powell, “A new optimal [18] L. A. McLay, “A maximum expected covering location model stepsize for approximate dynamic programming,” IEEE with two types of servers,” IIE Transactions, vol. 41, no. 8, Transactions on Automatic Control, vol. 60, no. 3, pp. 743– pp. 730–741, 2009. 758, 2015. [19] L. A. McLay and M. E. Mayorga, “A model for optimally dispatching ambulances to emergency calls with classiﬁcation errors in patient priorities,” IIE Transactions, vol. 45, no. 1, pp. 1–24, 2013. [20] J. Lee, Advanced Life Support Ambulance Will be Expanded for Emergency Patients, Doctorsnews, 2016, http://www.doctorsnews. co.kr/news/articleView.html?idxno�1101226. [21] L. A. McLay and M. E. Mayorga, “A dispatching model for server-to-customer systems that balances eﬃciency and eq- uity,” Manufacturing & Service Operations Management, vol. 15, no. 2, pp. 205–220, 2013. [22] E. Erkut, A. Ingolfsson, and G. Erdogan, ˘ “Ambulance location for maximum survival,” Naval Research Logistics, vol. 55, no. 1, pp. 42–58, 2008. [23] N. Geroliminis, M. G. Karlaftis, and A. Skabardonis, “A spatial queuing model for the emergency vehicle districting and location problem,” Transportation Research Part B: Method- ological, vol. 43, no. 7, pp. 798–811, 2009. [24] S. Jin, S. Jeong, J. Kim, and K. Kim, “A logistics model for the transport of disaster victims with various injuries and survival probabilities,” Annals of Operations Research, vol. 230, pp. 1–17, 2014. [25] V. A. Knight, P. R. Harper, and L. Smith, “Ambulance al- location for maximal survival with heterogeneous outcome measures,” Omega, vol. 40, no. 6, pp. 918–926, 2012. [26] M. P. Larsen, M. S. Eisenberg, R. O. Cummins, and A. P. Hallstrom, “Predicting survival from out-of-hospital cardiac arrest: a graphic model,” Annals of Emergency Med- icine, vol. 22, no. 11, pp. 1652–1658, 1993. [27] L. A. McLay and M. E. Mayorga, “Evaluating emergency medical service performance measures,” Health Care Man- agement Science, vol. 13, no. 2, pp. 124–136, 2010. [28] T. Mizumoto, W. Sun, K. Yasumoto, and M. Ito, “Trans- portation scheduling method for patients in mci using elec- tronic triage tag,” in Proceedings of the Bird International Conference on eHealth, Telemedicine, and Social Medicine, eTELEMED 2011, Gosier, Guadeloupe, France, February 2011. [29] C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei, “An eﬃcient heuristic for real-time ambulance redeployment,” Operations Research for Health Care, vol. 4, pp. 27–35, 2015. [30] S. S. W. Lam, C. B. L. Ng, F. N. H. L. Nguyen, Y. Y. Ng, and M. E. H. Ong, “Simulation-based decision support framework for dynamic ambulance redeployment in Singapore,” In- ternational Journal of Medical Informatics, vol. 106, pp. 37–47, [31] N. T. Argon, L. Ding, K. D. Glazebrook, and S. Ziya, “Dy- namic routing of customers with general delay costs in a multiserver queuing system,” Probability in the Engineering and Informational Sciences, vol. 23, no. 2, pp. 175–203, 2009. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Healthcare Engineering Hindawi Publishing Corporation http://www.deepdyve.com/lp/hindawi-publishing-corporation/two-tiered-ambulance-dispatch-and-redeployment-considering-patient-1i5hXFWo2v

Loading next page...

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher: Hindawi Publishing Corporation
Copyright: Copyright © 2019 Seong Hyeon Park and Young Hoon Lee. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN: 2040-2295
eISSN: 2040-2309
DOI: 10.1155/2019/6031789
Publisher site: See Article on Publisher Site

Abstract

Hindawi Journal of Healthcare Engineering Volume 2019, Article ID 6031789, 14 pages https://doi.org/10.1155/2019/6031789 Research Article Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors Seong Hyeon Park and Young Hoon Lee Department of Industrial Engineering, Yonsei University, D1010, 50, Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea Correspondence should be addressed to Seong Hyeon Park; s.park10@yonsei.ac.kr Received 10 July 2019; Accepted 11 November 2019; Published 9 December 2019 Academic Editor: Ping Zhou Copyright © 2019 Seong Hyeon Park and Young Hoon Lee. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A two-tiered ambulance system, consisting of advanced and basic life support for emergency and nonemergency patient care, respectively, can provide a cost-eﬃcient emergency medical service. However, such a system requires accurate classiﬁcation of patient severity to avoid complications. *us, this study considers a two-tiered ambulance dispatch and redeployment problem in which the average patient severity classiﬁcation errors are known. *is study builds on previous research into the ambulance dispatch and redeployment problem by additionally considering multiple types of patients and ambulances, and patient clas- siﬁcation errors. We formulate this dynamic decision-making problem as a semi-Markov decision process and propose a mini- batch monotone-approximate dynamic programming (ADP) algorithm to solve the problem within a reasonable computation time. Computational experiments using realistic system dynamics based on historical data from Seoul reveal that the proposed approach and algorithm reduce the risk level index (RLI) for all patients by an average of 11.2% compared to the greedy policy. In this numerical study, we identify the inﬂuence of certain system parameters such as the percentage of advanced-life support units among all ambulances and patient classiﬁcation errors. A key ﬁnding is that an increase in undertriage rates has a greater negative eﬀect on patient RLI than an increase in overtriage rates. *e proposed algorithm delivers an eﬃcient two-tiered ambulance management strategy. Furthermore, our ﬁndings could provide useful guidelines for practitioners, enabling them to classify patient severity in order to minimize undertriage rates. Emergency care and transport of patients should be both 1. Introduction highly ﬂexible and rapid because small time delays might have Ambulance operating methods are highly important for the a negative impact on emergency patients. However, in an emergency medical service (EMS) system as they directly EMS system where patient numbers are highly uncertain, aﬀect the patient survival rate and medical service quality. preplanned scheduling or operation solutions may not op- Two types of decision are required during ambulance timally respond to ﬂuctuating situations. *erefore, real-time decision-making is required, which must consider system operations: (1) the dispatch decision, i.e., which ambulance to send to an emergency call, and (2) the redeployment dynamics such as time-varying demands (emergency calls), time-varying traﬃc, and the diﬀerent ﬁrst-aid times required decision, i.e., the waiting location to which the ambulance that has just completed a patient-transport service should by patients. Another important consideration in ambulance be sent. *e goal of ambulance operations is to provide operations is the diﬀerent severity of the transported patients. patients with appropriate emergency treatment within a *e majority of patients are nonemergency patients. *ey short time period and then transport the patient to the request an ambulance because of a lack of transportation, hospital for speciﬁc advanced treatment. *erefore, an inability to ambulate, domestic violence, or poor social sit- eﬃcient strategy is required for dispatching and rede- uations while a few of them can either walk or use public ploying ambulances. transport to reach a hospital [1, 2]. Transfer of nonemergency 2 Journal of Healthcare Engineering patients into types based on their severity [6, 17, 18]. patients by ambulance can be delayed due to the preferential transfer of emergency patients because their deterioration rate However, these studies all assumed that patient severity can be immediately and accurately determined when the call is of health may be much lower. However, as only limited in- formation is delivered during calls to the emergency operator, received. Furthermore, few studies have considered the it is risky to designate a patient’s severity as low and delay the possibility of errors when classifying patient severity during dispatch of an ambulance to the patient. *erefore, all ambulance operations. McLay and Mayorga [19] mathe- emergency calls must be responded to immediately regardless matically addressed patient classiﬁcation errors during of the classiﬁed severity of patients; in South Korea, it is ambulance operations. *ey classiﬁed patient priorities in regulated by law. the all-ALS system into three levels and optimized the ambulance operation policy by using the Markov decision Based on the criteria used in South Korea, ambulances are classiﬁed into two types based on the patients’ level of process (MDP) model. *ey then compared two cases, in which middle-priority patients were classiﬁed as high-risk urgency [3]. (1) An advanced life support (ALS) vehicle is suitable for emergency-patient transport. It must be ac- and low-risk patients. In this context, we propose an approximate dynamic companied by paramedics who can perform more special- ized medical care and is designed with more stringent programming (ADP) model that runs on a discrete event standards, including the minimum area for the patient in the simulation to optimize the dispatch-and-redeployment ambulance and the medical equipment to be installed inside. policy of a two-tiered ambulance system by considering (2) A basic life support (BLS) vehicle is suitable for non- errors in patient-severity classiﬁcation. *e computational emergency patient transport. It provides basic medical experiment environment was created based on actual his- services with relatively little medical equipment and is ac- torical data from Seoul by considering the probability dis- tribution of demand-and-service time, time-varying companied by emergency medical technicians (EMTs). *erefore, high-risk emergency patients transported by BLS demand, and traﬃc speed. *e computational experiments show that our proposed algorithm performs better than the units would be at risk because they may not receive adequate care during transport. *e corresponding ambulance sys- greedy policy. In addition, we identify the inﬂuence and correlation between classiﬁcation errors and the ratio of ALS tems are also classiﬁed into two types: an “all-ALS system” that operates all ambulances as ALS vehicles and a “two- units to BLS units based on patient risk level. *is can tiered ambulance system (tiered system)” that uses a com- provide insights into patient-classiﬁcation attitudes and bination of ALS and BLS units. Previous research has de- ambulance management strategies. bated the superiority of all-ALS or mixed-ALS/BLS ambulance management systems according to their relative 2. Problem Description risks, treatment times, and cost eﬀectiveness [4–9]. To operate a two-tiered ambulance system eﬃciently, an In this study, we use an ADP algorithm to optimize am- emergency center should attempt to classify the severity of bulance dispatch and redeployment decisions in order to the patients during the emergency call. However, the lack of reduce the risk level of patients through rapid trans- information obtained from the call inevitably leads to patient portation. *e approach assumes that the strategic level of severity classiﬁcation errors, which could have a devastating decision-making, such as the location of the emergency impact on the patient risk level. However, although previous center and hospital and the number of ambulances, is ﬁxed. research has attempted to optimize ambulance dispatch and In addition, real-time dispatch and redeployment decisions redeployment strategies, they have not considered the ex- are dealt with at the operational level. *e ambulance op- istence of these classiﬁcation errors. For example, Brotcorne erating environment is assumed to comprise a two-tiered et al. [10] and Jagtenberg et al. [11] revealed that the greedy ambulance, two types of patient classes with diﬀerent se- policy of allocating the nearest ambulance to patients does verities, and patient classiﬁcation errors. *ese consider- not always yield the best performance. Moreover, research ations are not only key factors inﬂuencing decision-making into optimizing decisions in real time has achieved more but are also close to that of an actual ambulance operating realistic results [12]. Maxwell et al. [13], Nasrollahzadeh et al. environment. [14], Maxwell et al. [15], and Schmid [16] all showed that the Patients calling the emergency services are classiﬁed into approximate dynamic programming (ADP) model works two groups: high-and low-risk patients with high and low well as a real-time ambulance model of operational policy severity levels, respectively. We denote the severity of pa- A A optimization. However, although the ADP produced a near- tients as H (L ) if the actual severity of the patient is high C C optimal solution in limited experiments, all of these studies (low) risk, and H (L ) if the classiﬁed severity of the patient assumed one type of ambulance and no classiﬁcation errors. is high (low) risk. High-risk patients are described as life- *us, more sophisticated two-tiered ambulance opera- threatened if they do not receive adequate treatment within a tions are required that consider the existence of classiﬁcation given response time threshold (RTT). Although low-risk errors. Furthermore, it is important to determine (1) how the patients are not life-threatened, it is preferable to treat them optimal operation policy changes according to the classiﬁ- quickly to increase the service satisfaction level and prevent cation errors and (2) what type of classiﬁcation decision their treatment from becoming complicated and turning should be taken for ambiguous patients to minimize patient them into high-risk patients. risk. Some studies have considered the classiﬁcation of *e operation process of the ambulance and the time patient severity in mixed ALS/BLS systems by categorizing spent during the process are shown in Figure 1. *e Journal of Healthcare Engineering 3 Redeployment/dispatch Dispatch decision decision Ambulance arrival at scene Time Patient call arrival Service time Transport Service time Redeployment with patient time at hospital time Time required for proper care with adequate type of ambulance Time required for proper care when BLS is transporting a high-risk patient Figure 1: Flowchart of the ambulance operation process and decision-making points. ambulances typically remain at the emergency center. When *e response time (RT), which is typically used as an a patient is reported, the decision maker decides which evaluation measure of the EMS system, denotes the time ambulance to send to the patient using information of the from the patient report being obtained at the emergency severity classiﬁcation. When an ambulance arrives at the center to the ambulance arriving at the scene. However, in patient location, the actual severity of the patient becomes this study, we use the time required for proper care known, and the patient receives a ﬁrst-aid service. *e (RT_PC), which is the time from the patient report being ambulance then transports the patient to the nearest hospital obtained at the emergency center to the patient beginning to receive appropriate treatment. *at is, unless the ambulance emergency room. After the ambulance arrives at the hos- pital, the patient is transferred to the hospital staﬀ. After is the correct type to handle the severity of the patient, the patient only begins receiving appropriate treatment once the delivering the patient to the hospital, the decision maker determines whether there are any patients waiting to be ambulance arrives at the hospital. For example, if an ALS allocated an ambulance. If such a patient exists, the am- transports a high- or low-risk patient, or if a BLS transports a bulance is allocated to the patient; if a patient does not exist, low-risk patient, the RT_PC does not diﬀer from the original the decision maker determines which emergency center the RT. However, when a BLS transports a high-risk patient, ambulance should be relocated to. When a patient is re- providing appropriate treatment quickly is complicated by ported and no ambulance is available, which is a rare oc- the lack of specialized medical resources, such as a respirator currence in reality and has thus far not been noted in any or emergency medical staﬀ [20]. *us, the end time for the previous experiments, the patient is placed in a virtual RT_PC is the time that the ambulance arrives at the hospital. *e criterion for measuring RT_PC is also expressed in queue. In this situation, when an ambulance is about to be placed into an idle state, a high-risk patient is allocated at a Figure 1. In this study, we propose a risk level index (RLI) that higher priority than low-risk patients, regardless of the report-arrival time. For patients within the same risk level, reﬂects the diﬀerent risk levels of patient groups with dif- an ambulance is allocated on a ﬁrst-come-ﬁrst-served basis. ferent severity as another performance measure of the EMS If an ambulance is idle when a patient is waiting, the am- system. RLI is the response time adjusted to the risk of the bulance must respond to the patient, regardless of the lo- patient. *e RLI function f(RT, S ) is a function of RT_PC cation of the patient; i.e., a delay in ambulance allocation is and actual patient severity (S ), as shown in equation (1) and not allowed. Figure 2: A A ⎧ ⎨ C · RT_PC + 1􏼈RT C> RTT􏼉 · Penalty, if S � H , H P f􏼐RT, S 􏼑 � (1) A A C · RT_PC, if S � L . *e RLI increases linearly with RT_PC but with diﬀerent *e evaluation index of ambulance operations in EMS slopes depending on the severity of the patient. When the systems usually includes the RT [16, 23], the survival rate, RT_PC of high-risk patients exceeds the RTT, a penalty of which is a continuous function of RT [24–28], and the constant value is added. *e value of these parameters can be coverage level, which is the proportion of reports covered set according to the decision of an EMS system manager if within a predeﬁned RTT [29, 30]. However, these have some C ≥ C ≥ 0. *e RTT is typically set to 8 min or 9 min limitations. First, it is diﬃcult to use the RT index to consider H L [21, 22]. the diﬀerence among each patient group with diﬀerent 4 Journal of Healthcare Engineering Table 1: Probability of classiﬁcation errors for patient severity. Classiﬁed severity Probability C C High risk (H ) Low risk (L ) High risk (H ) 1 − α α Actual severity Low risk (L ) Β 1 − β Penalty (1 − α)Pr Pr A C � , H H Time required for (1 − α)Pr A + β 1 − Pr A􏼁 H H Response time threshold proper care (2) of high-risk patients (1 − β) 1 − Pr A􏼁 Pr A C � . High-risk patient L |L α · Pr A + (1 − β) 1 − Pr A􏼁 H H Low-risk patient Figure 2: Risk level index function. 3. Model and Solution Algorithm severities and determine whether the report is covered within the RTT. Second, the quantitative measurement of *e process of the EMS system is modeled as a semi-MDP survival rate over RT is not easily medically validated due to model that runs on discrete event simulation. *e state tran- the diﬀerent status levels of each patient; thus, previous sition function depends partly on the controllable decisions of studies used diﬀerent survival-rate functions. In addition, dispatch and redeployment and partly on unmanageable sto- higher priority might be assigned to a patient whose survival chastic events, such as patient arrival and service completion. A rate is high but rapidly decreasing than to a patient whose decision is made at the time an event occurs that requires new survival rate is already low; this raises an ethical issue. Lastly, decision-making. In other words, the simulation time jumps to as the coverage level only checks whether RT is within RTT, the real time of the next event instead of adding a constant unit it does not evaluate the exact RT; this might cause the time of time. *us, multiple calls are never received simultaneously. th immediately before the RTT to be labeled as the RT, Here, we let τ denote the time when the t event occurs. neglecting the condition of “the sooner, the better” and In a semi-MDP environment, dynamic programming potentially ignoring patients already waiting for longer than (DP) can be used to obtain optimal policies. DP uses the RTT. Conversely, the RLI used in this study has advantages state S, action a, contribution function C(S, a), and of including all characteristics, such as the patient’s severity, transition probabilities. In this study, S represents the state RT, and coverage level. of the EMS system associated with the ambulance and the *e proposed RLI is not an entirely new concept as patient. *e state of ambulance i is denoted by vector a � several studies have used an objective function that either {a1, a2, . . . , a6}, and the state of patient j is denoted by considers the risk associated with matching ambulance type vector p � 􏼈p1, p2, . . . , p4􏼉. Attributes a1–a6 represent and patient severity [6, 14] or that considers a linearly in- the ambulance type (ALS or BLS), ambulance location, creasing risk over time with a penalty for exceeding the time ambulance status (idle, moving toward patient, service in threshold [31]. *e RLI function for high-risk patients can be patient’s location, moving toward hospital, service in viewed not only as the response time adjusted by the patient hospital, and moving back toward emergency center), risk but also as a weighted sum of multiple objectives, the RT patient ID if the ambulance is assigned to the patient, and the coverage level, whereas the RLI function for low-risk destination (speciﬁc patient/hospital/emergency center) if patients is a relatively low-weighted RT. the ambulance is in transit, and time remaining until When classifying patients as high or low risk, two types arrival at destination, respectively. Attributes p1–p4 of error may occur (Table 1). *e undertriage rate α is the represent the patient’s location, time when an incident is probability of classifying a high-risk patient (H ) as a low- reported, status (waiting/in service), and classiﬁed se- risk patient (L ), and the overtriage rate β is the probability verity, respectively. *e set of all ambulances is A, and the of classifying a low-risk patient (L ) as a high-risk patient set of all patients is P. *e state S of event t at time τ is t t (H ). *e purpose of this study is not to determine the exact represented as a vector 􏽮a , p 􏽯 . Action a decides i j i∈A,j∈P value of these errors but to investigate the inﬂuence of these which idle ambulance to send to which waiting patient, or errors; thus, the authors assumed that errors α and β are which emergency center to relocate an ambulance to that known in advance by using historical data. has just been labeled idle after completing its service to a Moreover, the ratio of actual high-risk patients to all hospital. patients (Pr A) is assumed to be known from the historical *e contribution function C(S , a ) returns the value of t t data. In this study, Pr A � 24.8%, according to a survey by the reward given when action a is performed in state S . In t t Vandeventer et al. [32]. *erefore, if α, β, and Pr A are this study, we deﬁne the expected RLI of the patient as the known, we can calculate the probability of a patient being reward value, and the contribution function is described in A C correctly classiﬁed as high risk (Pr ) and vice versa equation (3). As the exact RT is not known when performing H |H (Pr A C): action a at state S , the average RT for the distance is used: L |L t t Slope C Slope C Risk level index Journal of Healthcare Engineering 5 A A C ⎪ Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if ALS carry H patient, H H L H ⎪ | | ⎪ A A C Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if ALS carry L patient, ⎨ H L L L | | C S , a􏼁 � (3) t t A A C ⎪ Pr A C · f RT, H 􏼁 + Pr A C · f RT, L 􏼁 , if BLS carry H patient, H H L H ⎪ | | ⎩ A A C Pr A C · f RT, H + Pr A C · f RT, L , if BLS carry L patient. 􏼁 􏼁 H L L L | | X (S ) is a function that returns the action to be taken in *e ADP further uses the postdecision state and ag- state S , when policy π is used. *e greedy policy π mini- gregation techniques to increase the computation speed. π a mizes C(S , X (S )); that is, at every decision point, an Postdecision state S represents the state immediately after t t t action is taken by only considering the reward that can be the decision to take action a at time τ and before the ex- received at the current state. However, we aim to obtain a ternal information W is received. *us, after a decision is t+1 policy that considers the eﬀects of the current action on made to perform action a in state S , as shown in Figure 3, t t future situations. *us, ADP is used to ﬁnd a policy that the time does not elapse and the process goes into post- minimizes the expected value of the total discounted sum of decision state S deterministically. Next, external in- t π the patient’s RLI over a long period, c C(S , X (S )), formation is received between τ and τ , and the process t�0 t t t t+1 where c∈ [0, 1] is a discount factor expressing how much goes into state S . *e ADP at the current time τ estimates t+1 t future rewards are worth in the present. As the time interval the value of postdecision state S instead of state S by t t+1 between rewards is not constant and cannot be precisely using equation (7) instead of equation (5); thus, calculation predicted in advance, the discount rate is set to a constant for of the expectation value in equation (5) can be omitted. *e simple application. V(S ) denotes the value of being in state ADP has a large computational advantage for estimating the S under policy π; that is, the expected value of the total value of being in a postdecision state, as it can use the discounted sum of the RLI. *en, V(S ) can be recursively deterministic value of postdecision state at the decision point expressed using the Bellman equation form, as in equation instead of computing possibilities of reaching the next state (4). W denotes the external information related to the status S for all possible states: t t+1 change known between τ and τ ; for example, obtaining a n a,n t− 1 t v � min C S , a􏼁 + cV S 􏼁 􏼁. t t t (7) new patient call and the ambulance arrival time at the patient t location or hospital. Let the state transition function be S , Furthermore, V is now a value function that returns an then S (S , a , W ) represents the state at time τ when t t t+1 t+1 a,n approximate value of being in postdecision state S and is external information W is received after taking action a t+1 t updated using equation (8) instead of equation (6): in state S , which is S : t t+1 a,n n,t a,n n,t V S 􏼁 ⟵ 􏼐1 − δ 􏼑V S 􏼁 + δ 􏽢 v. (8) M t s t s V S􏼁 � min 􏼐C S , a􏼁 + cΕ􏽨V􏼐S S , a , W 􏼁 􏼑􏽩􏼑. t t t t t t+1 (4) In addition, aggregation is used to reduce computation and generalize the evaluation of the value function across *e size of the state space increases rapidly as the other similar states. Diﬀerent but similar states are aggre- problem size becomes larger; i.e., as the dimensions of state S gated only to approximate the value function at the decision- and external information W increases. *us, the calculation making point. After the decision, the states are disaggregated of every V(S ) in the reverse direction starting from V(S ) at t T and proceed to the next simulation event. In this study, terminal time τ within a reasonable time is almost im- temporal and spatial aggregations are used. *e temporal possible as V(S ) is evaluated for all states S ∈ S. *erefore, t t TA and spatial aggregation sets are, respectively, denoted as ϕ we used ADP, which is a type of reinforcement learning and SA TA SA and ϕ , where the levels are |ϕ | � 3 and |ϕ | � 9. a powerful tool for solving stochastic and dynamic problems Temporal aggregation is achieved by dividing the day into and making real-time decisions [33]. ADP approximates TA TA three time zones as 01 : 00–08 : 00 (ϕ ), 08 : 00–11 : 00 (ϕ ), V(S ) iteratively and in the forward direction. It makes a 1 2 TA and 11 : 00–01 : 00 (ϕ ), depending on the incidents; each of decision to minimize 􏽢 v at each iteration and decision point. these time zones has similar demands (calls). For the spatial In equation (5), 􏽢 v is a sample estimate of the value of being in aggregation, the space is divided into a grid divided into nine state S obtained in iteration n at time τ , and V is a value t t squares with three equal sections along both the horizontal function that returns an approximate value of being in a and vertical axes. *e state’s attributes used for the evalu- certain state obtained from all previous steps: ation are the number of idle or relocating ambulances and n M n 􏽢 v � min 􏼐C S , a􏼁 + cΕ􏽨V􏼐S S , a , W 􏼁 􏼑􏽩􏼑. the number of patients waiting to be allocated an ambulance. t t t+1 (5) t t Other attributes of the state are omitted. In other words, the aggregated state that stores the value v is used to update V(S ) to make it more accurate, as n,t of the value function is a vector of 19 dimensions consisting shown in equation (6), and δ is the step size in iteration n at n,t of the number of idle ambulances and pending patients in time τ , where 0≤ δ ≤ 1. t s each of nine square regions and the time zone. *e value of n n,t n n,t n V S 􏼁 ⟵ 􏼐1 − δ 􏼑V S 􏼁 + δ 􏽢 v . (6) the value function for all aggregated states is stored in a t s t s t 6 Journal of Healthcare Engineering r r V⟵Π s , z , V􏼁 . (10) S : postdecision 1 M W : exogenous a state information However, if theΠ operator is used at every decision- S M a making instant, the time required per iteration is greatly increased, although the number of iterations is reduced because the reference state is compared with all other states. *erefore, in this study, the Π operator is applied sto- chastically to take advantage of the computational time. At the end of each iteration, we probabilistically sample ten states for each time zone, with the probability being pro- portional to the number of visits to that state. *en, with only the sampled states as reference points, all other states are updated through theΠ operator. Using the stochastic monotonicity-preserving projection, the approximation of the value function can be eﬀectively updated by applying the Figure 3: State-transition diagram. operator in a much more time-eﬃcient manner. Here, we propose the mini-batch monotone-ADP algo- lookup table. Aggregation reduces the size of the table, and rithm, which stochastically uses the monotonicity-preserving the use of the postdecision state reduces the number of times projection to modify the monotone-ADP algorithm proposed a table is queried. Algorithm development process in this in the study by Powell [33]. *e detailed algorithm is shown in study so far builds on previous research into the ambulance Figure 4. *e initial value of V aﬀects the tradeoﬀ relationship dispatch and redeployment problem by additionally con- between exploration and exploitation. In this study, as the sidering multiple types of patients and ambulances, and minimization problem is considered, the initial value of V is patient classiﬁcation errors; so, we recommend to see set to 0 to explore as many action decisions as possible. [16, 33] for full details. However, as it is still a large table, we also use the 4. Numerical Experiments and Results monotonicity-preserving projection operatorΠ introduced by Jiang and Powell [34]. If the expected contribution between 4.1. Experimental Design. *is study used actual data ob- some states can be compared in advance, this operator can be tained on March 2015 for Songpa-gu, Seoul, Korea. Songpa- used to reduce the computation by eﬃciently approximating gu is a high-density neighborhood with a population of the value function. In this study, if state S has (1) a greater or approximately 680,000 and an area of approximately 90 km . equal number of idle ALS vehicles at each emergency center, Actual historical data on patient arrival rate and traﬃc was (2) a greater or equal number of idle ambulances at each obtained from the South Korean Open Data Portal (data.- emergency center, (3) a lesser or equal number of pending go.kr) and the Seoul Traﬃc Information Center, respectively. high-risk patients in each region (aggregated space), and (4) *ese data reveal an average of 127.9 calls per day, of which fewer patients who have not been assigned an ambulance in 24.8% are assumed to be high risk [32] *e area contains each region than state S , then being in state S would result in three hospitals with emergency rooms, six ambulances, and better contributions than being in state S . If S dominates S as six ﬁre stations that function as waiting locations (Figure 5). described, S≽ S . In this study, because the aim is to minimize Actual data on the time-varying demand and changes in RLI, V(S), the expected value of being in state S, should be less ambulance speed over time were also used in the model. ′ ′ than V(S ) if S≽ S . Patient calls were generated from a Poisson process with r r Let s ∈ S be a reference state, z ∈ R be a reference diﬀerent parameters for each district, and the arrival time of r r value, and (s , z ) be a reference point for comparison. *e the calls at each district was also generated using a Poisson value function is V∈ R , and the monotonicity-preserving process with a time-varying parameter. *e average number d d projection operation is deﬁned asΠ : S × R × R ⟶ R . of patients who arrived from the entire Songpa-gu area over *e component of the output vector of Π at state s is time is shown in Figure 6, and the average speed of an deﬁned as ambulance in traﬃc is shown in Figure 7. r r z , if s � s , It is assumed that up to two ambulances can be placed in a ⎧ ⎪ waiting location at one time. *e coeﬃcients of the RLI r r r ⎨ z ∧ V (s), if s ≼ s, s≠ s , r r function were set to C � 1, C � 0.25, Penalty � 30, and Π s , z , V􏼁 (s) � (9) H L M t r r r z ∨ V (s), if s ≽ s, s≠ s , t RTT � 7 min, respectively, based on basic interviews with EMS practitioners. As the RLI can be viewed as an adjusted RT, this V (s), otherwise. setting means that exceeding the RTT is equivalent to a 30-min In general, for every iteration of ADP, theΠ operator is delay. Moreover, 4 min for a high-risk patient is equal to 1 min applied every time after updating the value function with for a low-risk patient. However, diﬀerent values can be applied r r value z for current state s . Jiang and Powell [34] showed depending on the practitioners’ opinion. *e service time at the that the value function converges quickly with fewer iter- patient’s location was assumed to follow a gamma distribution ations because the monotonicity of the state set is always with a scale parameter θ � 3.57 and a shape parameter k � 6.2, maintained by using theΠ operator as follows: with an average of 22.12 min. *e service time at the hospitals M Journal of Healthcare Engineering 7 Step 0. Initialize V = 0 ∀t Set n = 1, iteration termination time τ , learning termination criteria N Step 1. Set τ = 0 Get initial state S aer a warm-up period Step 2. For every decision point τ , Calculate the sample estimate: Step 2a. n a,n v = min (C(S , a ) + γV(S )) t t t Step 2b. Update the value function: a,n n,t–1 a,n n,t–1 V (S ) (1 – δ )V(S ) + δ v t–1 s t–1 s n a,n Step 2c. Take action argmin (C(S ,a ) + γV(S )); go to the next decision point t t t τ ; t+1 a,n Step 2d. If τ ≥ τ , go to Step 3; else, update state S to S t+1 T t t+1 TA Step 3. If n = N, terminate.; else, for each time -zone ϕ , Step 3a. Sample 10 states with a probability of nn TA a,m TA a,m p = ∑ ∑ 1 / ∑ ∑ 1 for each state s ∈  s τ ∈ϕ m=1 {s=S } s,τ ∈ϕ m=1 {s=S } t i t t i t Step 3b. Perform monotonicity projection operator on all sampled reference state S : r r V Π (s ,V(s ),V) Step 4. Increase n by 1 and return to Step 1 Figure 4: Details of the mini-batch monotone-ADP algorithm. 0 6 12 18 24 Time (h) Figure 6: Average arrival time of all emergency calls received in one day. Waiting location Hospital Figure 5: Locations of emergency centers and hospitals in Songpa- gu, Seoul. 0 6 12 18 24 Time (h) was also assumed to follow a gamma distribution with a scale Figure 7: Average vehicle speeds over a day. parameter θ � 5.02 and a shape parameter k � 3.0 with an average of 15.05 min, as inferred by Maxwell et al. [15]. *e step size of the proposed ADP algorithm was set to (see the studies by Jiang and Powell [34] and Ryzhov et al. n,t n δ � 1/ 1 a,m , which is the reciprocal of the [35] for details). *e discount factor c was set to 0.9, which is 􏽐 􏽐 s t m�1 s�S { } number of visiting states S from the beginning to time τ at future-oriented and showed the best performance in simple iteration n. *is step size almost deﬁnitely assures conver- tests. *e algorithm was implemented in Python, and all gence as the number of iterations increases with a well- experiments were run on a computer with an i5-4460 CPU. known result in stochastic approximation because it satisﬁes We varied the following three factors to determine their ∞ n,t ∞ n,t 2 􏽐 δ �∞ and 􏽐 (δ ) <∞, with some regularity inﬂuence on the RLI: the ratio of ALS to the total number of n�0 s n�0 s assumptions regarding the underlying stochastic processes ambulances (hereafter the ALS ratio), the undertriage rate α, Avg. speed of vehicle (km/h) Avg. number of calls received 8 Journal of Healthcare Engineering and the overtriage rate β. In this experiment, the ALS ratios paired t-test for the ADP and greedy policies, the former with respect to the six ambulances were 0.0, 0.17, 0.33, 0.5, performed signiﬁcantly better in 97 of 100 combinations at the 95% conﬁdence level. In most experiments, the p value 0.67, 0.83, and 1.0. Each error α and β was divided into ﬁve increments of 0.1, beginning at 0. A complete factorial ex- was less than 0.001, indicating that the dominant perfor- periment was performed for each combination of factors. mance was very signiﬁcant. Table 3 shows the diﬀerence in *e learning phase of the proposed ADP algorithm, RLI between the ADP and the greedy policy based on the which approximates the optimal value function, was ter- ALS ratio, which had the greatest eﬀect on patient risk level. minated based on a two-h limit instead of the number of Overall, the patient RLI decreased by 0.486 when using the iterations. We drew each point in Figure 8 to represent the ADP policy, which was an improvement of 11.2% over the average value of the RLI for 100 iterations. As Figure 8 greedy policy. shows, the RLI gradually decreased and converged after an average of 5387.7 iterations. *e policy optimized by the 4.3. Factors Aﬀecting the Risk Level Index. *e results of proposed ADP algorithm (hereafter the ADP policy) was multiway ANOVA tests on the RLI in the ADP policy are tested 100 times for each experiment. Each iteration of the shown in Table 4 and Figure 10. *e ANOVA was conducted learning phase and each test of optimized policy were run for using SAS software. As expected, the RLI decreased with seven simulation days after a warm-up time of one simu- increasing ALS ratio and decreasing undertriage rate α or lation day, which is suﬃcient time to eliminate the inﬂuence overtriage rate β. *e main eﬀects on error α, error β, and of an arbitrary initial position of the ambulances. ALS ratio were signiﬁcant, as were the interaction eﬀects of the α × ALS ratio and β × ALS ratio, with a signiﬁcance level of 0.01 and a p value of less than 0.001. *e interaction eﬀect 4.2. Comparison with Greedy Policy. As mentioned in Sec- of α × β was signiﬁcant with a p value of less than 0.05, but tion 3, the greedy policy moves the ambulance in a way that the magnitude of the eﬀect was negligible; therefore, a de- minimizes C(S , X (S )); thus, it only considers the con- t t tailed analysis was not conducted. *e interaction eﬀect of tribution at the current state and does not consider the α × β × ALS ratio was not signiﬁcant. *e ALS ratio had the eﬀects of the current action on future situations. However, greatest impact on the RLI, followed by α, the α × ALS ratio because it still considers the patient severity classiﬁcation interaction, the β × ALS ratio interaction, and β. Figure 10 errors, ambulance type, and the expected response time of shows that each factor has a nonlinear eﬀect on RLI. the current state, it is a basic and reasonable policy that is *e main eﬀects of the diﬀerent factors are summarized expected to perform at least better than a myopic policy that in Figure 11 using the averages of the experimental values for allocates the nearest ambulance to the patient and relocates all levels of the factors. *e eﬀect of undertriage rate was the former to the nearest available waiting location. distinctly nonlinear; RLI increased rapidly as α increased It is diﬃcult to compare the various policies with a small from 0 to 0.1. Moreover, when error α increased, there was number of simulation experiments because the RLI has high an increased frequency of assigning a BLS to misclassiﬁed variability due to the inherent uncertain nature of patient actual high-risk patients, leading to a negative impact on the numbers in the EMS system. *erefore, we used the common patient’s risk level. Conversely, the RLI increased linearly random number (CRN), a variance reduction technique, to with increasing β; however, this eﬀect was not large because eﬃciently compare alternative policies with a small number of assigning ALS to a misclassiﬁed actual low-risk patient does simulations. *is was made possible because the CRN method not immediately and directly increase that patient’s risk synchronizes a random number stream for some variables to level, but rather indirectly aﬀects the ability of future high- generate the same random number in every alternative policy risk patients to cope. Another reason for the small eﬀect is when running the simulation. In this study, we compared the that the absolute number of high-risk patients is relatively greedy policy and the ADP policy under the condition that the small. As the ALS ratio decreased, the RLI increased more random number streams of the patients’ occurrence times, rapidly. Figures 12 and 13 show the interaction eﬀect be- locations, and actual and classiﬁed severities were synchronized. tween the undertriage rate α or overtriage rate β and ALS To ﬁnd the minimum number of ALS vehicles capable of ratio on the RLI. As the ALS ratio increased, the RLI was less eﬀectively transporting high-risk patients, the average RLI aﬀected by both classiﬁcation errors; however, when the ALS according to the ALS ratio was analyzed as shown in Fig- ratio was relatively low, α generated a greater diﬀerence in ure 9. Figure 9 shows that the RLI increased sharply when RLI than β. the ALS ratio decreased to less than 0.5. *is was a result of the frequent assignment of BLS to high-risk patients because there were insuﬃcient ALSs to treat them. In a further 4.4. Operational Properties of the Improved Ambulance Op- experiment that restricted BLS from transporting high-risk eration Policy. Although the ADP policy performs better patients, these patients continued to accumulate in the queue than the greedy policy, understanding how ambulance if the ALS ratio was less than 0.5. *is indicates that the ALS operations based on the ADP policy diﬀer from those of the in the EMS system had insuﬃcient capacity; therefore, greedy policy is complex. *us, to gain a greater un- derstanding of operational properties and more general and subsequent analyses of the experimental results will only evaluate situations in which the ALS ratio is above 0.5. intuitive insights into decision-making in the proposed optimized ambulance operation policy, we developed and Table 2 shows the results of the RLI of the two policies for each of the four ALS ratios and ﬁve levels of α and β. In the analyzed additional indices other than RLI. Journal of Healthcare Engineering 9 3.9 3.5 3.4 3.8 3.3 3.7 3.2 3.6 3.1 3.5 3.0 3.4 2.9 2.8 3.3 Iteration Iteration (a) (b) Figure 8: Risk level index for iterations of the learning phase: (a) α � β � 0, ALS Ratio � 0.83 and (b) α � β � 0.4, ALS Ratio � 0.83. average, 30% of patients classiﬁed as low risk were assigned an ambulance other than the nearest ambulance. *e second index measured was the present orientation for patients classiﬁed as low-risk index (PLI), which refers to the ratio achieved when the ALS nearest to a low-risk patient is allocated when the nearest ambulance is that speciﬁc ALS and other idle ambulances are present. As the PLI value increased when a low-risk patient was assigned to the nearest ALS, a larger PLI value can be considered a more short- sighted dispatch approach, whereas a smaller PLI value is a more forward-looking dispatch approach. As a result, the PLI was minimally aﬀected by the overtriage rate β (Table 6) but increased with increasing undertriage rate α. *us, when the undertriage rate α was low, a patient classiﬁed as low risk was relatively frequently allocated an ambulance that is farther away, even if there was a closer ALS, in order to 0.00 0.17 0.33 0.50 0.67 0.83 1.00 prepare for potential high-risk patients in future. On the ALS ratio contrary, when the undertriage rate α was high, the nearest ALS was frequently assigned to a patient even if they were Greedy ADP classiﬁed as low risk. Furthermore, PLI exhibited nonlinear characteristics with undertriage rate α. When α increased Figure 9: Comparison of the risk level index for greedy and ADP from 0, the PLI increased considerably; however, when α patient transport policies based on the ALS (advanced life support) exceeded 0.2, the increase in PLI was reduced. ratio. Transporting a low-risk patient via ALS instead of BLS is a relatively ineﬃcient way of using ambulance resources *e ﬁrst of these indices, the future orientation for because it is an oversupply of the medical service. *us, patients classiﬁed as high-risk index (FHI), refers to the ratio the third index measured was the ineﬃciency of the ALS achieved when the nearest ALS is not allocated, or the index (IAI), which refers to the ratio achieved by allo- cating an ALS to a patient classiﬁed as low risk. Table 7 nearest ambulance is not allocated to a patient classiﬁed as high risk when other idle ALSs are present. *e FHI was shows the IAI for error α and error β, which is the average value of all experiments except for an ALS ratio of 1.0. IAI close to 0 in almost all situations (Table 5). As the availability of an ALS increased as the ALS ratio increased, the FHI increased as α increased but was not signiﬁcantly aﬀected by β. In other words, as the undertriage rate increased, the increased slightly from 0.05 to 0.09, which was slightly fu- ture-oriented but still very low. *is means that almost all ineﬃcient use of ALS vehicles increased as more ALSs patients classiﬁed as high risk, regardless of the magnitude of were assigned to patients classiﬁed as low risk. IAI also the error and the ALS ratio, were assigned the nearest ALS or increased considerably and nonlinearly with α, similar to a BLS if it was closer. In other words, ambulances tried to PLI. However, if the undertriage rate exceeded 0.2, the respond as quickly as possible to patients classiﬁed as high increase in IAI began to decrease. Finally, the average time risk in any situation. On the contrary, dispatching ambu- required to relocate an ambulance was 2.61 min for the greedy policy and 3.84 min for the ADP policy. *is in- lances to patients classiﬁed as low risk was less aﬀected by the distance between the patient and the ambulance. On dicates that although the greedy policy tried to relocate Risk level index Risk level index Risk level index 4600 10 Journal of Healthcare Engineering Table 2: Risk level index of each ambulance operation policy, ALS ratio, undertriage rate α, and overtriage rate β. Overtriage rate β ALS ratio Undertriage rate α 0 0.1 0.2 0.3 0.4 Greedy ADP Greedy ADP Greedy ADP Greedy ADP Greedy ADP 0 5.12 3.99 5.26 4.21 5.48 4.32 5.57 4.61 5.68 4.89 ∗ ∗∗ 0.1 5.58 5.11 5.81 5.31 5.91 5.60 6.12 5.93 6.19 6.12 ∗ ∗ ∗∗ ∗∗ 0.5 0.2 5.98 5.75 6.34 5.97 6.30 6.12 6.49 6.45 6.62 6.67 ∗ ∗ 0.3 6.50 6.26 6.74 6.38 6.73 6.47 6.97 6.69 7.10 6.79 0.4 7.00 6.45 7.19 6.49 7.41 6.68 7.46 6.78 7.52 6.90 0 4.07 3.43 4.13 3.45 4.18 3.52 4.20 3.59 4.30 3.70 0.1 4.47 3.99 4.47 4.14 4.51 4.23 4.60 4.32 4.61 4.46 0.67 0.2 4.78 4.38 4.77 4.47 4.89 4.40 4.90 4.52 4.93 4.68 0.3 5.12 4.45 5.06 4.60 5.24 4.72 5.16 4.65 5.32 4.75 0.4 5.43 4.55 5.49 4.75 5.48 4.70 5.69 4.69 5.59 4.84 0 3.47 2.99 3.47 3.03 3.57 3.09 3.55 3.12 3.58 3.09 0.1 3.66 3.31 3.78 3.29 3.77 3.41 3.77 3.38 3.79 3.43 0.83 0.2 3.90 3.39 3.88 3.40 3.90 3.45 3.89 3.43 3.95 3.47 0.3 4.03 3.43 4.08 3.47 4.04 3.47 4.08 3.56 4.08 3.46 0.4 4.17 3.46 4.23 3.45 4.17 3.54 4.30 3.46 4.36 3.49 0 3.18 2.71 3.17 2.72 3.19 2.8 3.15 2.76 3.16 2.73 0.1 3.18 2.74 3.20 2.79 3.17 2.8 3.17 2.77 3.18 2.80 1 0.2 3.18 2.74 3.18 2.76 3.17 2.83 3.17 2.77 3.17 2.83 0.3 3.18 2.78 3.17 2.80 3.17 2.80 3.16 2.78 3.17 2.78 0.4 3.14 2.81 3.16 2.80 3.16 2.85 3.17 2.82 3.16 2.79 ∗ ∗∗ Note. p is less than 0.001 in all experiments except p< 0.05; not signiﬁcant. Table 3: Diﬀerence in the risk level index between the ADP and greedy policies. Risk level index (ADP-greedy) ALS ratio Average Maximum Minimum 0.5 − 0.485 (8.0%) − 1.160 (22.0%) 0.000 (0.0%) 0.67 − 0.535 (11.0%) − 0.996 (17.5%) − 0.156 (3.4%) 0.83 − 0.536 (13.6%) − 0.872 (20.0%) − 0.346 (9.5%) 1 − 0.389 (12.3%) − 0.467 (14.7%) − 0.314 (9.9%) Average − 0.486 (11.2%) Table 4: ANOVA results of the risk level index. Dependent variable: risk level index (RLI) Source DF Sum of squares Mean square F value Pr> F Model 99 16158.86 163.22 402.25 <.0001 Error 9900 4017.07 0.41 Corrected total 9999 20175.93 R-square Coeﬀ var Root MSE RLI mean 0.801 15.592 0.637 4.085 Source DF Anova SS Mean square F value Pr> F ALS ratio 3 13722.50 4574.17 11272.90 <.0001 α 4 1257.75 314.44 774.93 <.0001 β 4 107.24 26.81 66.07 <.0001 ALS ratio × α 12 933.14 77.76 191.64 <.0001 ALS ratio × β 12 112.73 9.39 23.15 <.0001 α × β 16 10.95 0.68 1.69 0.042 ALS ratio × α × β 48 14.56 0.30 0.75 0.9013 ambulances to make them idle as quickly as possible, the 5. Discussion ADP policy tried to relocate ambulances to positions in One of the major diﬃculties of an EMS system that which they could better respond to future patients, which led to improved performance. transports patients by emergency ambulances is that they Error α Journal of Healthcare Engineering 11 1.0 6.5 6.0 0.9 5.5 0.8 5.0 0.7 4.5 0.6 4.0 0.5 3.5 0.4 0.3 3.0 0.0 0.1 0.2 0.2 0.1 0.3 0.0 0.4 Figure 10: Risk level index plot for the ALS ratio, error α, and error β. 4.40 4.50 4.20 4.10 3.90 3.60 3.30 3.80 0 0.1 0.2 0.3 0.4 0 0.1 0.2 0.3 0.4 Error β Error α (a) (b) 6.00 5.70 5.40 5.10 4.80 4.50 4.20 3.90 3.60 3.30 3.00 2.70 2.40 0.50 0.67 0.83 1 ALS ratio (c) Figure 11: Eﬀect of (a) error α, (b) error β, and (c) ALS ratio on the patient risk level index. have to respond as quickly as possible despite limited am- patients were categorized into two groups: high risk bulance resources. As not all patients are actual emergency (emergency) and low risk (nonemergency), where the ma- patients, it is clear that using a mixed ALS/BLS system based jority fall into the latter category. A mixed ALS/BLS (two- tiered ambulance) system in which ALS and BLS vehicles are on the severity of the patient’s condition is a more eﬃcient management strategy that will enable ambulances to re- suitable for transporting high-risk and low-risk patients, spond to patients more rapidly. However, a key limitation of respectively, was also considered. Two types of classiﬁcation mixed ALS/BLS systems is the high risk of errors when errors were assumed. *e undertriage rate α was the classifying the severity of the patient’s conditions. probability of false classiﬁcations of actual high-risk patients, *erefore, we developed an ADP model to optimize the and the overtriage rate β was the probability of false clas- ambulance dispatch and redeployment policy whilst in- siﬁcations of actual low-risk patients. To develop a realistic cluding patient severity classiﬁcation errors, which has not model, system dynamics such as the time-varying traﬃc and been suﬃciently addressed by previous research. *e frequency of patient occurrence and ambulance service time Error β Risk level index Risk level index Risk level index ALS ratio Risk level index 12 Journal of Healthcare Engineering 7.5 7.5 6.5 6.5 5.5 5.5 4.5 4.5 3.5 3.5 2.5 2.5 0 0.1 0.2 0.3 0.4 0.5 0.67 0.83 1 Error α ALS ratio AR = 0.5 AR = 0.83 α = 0.0 α = 0.3 AR = 0.67 AR = 1.0 α = 0.1 α = 0.4 α = 0.2 (a) (b) Figure 12: Interaction eﬀect of ALS ratio and error α on the risk level index. 7.0 7.0 6.5 6.5 6.0 6.0 5.5 5.5 5.0 5.0 4.5 4.5 4.0 4.0 3.5 3.5 3.0 3.0 2.5 2.5 0 0.1 0.2 0.3 0.4 0.5 0.67 0.83 1 Error β ALS ratio AR = 0.5 AR = 0.83 β = 0.0 β = 0.3 AR = 0.67 AR = 1.0 β = 0.1 β = 0.4 β = 0.2 (a) (b) Figure 13: Interaction eﬀect of ALS ratio and error β on the risk level index. Table 5: Future orientation for patients classiﬁed as high-risk index Table 7: Ineﬃciency of the ALS index (IAI) for classiﬁcation errors. (FHI) according to the ALS ratio. Undertriage rate α 0 0.1 0.2 0.3 0.4 ALS ratio 0.50 0.67 0.83 1 IAI 0.40 0.70 0.82 0.86 0.88 FHI 0.05 0.07 0.08 0.09 Overtriage rate β 0 0.1 0.2 0.3 0.4 IAI 0.73 0.73 0.73 0.74 0.74 Table 6: Present orientation for patients classiﬁed as low-risk index were based on historical data. As a result, the proposed ADP (PLI) for classiﬁcation errors. model reduced the risk level index (RLI) for all patients by an Undertriage rate α 0 0.1 0.2 0.3 0.4 average of 11.2% compared to the greedy policy. PLI 0.50 0.72 0.80 0.83 0.84 We also analyzed the magnitude and correlation of the Overtriage rate β 0 0.1 0.2 0.3 0.4 eﬀects of α, β, and the ALS ratio on the patient RLI under PLI 0.74 0.74 0.74 0.74 0.74 optimized ambulance dispatch and relocation policies. *e Risk level index Risk level index Risk level index Risk level index Journal of Healthcare Engineering 13 patient RLI decreases when the ALS ratio increases or either Conflicts of Interest classiﬁcation error decreases. ALS ratio has the greatest *e authors declare that they have no conﬂicts of interest. impact on RLI, followed by α, α × ALS ratio interaction, β × ALS ratio interaction, and β. *e interaction eﬀects show that the patient RLI is less aﬀected by changes in both Acknowledgments classiﬁcation errors as ALS ratio increases. Furthermore, a key observation is that α is much more sensitive than β in *is work was supported by the National Research Foun- terms of the patient RLI. *erefore, it is desirable to classify dation of Korea (NRF) grant funded by the Korea gov- patient severity in order to minimize the undertriage rate, ernment (MSIT) (2017R1E1A1A03070757). even though it may increase the overtriage rate. For ex- ample, a patient whose condition is unclear or ambiguous References and cannot be classiﬁed accurately would be classiﬁed as high risk. Furthermore, we evaluated the characteristics of [1] Y. Choi, Need to Find Measures to Reduce Non-emergency or the optimized ambulance operation policy. Patients clas- Habitual 911 Users, Minan Newspaper, 2016, http://www. siﬁed as high risk were almost always assigned the nearest minan21c.com/ezview/article_main.html?no�9319. ALS regardless of the error level or ALS ratio. However, [2] S. Lee, Do not use 911 Ambulances for Non-Emergency Pa- patients classiﬁed as low risk were more likely to be al- tients Anymore, Yangcheon Newspaper, 2016, http://ycnews. kr/sub_read.html?uid�7891&section�sc8&section2�%B1% located the nearest ALS as the undertriage rate increased. E2%B0%ED. Moreover, the manner in which ambulances operated was [3] Ministry of Health and Welfare Republic of Korea, Guidelines not signiﬁcantly aﬀected by the overtriage rate. *ese for Registration and Management of Ambulance, Ministry of ﬁndings could serve as useful guidelines for optimizing Health and Welfare Republic of Korea, Seoul, South Korea, ambulance operations when patient severity classiﬁcation 3rd edition, 2015. errors exist. Although the experimental environment was [4] J. P. Ornato, E. M. Racht, J. J. Fitch, and J. F. Berry, “*e need limited to Seoul, we expect that these results would not be for ALS in urban and suburban EMS systems,” Annals of signiﬁcantly diﬀerent in other regions with characteristics Emergency Medicine, vol. 19, no. 12, pp. 1469-1470, 1990. similar to Seoul, e.g., urban areas with a similar density of [5] B. Wilson, M. C. Gratton, J. Overton, and W. A. Watson, ambulance, base, and demand. However, in order to ﬁnd “Unexpected ALS procedures on non-emergency ambulance out the impact of speciﬁc regional characteristics on the calls: the value of a single-tier system,” Prehospital and Di- saster Medicine, vol. 7, no. 4, pp. 380–382, 1992. ADP policy, further research is required, such as learning a [6] K. C. Chong, S. G. Henderson, and M. E. Lewis, “*e vehicle new policy in the area with diﬀerent characteristics, e.g., mix decision in emergency medical service systems,” rural areas with fewer ambulances and demands, or ap- Manufacturing & Service Operations Management, vol. 18, plying transfer learning using a policy that has completed no. 3, pp. 347–360, 2016. learning in a similar area. [7] O. Braun, R. McCallion, and J. Fazackerley, “Characteristics of *e main goal of this study was to develop an algorithm midsized urban EMS systems,” Annals of Emergency Medi- to determine the optimal ambulance operation policy using cine, vol. 19, no. 5, pp. 536–546, 1990. a realistic model that includes patient severity classiﬁcation [8] C. M. Slovis, T. B. Carruth, W. J. Seitz, C. M. *omas, and errors and then provide insights into classifying patient W. R. Elsea, “A priority dispatch system for emergency severity and ambulance operational strategy by identifying medical services,” Annals of Emergency Medicine, vol. 14, no. 11, pp. 1055–1060, 1985. the eﬀects of several factors and useful indices under the [9] J. Stout, P. E. Pepe, and V. N. Mosesso, “All advanced life optimized policy. *erefore, the goal was not to demonstrate support vs tiered response a mbulance systems,” Prehospital the superiority of an all-ALS system versus a mixed-ALS/ Emergency Care, vol. 4, no. 1, pp. 1–6, 2000. BLS system or to increase the accuracy of patient classiﬁ- [10] L. Brotcorne, G. Laporte, and F. Semet, “Ambulance location cation. Although the total number of ambulances is assumed and relocation models,” European Journal of Operational to be constant in this study, the number of ambulances Research, vol. 147, no. 3, pp. 451–463, 2003. available under one budget can vary due to diﬀerences in [11] C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei, “Dynamic operating and purchasing costs between ALS and BLS. In- ambulance dispatching: is the closest-idle policy always op- creasing the total number of ambulances with a high ratio of timal?,” Health Care Management Science, vol. 20, pp. 1–15, BLS may contribute positively to reducing the patient risk level. In addition, if it was possible to lower the undertriage [12] D. Degel, L. Wiesche, S. Rachuba, and B. Werners, “Time- dependent ambulance allocation considering data-driven and overtriage rates, EMS system managers could consider empirically required coverage,” Health Care Management controlling these errors and the conﬁguration of ambulances Science, vol. 18, no. 4, pp. 444–458, 2015. to enable eﬀective decision-making that could minimize the [13] M. S. Maxwell, M. Restrepo, S. G. Henderson, and patient risk level within a limited budget. *e ﬁndings of this H. Topaloglu, “Approximate dynamic programming for study could be useful for such future research. ambulance redeployment,” INFORMS Journal on Computing, vol. 22, no. 2, pp. 266–281, 2010. Data Availability [14] A. A. Nasrollahzadeh, A. Khademi, and M. E. Mayorga, “Real- time ambulance dispatching and relocation,” Manufacturing *e data used to support the ﬁndings of this study are & Service Operations Management, vol. 20, no. 3, pp. 389–600, available from the corresponding author upon request. 2018. 14 Journal of Healthcare Engineering [15] M. S. Maxwell, S. G. Henderson, and H. Topaloglu, “Tuning [32] S. Vandeventer, J. R. Studnek, J. S. Garrett, S. R. Ward, approximate dynamic programming policies for ambulance K. Staley, and T. Blackwell, “*e association between am- bulance hospital turnaround times and patient Acuity, des- redeployment via direct search,” Stochastic Systems, vol. 3, tination hospital, and time of day,” Prehospital Emergency no. 2, pp. 322–361, 2013. Care, vol. 15, no. 3, pp. 366–370, 2011. [16] V. Schmid, “Solving the dynamic ambulance relocation and [33] W. B. Powell, Approximate Dynamic Programming for Op- dispatching problem using approximate dynamic pro- erations Research: Solving the Curses of Dimensionality, John gramming,” European Journal of Operational Research, Wiley & Sons, New York, NY, USA, 2nd edition, 2011. vol. 219, no. 3, pp. 611–621, 2012. [34] D. R. Jiang and W. B. Powell, “An approximate dynamic [17] D. Bandara, M. E. Mayorga, and L. A. McLay, “Priority programming algorithm for monotone value functions,” dispatching strategies for EMS systems,” Journal of the Op- Operations Research, vol. 63, no. 6, pp. 1489–1511, 2015. erational Research Society, vol. 65, no. 4, pp. 572–587, 2013. [35] I. O. Ryzhov, P. I. Frazier, and W. B. Powell, “A new optimal [18] L. A. McLay, “A maximum expected covering location model stepsize for approximate dynamic programming,” IEEE with two types of servers,” IIE Transactions, vol. 41, no. 8, Transactions on Automatic Control, vol. 60, no. 3, pp. 743– pp. 730–741, 2009. 758, 2015. [19] L. A. McLay and M. E. Mayorga, “A model for optimally dispatching ambulances to emergency calls with classiﬁcation errors in patient priorities,” IIE Transactions, vol. 45, no. 1, pp. 1–24, 2013. [20] J. Lee, Advanced Life Support Ambulance Will be Expanded for Emergency Patients, Doctorsnews, 2016, http://www.doctorsnews. co.kr/news/articleView.html?idxno�1101226. [21] L. A. McLay and M. E. Mayorga, “A dispatching model for server-to-customer systems that balances eﬃciency and eq- uity,” Manufacturing & Service Operations Management, vol. 15, no. 2, pp. 205–220, 2013. [22] E. Erkut, A. Ingolfsson, and G. Erdogan, ˘ “Ambulance location for maximum survival,” Naval Research Logistics, vol. 55, no. 1, pp. 42–58, 2008. [23] N. Geroliminis, M. G. Karlaftis, and A. Skabardonis, “A spatial queuing model for the emergency vehicle districting and location problem,” Transportation Research Part B: Method- ological, vol. 43, no. 7, pp. 798–811, 2009. [24] S. Jin, S. Jeong, J. Kim, and K. Kim, “A logistics model for the transport of disaster victims with various injuries and survival probabilities,” Annals of Operations Research, vol. 230, pp. 1–17, 2014. [25] V. A. Knight, P. R. Harper, and L. Smith, “Ambulance al- location for maximal survival with heterogeneous outcome measures,” Omega, vol. 40, no. 6, pp. 918–926, 2012. [26] M. P. Larsen, M. S. Eisenberg, R. O. Cummins, and A. P. Hallstrom, “Predicting survival from out-of-hospital cardiac arrest: a graphic model,” Annals of Emergency Med- icine, vol. 22, no. 11, pp. 1652–1658, 1993. [27] L. A. McLay and M. E. Mayorga, “Evaluating emergency medical service performance measures,” Health Care Man- agement Science, vol. 13, no. 2, pp. 124–136, 2010. [28] T. Mizumoto, W. Sun, K. Yasumoto, and M. Ito, “Trans- portation scheduling method for patients in mci using elec- tronic triage tag,” in Proceedings of the Bird International Conference on eHealth, Telemedicine, and Social Medicine, eTELEMED 2011, Gosier, Guadeloupe, France, February 2011. [29] C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei, “An eﬃcient heuristic for real-time ambulance redeployment,” Operations Research for Health Care, vol. 4, pp. 27–35, 2015. [30] S. S. W. Lam, C. B. L. Ng, F. N. H. L. Nguyen, Y. Y. Ng, and M. E. H. Ong, “Simulation-based decision support framework for dynamic ambulance redeployment in Singapore,” In- ternational Journal of Medical Informatics, vol. 106, pp. 37–47, [31] N. T. Argon, L. Ding, K. D. Glazebrook, and S. Ziya, “Dy- namic routing of customers with general delay costs in a multiserver queuing system,” Probability in the Engineering and Informational Sciences, vol. 23, no. 2, pp. 175–203, 2009. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal

Journal of Healthcare Engineering – Hindawi Publishing Corporation

Published: Dec 9, 2019

References

The need for ALS in urban and suburban EMS systems,

J. P. Ornato, E. M. Racht, J. J. Fitch, and J. F. Berry
Unexpected ALS procedures on non-emergency ambulance calls: the value of a single-tier system,

B. Wilson, M. C. Gratton, J. Overton, and W. A. Watson
The vehicle mix decision in emergency medical service systems,

K. C. Chong, S. G. Henderson, and M. E. Lewis
Characteristics of midsized urban EMS systems,

O. Braun, R. McCallion, and J. Fazackerley
A priority dispatch system for emergency medical services,

C. M. Slovis, T. B. Carruth, W. J. Seitz, C. M. Thomas, and W. R. Elsea
All advanced life support vs tiered response a mbulance systems,

J. Stout, P. E. Pepe, and V. N. Mosesso
Ambulance location and relocation models,

L. Brotcorne, G. Laporte, and F. Semet
Dynamic ambulance dispatching: is the closest-idle policy always optimal?

C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei
Time-dependent ambulance allocation considering data-driven empirically required coverage,

D. Degel, L. Wiesche, S. Rachuba, and B. Werners
Approximate dynamic programming for ambulance redeployment,

M. S. Maxwell, M. Restrepo, S. G. Henderson, and H. Topaloglu
Real-time ambulance dispatching and relocation,

A. A. Nasrollahzadeh, A. Khademi, and M. E. Mayorga
Tuning approximate dynamic programming policies for ambulance redeployment via direct search,

M. S. Maxwell, S. G. Henderson, and H. Topaloglu
Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming,

V. Schmid
Priority dispatching strategies for EMS systems,

D. Bandara, M. E. Mayorga, and L. A. McLay
A maximum expected covering location model with two types of servers,

L. A. McLay
A model for optimally dispatching ambulances to emergency calls with classification errors in patient priorities,

L. A. McLay and M. E. Mayorga
A dispatching model for server-to-customer systems that balances efficiency and equity,

L. A. McLay and M. E. Mayorga
Ambulance location for maximum survival,

E. Erkut, A. Ingolfsson, and G. Erdoğan
A spatial queuing model for the emergency vehicle districting and location problem,

N. Geroliminis, M. G. Karlaftis, and A. Skabardonis
A logistics model for the transport of disaster victims with various injuries and survival probabilities,

S. Jin, S. Jeong, J. Kim, and K. Kim
Ambulance allocation for maximal survival with heterogeneous outcome measures,

V. A. Knight, P. R. Harper, and L. Smith
Predicting survival from out-of-hospital cardiac arrest: a graphic model,

M. P. Larsen, M. S. Eisenberg, R. O. Cummins, and A. P. Hallstrom
Evaluating emergency medical service performance measures,

L. A. McLay and M. E. Mayorga
Transportation scheduling method for patients in mci using electronic triage tag,

T. Mizumoto, W. Sun, K. Yasumoto, and M. Ito
An efficient heuristic for real-time ambulance redeployment,

C. J. Jagtenberg, S. Bhulai, and R. D. van der Mei
Simulation-based decision support framework for dynamic ambulance redeployment in Singapore,

S. S. W. Lam, C. B. L. Ng, F. N. H. L. Nguyen, Y. Y. Ng, and M. E. H. Ong
Dynamic routing of customers with general delay costs in a multiserver queuing system,

N. T. Argon, L. Ding, K. D. Glazebrook, and S. Ziya
The association between ambulance hospital turnaround times and patient Acuity, destination hospital, and time of day,

S. Vandeventer, J. R. Studnek, J. S. Garrett, S. R. Ward, K. Staley, and T. Blackwell
An approximate dynamic programming algorithm for monotone value functions,

D. R. Jiang and W. B. Powell
A new optimal stepsize for approximate dynamic programming,

I. O. Ryzhov, P. I. Frazier, and W. B. Powell

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors

References

Abstract

Journal

Recommended Articles

References

Our policy towards the use of cookies