Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Approaches to Medical Decision-Making Based on Big Clinical Data

Approaches to Medical Decision-Making Based on Big Clinical Data Hindawi Journal of Healthcare Engineering Volume 2018, Article ID 3917659, 10 pages https://doi.org/10.1155/2018/3917659 Research Article Approaches to Medical Decision-Making Based on Big Clinical Data V. L. Malykh and S. V. Rudetskiy Medical Informatics Research Center, Ailamazyan Program Systems Institute of RAS, Pereslavl-Zalessky, Russia Correspondence should be addressed to V. L. Malykh; mvl@interin.ru Received 21 September 2017; Revised 14 February 2018; Accepted 30 April 2018; Published 4 June 2018 Academic Editor: Giedrius Vanagas Copyright © 2018 V. L. Malykh and S. V. Rudetskiy. +is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. +e paper discusses different approaches to building a medical decision support system based on big data. +e authors sought to abstain from any data reduction and apply universal teaching and big data processing methods independent of disease clas- sification standards. +e paper assesses and compares the accuracy of recommendations among three options: case-based reasoning, simple single-layer neural network, and probabilistic neural network. Further, the paper substantiates the assumption regarding the most efficient approach to solving the specified problem. this increasing subjectivity of MISs, we have proposed a new 1. Introduction term “active MIS” that emphasizes a certain degree of in- Providing support to medical decision-making is one of the dependence from users or subjectivity of the cyber system. most urgent issues in healthcare automation. It has been Kohane [4] presents the most “balanced” definition of repeatedly noted in different articles, reports, and forum personalized medicine, “personalized medicine is the discussions [1] both in Russia and abroad that MIS in- practice of clinical decision-making such that the decisions troduction requires a considerable extra effort from made maximize the outcomes that the patient most cares users—doctors in the first place—to enter primary data into about and minimize those that the patient fears the most, on the basis of as much knowledge about the individual’s state the system. Naturally, doctors expect practical intelligent outcomes from big clinical data accumulated by modern as is available.” +is perception of personal medicine is MISs. Handler et al. [2] present the operating paradigm of focused on clinical decision-making and once again exhibits 5th generation MISs, referred to as “MIS as Mentor.” Malykh the urgency and importance of scientific research in the area. et al. [3] adds one more qualitative characteristic to the +erefore, building an automated active mentor-type system above paradigm—“MIS as automated mentor.” “It is ad- that provides recommendations regarding treatment and visable to abandon the practice of active user dialogs typical diagnostic activities to the doctor is an urgent practical task. of expert systems, involving requests for data that the system Butko and Olshansky [5] and Kotov [6] provide a retro- considers missing from the user, and substitute the dialog spective overview of approaches to building medical decision with an automated nonintrusive algorithm that draws its support systems. +e applied approaches were restricted in own logical conclusions and generates recommendations in many respects by the abilities of computers at that time. Ac- cordingly, there was no such problem as processing big medical a completely automated manner based on available data, without involving the user in the process. +e user may data. Technologies have evolved to the point when big medical either accept or ignore the system’s prompts and recom- data (both on individuals and the population in general) mendations; however, they will not provoke rejection in collection and accumulation is finally feasible. At the same users if delivered automatically without requiring a dialog time, big data processing and intelligent system learning with the system.” To provide a brief qualitative description of methods were evolving as well. Along with “deep learning,” the 2 Journal of Healthcare Engineering systems. +is approach helps achieve top quality when term “deep patient” [7] was coined, meaning the opportunity to extract increasingly more complete, deep, and valuable in- solving isolated problems [6, 12]; however, it is almost impossible to apply it to big clinical data. formation about patients from big clinical data using deep learning methods. +e fourth approach that claims to have a global scope of Malykh et al. [8] mention the possibility of creating application is focused on building a cognitive system capable national-scale clinical data banks. Herrett et al. [9] provide of self-learning and knowledge digestion directly from an example of a database (DB) containing anonymous nonformalized text sources (IBM Watson http://www.ibm. medical records on primary healthcare services provided. com/smarterplanet/us/en/ibmwatson/). +is DB was created by a joint effort of 674 general prac- None of the reviewed approaches is immaculate. All of them require efforts of experts and regular updates of titioners and covers over 11.3 mn patients in Great Britain. Decision-making in hospitals has evolved from being knowledge bases. Moreover, each of the approaches is in fact tailored to specific purposes. opinion-based to being based on sound scientific evidence. +is decision-making is recognized as evidence-based +e latest Russian-language review [12] noted that clinical decision support systems have not become wide- practice. Perpetual publication of new evidence combined with the demands of everyday practice makes it difficult for spread in Russia. +is is due to the complexity of the de- health professionals to keep up to date [10]. velopment of such systems, the specific character of the A large number of publications are devoted to medical systems already developed, and the need to involve high- decision support systems (DSSs), including publications in class experts in the development. specialized scientific journals (Artificial Intelligence in In this paper, we will review general approaches to Medicine, BMC Medical Informatics and Decision Making, decision support system development based on nonreduced big clinical data. +e main expectations related to applica- International Journal of Medical Informatics, Medical De- cision Making, etc.). +e work does not aim to give an tion of general approaches ensue from the case-based nature of decision-making in healthcare, and the assumption that overview of different approaches to making of decision support systems, referring readers to the original reviews big clinical data already contain enough knowledge for ef- ficient decision-making. [11–13]. We can give a few definitions for decision support system from Wikipedia: “Clinical Decision Support systems +ere are two other factors that draw attention to sys- link health observations with health knowledge to influence tems based on machine learning or precedent approach. health choices by clinicians for improved health care” and First of them is that there are trends in the development “active knowledge systems, which use two or more items of of our civilization which include an explosive development patient data to generate case-specific advice.” No one doubts of information technologies (among them M2M, Big Data, the feasibility of such systems and that they have a positive and IoT), their strong need for formalized knowledge, and practical absence of qualified experts who could formalize impact on professional practice, patient outcomes, length of hospital stay, and hospital costs. +e main problem is to find that knowledge. +e chief editor of the Rational Enterprise Management (REM) magazine (Russia) holds regular dis- effective approaches to building such systems. A number of contemporary approaches to medical decision cussions on a wide range of problems including the above- support system development are listed by Malykh et al. [14]. mentioned ones. Results of the discussions are published in +e first one of these approaches involves provision of the REM editor’s column. +e guests of a recent discussion relevant data sources to doctors, helping them make decisions [16] included Igor Rudym (Intel), Dmitriy Tameev (PTC), independently. +e system does not recommend any final Alexander Belotserkovskiy (Microsoft), Igor Girkin (Cisco), solutions—instead, it suggests data sources to study and find and Igor Kulinitchev (IBM). All the participants agreed that, answers to current questions (Evidence-Based Clinical De- nowadays, the key challenge of IT development is not as- cision Support at the Point of Care | UpToDate URL: http:// sociated with hardware or software, but it needs break- through approaches to data analysis. www.uptodate.com/home). +e second approach is to use clinical pathways. Clinical As for the second factor, it is obvious that, nowadays, there are no qualified experts in the field of knowledge even pathways represent prescriptive models of the standard healthcare procedures that need to be undertaken for in key branches. +e actual situation is even more critical as a specific patient population. Instances of the clinical the experts who are able to solve at least a part of these pathways (also known as cases) describe the actual problems are not able to cope with ever increasing in- diagnostic-therapeutic cycle of an individual patient [15]. formation flow. From this point of view, precedent-based But even in the case of the use of clinical pathways, the DSSs practically need no experts. Experts may be needed for process of clinical decision-making has high complexity. enhancing or optimizing existing medical data bases and knowledge bases [14]. While the medical knowledge used in the decision process comes partially from published research contributions and widespread medical guidelines (with various kinds of evi- 2. Model and Methods dence levels), it is generally accepted that the decision process is profoundly influenced by the expertise and ex- We regard the diagnostic and treatment process (DTP) as periences of the involved medical experts [15]. a discrete controlled process with a memory. +e model was +e third approach involves development of a large first introduced by Malykh et al. [17, 18] written in Russian. number of individual narrow-focused decision support In English, the model is described by Malykh et al. [8, 19]. To Journal of Healthcare Engineering 3 ensure further understanding of the essence of the problem, be unknown to us. When applying different methods to the let us provide an extract from the source. model, we may need to digitize non-numerical values of components and identify missing values of monitored Modern medical information systems store electronic medical records and contain descriptions of millions of properties. various clinical cases. +e degree of formalization of clinical data stored in MISs varies. MISs model the diagnostic and treatment process as a sequence of controlling events 2.1. Definition of the Objective. We will review several methods that can be applied to build a cybernetic taught reflecting diagnostic and treatment activities, and a sequence of monitoring events describing the condition of the patient. system. +e input into the system will be a sequence of vectors describing a discrete DTP in accordance with the Controlling events are well formalized; medical organiza- tions keep statistical and business records of such events, presented model. +e output will consist of recommenda- tions proposing diagnostic and treatment options (choice of plan them, and allocate required resources. Medical data controls) for this particular state of the process. A diagram of related to monitoring of patients’ condition are less for- the system is presented in Figure 1. malized and may be partly available in the form of plain text Let us define the objective more accurately and assume medical documents. Previous studies provide evidence that is possible to that each DTP model is considered in the context of an already available predominant diagnosis. For each model, we model the DTP using controlled stochastic Markov processes [18]. +e model is based on the assumption that the DTP is have an array of earlier observed DTP implementations. Such implementations are sources of knowledge about a discrete controlled process. +e model introduces the no- tions of control U and state X. Controls are diagnostic and treatment of a particular nosology, and they are used to teach a cybernetic recommender system to operate in the given treatment decisions made and executed in future. Controls are different diagnostic and treatment activities prescribed by context. Based on available DTP implementations, we de- fined a glossary of controls and monitored properties for doctors, including diagnostic tests, medicines, surgical in- each model. Issues related to normalization of primary data, terventions, various procedures, and manipulations. +e outlier testing and exclusion, and approaches to data gen- choice of diagnostic and treatment activities is based on the eralization based on assignment of monitored properties to accumulated medical knowledge and the doctor’s individual experience. +e scope of potential diagnostic and treatment generic classes are beyond the scope of this paper [18]. It might also be necessary to extract data directly from the text activities comprises previously applied measures with proven efficiency. Controls are essentially precedent dependent. of medical documents. Once this enormous and useful effort is completed, we will have a bank of clinical data containing +e choice of control (X , U ) depends not only on the i i current state (X ) but also on the overall background of the sets of DTPs with homogeneous descriptions for each no- sology present in the bank. We would like to emphasize that process as well as controls applied at earlier DTP stages {i, i− 1, i− 2, . . .}. +is is due to the specific features and nature of the no primary data reduction is envisaged, such as focusing solely on properties meaningful in the context of the relevant treatment process. To take the process memory effect into nosology. Data are extracted from the MIS “as is”—exactly as account, it is proposed to include the integral property of the there were entered in the MIS by doctors, assuming such relevant control in the extended state of the discrete process. data will most likely contain significant and meaningful Each control in the DTP can be associated with some integral information for the relevant nosology. property of such control. For example, such integral properties Finally, let us provide examples of typical properties of include full dose of medicine taken by the patient at this stage of the DTP or full dose of radiation the patient was exposed to nonreduced primary data. We believe that a process ensemble in a data bank may reach 10^3–10^6 processes for an individual in the course of radiotherapy. +e frequency (number) of application of different control elements is also regarded as an nosology. +e dimension of a vector describing one step of a discrete DTP exceeds 10^3. +e dimension of a control integral property (e.g., the number of assigned ECGs). DTP modeling based on the Markov process appears (output of the cybernetic system) may also exceed 10^3. +e case-based approach, including its application to sufficiently substantiated [17, 18, 20], especially in cases medical decision support, has been described in sufficient involving DTP description for inpatients with strictly regular detail in multiple sources [6, 14, 19]. +e main idea of the monitoring and medical decision-making. case-based approach is quite simple—find a clinical case in +us, in the model, the DTP is represented by a sequence the DB similar to the one in focus and use it for medical of vectors of equal length and structure V split into two decision support purposes. Additionally, clinical cases used components—control U and monitored properties X. Con- trol components have non-negative numerical values. A zero as precedents during the search can be filtered taking into account such factors as reputation of medical organizations value of control at this stage of the process means that this kind of control has never been applied before, starting from that such cases originate from, reputation of doctors who created such cases, or relevance of the cases in view of the beginning of the process and up until this step inclusively. contemporary medical technologies. To ensure successful Components of monitored properties are of different nature. application of the case-based approach, it is necessary to +ey can be dimensional physical values or non-numerical, have representative DBs of clinical cases. for example, assignment of a property’s value to a specific Malykh et al. [14] present assessment results with respect class. Since it is almost impossible to monitor all the prop- to the accuracy of diagnostic and treatment activities erties at the same time, certain components of properties may 4 Journal of Healthcare Engineering Discrete process steps 1 2 … t X1 X1 X1 X2 X2 X2 . . . . . . Taught Xn Xn Xn recommender U1 U1 U1 Z1 system U2 U2 U2 Z2 . . . . . . . Um Um Um Zm Discrete diagnostic and treatment process Output (recommendations) Figure 1: Recommender system. R1 Random state t –2 Ni1 t –1 Ni2 Out In N1 t N3 Output controls Input state Input vector N2 t +1 t –2 t Transition between two states in a single process N1 t Small-world graphs, d (X,Y) — graph metrics Figure 2: Structure of a case-based system. recommended using case-based reasoning. e structure of graph (R1 in the example presented in Figure 2). From the cybernetic system chosen for the approach in focus is original nodes towards their closest neighbors, we go down presented in Figure 2. to the graph node minimizing locally the distance between We have a network and each node in it is presented by the node (R1→ Ni1→ Ni2→ t in Figure 2) and the input a single DTP state. Each individual DTP represents a speci�c state. e best of all the identi�ed local minimums is se- route within the network (routes are marked in Figure 2 by lected. It will be regarded as the closest neighbor of input orange arrows). In the model, each state is represented by state In. At this point, the recommended control can be vector V. A metric or distance d(X, Y) is de�ned for each calculated as the di“erence between integral properties of control components of two vectors. In Figure 2, these are state. Based on the de�ned metric or distance, a small-world graph is plotted [21]. For each node in the small-world state vectors (t + 1) and (t). e recommended control is U  U(t + 1)− U(t). graph, n (graph parameter) closest neighbors are identi�ed. In Figure 2, closest neighbors are marked with pointing blue It is easy to assess the scale of the network in focus. In the arrows; four closest neighbors are speci�ed for node t—N1, example with 1,000 processes for one main nosology with N2, N3, and Ni2. the average duration of the process equal to ten days, we will Here is how the recommender system operates. e need 10,000 network nodes. Each node will store a vector input into the system is a current state of the DTP: e with the dimension 1,000 or higher. Computational ex- situation when the input contains the entire implemented periments show that 0.5–1% of the total number of nodes is sequence of process states is beyond the scope of this paper. su›cient as random initial network nodes. In case with Several nodes are randomly selected on the small-world 10,000 nodes, the number of initial nodes will be 50–100. e Journal of Healthcare Engineering 5 State of the discrete diagnostic and treatment process at moment t Neural network weights X1 W11 X2 Adders Neurons with activation function W21 Wn1 . Xn ∩ Z1 U1 Z2 U2 . . W(n + m)1 W(n + m)m Zm Um ⌒ Output (recommendations) Figure 3: Neural network. descent along the small-world graph was quick, and the Let us refer to the network scale as an example. Let the routes did not exceed 10 steps on average. e number of dimension of input vector be 1,000 and that of the control edges originating from each node in the small-world graph component 500. In such case the teaching process will in- was equal to 8. e top-down assessment of the number of volve de�nition of 1,000∗500 weights. Let us remark that no metric calculations in this case equals to 100∗10∗8. It is major reduction of the neural network is possible to solve the possible to accelerate the calculations by splitting the small- above problem. e reason is that the dimension of the world graph into layers corresponding to speci�c DTP control component is the number of diagnostic and treat- lengths and searching for closest neighbors within the layer ment activities that can be prescribed for this nosology, corresponding to the input state. In the above example, we including coexisting illnesses. And this number is enormous. would have layers consisting of 1,000 states, and we would Adding new layers to the neural network will only make search for closest neighbors starting from 5 to 10 randomly matters worse by increasing the number of taught selected nodes. is is fully acceptable in view of the parameters. computational requirements: computational experiments Let us examine the network teaching process. Initially, show that, in this case, computations can be performed a certain set of DTPs is selected and used for network teaching almost real-time. purposes, including calculation of weights. New DTP Let us review the network teaching process. Teaching implementations emerge. How should we use this new means adding new DTP implementations to the network. knowledge? If a su›ciently large volume of DTP imple- e number of metric calculations d when adding k states of mentations was used to teach the network (1,000 to 10,000) a new process to the network containing m states equals to and new implementations constitute an insigni�cant share of k∗m. is is absolutely acceptable in view of the compu- the teaching sample (e.g., 100 new implementations versus tational requirements As a result, new knowledge will be 10,000 is merely 1%), it can be asserted that network re- added to the network, and it will be extended by k new nodes teaching will not result in any noticeable changes in teaching and (k− 1+ k∗n) edges. It is essential to emphasize the parameters, and consequently, any major variations in the network’s sensitivity to new knowledge. Apparently, any network’s output. is kind of network is rough and con- newly added DTP implementation may have a signi�cant servative; it can “digest” new knowledge only when the impact on the decision recommended by the system if the volume of such is su›cient. In this respect, neural networks closest neighbor is selected from the added implementation. are not as good as networks applying the case-based approach. It may be asserted that the network digests new knowledge As another alternative approach, let us consider and starts applying it immediately. We will not see this in a probabilistic neural network. e structure of the network approaches described below. is outlined in Figure 4. For each state (state vector V), there is As an alternative approach, let us consider a basic neural one kernel function f(V) common for all the states. In our network with a single layer. e structure of the network is case, we used a multivariate Gaussian distribution function outlined in Figure 3. with a diagonal covariance matrix. e kernel function Current DTP state is used as input to a basic one-layer includes parameter σ a“ecting the function’s width. Each neural network. e network contains m adders and m state is classi�ed into 2m classes, where m is the dimension of neurons in accordance with the dimension of control com- the control component. If a doctor applies control L to state ponent U. In the output, each neuron has either one of the t, then t belongs to class KL1; otherwise, it belongs to class values {0,1}. Output 1 of neuron i means the system recom- KL0. mends control U for this state. Output 0 of neuron i means the Figure 5 shows the impact of control parameter σ on the system refuses to recommend control U for this state. type of distribution. i 6 Journal of Healthcare Engineering K10 K11 K20 K21 Km0 Km1 t –1 Out In t +1 Output controls Input state Input vector Kernel function t –1 t Transition between two states in a single process KL0 Control L off KL1 Control L on t KL1 State t belongs to class KL1 Figure 4: Probabilistic neural network. and generate recommendations regarding the choice of diagnostic and treatment activities for this state. Let us refer to the network scale as an example. Let the dimension of the input vector be 1,000, the dimension of the 0.8 control component be 500, and the teaching sample contain 0.6 1,000 processes with 10 states in each. We will need to 0.4 calculate 10,000 kernel functions and then calculate 1,000 6.3 0.2 posterior probabilities of the input vector belonging to each 4.2 class for various distributions of kernel function supports for 2.1 500∗2 di“erent classes. 0.6 1.2 1.8 2.4 3.6 Let us examine the network teaching process. e 4.2 4.8 5.4 0 6.6 7.2 teaching process is focused on adding new DTP imple- 7.8 mentations to the network, including assignment of states to 0–0.2 0.6–0.8 di“erent classes. If the number of new implementations is 0.2–0.4 0.8–1 a small share of the teaching sample used earlier, it can be 0.4–0.6 asserted that adding new implementations will have no (a) major impact on the network’s output. e probabilistic neural network proves to be rough and conservative; it can “digest” new knowledge only when the volume of such is 1.2 su›cient. In this respect, probabilistic neural networks are not as good as networks applying the case-based approach. 0.8 0.6 3. Results 0.4 6.3 0.2 We performed computational experiments for a network 4.2 built using the case-based approach in 2015-2016. e results 2.1 0.6 1.2 1.8 were published in Malykh et al. [14]. To compare di“erent 2.4 3.6 4.2 4.8 5.4 6 approaches to the problem, we will present the results of 6.6 7.2 7.8 paper [14] in a slightly modi�ed format. 0–0.2 0.6–0.8 Table 1 shows that the number of correct recommen- 0.2–0.4 0.8–1 dations (TP True Positive) varies from 58.7 to 94.9% 0.4–0.6 1–1.2 depending on the type of nosology. e majority of rec- (b) ommendations match the doctor’s actions. In the matter of neural networks, computational ex- Figure 5: Impact of control parameter σ on kernel functions and periments for all nosologies listed in Table 1 required quite type of distribution. a lot of time and computing power. e practical value of such full-scale experiments was unclear. erefore, it was Now, a probability density function can be “restored” for decided to limit computational experiments to estimations each class. For input vector In, we apply Bayes’ formula to for nosology J13. Table 2 contains general information about calculate the posterior probability of belonging to each class the experiment with a single-layer neural network. Journal of Healthcare Engineering 7 Table 1: Accuracy assessment of recommended diagnostic and treatment activities for seven nosologies using the case-based approach. Number of diagnostic and Total number of Number of correct treatment activities the clinical Number of recommendations recommendations decision support system was precedents/number with a different control level among control unable to provide of control among control precedents precedents recommendations for among MKB-10 code/nosology precedents control precedents Absolute value/share Number of Absolute value/share in the Absolute value/share in the in the total number states/number of total number of diagnostic total number of diagnostic of diagnostic and controlled variables and treatment activities and treatment activities treatment activities J13/pneumonia due to 166/11 6788/81.6% 3923/47.2% 1530/18.4% Streptococcus pneumoniae 2938/118 K80.1/calculus of gallbladder 1018/128 34468/76.7% 18390/40.9% 10490/23.3% with other cholecystitis 12853/931 H25.1/age-related nuclear 1205/121 3522/94.9% 539/14.5% 189/5.1% cataract 5509/293% 1255/126 H26.2/complicated cataract 4362/91.4% 1617/33.9% 408/8.6% 5778/249% I67.4/hypertensive 1336/134 65678/72.4% 37563/41.4% 25060/27.6% encephalopathy 23165/1431 I67.9/cerebrovascular disease, 1403/141 58649/75.4% 32447/41.7% 19117/24.6% unspecified 24875/1518 1632/164 N20.1/calculus of ureter 17489/58.7% 9948/58.7% 12291/41.3% 15922/205 Table 2: Accuracy assessment of recommended diagnostic and treatment activities for nosology J13 based on a single-layer neural network. Total number of negative Number of recommendations/total Number of correct incorrect number Total number of clinical positive positive of positive recommendations precedents/number of recommendations recommendations Share of correct negative control precedents among among recommendations/share of control precedents MKB-10 code/nosology control precedents correct positive recommendations Absolute Absolute Number of neural network value/share in value/share in inputs/number of neural the total number the total number of Absolute value/percent network outputs (number of of positive positive controlled variables) recommendations recommendations J13/pneumonia due to 266/11 35567/841 339/40.31% 502/59.69% Streptococcus pneumoniae 224/222 98.55%/40.31% Let us emphasize that the volume of statistics on this (standards of the Russian Ministry of Health). Different illness stored in the DB has increased compared to an earlier insurance programs also limit integral values of controls. experiment involving the same nosology—from 166 to 266 Healthcare providers will not exceed these limits unless they completed clinical processes. Controls included all types of find it necessary. Formally, with respect to the model, it drug prescriptions (222 different pharmaceutical products in means that once an integral property of a control reaches a certain limit, it stops growing further or such growth is our case). Data normalization involved adjustment of pre- scribed dosages of pharmaceutical products to unified dose highly unlikely. +e gradient of the target function with respect to weights was calculated explicitly, and the steepest units. +e only monitored variable was “inpatient days.” Inputs also included bias. 49,728 weights had to be de- descent method was applied. Teaching included 1,006 de- termined. +e optimized target function was a quadratic scent steps. Criteria reflecting the accuracy of the neural residual between neural network output and control com- network are presented in Table 3. ponents monitored in control samples, adjusted to (0, 1). We +e relevant receiver operating characteristic (ROC) used a nonstandard neurons activation bell curve (Gaussian error curve is shown in Figure 6. function). +is choice of activation function was based on Results of the experiment based on a probabilistic neural the fact that integral values of many controls had apparent network are presented in Table 4. +e state vector dimension limits stipulated by Russian federal healthcare standards was equal to 639. +e control component included 125 8 Journal of Healthcare Engineering Table 3: Accuracy of recommended diagnostic and treatment Table 4: Accuracy of recommended diagnostic and treatment activities for nosology J13 based on a single-layer neural network activities for nosology J13 based on a probabilistic neural network with an activation threshold equal to 0.1. with σ  2.5. Absolute values (neuron activation threshold equal to 0.1) Absolute values (σ  2.5) TP 339 502 FP TP 233 191 FP TN 35,052 515 FN TN 35,376 608 FN Percent (neuron activation threshold equal to 0.1) Percent (σ  2.5) TP 40.31% 59.69% FP TP 55.0% 45.0 FP TN 98.55% 1.45% FN TN 98.31% 1.69% FN TP, true positive; FP, false positive; TN, true negative; FN, false negative. such modeling or knowledge extraction from data. Data are 1.2 extracted from the MIS without reduction, “as is.” It is assumed that the data contain signi�cant information 1.0 re±ecting medical knowledge and contemporary medical 0.8 treatment technologies. ree di“erent approaches to big 0.6 clinical data processing were examined: (1) case-based 0.4 reasoning for decision-making; (2) decision-making based 0.2 on a single-layer neural network; and (3) decision-making based on a probabilistic neural network. Experimental 0.0 0 0.2 0.4 0.6 0.8 1 1.2 calculations were performed to assess the accuracy of rec- FPR ommendations generated using di“erent approaches. Drawbacks of the above neural networks with respect to Figure 6: ROC error curve. the given problem were identi�ed. e overall accuracy of provided recommendations was rather high. Moreover, the diagnostic tests, 200 laboratory tests (di“erent kinds), 222 di“erent pharmaceuticals, 87 medical treatments, and 4 accuracy of negative recommendations that the neural networks learned to provide was very high (98–99%). controls classi�ed as “others.” e only monitored property However, the accuracy of positive recommendations pro- was “inpatient days.” e number of kernels (states) in the vided by the neural networks was not so high (40–55%, teaching sample of 266 processes was 4,361. e dimension of which is obviously insu›cient for successful practical ap- the state vector in the probabilistic neural network was almost plication). Another disadvantage of neural networks is their three times the dimension of the state vector in the single- rough and conservative nature, particularly when digesting layer neural network (639 versus 223). To make the results of isolated portions of new data with the volume insigni�cant both networks comparable, the output of the probabilistic compared to previously available data. neural network was considered to be the same as for the �rst e case-based approach to decision-making yielded neural network. e output was a vector with a dimension of more accurate recommendations (59–95%), which is su›- 222, related to prescription of di“erent pharmaceuticals. Both neural networks generated 36,408 positive and negative cient for its successful practical application. Another ad- vantage of the case-based approach is its sensitivity to new recommendations for the control sample. e experiment data. With respect to calculations, the case-based approach is involved one control parameter σ, a multiplier for a diagonal also more e›cient compared to other options under con- covariance matrix used in the kernel function (multivariate sideration as it ensures a high operating speed of the decision Gaussian distribution of independent random variables). A support system, thus making it acceptable for practical ap- value grid was predetermined for the parameter σ, and the plication. ese are the key �ndings of the study conducted. best value of the parameter was chosen based on experimental is o“ers encouraging prospects for designing and calculation results [22]. Calculations were performed for the developing decision support systems for physicians based on following values of σ: (0.1, 0.5, 1, and 2.5). e best results empirical components of medical knowledge. is approach were obtained for σ  2.5. ey are presented in Table 4. Let us also corresponds to existing case-based character of man- emphasize that standard deviation values of the state vector components calculated for the teaching sample were signif- agement and decision-making in medical practice. So far, the results indicate that precedent-based approach has a high icant and often exceeded average values. e multiplier equal e“ectiveness and could naturally enhance other approaches to 2.5 yields “wide” kernel functions (see the rightmost dis- to supporting physicians’ decision-making, particularly tribution in Figure 5). With “sharp” kernel functions (σ  0.1), knowledge-based ones. e obvious practical value of this the results were obviously worse. approach lies in the fact that it can be complementary to other knowledge-based approaches (clinical pathways, 4. Summary Evidence-Based Clinical Decision Support, expert systems, e focus of this paper was how to build a medical decision Watson, etc.). e doctor will be able to make decisions support system based on big clinical data. e authors review based on the best examples of medical practice, �nding general approaches to the problem that do not involve in- precedents of clinical cases close to the given case. e constraints of precedent-based approach include the dividual models for speci�c nosologies and neither do they require engagement of experts in the relevant subject area to need for a representative database of veri�ed precedents TPR Journal of Healthcare Engineering 9 [3] V. L. Malykh, S. V. Rudetskiy, and M. I. Khatkevich, Active excluding medical errors. From another perspective, pre- MIS. Information Technologies for the Physician, Publisher cedents with corrected errors are of particular interest to House “Public Health Manager”, Moscow, Russia, no. 6, pp. physicians training and further prevention of such errors. 16–24, 2016, in Russian. +e information about the results of these errors and pos- [4] I. S. Kohane, “+e twin questions of personalized medicine: sible ways of correcting them is also valuable. +us, who are you and whom do you most resemble?,” Genome precedent-based approach could be widely spread as an Medicine, vol. 1, no. 1, p. 4, 2009. educational tool. On the other hand, the precedent-based [5] S. N. Butko and V. K. Olshansky, New Decision Support approach does not imply formalization of medical knowl- Systems in Foreign Healthcare. Automation and Remote edge, which entails poor cognitive justification of generated Control, Springer, Berlin, Germany, 1990. recommendations. Consequently, justifications only de- [6] Y. B. Kotov, New Mathematical Approaches to Medical Di- scribe how other patients were treated in similar clinical agnostics, Editorial URSS, Moscow, Russia, 2004. cases. +ere are also problems with optimization of provided [7] R. Miotto, L. Li, B. A. Kidd, and J. T. Dudley, “Deep patient: an unsupervised representation to predict the future of patients metrics, compression of state descriptions, and construction from the electronic health records,” Scientific Reports, vol. 6, of training procedures. +ese problems are connected with no. 1, p. 26094, 2016. high dimensionality of the space of state characteristics and [8] V. L. Malykh and D. V. Belyshev, “Case-based reasoning in samples of clinical precedents. However, discussion of these clinical processes using clinical data banks,” in Proceedings of issues and possible ways of addressing them has been left 2015 International Conference on Biomedical Engineering and outside of this research [14]. Computational Technologies (SIBIRCON), Technopark of In further studies, we are going to focus on detailed Novosibirsk Akademgorodok, pp. 211–216, Novosibirsk, application of the case-based approach, analyze metrics, and Russia, 2015. distances not only for pairs of vectors but also for pairs of [9] E. Herrett, A. M. Gallagher, K. Bhaskaran et al., “Data re- vector sequences, and examine issues concerned with in- source profile: clinical practice research datalink (CPRD),” telligent normalization of primary data and data extraction International Journal of Epidemiology, vol. 44, no. 3, from plain texts of medical documents. pp. 827–836, 2015. [10] T. Rotter, L. Kinsman, E. James et al., “Clinical pathways: effects on professional practice, patient outcomes, length of Disclosure stay and hospital costs,” Cochrane Database System Review, vol. 3, p. CD006632, 2010. UDC 007.52 (Automatically operated systems without any [11] AHRQ, Clinical Decision Support Systems: State of the Art- humans among system links, robots, and automated machines). Agency for Healthcare Research and Quality U.S, AHRQ Publication No. 09-0069-EF, Rockville, MD, USA, 2009. [12] I. V. Efimenko and V. F. Khoroshevsky, “Intelligent decision Conflicts of Interest support systems in medicine: state of the art and beyond in Russian,” in Proceedings of OSTIS, Open Semantic Technol- +e authors declare that they have no conflicts of interest. ogies for Intelligent Systems, Minsk, Belarus, 2017, http:// proc.ostis.net/proc/Proceedings%20OSTIS-2017.pdf. Acknowledgments [13] Wikipedia, Clinical Decision Support System, http://en. academic.ru/dic.nsf/enwiki/1126747, 2017. +e authors would like to thank Professor V. M. Khachumov, [14] V. L. Malykh, I. N. Kononenko, and S. V. Rudetskiy, “Esti- a doctor of engineering sciences, and V. P. Fralenko, mation of accuracy of recommended diagnostic and treat- a candidate of engineering sciences, for the consulting ment actions based on precedent approach,” in Proceedings of support on neural networks and teaching methods, as well the International Conference e-Health 2016, pp. 52–58, Ma- as Professor N. N. Nepevoda, a doctor of physical and deira, Portugal, July 2016. [15] F. Caron, J. Vanthienen, and B. Baesens, “Healthcare ana- mathematical sciences, for the discussion and assessment of lytics: examine the diagnosis-treatment cycle,” Procedia the outcomes. Some of the outcomes presented in the paper Technology, vol. 9, pp. 996–1004, 2013. were achieved earlier under the support of the Ministry of [16] E. Vasilyeva, Industrial Internet of @ings (IoT), Rational Education and Science of the Russian Federation (Project Enterprise Management, Athens, GA, USA, 2015, in Russian. RFMEFI60714X0089) and in the context of Grant 13-07- [17] V. L. Malykh and Y. I. Guliev, “Controlled stochastic precedent 12012 provided by the Russian Foundation for Basic process with memory as a mathematical model of the di- Research. agnostic and treatment process,” Information Technologies and Computational Systems, vol. 2, pp. 62–72, 2014, in Russian. [18] V. L. Malykh, Y. I. Guliev, A. V. Eremin, and S. V. Rudetskyi, References “Management and decision making in clinical processes,” in Proceedings of XII All-Russian Conference on Problems of [1] MedSoft, “Medical software,” in Conference Presentations of the 12th International Forum MedSoft-2016, in Russian, Management of VSPU-2014, pp. 6518, Moscow, Russia, 2014, in Russian. Moscow, Russia, 2016, http://www.armit.ru/medsoft/2016/ conference/prog. [19] V. L. Malykh and Y. I. Guliev, “Precedent approach to decisionaking in clinical processes,”, in MEDINFO 2015: [2] B. S. Handler, J. +omas, and B. R. Hieb, Gartner’s 2007 Criteria for the Enterprise CPR, 2014, http://rsept.wikispaces. Health-nabled Health, Studies in Health Technology and In- formatics, Vol. 216, Ed., IMIA and IOS Press, Amsterdam, com/file/view/Gartner_Criteria_for_the_Enterprise_CPR_ 2007.pdf. Netherlands, 2015. 10 Journal of Healthcare Engineering [20] C. Bennett and K. Hauser, “Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach,” Artificial Intelligence in Medicine, vol. 57, no. 1, pp. 9–19, 2013. [21] Y. Malkov, A. Ponomarenko, A. Logvinov, and V. Krylov, “Approximate nearest neighbor algorithm based on navigable small world graphs,” Information Systems, vol. 45, pp. 61–68, [22] R. N. Kvetniy, V. V. Kabachiy, and O. O. Chumachenko, Probabilistic Neural Networks in Time Series Identification, Information Technologies and Computers, Scientific works of Vinnytsia National Technical University, Vinnytsia, Ukraine, 2010, in Russian. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Healthcare Engineering Hindawi Publishing Corporation

Approaches to Medical Decision-Making Based on Big Clinical Data

Loading next page...
 
/lp/hindawi-publishing-corporation/approaches-to-medical-decision-making-based-on-big-clinical-data-4YHlGBwhz2

References (26)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2018 V. L. Malykh and S. V. Rudetskiy. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
2040-2295
eISSN
2040-2309
DOI
10.1155/2018/3917659
Publisher site
See Article on Publisher Site

Abstract

Hindawi Journal of Healthcare Engineering Volume 2018, Article ID 3917659, 10 pages https://doi.org/10.1155/2018/3917659 Research Article Approaches to Medical Decision-Making Based on Big Clinical Data V. L. Malykh and S. V. Rudetskiy Medical Informatics Research Center, Ailamazyan Program Systems Institute of RAS, Pereslavl-Zalessky, Russia Correspondence should be addressed to V. L. Malykh; mvl@interin.ru Received 21 September 2017; Revised 14 February 2018; Accepted 30 April 2018; Published 4 June 2018 Academic Editor: Giedrius Vanagas Copyright © 2018 V. L. Malykh and S. V. Rudetskiy. +is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. +e paper discusses different approaches to building a medical decision support system based on big data. +e authors sought to abstain from any data reduction and apply universal teaching and big data processing methods independent of disease clas- sification standards. +e paper assesses and compares the accuracy of recommendations among three options: case-based reasoning, simple single-layer neural network, and probabilistic neural network. Further, the paper substantiates the assumption regarding the most efficient approach to solving the specified problem. this increasing subjectivity of MISs, we have proposed a new 1. Introduction term “active MIS” that emphasizes a certain degree of in- Providing support to medical decision-making is one of the dependence from users or subjectivity of the cyber system. most urgent issues in healthcare automation. It has been Kohane [4] presents the most “balanced” definition of repeatedly noted in different articles, reports, and forum personalized medicine, “personalized medicine is the discussions [1] both in Russia and abroad that MIS in- practice of clinical decision-making such that the decisions troduction requires a considerable extra effort from made maximize the outcomes that the patient most cares users—doctors in the first place—to enter primary data into about and minimize those that the patient fears the most, on the basis of as much knowledge about the individual’s state the system. Naturally, doctors expect practical intelligent outcomes from big clinical data accumulated by modern as is available.” +is perception of personal medicine is MISs. Handler et al. [2] present the operating paradigm of focused on clinical decision-making and once again exhibits 5th generation MISs, referred to as “MIS as Mentor.” Malykh the urgency and importance of scientific research in the area. et al. [3] adds one more qualitative characteristic to the +erefore, building an automated active mentor-type system above paradigm—“MIS as automated mentor.” “It is ad- that provides recommendations regarding treatment and visable to abandon the practice of active user dialogs typical diagnostic activities to the doctor is an urgent practical task. of expert systems, involving requests for data that the system Butko and Olshansky [5] and Kotov [6] provide a retro- considers missing from the user, and substitute the dialog spective overview of approaches to building medical decision with an automated nonintrusive algorithm that draws its support systems. +e applied approaches were restricted in own logical conclusions and generates recommendations in many respects by the abilities of computers at that time. Ac- cordingly, there was no such problem as processing big medical a completely automated manner based on available data, without involving the user in the process. +e user may data. Technologies have evolved to the point when big medical either accept or ignore the system’s prompts and recom- data (both on individuals and the population in general) mendations; however, they will not provoke rejection in collection and accumulation is finally feasible. At the same users if delivered automatically without requiring a dialog time, big data processing and intelligent system learning with the system.” To provide a brief qualitative description of methods were evolving as well. Along with “deep learning,” the 2 Journal of Healthcare Engineering systems. +is approach helps achieve top quality when term “deep patient” [7] was coined, meaning the opportunity to extract increasingly more complete, deep, and valuable in- solving isolated problems [6, 12]; however, it is almost impossible to apply it to big clinical data. formation about patients from big clinical data using deep learning methods. +e fourth approach that claims to have a global scope of Malykh et al. [8] mention the possibility of creating application is focused on building a cognitive system capable national-scale clinical data banks. Herrett et al. [9] provide of self-learning and knowledge digestion directly from an example of a database (DB) containing anonymous nonformalized text sources (IBM Watson http://www.ibm. medical records on primary healthcare services provided. com/smarterplanet/us/en/ibmwatson/). +is DB was created by a joint effort of 674 general prac- None of the reviewed approaches is immaculate. All of them require efforts of experts and regular updates of titioners and covers over 11.3 mn patients in Great Britain. Decision-making in hospitals has evolved from being knowledge bases. Moreover, each of the approaches is in fact tailored to specific purposes. opinion-based to being based on sound scientific evidence. +is decision-making is recognized as evidence-based +e latest Russian-language review [12] noted that clinical decision support systems have not become wide- practice. Perpetual publication of new evidence combined with the demands of everyday practice makes it difficult for spread in Russia. +is is due to the complexity of the de- health professionals to keep up to date [10]. velopment of such systems, the specific character of the A large number of publications are devoted to medical systems already developed, and the need to involve high- decision support systems (DSSs), including publications in class experts in the development. specialized scientific journals (Artificial Intelligence in In this paper, we will review general approaches to Medicine, BMC Medical Informatics and Decision Making, decision support system development based on nonreduced big clinical data. +e main expectations related to applica- International Journal of Medical Informatics, Medical De- cision Making, etc.). +e work does not aim to give an tion of general approaches ensue from the case-based nature of decision-making in healthcare, and the assumption that overview of different approaches to making of decision support systems, referring readers to the original reviews big clinical data already contain enough knowledge for ef- ficient decision-making. [11–13]. We can give a few definitions for decision support system from Wikipedia: “Clinical Decision Support systems +ere are two other factors that draw attention to sys- link health observations with health knowledge to influence tems based on machine learning or precedent approach. health choices by clinicians for improved health care” and First of them is that there are trends in the development “active knowledge systems, which use two or more items of of our civilization which include an explosive development patient data to generate case-specific advice.” No one doubts of information technologies (among them M2M, Big Data, the feasibility of such systems and that they have a positive and IoT), their strong need for formalized knowledge, and practical absence of qualified experts who could formalize impact on professional practice, patient outcomes, length of hospital stay, and hospital costs. +e main problem is to find that knowledge. +e chief editor of the Rational Enterprise Management (REM) magazine (Russia) holds regular dis- effective approaches to building such systems. A number of contemporary approaches to medical decision cussions on a wide range of problems including the above- support system development are listed by Malykh et al. [14]. mentioned ones. Results of the discussions are published in +e first one of these approaches involves provision of the REM editor’s column. +e guests of a recent discussion relevant data sources to doctors, helping them make decisions [16] included Igor Rudym (Intel), Dmitriy Tameev (PTC), independently. +e system does not recommend any final Alexander Belotserkovskiy (Microsoft), Igor Girkin (Cisco), solutions—instead, it suggests data sources to study and find and Igor Kulinitchev (IBM). All the participants agreed that, answers to current questions (Evidence-Based Clinical De- nowadays, the key challenge of IT development is not as- cision Support at the Point of Care | UpToDate URL: http:// sociated with hardware or software, but it needs break- through approaches to data analysis. www.uptodate.com/home). +e second approach is to use clinical pathways. Clinical As for the second factor, it is obvious that, nowadays, there are no qualified experts in the field of knowledge even pathways represent prescriptive models of the standard healthcare procedures that need to be undertaken for in key branches. +e actual situation is even more critical as a specific patient population. Instances of the clinical the experts who are able to solve at least a part of these pathways (also known as cases) describe the actual problems are not able to cope with ever increasing in- diagnostic-therapeutic cycle of an individual patient [15]. formation flow. From this point of view, precedent-based But even in the case of the use of clinical pathways, the DSSs practically need no experts. Experts may be needed for process of clinical decision-making has high complexity. enhancing or optimizing existing medical data bases and knowledge bases [14]. While the medical knowledge used in the decision process comes partially from published research contributions and widespread medical guidelines (with various kinds of evi- 2. Model and Methods dence levels), it is generally accepted that the decision process is profoundly influenced by the expertise and ex- We regard the diagnostic and treatment process (DTP) as periences of the involved medical experts [15]. a discrete controlled process with a memory. +e model was +e third approach involves development of a large first introduced by Malykh et al. [17, 18] written in Russian. number of individual narrow-focused decision support In English, the model is described by Malykh et al. [8, 19]. To Journal of Healthcare Engineering 3 ensure further understanding of the essence of the problem, be unknown to us. When applying different methods to the let us provide an extract from the source. model, we may need to digitize non-numerical values of components and identify missing values of monitored Modern medical information systems store electronic medical records and contain descriptions of millions of properties. various clinical cases. +e degree of formalization of clinical data stored in MISs varies. MISs model the diagnostic and treatment process as a sequence of controlling events 2.1. Definition of the Objective. We will review several methods that can be applied to build a cybernetic taught reflecting diagnostic and treatment activities, and a sequence of monitoring events describing the condition of the patient. system. +e input into the system will be a sequence of vectors describing a discrete DTP in accordance with the Controlling events are well formalized; medical organiza- tions keep statistical and business records of such events, presented model. +e output will consist of recommenda- tions proposing diagnostic and treatment options (choice of plan them, and allocate required resources. Medical data controls) for this particular state of the process. A diagram of related to monitoring of patients’ condition are less for- the system is presented in Figure 1. malized and may be partly available in the form of plain text Let us define the objective more accurately and assume medical documents. Previous studies provide evidence that is possible to that each DTP model is considered in the context of an already available predominant diagnosis. For each model, we model the DTP using controlled stochastic Markov processes [18]. +e model is based on the assumption that the DTP is have an array of earlier observed DTP implementations. Such implementations are sources of knowledge about a discrete controlled process. +e model introduces the no- tions of control U and state X. Controls are diagnostic and treatment of a particular nosology, and they are used to teach a cybernetic recommender system to operate in the given treatment decisions made and executed in future. Controls are different diagnostic and treatment activities prescribed by context. Based on available DTP implementations, we de- fined a glossary of controls and monitored properties for doctors, including diagnostic tests, medicines, surgical in- each model. Issues related to normalization of primary data, terventions, various procedures, and manipulations. +e outlier testing and exclusion, and approaches to data gen- choice of diagnostic and treatment activities is based on the eralization based on assignment of monitored properties to accumulated medical knowledge and the doctor’s individual experience. +e scope of potential diagnostic and treatment generic classes are beyond the scope of this paper [18]. It might also be necessary to extract data directly from the text activities comprises previously applied measures with proven efficiency. Controls are essentially precedent dependent. of medical documents. Once this enormous and useful effort is completed, we will have a bank of clinical data containing +e choice of control (X , U ) depends not only on the i i current state (X ) but also on the overall background of the sets of DTPs with homogeneous descriptions for each no- sology present in the bank. We would like to emphasize that process as well as controls applied at earlier DTP stages {i, i− 1, i− 2, . . .}. +is is due to the specific features and nature of the no primary data reduction is envisaged, such as focusing solely on properties meaningful in the context of the relevant treatment process. To take the process memory effect into nosology. Data are extracted from the MIS “as is”—exactly as account, it is proposed to include the integral property of the there were entered in the MIS by doctors, assuming such relevant control in the extended state of the discrete process. data will most likely contain significant and meaningful Each control in the DTP can be associated with some integral information for the relevant nosology. property of such control. For example, such integral properties Finally, let us provide examples of typical properties of include full dose of medicine taken by the patient at this stage of the DTP or full dose of radiation the patient was exposed to nonreduced primary data. We believe that a process ensemble in a data bank may reach 10^3–10^6 processes for an individual in the course of radiotherapy. +e frequency (number) of application of different control elements is also regarded as an nosology. +e dimension of a vector describing one step of a discrete DTP exceeds 10^3. +e dimension of a control integral property (e.g., the number of assigned ECGs). DTP modeling based on the Markov process appears (output of the cybernetic system) may also exceed 10^3. +e case-based approach, including its application to sufficiently substantiated [17, 18, 20], especially in cases medical decision support, has been described in sufficient involving DTP description for inpatients with strictly regular detail in multiple sources [6, 14, 19]. +e main idea of the monitoring and medical decision-making. case-based approach is quite simple—find a clinical case in +us, in the model, the DTP is represented by a sequence the DB similar to the one in focus and use it for medical of vectors of equal length and structure V split into two decision support purposes. Additionally, clinical cases used components—control U and monitored properties X. Con- trol components have non-negative numerical values. A zero as precedents during the search can be filtered taking into account such factors as reputation of medical organizations value of control at this stage of the process means that this kind of control has never been applied before, starting from that such cases originate from, reputation of doctors who created such cases, or relevance of the cases in view of the beginning of the process and up until this step inclusively. contemporary medical technologies. To ensure successful Components of monitored properties are of different nature. application of the case-based approach, it is necessary to +ey can be dimensional physical values or non-numerical, have representative DBs of clinical cases. for example, assignment of a property’s value to a specific Malykh et al. [14] present assessment results with respect class. Since it is almost impossible to monitor all the prop- to the accuracy of diagnostic and treatment activities erties at the same time, certain components of properties may 4 Journal of Healthcare Engineering Discrete process steps 1 2 … t X1 X1 X1 X2 X2 X2 . . . . . . Taught Xn Xn Xn recommender U1 U1 U1 Z1 system U2 U2 U2 Z2 . . . . . . . Um Um Um Zm Discrete diagnostic and treatment process Output (recommendations) Figure 1: Recommender system. R1 Random state t –2 Ni1 t –1 Ni2 Out In N1 t N3 Output controls Input state Input vector N2 t +1 t –2 t Transition between two states in a single process N1 t Small-world graphs, d (X,Y) — graph metrics Figure 2: Structure of a case-based system. recommended using case-based reasoning. e structure of graph (R1 in the example presented in Figure 2). From the cybernetic system chosen for the approach in focus is original nodes towards their closest neighbors, we go down presented in Figure 2. to the graph node minimizing locally the distance between We have a network and each node in it is presented by the node (R1→ Ni1→ Ni2→ t in Figure 2) and the input a single DTP state. Each individual DTP represents a speci�c state. e best of all the identi�ed local minimums is se- route within the network (routes are marked in Figure 2 by lected. It will be regarded as the closest neighbor of input orange arrows). In the model, each state is represented by state In. At this point, the recommended control can be vector V. A metric or distance d(X, Y) is de�ned for each calculated as the di“erence between integral properties of control components of two vectors. In Figure 2, these are state. Based on the de�ned metric or distance, a small-world graph is plotted [21]. For each node in the small-world state vectors (t + 1) and (t). e recommended control is U  U(t + 1)− U(t). graph, n (graph parameter) closest neighbors are identi�ed. In Figure 2, closest neighbors are marked with pointing blue It is easy to assess the scale of the network in focus. In the arrows; four closest neighbors are speci�ed for node t—N1, example with 1,000 processes for one main nosology with N2, N3, and Ni2. the average duration of the process equal to ten days, we will Here is how the recommender system operates. e need 10,000 network nodes. Each node will store a vector input into the system is a current state of the DTP: e with the dimension 1,000 or higher. Computational ex- situation when the input contains the entire implemented periments show that 0.5–1% of the total number of nodes is sequence of process states is beyond the scope of this paper. su›cient as random initial network nodes. In case with Several nodes are randomly selected on the small-world 10,000 nodes, the number of initial nodes will be 50–100. e Journal of Healthcare Engineering 5 State of the discrete diagnostic and treatment process at moment t Neural network weights X1 W11 X2 Adders Neurons with activation function W21 Wn1 . Xn ∩ Z1 U1 Z2 U2 . . W(n + m)1 W(n + m)m Zm Um ⌒ Output (recommendations) Figure 3: Neural network. descent along the small-world graph was quick, and the Let us refer to the network scale as an example. Let the routes did not exceed 10 steps on average. e number of dimension of input vector be 1,000 and that of the control edges originating from each node in the small-world graph component 500. In such case the teaching process will in- was equal to 8. e top-down assessment of the number of volve de�nition of 1,000∗500 weights. Let us remark that no metric calculations in this case equals to 100∗10∗8. It is major reduction of the neural network is possible to solve the possible to accelerate the calculations by splitting the small- above problem. e reason is that the dimension of the world graph into layers corresponding to speci�c DTP control component is the number of diagnostic and treat- lengths and searching for closest neighbors within the layer ment activities that can be prescribed for this nosology, corresponding to the input state. In the above example, we including coexisting illnesses. And this number is enormous. would have layers consisting of 1,000 states, and we would Adding new layers to the neural network will only make search for closest neighbors starting from 5 to 10 randomly matters worse by increasing the number of taught selected nodes. is is fully acceptable in view of the parameters. computational requirements: computational experiments Let us examine the network teaching process. Initially, show that, in this case, computations can be performed a certain set of DTPs is selected and used for network teaching almost real-time. purposes, including calculation of weights. New DTP Let us review the network teaching process. Teaching implementations emerge. How should we use this new means adding new DTP implementations to the network. knowledge? If a su›ciently large volume of DTP imple- e number of metric calculations d when adding k states of mentations was used to teach the network (1,000 to 10,000) a new process to the network containing m states equals to and new implementations constitute an insigni�cant share of k∗m. is is absolutely acceptable in view of the compu- the teaching sample (e.g., 100 new implementations versus tational requirements As a result, new knowledge will be 10,000 is merely 1%), it can be asserted that network re- added to the network, and it will be extended by k new nodes teaching will not result in any noticeable changes in teaching and (k− 1+ k∗n) edges. It is essential to emphasize the parameters, and consequently, any major variations in the network’s sensitivity to new knowledge. Apparently, any network’s output. is kind of network is rough and con- newly added DTP implementation may have a signi�cant servative; it can “digest” new knowledge only when the impact on the decision recommended by the system if the volume of such is su›cient. In this respect, neural networks closest neighbor is selected from the added implementation. are not as good as networks applying the case-based approach. It may be asserted that the network digests new knowledge As another alternative approach, let us consider and starts applying it immediately. We will not see this in a probabilistic neural network. e structure of the network approaches described below. is outlined in Figure 4. For each state (state vector V), there is As an alternative approach, let us consider a basic neural one kernel function f(V) common for all the states. In our network with a single layer. e structure of the network is case, we used a multivariate Gaussian distribution function outlined in Figure 3. with a diagonal covariance matrix. e kernel function Current DTP state is used as input to a basic one-layer includes parameter σ a“ecting the function’s width. Each neural network. e network contains m adders and m state is classi�ed into 2m classes, where m is the dimension of neurons in accordance with the dimension of control com- the control component. If a doctor applies control L to state ponent U. In the output, each neuron has either one of the t, then t belongs to class KL1; otherwise, it belongs to class values {0,1}. Output 1 of neuron i means the system recom- KL0. mends control U for this state. Output 0 of neuron i means the Figure 5 shows the impact of control parameter σ on the system refuses to recommend control U for this state. type of distribution. i 6 Journal of Healthcare Engineering K10 K11 K20 K21 Km0 Km1 t –1 Out In t +1 Output controls Input state Input vector Kernel function t –1 t Transition between two states in a single process KL0 Control L off KL1 Control L on t KL1 State t belongs to class KL1 Figure 4: Probabilistic neural network. and generate recommendations regarding the choice of diagnostic and treatment activities for this state. Let us refer to the network scale as an example. Let the dimension of the input vector be 1,000, the dimension of the 0.8 control component be 500, and the teaching sample contain 0.6 1,000 processes with 10 states in each. We will need to 0.4 calculate 10,000 kernel functions and then calculate 1,000 6.3 0.2 posterior probabilities of the input vector belonging to each 4.2 class for various distributions of kernel function supports for 2.1 500∗2 di“erent classes. 0.6 1.2 1.8 2.4 3.6 Let us examine the network teaching process. e 4.2 4.8 5.4 0 6.6 7.2 teaching process is focused on adding new DTP imple- 7.8 mentations to the network, including assignment of states to 0–0.2 0.6–0.8 di“erent classes. If the number of new implementations is 0.2–0.4 0.8–1 a small share of the teaching sample used earlier, it can be 0.4–0.6 asserted that adding new implementations will have no (a) major impact on the network’s output. e probabilistic neural network proves to be rough and conservative; it can “digest” new knowledge only when the volume of such is 1.2 su›cient. In this respect, probabilistic neural networks are not as good as networks applying the case-based approach. 0.8 0.6 3. Results 0.4 6.3 0.2 We performed computational experiments for a network 4.2 built using the case-based approach in 2015-2016. e results 2.1 0.6 1.2 1.8 were published in Malykh et al. [14]. To compare di“erent 2.4 3.6 4.2 4.8 5.4 6 approaches to the problem, we will present the results of 6.6 7.2 7.8 paper [14] in a slightly modi�ed format. 0–0.2 0.6–0.8 Table 1 shows that the number of correct recommen- 0.2–0.4 0.8–1 dations (TP True Positive) varies from 58.7 to 94.9% 0.4–0.6 1–1.2 depending on the type of nosology. e majority of rec- (b) ommendations match the doctor’s actions. In the matter of neural networks, computational ex- Figure 5: Impact of control parameter σ on kernel functions and periments for all nosologies listed in Table 1 required quite type of distribution. a lot of time and computing power. e practical value of such full-scale experiments was unclear. erefore, it was Now, a probability density function can be “restored” for decided to limit computational experiments to estimations each class. For input vector In, we apply Bayes’ formula to for nosology J13. Table 2 contains general information about calculate the posterior probability of belonging to each class the experiment with a single-layer neural network. Journal of Healthcare Engineering 7 Table 1: Accuracy assessment of recommended diagnostic and treatment activities for seven nosologies using the case-based approach. Number of diagnostic and Total number of Number of correct treatment activities the clinical Number of recommendations recommendations decision support system was precedents/number with a different control level among control unable to provide of control among control precedents precedents recommendations for among MKB-10 code/nosology precedents control precedents Absolute value/share Number of Absolute value/share in the Absolute value/share in the in the total number states/number of total number of diagnostic total number of diagnostic of diagnostic and controlled variables and treatment activities and treatment activities treatment activities J13/pneumonia due to 166/11 6788/81.6% 3923/47.2% 1530/18.4% Streptococcus pneumoniae 2938/118 K80.1/calculus of gallbladder 1018/128 34468/76.7% 18390/40.9% 10490/23.3% with other cholecystitis 12853/931 H25.1/age-related nuclear 1205/121 3522/94.9% 539/14.5% 189/5.1% cataract 5509/293% 1255/126 H26.2/complicated cataract 4362/91.4% 1617/33.9% 408/8.6% 5778/249% I67.4/hypertensive 1336/134 65678/72.4% 37563/41.4% 25060/27.6% encephalopathy 23165/1431 I67.9/cerebrovascular disease, 1403/141 58649/75.4% 32447/41.7% 19117/24.6% unspecified 24875/1518 1632/164 N20.1/calculus of ureter 17489/58.7% 9948/58.7% 12291/41.3% 15922/205 Table 2: Accuracy assessment of recommended diagnostic and treatment activities for nosology J13 based on a single-layer neural network. Total number of negative Number of recommendations/total Number of correct incorrect number Total number of clinical positive positive of positive recommendations precedents/number of recommendations recommendations Share of correct negative control precedents among among recommendations/share of control precedents MKB-10 code/nosology control precedents correct positive recommendations Absolute Absolute Number of neural network value/share in value/share in inputs/number of neural the total number the total number of Absolute value/percent network outputs (number of of positive positive controlled variables) recommendations recommendations J13/pneumonia due to 266/11 35567/841 339/40.31% 502/59.69% Streptococcus pneumoniae 224/222 98.55%/40.31% Let us emphasize that the volume of statistics on this (standards of the Russian Ministry of Health). Different illness stored in the DB has increased compared to an earlier insurance programs also limit integral values of controls. experiment involving the same nosology—from 166 to 266 Healthcare providers will not exceed these limits unless they completed clinical processes. Controls included all types of find it necessary. Formally, with respect to the model, it drug prescriptions (222 different pharmaceutical products in means that once an integral property of a control reaches a certain limit, it stops growing further or such growth is our case). Data normalization involved adjustment of pre- scribed dosages of pharmaceutical products to unified dose highly unlikely. +e gradient of the target function with respect to weights was calculated explicitly, and the steepest units. +e only monitored variable was “inpatient days.” Inputs also included bias. 49,728 weights had to be de- descent method was applied. Teaching included 1,006 de- termined. +e optimized target function was a quadratic scent steps. Criteria reflecting the accuracy of the neural residual between neural network output and control com- network are presented in Table 3. ponents monitored in control samples, adjusted to (0, 1). We +e relevant receiver operating characteristic (ROC) used a nonstandard neurons activation bell curve (Gaussian error curve is shown in Figure 6. function). +is choice of activation function was based on Results of the experiment based on a probabilistic neural the fact that integral values of many controls had apparent network are presented in Table 4. +e state vector dimension limits stipulated by Russian federal healthcare standards was equal to 639. +e control component included 125 8 Journal of Healthcare Engineering Table 3: Accuracy of recommended diagnostic and treatment Table 4: Accuracy of recommended diagnostic and treatment activities for nosology J13 based on a single-layer neural network activities for nosology J13 based on a probabilistic neural network with an activation threshold equal to 0.1. with σ  2.5. Absolute values (neuron activation threshold equal to 0.1) Absolute values (σ  2.5) TP 339 502 FP TP 233 191 FP TN 35,052 515 FN TN 35,376 608 FN Percent (neuron activation threshold equal to 0.1) Percent (σ  2.5) TP 40.31% 59.69% FP TP 55.0% 45.0 FP TN 98.55% 1.45% FN TN 98.31% 1.69% FN TP, true positive; FP, false positive; TN, true negative; FN, false negative. such modeling or knowledge extraction from data. Data are 1.2 extracted from the MIS without reduction, “as is.” It is assumed that the data contain signi�cant information 1.0 re±ecting medical knowledge and contemporary medical 0.8 treatment technologies. ree di“erent approaches to big 0.6 clinical data processing were examined: (1) case-based 0.4 reasoning for decision-making; (2) decision-making based 0.2 on a single-layer neural network; and (3) decision-making based on a probabilistic neural network. Experimental 0.0 0 0.2 0.4 0.6 0.8 1 1.2 calculations were performed to assess the accuracy of rec- FPR ommendations generated using di“erent approaches. Drawbacks of the above neural networks with respect to Figure 6: ROC error curve. the given problem were identi�ed. e overall accuracy of provided recommendations was rather high. Moreover, the diagnostic tests, 200 laboratory tests (di“erent kinds), 222 di“erent pharmaceuticals, 87 medical treatments, and 4 accuracy of negative recommendations that the neural networks learned to provide was very high (98–99%). controls classi�ed as “others.” e only monitored property However, the accuracy of positive recommendations pro- was “inpatient days.” e number of kernels (states) in the vided by the neural networks was not so high (40–55%, teaching sample of 266 processes was 4,361. e dimension of which is obviously insu›cient for successful practical ap- the state vector in the probabilistic neural network was almost plication). Another disadvantage of neural networks is their three times the dimension of the state vector in the single- rough and conservative nature, particularly when digesting layer neural network (639 versus 223). To make the results of isolated portions of new data with the volume insigni�cant both networks comparable, the output of the probabilistic compared to previously available data. neural network was considered to be the same as for the �rst e case-based approach to decision-making yielded neural network. e output was a vector with a dimension of more accurate recommendations (59–95%), which is su›- 222, related to prescription of di“erent pharmaceuticals. Both neural networks generated 36,408 positive and negative cient for its successful practical application. Another ad- vantage of the case-based approach is its sensitivity to new recommendations for the control sample. e experiment data. With respect to calculations, the case-based approach is involved one control parameter σ, a multiplier for a diagonal also more e›cient compared to other options under con- covariance matrix used in the kernel function (multivariate sideration as it ensures a high operating speed of the decision Gaussian distribution of independent random variables). A support system, thus making it acceptable for practical ap- value grid was predetermined for the parameter σ, and the plication. ese are the key �ndings of the study conducted. best value of the parameter was chosen based on experimental is o“ers encouraging prospects for designing and calculation results [22]. Calculations were performed for the developing decision support systems for physicians based on following values of σ: (0.1, 0.5, 1, and 2.5). e best results empirical components of medical knowledge. is approach were obtained for σ  2.5. ey are presented in Table 4. Let us also corresponds to existing case-based character of man- emphasize that standard deviation values of the state vector components calculated for the teaching sample were signif- agement and decision-making in medical practice. So far, the results indicate that precedent-based approach has a high icant and often exceeded average values. e multiplier equal e“ectiveness and could naturally enhance other approaches to 2.5 yields “wide” kernel functions (see the rightmost dis- to supporting physicians’ decision-making, particularly tribution in Figure 5). With “sharp” kernel functions (σ  0.1), knowledge-based ones. e obvious practical value of this the results were obviously worse. approach lies in the fact that it can be complementary to other knowledge-based approaches (clinical pathways, 4. Summary Evidence-Based Clinical Decision Support, expert systems, e focus of this paper was how to build a medical decision Watson, etc.). e doctor will be able to make decisions support system based on big clinical data. e authors review based on the best examples of medical practice, �nding general approaches to the problem that do not involve in- precedents of clinical cases close to the given case. e constraints of precedent-based approach include the dividual models for speci�c nosologies and neither do they require engagement of experts in the relevant subject area to need for a representative database of veri�ed precedents TPR Journal of Healthcare Engineering 9 [3] V. L. Malykh, S. V. Rudetskiy, and M. I. Khatkevich, Active excluding medical errors. From another perspective, pre- MIS. Information Technologies for the Physician, Publisher cedents with corrected errors are of particular interest to House “Public Health Manager”, Moscow, Russia, no. 6, pp. physicians training and further prevention of such errors. 16–24, 2016, in Russian. +e information about the results of these errors and pos- [4] I. S. Kohane, “+e twin questions of personalized medicine: sible ways of correcting them is also valuable. +us, who are you and whom do you most resemble?,” Genome precedent-based approach could be widely spread as an Medicine, vol. 1, no. 1, p. 4, 2009. educational tool. On the other hand, the precedent-based [5] S. N. Butko and V. K. Olshansky, New Decision Support approach does not imply formalization of medical knowl- Systems in Foreign Healthcare. Automation and Remote edge, which entails poor cognitive justification of generated Control, Springer, Berlin, Germany, 1990. recommendations. Consequently, justifications only de- [6] Y. B. Kotov, New Mathematical Approaches to Medical Di- scribe how other patients were treated in similar clinical agnostics, Editorial URSS, Moscow, Russia, 2004. cases. +ere are also problems with optimization of provided [7] R. Miotto, L. Li, B. A. Kidd, and J. T. Dudley, “Deep patient: an unsupervised representation to predict the future of patients metrics, compression of state descriptions, and construction from the electronic health records,” Scientific Reports, vol. 6, of training procedures. +ese problems are connected with no. 1, p. 26094, 2016. high dimensionality of the space of state characteristics and [8] V. L. Malykh and D. V. Belyshev, “Case-based reasoning in samples of clinical precedents. However, discussion of these clinical processes using clinical data banks,” in Proceedings of issues and possible ways of addressing them has been left 2015 International Conference on Biomedical Engineering and outside of this research [14]. Computational Technologies (SIBIRCON), Technopark of In further studies, we are going to focus on detailed Novosibirsk Akademgorodok, pp. 211–216, Novosibirsk, application of the case-based approach, analyze metrics, and Russia, 2015. distances not only for pairs of vectors but also for pairs of [9] E. Herrett, A. M. Gallagher, K. Bhaskaran et al., “Data re- vector sequences, and examine issues concerned with in- source profile: clinical practice research datalink (CPRD),” telligent normalization of primary data and data extraction International Journal of Epidemiology, vol. 44, no. 3, from plain texts of medical documents. pp. 827–836, 2015. [10] T. Rotter, L. Kinsman, E. James et al., “Clinical pathways: effects on professional practice, patient outcomes, length of Disclosure stay and hospital costs,” Cochrane Database System Review, vol. 3, p. CD006632, 2010. UDC 007.52 (Automatically operated systems without any [11] AHRQ, Clinical Decision Support Systems: State of the Art- humans among system links, robots, and automated machines). Agency for Healthcare Research and Quality U.S, AHRQ Publication No. 09-0069-EF, Rockville, MD, USA, 2009. [12] I. V. Efimenko and V. F. Khoroshevsky, “Intelligent decision Conflicts of Interest support systems in medicine: state of the art and beyond in Russian,” in Proceedings of OSTIS, Open Semantic Technol- +e authors declare that they have no conflicts of interest. ogies for Intelligent Systems, Minsk, Belarus, 2017, http:// proc.ostis.net/proc/Proceedings%20OSTIS-2017.pdf. Acknowledgments [13] Wikipedia, Clinical Decision Support System, http://en. academic.ru/dic.nsf/enwiki/1126747, 2017. +e authors would like to thank Professor V. M. Khachumov, [14] V. L. Malykh, I. N. Kononenko, and S. V. Rudetskiy, “Esti- a doctor of engineering sciences, and V. P. Fralenko, mation of accuracy of recommended diagnostic and treat- a candidate of engineering sciences, for the consulting ment actions based on precedent approach,” in Proceedings of support on neural networks and teaching methods, as well the International Conference e-Health 2016, pp. 52–58, Ma- as Professor N. N. Nepevoda, a doctor of physical and deira, Portugal, July 2016. [15] F. Caron, J. Vanthienen, and B. Baesens, “Healthcare ana- mathematical sciences, for the discussion and assessment of lytics: examine the diagnosis-treatment cycle,” Procedia the outcomes. Some of the outcomes presented in the paper Technology, vol. 9, pp. 996–1004, 2013. were achieved earlier under the support of the Ministry of [16] E. Vasilyeva, Industrial Internet of @ings (IoT), Rational Education and Science of the Russian Federation (Project Enterprise Management, Athens, GA, USA, 2015, in Russian. RFMEFI60714X0089) and in the context of Grant 13-07- [17] V. L. Malykh and Y. I. Guliev, “Controlled stochastic precedent 12012 provided by the Russian Foundation for Basic process with memory as a mathematical model of the di- Research. agnostic and treatment process,” Information Technologies and Computational Systems, vol. 2, pp. 62–72, 2014, in Russian. [18] V. L. Malykh, Y. I. Guliev, A. V. Eremin, and S. V. Rudetskyi, References “Management and decision making in clinical processes,” in Proceedings of XII All-Russian Conference on Problems of [1] MedSoft, “Medical software,” in Conference Presentations of the 12th International Forum MedSoft-2016, in Russian, Management of VSPU-2014, pp. 6518, Moscow, Russia, 2014, in Russian. Moscow, Russia, 2016, http://www.armit.ru/medsoft/2016/ conference/prog. [19] V. L. Malykh and Y. I. Guliev, “Precedent approach to decisionaking in clinical processes,”, in MEDINFO 2015: [2] B. S. Handler, J. +omas, and B. R. Hieb, Gartner’s 2007 Criteria for the Enterprise CPR, 2014, http://rsept.wikispaces. Health-nabled Health, Studies in Health Technology and In- formatics, Vol. 216, Ed., IMIA and IOS Press, Amsterdam, com/file/view/Gartner_Criteria_for_the_Enterprise_CPR_ 2007.pdf. Netherlands, 2015. 10 Journal of Healthcare Engineering [20] C. Bennett and K. Hauser, “Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach,” Artificial Intelligence in Medicine, vol. 57, no. 1, pp. 9–19, 2013. [21] Y. Malkov, A. Ponomarenko, A. Logvinov, and V. Krylov, “Approximate nearest neighbor algorithm based on navigable small world graphs,” Information Systems, vol. 45, pp. 61–68, [22] R. N. Kvetniy, V. V. Kabachiy, and O. O. Chumachenko, Probabilistic Neural Networks in Time Series Identification, Information Technologies and Computers, Scientific works of Vinnytsia National Technical University, Vinnytsia, Ukraine, 2010, in Russian. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal

Journal of Healthcare EngineeringHindawi Publishing Corporation

Published: Jun 4, 2018

There are no references for this article.