Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Serious Game–Derived Index for Detecting Children With Heterogeneous Developmental Disabilities: Randomized Controlled Trial

A Serious Game–Derived Index for Detecting Children With Heterogeneous Developmental... Background: Developmental disabilities are a set of heterogeneous delays or difficulties in one or more areas of neuropsychological development. Considering that childhood is an essential stage of brain development and developmental delays lead to personal or social burdens, the early detection of childhood developmental disabilities is important. However, early screening for developmental disabilities has been a challenge because of the fear of positive results, expensive tests, differences in diagnosis depending on examiners’ abilities, and difficulty in diagnosis arising from the need for long-term follow-up observation. Objective: This study aimed to assess the feasibility of using a serious game–derived index to identify heterogeneous developmental disabilities. This study also examines the correlation between the game-derived index and existing neuropsychological test results. Methods: The randomized controlled trial involved 48 children with either normal development or developmental disabilities. In this clinical trial, we used 19 features (6 from the Korean-Wechsler Preschool and Primary Scale of Intelligence, 8 from the Psychoeducational Profile Revised, 2 from the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition, and 3 from the Pediatric Evaluation of Disability Inventory) from neuropsychological tests and 9 (7 game scores, path accuracy, and completion rate) from the serious game, DoBrain. The following analysis was conducted based on participants’ baseline information and neuropsychological test and game-derived index data for one week: (1) we compared the baseline information between the normal development and developmental disabilities groups; (2) then we measured the correlation between the game-derived index and the neuropsychological test scores for each group; and (3) we built a classifier based on the game-derived index with a Gaussian process method and then compared the area under the curve (AUC) with a model based on neuropsychological test results. Results: A total of 16 children (normal development=9; developmental disabilities=7) were analyzed after selection. Their developmental abilities were assessed before they started to play the serious games, and statistically significant differences were found in both groups. Specifically, the normal development group was more developed than the developmental disabilities group in terms of social function, gross motor function, full-scale IQ, and visual motor imitation, in that order. Similarly, the normal development group obtained a higher score on the game-derived index than the developmental disabilities group. In the correlation analysis between the game-derived index and the neuropsychological tests, the normal development group showed greater correlation with more variables than the developmental disabilities group. The game-derived index–based model had an AUC=0.9, a similar detection value as the neuropsychological test–based model’s AUC=0.86. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Conclusions: A game-derived index based on serious games can detect children with heterogenous developmental disabilities. This suggests that serious games can be used as a potential screening tool for developmental disabilities. Trial Registration: Clinical Research Information Service KCT0003247; https://cris.nih.go.kr/cris/en/search/search_result_st01 .jsp?seq=12365 (JMIR Serious Games 2019;7(4):e14924) doi: 10.2196/14924 KEYWORDS serious game; developmental disabilities; mobile game; cognitive screening tool; machine learning developmental disabilities, lack of time among specialists, fear Introduction of positive screen results, and failure to consider testing for developmental disabilities necessary [8]. In fact, even when Developmental disabilities are among the most common diseases early diagnosis and intervention programs are conducted in a in children younger than 5 years old. Developmental disabilities timely manner, many instruments for diagnosis are administered are a set of heterogenous delays or difficulties in one or more in an environment distinct from an everyday setting. For developmental milestones, including learning, self-care, social example, children are directed to perform tasks in a laboratory interactions, and movement. The global number of children setting or under the supervision of unfamiliar investigators. under 5 years old with developmental disabilities was 52.9 million in 2016, accounting for 13.3% of total years living with To resolve these problems, approaches using serious games disability for these children. In 1990, 53.0 million children were have been proposed for the early detection of developmental living with developmental disabilities, indicating that there has disabilities in children. Serious games are games that do not not been much change since then [1]. In the United States, the have enjoyment, entertain[ment] or fun as their primary purpose prevalence of children diagnosed with developmental disabilities [9]. increased remarkably from 5.76% in 2014 to 6.99% in 2016 Serious games targeted for health care use have seen a surge in [2]. use since 2004 [10]. Health care–targeted serious games are The brain is sensitive to stimulation during childhood, which intended for health checks, disease detection, or rehabilitation. is an essential stage of human development. It is the foundation Some studies have been conducted that investigated games for successive educational and vocational achievements, as well created for detecting various disorders, such as Parkinson as society’s human capital development [3]. However, children disease, Alzheimer disease, or early dementia [11-14]. Other with developmental disabilities are at a higher risk of serious games have been targeted towards detecting substandard educational accomplishment, health status, and developmental disabilities in children. For example, Anzulewicz social relationships. More specifically, children with created a machine learning model that can identify children with developmental disabilities have heightened difficulties reading, autism [15]. Alchalabi also studied ways to detect children with spelling, and counting due to shortages of phonological attention deficit hyperactivity disorder (ADHD) using a serious short-term memory or central, executive-loaded, working game integrated with electroencephalogram signals [16]. memory [4]. Additionally, developmental disabilities lead to Additionally, many researchers have targeted adult cognitive sedentary lifestyles and seven times greater reported substandard impairment detection [17-21]. emotional support in adulthood than in adults without However, thus far there have been no studies about detecting disabilities. Consecutive developmental disabilities are risk general and heterogeneous developmental disabilities in children factors of chronic health conditions such as high blood pressure, using a serious game, despite how important it is to detect these cardiovascular disease, diabetes, and chronic pain [5]. developmental disabilities. Referring children to specialists for If a child fails to achieve a certain milestone, it is extremely clinical evaluation is also the most essential part of an early hard to recover that milestone later in life. Therefore, late diagnosis of developmental disabilities. Therefore, this study identification of developmental disabilities in children may aimed to develop a classifier that can distinguish between require schools and families to pay for expensive programs [6]. children with heterogeneous developmental disabilities and Additionally, an understanding of the subsequent characteristics those with normal development using a game-derived index of a child’s developmental process requires a sensitive based on a serious game for improving cognitive ability. In this longitudinal examination by caregivers or specialists. study, we collected gameplay data of children from Do Brain, a smart device-based serious game intended for the cognitive Given such a context, identifying developmental disabilities as enhancement of children. early as possible has been deemed crucial. Multiple screening programs occurring in conjunction are recommended to detect Methods early-stage developmental disabilities. For example, the American Academy of Pediatrics recommends frequent Summary counseling, preventive care visits, and treatment visits for This study was based on a single-blinded, parallel randomized children [7]. Nevertheless, the early detection of developmental controlled trial under the supervision of an independent data disabilities in children has not been performed adequately for management team in the Asan Medical Center (Seoul, South many reasons, such as a lack of specific training for screening http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Korea) The study consists of an intervention and a control group, There were no human interventions other than early, functional, with an allocation ratio of 1:1. Participants were openly recruited game usage training. To encourage sustainable use, we sent a and enrolled in clinical studies via face-to-face evaluation by text message to the representatives of participants who had not physicians and inspectors of neuropsychological testing. used it once in the preceding week. All clinical trial data were Children with normal development were 5-6 years old and were documented, coded, and stored on a computer at Asan Medical confirmed to have developed normally via examinations and Center in Seoul, Korea. specialists. Children with developmental disabilities were 5-7 In the current study, we wanted to explore the possibility of years old but had a cognitive age of 4-6 years old. The differentiating children with developmental disabilities from recruitment period for this study was from October 2018 to the normal population by analyzing gameplay patterns. We used January 2019. If the child was assigned to a group after the the results obtained in the first week of the intervention because, face-to-face evaluation, intervention was provided for 6 months. as the intervention progressed, the cognitive function of the Our study only used the results obtained in the first week of the participants changed. This is because the cognitive function of intervention. children changes much faster than that of adults, and the game Participants were too young to give consent, so we received Do Brain was intended to enhance the cognitive function of “Subject explanatory note and consent forms” from the children. The intervention was designed for a particular time participants’ representatives (see Multimedia Appendix 1). The and frequency of play (40 minutes twice a week). randomization, using a block size of four, was stratified Initially, 106 participants were recruited. The medical staff depending on intervention and developmental disability. screened 48 children who met the recruitment criteria. Therefore, Randomization was done with the use of opaque, sealed the study group consisted of 48 children with either normal envelopes. The statistician of the data management team development or developmental disabilities (Figure 1). generated the randomly allocated sequence with the use of the Twenty-two of the children underwent intervention for 12 weeks R program (R Core Team, Vienna, Austria). Physicians enrolled (experimental group) and the other 26 participants were the the patients and opened the envelope with the lowest available control group. The control group that did not undergo registration number within the appropriate stratum. Participants intervention and children who could not carry out level C of knew the group to which they belonged, but the physicians and the serious game were excluded from this study. The baseline the inspectors of the neuropsychological tests were blinded until information and data of 16 eligible participants during the first the end of the trial. week of the intervention were analyzed. This study was The intervention of this study was based on a serious game and approved by the Institutional Review Board of Asan Medical continued for 12 weeks for 40 minutes twice a week. The Center (IRB #2018-0989; Seoul, South Korea). intervention was based on mobile serious gameplay at home. Figure 1. Participant selection flow chart. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al at home. A more detailed description is available at Do Brain Neuropsychological Tests homepage [26]. A set of neuropsychological tests was conducted to assess children’s developmental ability before the intervention period. Game-Derived Index This study employed the Korean-Wechsler Preschool and In the program, there are sections that each contain 6-8 games. Primary Scale of Intelligence (K-WPPSI-IV) [22], a child’s Each game of a section tests for each of the categories of intelligence test that provides a comprehensive assessment of development, which includes: Spatial Perception, Mathematical overall intellectual capabilities. The target age of the test is from Thinking, Attention Memory, Logical Reasoning, Constructional 1 year and 6 months to 7 years and 7 months. This study also Ability, Discernment, and Reaction (Figure 2). Each game has used the Full-Scale IQ, which provides five basic indicators of 1-3 stages. In the first week of the intervention participants intellectual function of a particular cognitive domain, including played through nine sections, which consisted of 61 games and Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), 128 stages. For the first week of the intervention we computed Fluid Reasoning Index (FRI), Working Memory Index (WMI), the game score, path accuracy, and completion rate. and the Processing Speed Index (PSI). In addition, the Psychoeducational Profile Revised (PEP-R) [23], which us used The game score was calculated using both the duration and the to assess the treatment capacity of children between 1 and 7.5 incorrect answer count. Duration referred to the time it took a years old with autism and related developmental disabilities, participant to answer correctly, and incorrect answer count to was used to plan treatment programs. Additionally, the the number of wrong answers/attempts. The game score Bruininks-Oseretsky Test of Motor Proficiency, Second Edition increased as the participant completed a certain stage within a (BOT-2) [24], which is administered in participants aged shorter time and with fewer incorrect answers. Therefore, the between 4 and 22 years, was used to check their motor higher the score a participant obtained, the better they played. development; it can also measure large and small muscle skills. Below is the equation for the game score: The age range for the Pediatric Evaluation of Disability Inventory (PEDI) [25] is from 6 months to 7.5 years, and this test is used as a tool for evaluating independence in daily life through structured interviews with parents or caregivers. There were 16 drag-and-drop games in the 128 stages of the Serious Game first week of the intervention, for which we computed path Do Brain is a serious game based on an animated cartoon. The accuracy from both intended path and actual path. Intended path game is a smart device–based application certified for child referred to the geometric distance between the start and end suitability from the iOS App store and the Google Play Store. points and actual path to the distance of the finger’s movement. It consists of games that require simple touch inputs, such as The path accuracy value ranged from 0 to 1, with values closer one-point touch, drag and drop, and rub. The object of the game to 1 indicating that a participant moved their finger more is to enhance primary cognitive capacity (attention, orientation, precisely. Below is the equation for path accuracy: memory), higher-level thinking abilities (problem solving, reasoning, concept formation), and meta-processing abilities (executive function, self-awareness). The application was downloaded to the personal smart device of each participant’s caregiver from the iOS App store or Google Play Store. The Meanwhile, completion rate was the ratio of the number of participants played the game in a natural environment, such as games played by each participant in a week divided by the quota for that week. It measured a participant’s compliance. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Figure 2. Screenshots of the serious game by game categories. heterogeneous developmental disabilities. A classification model Statistical Tests was built with a Gaussian process classifier, and model In our study, we compared the baseline information between validation using leave-one-out cross validation (LOOCV) was the normal development and developmental disabilities groups conducted. LOOCV is a method for evaluating the prediction using the Mann-Whitney U test. We then measured the quality of a model built from a small dataset [27,28]. For the correlation between the game-derived index and a child’s game-derived index–based model, we used nine features (7 neuropsychological test scores for each group using Pearson’s game scores, path accuracy, and completion rate), and 19 correlation method. Finally, we assessed the feasibility of using features for the neuropsychological test–based model (6 this serious game as a tool for detecting children with K-WPPSI, 8 PEP-R, 2 BOT-2, 3 PEDI). In this study the http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al multi-variate Gaussian process classifier of Rasmussen and proportion of males in the developmental disabilities group was Williams was used [29]. The classifier is based on Laplace higher than in the normal development group. The K-WPPSI-IV approximation and makes predictions based on finite test results, which measured a child’s cognitive ability, showed combinations of all random variables that have multivariate that full scale IQ was significantly different between the normal normal distributions. The receiver operating characteristic development and developmental disabilities groups (P=.008). (ROC) and precision-recall (PR) curves were drawn from the Moreover, the normal development group surpassed the validation results, and then the area under the curve (AUC) was developmental disabilities group in all subcategories of calculated for each curve. Sensitivity, specificity, precision, true K-WPPSI-IV. In particular, the normal development and positive rate, and true negative rate were also measured. We set developmental disabilities groups showed statistically significant a significance level of 0.05. Data were processed and analyzed differences in VCI (P=.01), VSI (P=.01), and FRI (P=.01; Table using R version 3.5.0 (R Core Team, Vienna, Austria), and 1). Python 3.6 (Python Software Foundation, Wilmington, The developmental ages of the normal development and Delaware, United States; including the Pandas 0.22.0, NumPy developmental disabilities groups were significantly different 1.14.3, and Jupyter 1.0.0 packages). (P=.007). For all subcategories of PEP-R, the normal development group outperformed the developmental disabilities Results group. Specifically, statistically significant differences in Cognitive Verbal (P=.01), Cognitive Motion (P=.01), Fine Participant Characteristics Motor (P=.01), and Imitation (P=.03) categories were observed. Baseline Information Moreover, Visual Motor Imitation (P=.009) and Gross Motor (P=.005) showed more significant differences than the other The normal development and developmental disabilities groups categories (Table 1). For the data presented in Table 1, the were comprised of nine and seven children, respectively. Mann-Whitney U test was used as the statistical significance Chronological age was not significantly different between the test. two groups, but sex was significantly different in that the http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Table 1. Comparison of the baseline information and neuropsychological tests of the normal development and developmental disability groups. Normal development group (n=9) Developmental disability group (n=7) P value Sex, n (%) Male 4 (44.4) 6 (85.7) Female 5 (55.6) 1 (14.3) — Chronological age, mean (SD) 71.4 (6.20) 70.1 (8.01) .35 K-WPPSI-IV , mean (SD) Full scale IQ 110.7 (18.3) 86.7 (13.7) .008 Verbal comprehension index 113.2 (16.7) 89.1 (15.7) .02 Visual spatial index 109.4 (13.8) 89.1 (19.0) .01 Fluid reasoning index 104.6 (11.6) 89.0 (13.7) .01 Working memory index 118.4 (16.8) 106.1 (15.5) .14 Processing speed index 99.3 (16.2) 88.1 (14.5) .14 PEP-R , mean (SD) Developmental age 68.7 (5.0) 62.0 (5.4) .007 Cognitive verbal 73.6 (4.3) 64.4 (8.5) .01 Cognitive motion 70.0 (4.8) 63.4 (6.1) .01 Visual motor imitation 67.2 (6.2) 58.6 (5.4) .009 Gross motor 59.6 (0.9) 52.4 (11.7) .005 Fine motor 63.3 (4.2) 56.6 (5.4) .01 Perception 59.4 (2.9) 58.5 (2.9) .07 Imitation 63.1 (3.5) 60.6 (3.5) .03 BOT-2 , mean (SD) Fine motor 46.4 (19.9) 29.1 (13.8) .05 Manual coordination 30.2 (9.0) 22.3 (8.8) .08 PEDI , mean (SD) Self-care 70.5 (1.5) 66.0 (5.6) .10 Movement 57.0 (3.7) 54.4 (5.6) .20 Social function 62.7 (2.5) 56.7 (4.6) .005 Not applicable. K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence. PEP-R: Psychoeducational Profile Revised. BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition. PEDI: Pediatric Evaluation of Disability Inventory. and Attention Memory (P=.04) were statistically significant in Game-Derived Index the two groups. In all subcategories of the game-derived index, Overall, the normal development group showed better game the developmental disabilities group’s standard deviation was scores than the developmental disabilities group on the larger than in the normal development group. For the data game-derived index (Table 2). Logical Reasoning (P=.03), presented in Table 2, the Mann-Whitney U test was used as the Constructional Ability (P=.04), Mathematical Thinking (P=.01), statistical significance test. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Table 2. Comparison of the game-derived index between the normal development and developmental disability groups. Variable Normal development group (n=9) Developmental disability group (n=7) P value Game score, mean (SD) Logical reasoning 8.28 (0.66) 7.16 (1.16) .03 Constructional ability 7.82 (0.69) 7.04 (1.01) .04 Mathematical thinking 9.12 (0.54) 8.13 (0.88) .01 Attention memory 9.20 (0.68) 8.50 (0.98) .04 Spatial perception 8.67 (0.58) 7.76 (0.97) .06 Reaction 8.31 (0.78) 7.89 (0.30) .05 Discernment 8.71 (0.55) 8.07 (1.07) .19 Path accuracy, mean (SD) 0.795 (0.089) 0.750 (0.072) .17 Completion rate, mean (SD) 0.991 (0.026) 0.910 (0.203) .10 Game-Derived Index– and Neuropsychological Correlations Between the Neuropsychological Tests Test–Based Classifiers and Game-Derived Index As shown in Figures 4 and 5, the game-derived index–based, Correlations between the neuropsychological tests and classifier has a similar AUC to the neuropsychological game-derived index results were computed for each group using test–based classifier. Sensitivity and specificity showed similar Pearson’s correlation method (Figure 3). Overall, the patterns. The game-derived index classifier had a sensitivity of game-derived index and neuropsychological tests showed higher 0.714 and a specificity of 0.778, whereas the neuropsychological correlation values for the normal development group than the test–based classifier had a sensitivity of 0.857 and a specificity developmental disabilities group. Logical reassignment and of 0.778. completion rate in the game-derived index showed a high correlation with neuropsychological tests. In the developmental No serious adverse events or side effects were observed in the disabilities group, the constructional ability of the game-derived normal development and developmental disabilities groups. index showed the highest correlation with VSI (0.92). Figure 3. Correlation matrix for the game-derived index and neuropsychological tests of the normal development and developmental disability groups. K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence; PEP-R: Psychoeducational Profile Revised; BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition; PEDI: Pediatric Evaluation of Disability Inventory. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Figure 4. The receiver operating characteristic and precision-recall curves for leave-one-out cross validation of the model based on the game-derived index from the serious game. ROC: receiver operating characteristic; DD: developmental disabilities; ND: normal development; PR: precision recall; GDI: game-derived index. Figure 5. The receiver operating characteristic and precision-recall curves for leave-one-out cross-validation of the model based on the neuropsychological tests. ROC: receiver operating characteristic; DD: developmental disabilities; ND: normal development; PR: precision recall. the positive diagnosis of developmental disabilities should be Discussion conducted through a specialist’s exam, our classifier that detects general and heterogeneous developmental disabilities could be Principal Results used to refer children with developmental disabilities to We built a classifier from the game-derived index that could specialists for more specific and expert exams. distinguish children with developmental disabilities from those Additionally, we compared the game-derived index with the with normal development. In contrast to some models that can results of the neuropsychological tests to determine the detect a specific type of disease, such as autism or ADHD, our characteristics of the normal development group that helped classifier is the first model to detect children with heterogeneous them obtain a better game-derived index than the developmental types of developmental disabilities. This suggests that serious disabilities group. As many serious games that assess the games have the potential for screening the general cognitive function of humans lack psychometric analysis and developmental disability status of children. Considering that http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al comparisons with standardized studies [30], the comparison intentional path. However, the normal development group’s between the game-derived index and the neuropsychological path accuracy did not correlate well with Fine Motor index of tests in this study provides a more comprehensive interpretation PEP-R and BOT-2, mainly because testing of PEP-R or BOT-2 of the serious gameplay of children. differs from a drag-and-drop game. Moreover, the developmental disabilities group’s path accuracy had negative Differences in Game-Derived Index Between Normal correlation with the neuropsychological tests’ VSI and Fine Development and Developmental Disabilities Groups Motor index. This negative correlation can be attributed to the Game score, path accuracy, and completion rate were computed developmental disabilities group’s heterogeneity and small to assess participants’ gameplay. Although some features of the sample size. game-derived index lacked statistical significance, the overall Completion rate is a measure of participant compliance. In this pattern showed that the normal development group played better serious game, participants may skip a stage when they find it than the developmental disabilities group. difficult. As can be observed in the game score results, the Game score represents how fast and correctly a participant normal development group outperformed the developmental solved a problem. The serious game used in this study is based disabilities group in gameplay. Additionally, the normal on a cognitive counseling program for children. As the normal development group’s mean completion rate was higher than the development group showed higher intelligence scores on the developmental disabilities group’s. Notably, the standard neuropsychological tests, this means that they answered deviation differed greatly between the normal development correctly with fewer attempts and in less time. For the normal (0.026) and developmental disabilities (0.203) groups. The large development group, many game-derived index features showed standard deviation of the developmental disabilities group’s a positive linear correlation with neuropsychological tests. More completion rate is due to some developmental disabilities specifically, Logical Reasoning was correlated with Full Scale participants skipping many more stages than the others. IQ and Developmental Age with coefficients of 0.86 and 0.85, Classification Model respectively. The model built from the game-derived index performed By contrast, most game-derived index features of the adequately compared with that built from neuropsychological developmental disabilities group did not show linear correlations tests, showing the discriminating power of the game-derived or had negative correlations. These results can be attributed to index. This result suggests that serious games have the potential the following reasons. First, the developmental disabilities were for detecting children with developmental disabilities. As heterogenous in the developmental disabilities group. Also, the children with heterogeneous developmental disabilities show developmental disabilities group had a small, restricted sample. complicated characteristics, characterizing them with high Moreover, some studies indicate that cognitive assessment is sensitivity is difficult. In contrast to models that detect specific difficult for patients suffering from neuropsychological disorders developmental disorders developed in previous studies, the [31-33]. More specifically, when converting raw scores to model built in this study is the first to detect general standardized scores, performance variations that need to be developmental disabilities. detected often become obscure due to the flooring effect [34,35]. The flooring effect refers to a phenomenon in which the Comparison With Previous Studies measuring tool cannot discriminate among those who belong Some studies have been conducted regarding developmental to the lower level of the characteristic to be measured [36]. The delay detection with serious games. Previous works utilized flooring effect may occur when the score range itself is limited, more devices, required more resources, or could detect only or when inspection is too difficult. As standardized tests have specific diseases. such limitations, recent trends in intellectual disability research Alchalabi’s study that detected ADHD patients with an emphasizes narrating the cognitive signatures of conditions Electroencephalogram (EEG)-based serious game [16] required throughout their lifetime rather than depending on test scores an EEG reader and a special gaming setting. However, our study [31]. Although many game-derived index features did not have only utilizes smart devices and is done in the home setting. Like positive correlations, constructional ability, spatial perception, our study’s use of neuropsychological tests for clinical diagnosis, and completion rate had correlation coefficients of 0.69, 0.67, Alchalabi’s study was also meaningful in that it used EEG with and 0.62, respectively. some diagnostic value. Path accuracy indicates fine motor function and visuospatial Elhady build a speech disability detection game with limited ability. According to Vatavu, children who are better developed speech resources [38]. Our study uses touch input from a smart in visuospatial processing have a high path accuracy score [37]. device while it records the voice. However, they targeted a very In our study, the normal development group’s path accuracy specific speech disability population suffering from the score was higher than the developmental disabilities group, but phonemes /s/ and /r/. There is a need to cover broader speech the difference was not statistically significant. Path accuracy disabilities, as there exist heterogenous speech disabilities [39]. scores were highly correlated with the VSI of the K-WPPSI in the normal development group (r=0.87), which shows a similar Garcia and Ruiz created a children’s psychomotor delay pattern to Vatavu’s findings. These findings indicate that screening process called Ubiquitous Detection Ecosystem to children with better VSI scores are better at interpreting Care and Early Stimulation for Children with Developmental geometrical relationships. As a result, they understand Disorders (EDUCARE) [40] based on a smart toy. In drag-and-drop games better and can move their finger in a more http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 10 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al EDUCARE, the video that children played on the toy had been In a clinical situation, a neuropsychological test is essential for analyzed by developmental experts. However, EDUCARE only the diagnosis of developmental disabilities. Therefore, the assessed motor function and depended on lengthy screening by estimation of neuropsychological test results with a experts. Our study overcomes this by assessing more game-derived index could provide caregivers and specialists developmental areas and immediately giving numerical with more information. However, our study did not build a developmental indices to caregivers. In a different respect, model that estimates neuropsychological test results from the EDUCARE utilized methods more related to everyday objects, game-derived index because of the small sample size. Future such as stackable cubes, than to our game-based approach. studies must build a neuropsychological test estimation model more complex than our simple linear correlation model. Limitations Conclusion Our study population is insufficient for generalizability of our findings to the general population. However, our findings show The game-derived index patterns observed in serious gameplay the feasibility of using this serious game to screen for general in the normal development and developmental disabilities groups and heterogeneous developmental disabilities. As children were different. In particular, the neuropsychological tests and continue to develop in various areas in the earlier stages of game-derived index have significant differences in the developmental disabilities, a definite diagnosis of the disease correlations between groups. This result is a potential indicator is difficult. Our model is useful because of the flexibility and that game-derived index can distinguish developmental heterogeneous characteristics of developmental disabilities. In disabilities from normal development, and the model based on the future, we will widen our study population to enhance the it showed similar performance to neuropsychological tests that generalizability of our findings. constitute conventional developmental ability tests. This suggests that serious games can be used as a potential screening tool for developmental disabilities. Acknowledgments CB is supported by the Seok-San Biomedical Science Scholarships, Yonsei University College of Medicine. This study was supported by the student research fund from Yonsei University College of Medicine in 2019, by the Basic Science Research Program through the National Research Foundation of Korea (NRF), which is funded by the Ministry of Education (NRF-2017R1D1A1B03035762), and by the Foundational Technology Development Program (NRF-2019M3E5D4064682) through the Ministry of Science and ICT in the Republic of Korea. Conflicts of Interest WL, BK, and YC are employees of Do Brain Co, Ltd. Multimedia Appendix 1 Informed consent for the randomized controlled trial. [DOCX File , 55 KB-Multimedia Appendix 1] Multimedia Appendix 2 CONSORT‐EHEALTH checklist (V 1.6.1). [PDF File (Adobe PDF File), 3666 KB-Multimedia Appendix 2] References 1. Global Research on Developmental Disabilities Collaborators. Developmental disabilities among children younger than 5 years in 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Glob Health 2018 Oct;6(10):e1100-e1121 [FREE Full text] [doi: 10.1016/S2214-109X(18)30309-7] [Medline: 30172774] 2. Zablotsky B, Black LI, Blumberg SJ. Estimated Prevalence of Children With Diagnosed Developmental Disabilities in the United States, 2014-2016. NCHS Data Brief 2017 Nov(291):1-8 [FREE Full text] [Medline: 29235982] 3. Shonkoff JP, Phillips DA. From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press; 2000. 4. Henry L, Winfield J. Working memory and educational achievement in children with intellectual disabilities. J Intellect Disabil Res 2010 Apr;54(4):354-365. [doi: 10.1111/j.1365-2788.2010.01264.x] [Medline: 20236191] 5. Havercamp SM, Scandlin D, Roth M. Health disparities among adults with developmental disabilities, adults with other disabilities, and adults not reporting disability in North Carolina. Public Health Rep 2004;119(4):418-426 [FREE Full text] [doi: 10.1016/j.phr.2004.05.006] [Medline: 15219799] 6. The importance of caregiver-child interactions for the survival and healthy development of young children. In: Developmental difficulties in early childhood. Boston, US: World Health Organization; 2012. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 11 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al 7. Committee On Practice and Ambulatory Medicine, Bright Futures Periodicity Schedule Workgroup. 2019 Recommendations for Preventive Pediatric Health Care. Pediatrics 2019 Mar 25;143(3):e20183971. [doi: 10.1542/peds.2018-3971] [Medline: 30804073] 8. Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, Landes C. Developmental Stages of Developmental Screening: Steps to Implementation of a Successful Program. Am J Public Health 2005 Nov;95(11):1928-1932. [doi: 10.2105/ajph.2004.052167] 9. Michael D, Chen S. Serious Games: Games that Educate, Train and Inform. Boston: Thomson Course Technology; 2006. 10. Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertainment Computing 2013 Dec;4(4):231-247. [doi: 10.1016/j.entcom.2013.09.002] 11. van der Meulen E, Cidota MA, Lukosch SG, Bank PJM, van der Helm AJC, Visch VT. A Haptic Serious Augmented Reality Game for Motor Assessment of Parkinson's Disease Patients. 2016 Presented at: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct); 19-23 Sept. 2016; Merida p. 104. 12. Vallejo V, Wyss P, Rampa L, Mitache AV, Müri RM, Mosimann UP, et al. Evaluation of a novel Serious Game based assessment tool for patients with Alzheimer's disease. PLoS One 2017;12(5):e0175999 [FREE Full text] [doi: 10.1371/journal.pone.0175999] [Medline: 28472049] 13. Tarnanas I, Laskaris N, Tsolaki M, Muri R, Nef T, Mosimann UP. On the comparison of a novel serious game and electroencephalography biomarkers for early dementia screening. Adv Exp Med Biol 2015;821:63-77. [doi: 10.1007/978-3-319-08939-3_11] [Medline: 25416111] 14. Zucchella C, Sinforiani E, Tassorelli C, Cavallini E, Tost-Pardell D, Grau S, et al. Serious games for screening pre-dementia conditions: from virtuality to reality? A pilot project. Funct Neurol 2014;29(3):153-158 [FREE Full text] [Medline: 25473734] 15. Anzulewicz A, Sobota K, Delafield-Butt JT. Toward the Autism Motor Signature: Gesture patterns during smart tablet gameplay identify children with autism. Sci Rep 2016 Aug 24;6(1):31107 [FREE Full text] [doi: 10.1038/srep31107] [Medline: 27553971] 16. Alchalabi AE, Shirmohammadi S, Eddin AN, Elsharnouby M. FOCUS: Detecting ADHD Patients by an EEG-Based Serious Game. IEEE Trans. Instrum. Meas 2018 Jul;67(7):1512-1520. [doi: 10.1109/tim.2018.2838158] 17. Valladares-Rodriguez S, Perez-Rodriguez R, Facal D, Fernandez-Iglesias MJ, Anido-Rifon L, Mouriño-Garcia M. Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults. PeerJ 2017;5:e3508 [FREE Full text] [doi: 10.7717/peerj.3508] [Medline: 28674661] 18. Vallejo V, Wyss P, Chesham A, Mitache AV, Müri RM, Mosimann UP, et al. Evaluation of a new serious game based multitasking assessment tool for cognition and activities of daily living: Comparison with a real cooking task. Computers in Human Behavior 2017 May;70:500-506. [doi: 10.1016/j.chb.2017.01.021] 19. Tong T, Chignell M, Tierney MC, Lee J. A Serious Game for Clinical Assessment of Cognitive Status: Validation Study. JMIR Serious Games 2016 May 27;4(1):e7 [FREE Full text] [doi: 10.2196/games.5006] [Medline: 27234145] 20. Byun S, Park C. Serious Game for Cognitive Testing of Elderly. In: HCI 2011: HCI International 2011–Posters’ Extended Abstracts. Berlin: Springer; 2019:354-357. 21. Boletsis C, McCallum S. Smartkuber: A Serious Game for Cognitive Health Screening of Elderly Players. Games Health J 2016 Aug;5(4):241-251. [doi: 10.1089/g4h.2015.0107] [Medline: 27192473] 22. Park H, Lee K. Intellectual Characteristics of Korean Children: Analyses of K-WPPSI-IV Performance. Korean J Child Stud 2016 Dec 31;37(6):157-168. [doi: 10.5723/kjcs.2016.37.6.157] 23. Schopler E. Psychoeducational Profile Revised (PEP-R). In: Encyclopedia of Autism Spectrum Disorders: 5 Volume Set. New York: Springer; 2019. 24. Deitz JC, Kartin D, Kopp K. Review of the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition (BOT-2). Phys Occup Ther Pediatr 2007;27(4):87-102. [Medline: 18032151] 25. Haley SM, Coster WJ, Ludlow LH, Haltiwanger JT, Andrellos PJ. Pediatric evaluation of disability inventory (PEDI): Development, standardization and administration manual. New York: The Psychological Corporation USA; 1997. 26. DOBY-Early Start in Math. 2019. URL: https://dobrain.co/ [accessed 2019-10-09] 27. Chen X, Huang L, Xie D, Zhao Q. EGBMMDA: Extreme Gradient Boosting Machine for MiRNA-Disease Association prediction. Cell Death Dis 2018 Jan 05;9(1):3 [FREE Full text] [doi: 10.1038/s41419-017-0003-x] [Medline: 29305594] 28. You Z, Huang Z, Zhu Z, Yan G, Li Z, Wen Z, et al. PBMDA: A novel and effective path-based computational model for miRNA-disease association prediction. PLoS Comput Biol 2017 Mar 24;13(3):e1005455 [FREE Full text] [doi: 10.1371/journal.pcbi.1005455] [Medline: 28339468] 29. Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. Cambridge: MIT Press; 2005. 30. Valladares-Rodríguez S, Pérez-Rodríguez R, Anido-Rifón L, Fernández-Iglesias M. Trends on the application of serious games to neuropsychological evaluation: A scoping review. J Biomed Inform 2016 Dec;64:296-319 [FREE Full text] [doi: 10.1016/j.jbi.2016.10.019] [Medline: 27815228] 31. Kogan CS, Boutet I, Cornish K, Graham GE, Berry-Kravis E, Drouin A, et al. A comparative neuropsychological test battery differentiates cognitive signatures of Fragile X and Down syndrome. J Intellect Disabil Res 2009 Feb;53(2):125-142. [doi: 10.1111/j.1365-2788.2008.01135.x] [Medline: 19054268] http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 12 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al 32. Hessl D, Nguyen DV, Green C, Chavez A, Tassone F, Hagerman RJ, et al. A solution to limitations of cognitive testing in children with intellectual disabilities: the case of fragile X syndrome. J Neurodev Disord 2009 Mar 22;1(1):33-45 [FREE Full text] [doi: 10.1007/s11689-008-9001-8] [Medline: 19865612] 33. Ozonoff S. Reliability and validity of the Wisconsin Card Sorting Test in studies of autism. Neuropsychology 1995;9(4):491-500. [doi: 10.1037//0894-4105.9.4.491] 34. Groth-marnat G. Handbook of Psychological Assessment. Hoboken: Wiley; 2009. 35. Sternberg JR, Kaufman BS. The Cambridge Handbook of Intelligence. Cambridge: Cambridge University Press; 2019. 36. Everitt BS, Skrondal A. The Cambridge Dictionary of Statistics. Cambridge: Cambridge University Press; 2010. 37. Vatavu R, Cramariuc G, Schipor DM. Touch interaction for children aged 3 to 6 years: Experimental findings and relationship to motor skills. International Journal of Human-Computer Studies 2015 Feb;74:54-76. [doi: 10.1016/j.ijhcs.2014.10.007] 38. Reem E, Mohamed E, Injy H, Slim A. A Game with a Purpose for Automatic Detection of Children’s Speech Disabilities Using Limited Speech Resources. 2018 Jan 05 Presented at: International Conference on Arabic Language Processing; 11-12 October 2017; Fez, Morocco p. 89. [doi: 10.1007/978-3-319-73500-9_6] 39. Verdolini K, Rosen CA, Branski RC. Classification Manual For Voice Disorders-I. Abingdon-on-Thames: Routledge; 40. Gutiérrez García MA, Martín Ruiz ML, Rivera D, Vadillo L, Valero Duboy MA. A Smart Toy to Enhance the Decision-Making Process at Children's Psychomotor Delay Screenings: A Pilot Study. J Med Internet Res 2017 May 19;19(5):e171 [FREE Full text] [doi: 10.2196/jmir.7533] [Medline: 28526666] Abbreviations ADHD: attention deficit hyperactivity disorder AUC: area under the curve BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition EDUCARE: Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders EEG: electroencephalogram FRI: Fluid Reasoning Index K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence LOOCV: leave-one-out cross validation PEDI: Pediatric Evaluation of Disability Inventory PEP-R: Psychoeducational Profile Revised PR: precision recall PSI: Processing Speed Index ROC: receiver operating characteristic VCI: Verbal Comprehension Index VSI: Visual Spatial Index WMI: Working Memory Index Edited by G Eysenbach; submitted 04.06.19; peer-reviewed by R Haghighi Osgouei, P Urwyler; comments to author 01.07.19; revised version received 26.08.19; accepted 24.09.19; published 24.10.19 Please cite as: Bang C, Nam Y, Ko EJ, Lee W, Kim B, Choi Y, Park YR A Serious Game–Derived Index for Detecting Children With Heterogeneous Developmental Disabilities: Randomized Controlled Trial JMIR Serious Games 2019;7(4):e14924 URL: http://games.jmir.org/2019/4/e14924/ doi: 10.2196/14924 PMID: 31651408 ©Changbae Bang, Yelin Nam, Eun Jae Ko, Wooseong Lee, Byungjae Kim, Yejin Choi, Yu Rang Park. Originally published in JMIR Serious Games (http://games.jmir.org), 24.10.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 13 (page number not for citation purposes) XSL FO RenderX http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JMIR Serious Games JMIR Publications

A Serious Game–Derived Index for Detecting Children With Heterogeneous Developmental Disabilities: Randomized Controlled Trial

Download PDF
13 pages

Loading next page...
 
/lp/jmir-publications/a-serious-game-derived-index-for-detecting-children-with-heterogeneous-4v0hts1itb
Publisher
JMIR Publications
Copyright
Copyright © The Author(s). Licensed under Creative Commons Attribution cc-by 4.0
ISSN
2291-9279
DOI
10.2196/14924
Publisher site
See Article on Publisher Site

Abstract

Background: Developmental disabilities are a set of heterogeneous delays or difficulties in one or more areas of neuropsychological development. Considering that childhood is an essential stage of brain development and developmental delays lead to personal or social burdens, the early detection of childhood developmental disabilities is important. However, early screening for developmental disabilities has been a challenge because of the fear of positive results, expensive tests, differences in diagnosis depending on examiners’ abilities, and difficulty in diagnosis arising from the need for long-term follow-up observation. Objective: This study aimed to assess the feasibility of using a serious game–derived index to identify heterogeneous developmental disabilities. This study also examines the correlation between the game-derived index and existing neuropsychological test results. Methods: The randomized controlled trial involved 48 children with either normal development or developmental disabilities. In this clinical trial, we used 19 features (6 from the Korean-Wechsler Preschool and Primary Scale of Intelligence, 8 from the Psychoeducational Profile Revised, 2 from the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition, and 3 from the Pediatric Evaluation of Disability Inventory) from neuropsychological tests and 9 (7 game scores, path accuracy, and completion rate) from the serious game, DoBrain. The following analysis was conducted based on participants’ baseline information and neuropsychological test and game-derived index data for one week: (1) we compared the baseline information between the normal development and developmental disabilities groups; (2) then we measured the correlation between the game-derived index and the neuropsychological test scores for each group; and (3) we built a classifier based on the game-derived index with a Gaussian process method and then compared the area under the curve (AUC) with a model based on neuropsychological test results. Results: A total of 16 children (normal development=9; developmental disabilities=7) were analyzed after selection. Their developmental abilities were assessed before they started to play the serious games, and statistically significant differences were found in both groups. Specifically, the normal development group was more developed than the developmental disabilities group in terms of social function, gross motor function, full-scale IQ, and visual motor imitation, in that order. Similarly, the normal development group obtained a higher score on the game-derived index than the developmental disabilities group. In the correlation analysis between the game-derived index and the neuropsychological tests, the normal development group showed greater correlation with more variables than the developmental disabilities group. The game-derived index–based model had an AUC=0.9, a similar detection value as the neuropsychological test–based model’s AUC=0.86. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Conclusions: A game-derived index based on serious games can detect children with heterogenous developmental disabilities. This suggests that serious games can be used as a potential screening tool for developmental disabilities. Trial Registration: Clinical Research Information Service KCT0003247; https://cris.nih.go.kr/cris/en/search/search_result_st01 .jsp?seq=12365 (JMIR Serious Games 2019;7(4):e14924) doi: 10.2196/14924 KEYWORDS serious game; developmental disabilities; mobile game; cognitive screening tool; machine learning developmental disabilities, lack of time among specialists, fear Introduction of positive screen results, and failure to consider testing for developmental disabilities necessary [8]. In fact, even when Developmental disabilities are among the most common diseases early diagnosis and intervention programs are conducted in a in children younger than 5 years old. Developmental disabilities timely manner, many instruments for diagnosis are administered are a set of heterogenous delays or difficulties in one or more in an environment distinct from an everyday setting. For developmental milestones, including learning, self-care, social example, children are directed to perform tasks in a laboratory interactions, and movement. The global number of children setting or under the supervision of unfamiliar investigators. under 5 years old with developmental disabilities was 52.9 million in 2016, accounting for 13.3% of total years living with To resolve these problems, approaches using serious games disability for these children. In 1990, 53.0 million children were have been proposed for the early detection of developmental living with developmental disabilities, indicating that there has disabilities in children. Serious games are games that do not not been much change since then [1]. In the United States, the have enjoyment, entertain[ment] or fun as their primary purpose prevalence of children diagnosed with developmental disabilities [9]. increased remarkably from 5.76% in 2014 to 6.99% in 2016 Serious games targeted for health care use have seen a surge in [2]. use since 2004 [10]. Health care–targeted serious games are The brain is sensitive to stimulation during childhood, which intended for health checks, disease detection, or rehabilitation. is an essential stage of human development. It is the foundation Some studies have been conducted that investigated games for successive educational and vocational achievements, as well created for detecting various disorders, such as Parkinson as society’s human capital development [3]. However, children disease, Alzheimer disease, or early dementia [11-14]. Other with developmental disabilities are at a higher risk of serious games have been targeted towards detecting substandard educational accomplishment, health status, and developmental disabilities in children. For example, Anzulewicz social relationships. More specifically, children with created a machine learning model that can identify children with developmental disabilities have heightened difficulties reading, autism [15]. Alchalabi also studied ways to detect children with spelling, and counting due to shortages of phonological attention deficit hyperactivity disorder (ADHD) using a serious short-term memory or central, executive-loaded, working game integrated with electroencephalogram signals [16]. memory [4]. Additionally, developmental disabilities lead to Additionally, many researchers have targeted adult cognitive sedentary lifestyles and seven times greater reported substandard impairment detection [17-21]. emotional support in adulthood than in adults without However, thus far there have been no studies about detecting disabilities. Consecutive developmental disabilities are risk general and heterogeneous developmental disabilities in children factors of chronic health conditions such as high blood pressure, using a serious game, despite how important it is to detect these cardiovascular disease, diabetes, and chronic pain [5]. developmental disabilities. Referring children to specialists for If a child fails to achieve a certain milestone, it is extremely clinical evaluation is also the most essential part of an early hard to recover that milestone later in life. Therefore, late diagnosis of developmental disabilities. Therefore, this study identification of developmental disabilities in children may aimed to develop a classifier that can distinguish between require schools and families to pay for expensive programs [6]. children with heterogeneous developmental disabilities and Additionally, an understanding of the subsequent characteristics those with normal development using a game-derived index of a child’s developmental process requires a sensitive based on a serious game for improving cognitive ability. In this longitudinal examination by caregivers or specialists. study, we collected gameplay data of children from Do Brain, a smart device-based serious game intended for the cognitive Given such a context, identifying developmental disabilities as enhancement of children. early as possible has been deemed crucial. Multiple screening programs occurring in conjunction are recommended to detect Methods early-stage developmental disabilities. For example, the American Academy of Pediatrics recommends frequent Summary counseling, preventive care visits, and treatment visits for This study was based on a single-blinded, parallel randomized children [7]. Nevertheless, the early detection of developmental controlled trial under the supervision of an independent data disabilities in children has not been performed adequately for management team in the Asan Medical Center (Seoul, South many reasons, such as a lack of specific training for screening http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Korea) The study consists of an intervention and a control group, There were no human interventions other than early, functional, with an allocation ratio of 1:1. Participants were openly recruited game usage training. To encourage sustainable use, we sent a and enrolled in clinical studies via face-to-face evaluation by text message to the representatives of participants who had not physicians and inspectors of neuropsychological testing. used it once in the preceding week. All clinical trial data were Children with normal development were 5-6 years old and were documented, coded, and stored on a computer at Asan Medical confirmed to have developed normally via examinations and Center in Seoul, Korea. specialists. Children with developmental disabilities were 5-7 In the current study, we wanted to explore the possibility of years old but had a cognitive age of 4-6 years old. The differentiating children with developmental disabilities from recruitment period for this study was from October 2018 to the normal population by analyzing gameplay patterns. We used January 2019. If the child was assigned to a group after the the results obtained in the first week of the intervention because, face-to-face evaluation, intervention was provided for 6 months. as the intervention progressed, the cognitive function of the Our study only used the results obtained in the first week of the participants changed. This is because the cognitive function of intervention. children changes much faster than that of adults, and the game Participants were too young to give consent, so we received Do Brain was intended to enhance the cognitive function of “Subject explanatory note and consent forms” from the children. The intervention was designed for a particular time participants’ representatives (see Multimedia Appendix 1). The and frequency of play (40 minutes twice a week). randomization, using a block size of four, was stratified Initially, 106 participants were recruited. The medical staff depending on intervention and developmental disability. screened 48 children who met the recruitment criteria. Therefore, Randomization was done with the use of opaque, sealed the study group consisted of 48 children with either normal envelopes. The statistician of the data management team development or developmental disabilities (Figure 1). generated the randomly allocated sequence with the use of the Twenty-two of the children underwent intervention for 12 weeks R program (R Core Team, Vienna, Austria). Physicians enrolled (experimental group) and the other 26 participants were the the patients and opened the envelope with the lowest available control group. The control group that did not undergo registration number within the appropriate stratum. Participants intervention and children who could not carry out level C of knew the group to which they belonged, but the physicians and the serious game were excluded from this study. The baseline the inspectors of the neuropsychological tests were blinded until information and data of 16 eligible participants during the first the end of the trial. week of the intervention were analyzed. This study was The intervention of this study was based on a serious game and approved by the Institutional Review Board of Asan Medical continued for 12 weeks for 40 minutes twice a week. The Center (IRB #2018-0989; Seoul, South Korea). intervention was based on mobile serious gameplay at home. Figure 1. Participant selection flow chart. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al at home. A more detailed description is available at Do Brain Neuropsychological Tests homepage [26]. A set of neuropsychological tests was conducted to assess children’s developmental ability before the intervention period. Game-Derived Index This study employed the Korean-Wechsler Preschool and In the program, there are sections that each contain 6-8 games. Primary Scale of Intelligence (K-WPPSI-IV) [22], a child’s Each game of a section tests for each of the categories of intelligence test that provides a comprehensive assessment of development, which includes: Spatial Perception, Mathematical overall intellectual capabilities. The target age of the test is from Thinking, Attention Memory, Logical Reasoning, Constructional 1 year and 6 months to 7 years and 7 months. This study also Ability, Discernment, and Reaction (Figure 2). Each game has used the Full-Scale IQ, which provides five basic indicators of 1-3 stages. In the first week of the intervention participants intellectual function of a particular cognitive domain, including played through nine sections, which consisted of 61 games and Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), 128 stages. For the first week of the intervention we computed Fluid Reasoning Index (FRI), Working Memory Index (WMI), the game score, path accuracy, and completion rate. and the Processing Speed Index (PSI). In addition, the Psychoeducational Profile Revised (PEP-R) [23], which us used The game score was calculated using both the duration and the to assess the treatment capacity of children between 1 and 7.5 incorrect answer count. Duration referred to the time it took a years old with autism and related developmental disabilities, participant to answer correctly, and incorrect answer count to was used to plan treatment programs. Additionally, the the number of wrong answers/attempts. The game score Bruininks-Oseretsky Test of Motor Proficiency, Second Edition increased as the participant completed a certain stage within a (BOT-2) [24], which is administered in participants aged shorter time and with fewer incorrect answers. Therefore, the between 4 and 22 years, was used to check their motor higher the score a participant obtained, the better they played. development; it can also measure large and small muscle skills. Below is the equation for the game score: The age range for the Pediatric Evaluation of Disability Inventory (PEDI) [25] is from 6 months to 7.5 years, and this test is used as a tool for evaluating independence in daily life through structured interviews with parents or caregivers. There were 16 drag-and-drop games in the 128 stages of the Serious Game first week of the intervention, for which we computed path Do Brain is a serious game based on an animated cartoon. The accuracy from both intended path and actual path. Intended path game is a smart device–based application certified for child referred to the geometric distance between the start and end suitability from the iOS App store and the Google Play Store. points and actual path to the distance of the finger’s movement. It consists of games that require simple touch inputs, such as The path accuracy value ranged from 0 to 1, with values closer one-point touch, drag and drop, and rub. The object of the game to 1 indicating that a participant moved their finger more is to enhance primary cognitive capacity (attention, orientation, precisely. Below is the equation for path accuracy: memory), higher-level thinking abilities (problem solving, reasoning, concept formation), and meta-processing abilities (executive function, self-awareness). The application was downloaded to the personal smart device of each participant’s caregiver from the iOS App store or Google Play Store. The Meanwhile, completion rate was the ratio of the number of participants played the game in a natural environment, such as games played by each participant in a week divided by the quota for that week. It measured a participant’s compliance. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Figure 2. Screenshots of the serious game by game categories. heterogeneous developmental disabilities. A classification model Statistical Tests was built with a Gaussian process classifier, and model In our study, we compared the baseline information between validation using leave-one-out cross validation (LOOCV) was the normal development and developmental disabilities groups conducted. LOOCV is a method for evaluating the prediction using the Mann-Whitney U test. We then measured the quality of a model built from a small dataset [27,28]. For the correlation between the game-derived index and a child’s game-derived index–based model, we used nine features (7 neuropsychological test scores for each group using Pearson’s game scores, path accuracy, and completion rate), and 19 correlation method. Finally, we assessed the feasibility of using features for the neuropsychological test–based model (6 this serious game as a tool for detecting children with K-WPPSI, 8 PEP-R, 2 BOT-2, 3 PEDI). In this study the http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al multi-variate Gaussian process classifier of Rasmussen and proportion of males in the developmental disabilities group was Williams was used [29]. The classifier is based on Laplace higher than in the normal development group. The K-WPPSI-IV approximation and makes predictions based on finite test results, which measured a child’s cognitive ability, showed combinations of all random variables that have multivariate that full scale IQ was significantly different between the normal normal distributions. The receiver operating characteristic development and developmental disabilities groups (P=.008). (ROC) and precision-recall (PR) curves were drawn from the Moreover, the normal development group surpassed the validation results, and then the area under the curve (AUC) was developmental disabilities group in all subcategories of calculated for each curve. Sensitivity, specificity, precision, true K-WPPSI-IV. In particular, the normal development and positive rate, and true negative rate were also measured. We set developmental disabilities groups showed statistically significant a significance level of 0.05. Data were processed and analyzed differences in VCI (P=.01), VSI (P=.01), and FRI (P=.01; Table using R version 3.5.0 (R Core Team, Vienna, Austria), and 1). Python 3.6 (Python Software Foundation, Wilmington, The developmental ages of the normal development and Delaware, United States; including the Pandas 0.22.0, NumPy developmental disabilities groups were significantly different 1.14.3, and Jupyter 1.0.0 packages). (P=.007). For all subcategories of PEP-R, the normal development group outperformed the developmental disabilities Results group. Specifically, statistically significant differences in Cognitive Verbal (P=.01), Cognitive Motion (P=.01), Fine Participant Characteristics Motor (P=.01), and Imitation (P=.03) categories were observed. Baseline Information Moreover, Visual Motor Imitation (P=.009) and Gross Motor (P=.005) showed more significant differences than the other The normal development and developmental disabilities groups categories (Table 1). For the data presented in Table 1, the were comprised of nine and seven children, respectively. Mann-Whitney U test was used as the statistical significance Chronological age was not significantly different between the test. two groups, but sex was significantly different in that the http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Table 1. Comparison of the baseline information and neuropsychological tests of the normal development and developmental disability groups. Normal development group (n=9) Developmental disability group (n=7) P value Sex, n (%) Male 4 (44.4) 6 (85.7) Female 5 (55.6) 1 (14.3) — Chronological age, mean (SD) 71.4 (6.20) 70.1 (8.01) .35 K-WPPSI-IV , mean (SD) Full scale IQ 110.7 (18.3) 86.7 (13.7) .008 Verbal comprehension index 113.2 (16.7) 89.1 (15.7) .02 Visual spatial index 109.4 (13.8) 89.1 (19.0) .01 Fluid reasoning index 104.6 (11.6) 89.0 (13.7) .01 Working memory index 118.4 (16.8) 106.1 (15.5) .14 Processing speed index 99.3 (16.2) 88.1 (14.5) .14 PEP-R , mean (SD) Developmental age 68.7 (5.0) 62.0 (5.4) .007 Cognitive verbal 73.6 (4.3) 64.4 (8.5) .01 Cognitive motion 70.0 (4.8) 63.4 (6.1) .01 Visual motor imitation 67.2 (6.2) 58.6 (5.4) .009 Gross motor 59.6 (0.9) 52.4 (11.7) .005 Fine motor 63.3 (4.2) 56.6 (5.4) .01 Perception 59.4 (2.9) 58.5 (2.9) .07 Imitation 63.1 (3.5) 60.6 (3.5) .03 BOT-2 , mean (SD) Fine motor 46.4 (19.9) 29.1 (13.8) .05 Manual coordination 30.2 (9.0) 22.3 (8.8) .08 PEDI , mean (SD) Self-care 70.5 (1.5) 66.0 (5.6) .10 Movement 57.0 (3.7) 54.4 (5.6) .20 Social function 62.7 (2.5) 56.7 (4.6) .005 Not applicable. K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence. PEP-R: Psychoeducational Profile Revised. BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition. PEDI: Pediatric Evaluation of Disability Inventory. and Attention Memory (P=.04) were statistically significant in Game-Derived Index the two groups. In all subcategories of the game-derived index, Overall, the normal development group showed better game the developmental disabilities group’s standard deviation was scores than the developmental disabilities group on the larger than in the normal development group. For the data game-derived index (Table 2). Logical Reasoning (P=.03), presented in Table 2, the Mann-Whitney U test was used as the Constructional Ability (P=.04), Mathematical Thinking (P=.01), statistical significance test. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Table 2. Comparison of the game-derived index between the normal development and developmental disability groups. Variable Normal development group (n=9) Developmental disability group (n=7) P value Game score, mean (SD) Logical reasoning 8.28 (0.66) 7.16 (1.16) .03 Constructional ability 7.82 (0.69) 7.04 (1.01) .04 Mathematical thinking 9.12 (0.54) 8.13 (0.88) .01 Attention memory 9.20 (0.68) 8.50 (0.98) .04 Spatial perception 8.67 (0.58) 7.76 (0.97) .06 Reaction 8.31 (0.78) 7.89 (0.30) .05 Discernment 8.71 (0.55) 8.07 (1.07) .19 Path accuracy, mean (SD) 0.795 (0.089) 0.750 (0.072) .17 Completion rate, mean (SD) 0.991 (0.026) 0.910 (0.203) .10 Game-Derived Index– and Neuropsychological Correlations Between the Neuropsychological Tests Test–Based Classifiers and Game-Derived Index As shown in Figures 4 and 5, the game-derived index–based, Correlations between the neuropsychological tests and classifier has a similar AUC to the neuropsychological game-derived index results were computed for each group using test–based classifier. Sensitivity and specificity showed similar Pearson’s correlation method (Figure 3). Overall, the patterns. The game-derived index classifier had a sensitivity of game-derived index and neuropsychological tests showed higher 0.714 and a specificity of 0.778, whereas the neuropsychological correlation values for the normal development group than the test–based classifier had a sensitivity of 0.857 and a specificity developmental disabilities group. Logical reassignment and of 0.778. completion rate in the game-derived index showed a high correlation with neuropsychological tests. In the developmental No serious adverse events or side effects were observed in the disabilities group, the constructional ability of the game-derived normal development and developmental disabilities groups. index showed the highest correlation with VSI (0.92). Figure 3. Correlation matrix for the game-derived index and neuropsychological tests of the normal development and developmental disability groups. K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence; PEP-R: Psychoeducational Profile Revised; BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition; PEDI: Pediatric Evaluation of Disability Inventory. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al Figure 4. The receiver operating characteristic and precision-recall curves for leave-one-out cross validation of the model based on the game-derived index from the serious game. ROC: receiver operating characteristic; DD: developmental disabilities; ND: normal development; PR: precision recall; GDI: game-derived index. Figure 5. The receiver operating characteristic and precision-recall curves for leave-one-out cross-validation of the model based on the neuropsychological tests. ROC: receiver operating characteristic; DD: developmental disabilities; ND: normal development; PR: precision recall. the positive diagnosis of developmental disabilities should be Discussion conducted through a specialist’s exam, our classifier that detects general and heterogeneous developmental disabilities could be Principal Results used to refer children with developmental disabilities to We built a classifier from the game-derived index that could specialists for more specific and expert exams. distinguish children with developmental disabilities from those Additionally, we compared the game-derived index with the with normal development. In contrast to some models that can results of the neuropsychological tests to determine the detect a specific type of disease, such as autism or ADHD, our characteristics of the normal development group that helped classifier is the first model to detect children with heterogeneous them obtain a better game-derived index than the developmental types of developmental disabilities. This suggests that serious disabilities group. As many serious games that assess the games have the potential for screening the general cognitive function of humans lack psychometric analysis and developmental disability status of children. Considering that http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al comparisons with standardized studies [30], the comparison intentional path. However, the normal development group’s between the game-derived index and the neuropsychological path accuracy did not correlate well with Fine Motor index of tests in this study provides a more comprehensive interpretation PEP-R and BOT-2, mainly because testing of PEP-R or BOT-2 of the serious gameplay of children. differs from a drag-and-drop game. Moreover, the developmental disabilities group’s path accuracy had negative Differences in Game-Derived Index Between Normal correlation with the neuropsychological tests’ VSI and Fine Development and Developmental Disabilities Groups Motor index. This negative correlation can be attributed to the Game score, path accuracy, and completion rate were computed developmental disabilities group’s heterogeneity and small to assess participants’ gameplay. Although some features of the sample size. game-derived index lacked statistical significance, the overall Completion rate is a measure of participant compliance. In this pattern showed that the normal development group played better serious game, participants may skip a stage when they find it than the developmental disabilities group. difficult. As can be observed in the game score results, the Game score represents how fast and correctly a participant normal development group outperformed the developmental solved a problem. The serious game used in this study is based disabilities group in gameplay. Additionally, the normal on a cognitive counseling program for children. As the normal development group’s mean completion rate was higher than the development group showed higher intelligence scores on the developmental disabilities group’s. Notably, the standard neuropsychological tests, this means that they answered deviation differed greatly between the normal development correctly with fewer attempts and in less time. For the normal (0.026) and developmental disabilities (0.203) groups. The large development group, many game-derived index features showed standard deviation of the developmental disabilities group’s a positive linear correlation with neuropsychological tests. More completion rate is due to some developmental disabilities specifically, Logical Reasoning was correlated with Full Scale participants skipping many more stages than the others. IQ and Developmental Age with coefficients of 0.86 and 0.85, Classification Model respectively. The model built from the game-derived index performed By contrast, most game-derived index features of the adequately compared with that built from neuropsychological developmental disabilities group did not show linear correlations tests, showing the discriminating power of the game-derived or had negative correlations. These results can be attributed to index. This result suggests that serious games have the potential the following reasons. First, the developmental disabilities were for detecting children with developmental disabilities. As heterogenous in the developmental disabilities group. Also, the children with heterogeneous developmental disabilities show developmental disabilities group had a small, restricted sample. complicated characteristics, characterizing them with high Moreover, some studies indicate that cognitive assessment is sensitivity is difficult. In contrast to models that detect specific difficult for patients suffering from neuropsychological disorders developmental disorders developed in previous studies, the [31-33]. More specifically, when converting raw scores to model built in this study is the first to detect general standardized scores, performance variations that need to be developmental disabilities. detected often become obscure due to the flooring effect [34,35]. The flooring effect refers to a phenomenon in which the Comparison With Previous Studies measuring tool cannot discriminate among those who belong Some studies have been conducted regarding developmental to the lower level of the characteristic to be measured [36]. The delay detection with serious games. Previous works utilized flooring effect may occur when the score range itself is limited, more devices, required more resources, or could detect only or when inspection is too difficult. As standardized tests have specific diseases. such limitations, recent trends in intellectual disability research Alchalabi’s study that detected ADHD patients with an emphasizes narrating the cognitive signatures of conditions Electroencephalogram (EEG)-based serious game [16] required throughout their lifetime rather than depending on test scores an EEG reader and a special gaming setting. However, our study [31]. Although many game-derived index features did not have only utilizes smart devices and is done in the home setting. Like positive correlations, constructional ability, spatial perception, our study’s use of neuropsychological tests for clinical diagnosis, and completion rate had correlation coefficients of 0.69, 0.67, Alchalabi’s study was also meaningful in that it used EEG with and 0.62, respectively. some diagnostic value. Path accuracy indicates fine motor function and visuospatial Elhady build a speech disability detection game with limited ability. According to Vatavu, children who are better developed speech resources [38]. Our study uses touch input from a smart in visuospatial processing have a high path accuracy score [37]. device while it records the voice. However, they targeted a very In our study, the normal development group’s path accuracy specific speech disability population suffering from the score was higher than the developmental disabilities group, but phonemes /s/ and /r/. There is a need to cover broader speech the difference was not statistically significant. Path accuracy disabilities, as there exist heterogenous speech disabilities [39]. scores were highly correlated with the VSI of the K-WPPSI in the normal development group (r=0.87), which shows a similar Garcia and Ruiz created a children’s psychomotor delay pattern to Vatavu’s findings. These findings indicate that screening process called Ubiquitous Detection Ecosystem to children with better VSI scores are better at interpreting Care and Early Stimulation for Children with Developmental geometrical relationships. As a result, they understand Disorders (EDUCARE) [40] based on a smart toy. In drag-and-drop games better and can move their finger in a more http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 10 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al EDUCARE, the video that children played on the toy had been In a clinical situation, a neuropsychological test is essential for analyzed by developmental experts. However, EDUCARE only the diagnosis of developmental disabilities. Therefore, the assessed motor function and depended on lengthy screening by estimation of neuropsychological test results with a experts. Our study overcomes this by assessing more game-derived index could provide caregivers and specialists developmental areas and immediately giving numerical with more information. However, our study did not build a developmental indices to caregivers. In a different respect, model that estimates neuropsychological test results from the EDUCARE utilized methods more related to everyday objects, game-derived index because of the small sample size. Future such as stackable cubes, than to our game-based approach. studies must build a neuropsychological test estimation model more complex than our simple linear correlation model. Limitations Conclusion Our study population is insufficient for generalizability of our findings to the general population. However, our findings show The game-derived index patterns observed in serious gameplay the feasibility of using this serious game to screen for general in the normal development and developmental disabilities groups and heterogeneous developmental disabilities. As children were different. In particular, the neuropsychological tests and continue to develop in various areas in the earlier stages of game-derived index have significant differences in the developmental disabilities, a definite diagnosis of the disease correlations between groups. This result is a potential indicator is difficult. Our model is useful because of the flexibility and that game-derived index can distinguish developmental heterogeneous characteristics of developmental disabilities. In disabilities from normal development, and the model based on the future, we will widen our study population to enhance the it showed similar performance to neuropsychological tests that generalizability of our findings. constitute conventional developmental ability tests. This suggests that serious games can be used as a potential screening tool for developmental disabilities. Acknowledgments CB is supported by the Seok-San Biomedical Science Scholarships, Yonsei University College of Medicine. This study was supported by the student research fund from Yonsei University College of Medicine in 2019, by the Basic Science Research Program through the National Research Foundation of Korea (NRF), which is funded by the Ministry of Education (NRF-2017R1D1A1B03035762), and by the Foundational Technology Development Program (NRF-2019M3E5D4064682) through the Ministry of Science and ICT in the Republic of Korea. Conflicts of Interest WL, BK, and YC are employees of Do Brain Co, Ltd. Multimedia Appendix 1 Informed consent for the randomized controlled trial. [DOCX File , 55 KB-Multimedia Appendix 1] Multimedia Appendix 2 CONSORT‐EHEALTH checklist (V 1.6.1). [PDF File (Adobe PDF File), 3666 KB-Multimedia Appendix 2] References 1. Global Research on Developmental Disabilities Collaborators. Developmental disabilities among children younger than 5 years in 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Glob Health 2018 Oct;6(10):e1100-e1121 [FREE Full text] [doi: 10.1016/S2214-109X(18)30309-7] [Medline: 30172774] 2. Zablotsky B, Black LI, Blumberg SJ. Estimated Prevalence of Children With Diagnosed Developmental Disabilities in the United States, 2014-2016. NCHS Data Brief 2017 Nov(291):1-8 [FREE Full text] [Medline: 29235982] 3. Shonkoff JP, Phillips DA. From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press; 2000. 4. Henry L, Winfield J. Working memory and educational achievement in children with intellectual disabilities. J Intellect Disabil Res 2010 Apr;54(4):354-365. [doi: 10.1111/j.1365-2788.2010.01264.x] [Medline: 20236191] 5. Havercamp SM, Scandlin D, Roth M. Health disparities among adults with developmental disabilities, adults with other disabilities, and adults not reporting disability in North Carolina. Public Health Rep 2004;119(4):418-426 [FREE Full text] [doi: 10.1016/j.phr.2004.05.006] [Medline: 15219799] 6. The importance of caregiver-child interactions for the survival and healthy development of young children. In: Developmental difficulties in early childhood. Boston, US: World Health Organization; 2012. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 11 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al 7. Committee On Practice and Ambulatory Medicine, Bright Futures Periodicity Schedule Workgroup. 2019 Recommendations for Preventive Pediatric Health Care. Pediatrics 2019 Mar 25;143(3):e20183971. [doi: 10.1542/peds.2018-3971] [Medline: 30804073] 8. Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, Landes C. Developmental Stages of Developmental Screening: Steps to Implementation of a Successful Program. Am J Public Health 2005 Nov;95(11):1928-1932. [doi: 10.2105/ajph.2004.052167] 9. Michael D, Chen S. Serious Games: Games that Educate, Train and Inform. Boston: Thomson Course Technology; 2006. 10. Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertainment Computing 2013 Dec;4(4):231-247. [doi: 10.1016/j.entcom.2013.09.002] 11. van der Meulen E, Cidota MA, Lukosch SG, Bank PJM, van der Helm AJC, Visch VT. A Haptic Serious Augmented Reality Game for Motor Assessment of Parkinson's Disease Patients. 2016 Presented at: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct); 19-23 Sept. 2016; Merida p. 104. 12. Vallejo V, Wyss P, Rampa L, Mitache AV, Müri RM, Mosimann UP, et al. Evaluation of a novel Serious Game based assessment tool for patients with Alzheimer's disease. PLoS One 2017;12(5):e0175999 [FREE Full text] [doi: 10.1371/journal.pone.0175999] [Medline: 28472049] 13. Tarnanas I, Laskaris N, Tsolaki M, Muri R, Nef T, Mosimann UP. On the comparison of a novel serious game and electroencephalography biomarkers for early dementia screening. Adv Exp Med Biol 2015;821:63-77. [doi: 10.1007/978-3-319-08939-3_11] [Medline: 25416111] 14. Zucchella C, Sinforiani E, Tassorelli C, Cavallini E, Tost-Pardell D, Grau S, et al. Serious games for screening pre-dementia conditions: from virtuality to reality? A pilot project. Funct Neurol 2014;29(3):153-158 [FREE Full text] [Medline: 25473734] 15. Anzulewicz A, Sobota K, Delafield-Butt JT. Toward the Autism Motor Signature: Gesture patterns during smart tablet gameplay identify children with autism. Sci Rep 2016 Aug 24;6(1):31107 [FREE Full text] [doi: 10.1038/srep31107] [Medline: 27553971] 16. Alchalabi AE, Shirmohammadi S, Eddin AN, Elsharnouby M. FOCUS: Detecting ADHD Patients by an EEG-Based Serious Game. IEEE Trans. Instrum. Meas 2018 Jul;67(7):1512-1520. [doi: 10.1109/tim.2018.2838158] 17. Valladares-Rodriguez S, Perez-Rodriguez R, Facal D, Fernandez-Iglesias MJ, Anido-Rifon L, Mouriño-Garcia M. Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults. PeerJ 2017;5:e3508 [FREE Full text] [doi: 10.7717/peerj.3508] [Medline: 28674661] 18. Vallejo V, Wyss P, Chesham A, Mitache AV, Müri RM, Mosimann UP, et al. Evaluation of a new serious game based multitasking assessment tool for cognition and activities of daily living: Comparison with a real cooking task. Computers in Human Behavior 2017 May;70:500-506. [doi: 10.1016/j.chb.2017.01.021] 19. Tong T, Chignell M, Tierney MC, Lee J. A Serious Game for Clinical Assessment of Cognitive Status: Validation Study. JMIR Serious Games 2016 May 27;4(1):e7 [FREE Full text] [doi: 10.2196/games.5006] [Medline: 27234145] 20. Byun S, Park C. Serious Game for Cognitive Testing of Elderly. In: HCI 2011: HCI International 2011–Posters’ Extended Abstracts. Berlin: Springer; 2019:354-357. 21. Boletsis C, McCallum S. Smartkuber: A Serious Game for Cognitive Health Screening of Elderly Players. Games Health J 2016 Aug;5(4):241-251. [doi: 10.1089/g4h.2015.0107] [Medline: 27192473] 22. Park H, Lee K. Intellectual Characteristics of Korean Children: Analyses of K-WPPSI-IV Performance. Korean J Child Stud 2016 Dec 31;37(6):157-168. [doi: 10.5723/kjcs.2016.37.6.157] 23. Schopler E. Psychoeducational Profile Revised (PEP-R). In: Encyclopedia of Autism Spectrum Disorders: 5 Volume Set. New York: Springer; 2019. 24. Deitz JC, Kartin D, Kopp K. Review of the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition (BOT-2). Phys Occup Ther Pediatr 2007;27(4):87-102. [Medline: 18032151] 25. Haley SM, Coster WJ, Ludlow LH, Haltiwanger JT, Andrellos PJ. Pediatric evaluation of disability inventory (PEDI): Development, standardization and administration manual. New York: The Psychological Corporation USA; 1997. 26. DOBY-Early Start in Math. 2019. URL: https://dobrain.co/ [accessed 2019-10-09] 27. Chen X, Huang L, Xie D, Zhao Q. EGBMMDA: Extreme Gradient Boosting Machine for MiRNA-Disease Association prediction. Cell Death Dis 2018 Jan 05;9(1):3 [FREE Full text] [doi: 10.1038/s41419-017-0003-x] [Medline: 29305594] 28. You Z, Huang Z, Zhu Z, Yan G, Li Z, Wen Z, et al. PBMDA: A novel and effective path-based computational model for miRNA-disease association prediction. PLoS Comput Biol 2017 Mar 24;13(3):e1005455 [FREE Full text] [doi: 10.1371/journal.pcbi.1005455] [Medline: 28339468] 29. Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. Cambridge: MIT Press; 2005. 30. Valladares-Rodríguez S, Pérez-Rodríguez R, Anido-Rifón L, Fernández-Iglesias M. Trends on the application of serious games to neuropsychological evaluation: A scoping review. J Biomed Inform 2016 Dec;64:296-319 [FREE Full text] [doi: 10.1016/j.jbi.2016.10.019] [Medline: 27815228] 31. Kogan CS, Boutet I, Cornish K, Graham GE, Berry-Kravis E, Drouin A, et al. A comparative neuropsychological test battery differentiates cognitive signatures of Fragile X and Down syndrome. J Intellect Disabil Res 2009 Feb;53(2):125-142. [doi: 10.1111/j.1365-2788.2008.01135.x] [Medline: 19054268] http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 12 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Bang et al 32. Hessl D, Nguyen DV, Green C, Chavez A, Tassone F, Hagerman RJ, et al. A solution to limitations of cognitive testing in children with intellectual disabilities: the case of fragile X syndrome. J Neurodev Disord 2009 Mar 22;1(1):33-45 [FREE Full text] [doi: 10.1007/s11689-008-9001-8] [Medline: 19865612] 33. Ozonoff S. Reliability and validity of the Wisconsin Card Sorting Test in studies of autism. Neuropsychology 1995;9(4):491-500. [doi: 10.1037//0894-4105.9.4.491] 34. Groth-marnat G. Handbook of Psychological Assessment. Hoboken: Wiley; 2009. 35. Sternberg JR, Kaufman BS. The Cambridge Handbook of Intelligence. Cambridge: Cambridge University Press; 2019. 36. Everitt BS, Skrondal A. The Cambridge Dictionary of Statistics. Cambridge: Cambridge University Press; 2010. 37. Vatavu R, Cramariuc G, Schipor DM. Touch interaction for children aged 3 to 6 years: Experimental findings and relationship to motor skills. International Journal of Human-Computer Studies 2015 Feb;74:54-76. [doi: 10.1016/j.ijhcs.2014.10.007] 38. Reem E, Mohamed E, Injy H, Slim A. A Game with a Purpose for Automatic Detection of Children’s Speech Disabilities Using Limited Speech Resources. 2018 Jan 05 Presented at: International Conference on Arabic Language Processing; 11-12 October 2017; Fez, Morocco p. 89. [doi: 10.1007/978-3-319-73500-9_6] 39. Verdolini K, Rosen CA, Branski RC. Classification Manual For Voice Disorders-I. Abingdon-on-Thames: Routledge; 40. Gutiérrez García MA, Martín Ruiz ML, Rivera D, Vadillo L, Valero Duboy MA. A Smart Toy to Enhance the Decision-Making Process at Children's Psychomotor Delay Screenings: A Pilot Study. J Med Internet Res 2017 May 19;19(5):e171 [FREE Full text] [doi: 10.2196/jmir.7533] [Medline: 28526666] Abbreviations ADHD: attention deficit hyperactivity disorder AUC: area under the curve BOT-2: Bruininks-Oseretsky Test of Motor Proficiency, Second Edition EDUCARE: Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders EEG: electroencephalogram FRI: Fluid Reasoning Index K-WPPSI-IV: Korean-Wechsler Preschool and Primary Scale of Intelligence LOOCV: leave-one-out cross validation PEDI: Pediatric Evaluation of Disability Inventory PEP-R: Psychoeducational Profile Revised PR: precision recall PSI: Processing Speed Index ROC: receiver operating characteristic VCI: Verbal Comprehension Index VSI: Visual Spatial Index WMI: Working Memory Index Edited by G Eysenbach; submitted 04.06.19; peer-reviewed by R Haghighi Osgouei, P Urwyler; comments to author 01.07.19; revised version received 26.08.19; accepted 24.09.19; published 24.10.19 Please cite as: Bang C, Nam Y, Ko EJ, Lee W, Kim B, Choi Y, Park YR A Serious Game–Derived Index for Detecting Children With Heterogeneous Developmental Disabilities: Randomized Controlled Trial JMIR Serious Games 2019;7(4):e14924 URL: http://games.jmir.org/2019/4/e14924/ doi: 10.2196/14924 PMID: 31651408 ©Changbae Bang, Yelin Nam, Eun Jae Ko, Wooseong Lee, Byungjae Kim, Yejin Choi, Yu Rang Park. Originally published in JMIR Serious Games (http://games.jmir.org), 24.10.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included. http://games.jmir.org/2019/4/e14924/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e14924 | p. 13 (page number not for citation purposes) XSL FO RenderX

Journal

JMIR Serious Games – JMIR Publications

Published: Oct 24, 2019

Keywords: serious game; developmental disabilities; mobile game; cognitive screening tool; machine learning

There are no references for this article.