Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4-8 Years

A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward... Background: A computer-based game, named Timo’s Adventure, was developed to assess specific cognitive functions (eg, attention, planning, and working memory), time perception, and reward mechanisms in young school-aged children. The game consists of 6 mini-games embedded in a story line and includes fantasy elements to enhance motivation. Objective: The aim of this study was to investigate the validity of Timo’s Adventure in normally developing children and in children with attention-deficit/hyperactivity disorder (ADHD). Methods: A total of 96 normally developing children aged 4-8 years and 40 children with ADHD were assessed using the game. Clinical validity was investigated by examining the effects of age on performances within the normally developing children, as well as performance differences between the healthy controls and the ADHD group. Results: Our analyses in the normally developing children showed developmental effects; that is, older children made fewer inhibition mistakes (r=−.33, P=.001), had faster (and therefore better) reaction times (r=−.49, P<.001), and were able to produce time intervals more accurately than younger children (ρ=.35, P<.001). Discriminant analysis showed that Timo’s Adventure was accurate in most classifications whether a child belonged to the ADHD group or the normally developing group: 78% (76/97) of the children were correctly classified as having ADHD or as being in the normally developing group. The classification results showed that 72% (41/57) children in the control group were correctly classified, and 88% (35/40) of the children in the ADHD group were correctly classified as having ADHD. Sensitivity (0.89) and specificity (0.69) of Timo’s Adventure were satisfying. Conclusions: Computer-based games seem to be a valid tool to assess specific strengths and weaknesses in young children with ADHD. (JMIR Serious Games 2016;4(2):e15) doi: 10.2196/games.5997 KEYWORDS experimental games; ADHD; children; neuropsychological test http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al For this purpose, age-related differences in a group of normally Introduction developing children (N=96) were examined. Assessment of children’s cognitive strengths and weaknesses Next, the scores on the mini-games of normally developing is an important focus of clinical child neuropsychological children were compared with those of children with research and clinical care worldwide [1]. Cognitive abilities are attention-deficit/hyperactivity disorder (ADHD; N=40). quantified traditionally by use of, for example, paper-and-pencil Attention-deficit/hyperactivity disorder is a developmental performance tests and more recently with computer-assisted disorder that is associated with academic difficulties and social tools [2,3]. Performances on traditional cognitive tests are disadvantage [15]. According to the Diagnostic and Statistical believed to be influenced significantly by noncognitive Manual of Mental Disorders (Fifth Edition; DSM-V) [16], 2 functions, such as motivation and perseverance [4]. Therefore, main areas of impairment in children with ADHD exist: a lower score on, for instance, a working memory test might inattention (eg, difficulty in maintaining attention during a task indicate a memory problem but also, for example, a decreased or problems in dividing attention) and hyperactivity and motivation. This overall performance score therefore only impulsive behavior (eg, acting out before thinking about the limitedly reflects the underlying “cause” in case of a decreased consequences). Previous research has found that cognitive performance, which makes this score difficult to interpret. In difficulties, more specifically in the domain of working memory order to test one’s cognitive abilities more purely, it would be and attention, occur in children with ADHD [17]. However, preferable to (1) optimize motivation in the test situation and according to the triple-pathway model by Sonuga-Barke et al (2) assess motivation in a separate test additionally— [18], not all children with ADHD have cognitive weaknesses. unfortunately, tests measuring motivation in children are In this model 3 distinct patterns of ADHD deficits are relatively scarcely used in the clinics. distinguished. The first pathway is related to cognitive functions and is called the inhibitory-based executive dysfunction. This On the basis of the literature, it is known that introducing pathway views ADHD as a disorder of dysregulation of thought immediate (vs delayed) rewards and the adaptation of item and action associated with diminished inhibitory control (ie, difficulty levels to the child’s abilities are likely to increase executive functions). In the second pathway, ADHD is explained motivation in children and help the child to stay focused on the as a motivational style associated with fundamental alterations tasks that he or she needs to do [5]. Also, introducing a context in reward mechanisms. Children with ADHD are assumed to (eg, by introducing cognitive tests in the context of a story) may prefer small immediate rewards over large delayed rewards, increase motivation, although data on the effects of including which results in inattentive, overactive, and impulsive behaviors a story line on motivation or engagement in games are so far [19]. The third pathway states that deficits in time perception, inconclusive [6]. The introduction of a story line improves the for instance, deficits in distinguishing between two time child’s feeling of being part of a gaming environment [7]. In intervals, producing time intervals, and estimating time, are addition, the use of intrinsic fantasy elements has been found another component of ADHD. Indeed, time perception deficits to improve motivation to conduct a specific task [8,9]. have been reported for children with ADHD [20]. However, the The above-mentioned increase in popularity of results are not consistent; that is, some authors report no computer-assisted assessment tools is partly caused by the fact ADHD-related deficits [21]. All 3 pathways are believed to that it is relatively easy to implement these immediate rewards, have their own neural substrate [18]. Sonuga-Barke and to adapt difficulty levels, to implement a story, and to use colleagues [18] found in children with ADHD aged between 6 intrinsic fantasy elements. This leads to a situation in which the and 17 years that delay aversion, poor executive functions, and child does not have the feeling of being assessed but instead poor time perception are core, but unrelated and independent, thinks that he or she is playing a game [10]. This is especially characteristics of ADHD. A person with ADHD can have important in tests that are designed for children because it is deficits in one of the pathways or a combination of pathways. known that children can behave differently when they know Neuropsychological measurements (which are used to examine they are being studied (also known as the Hawthorne effect possible deficits in one of the pathways) usually focus primarily [11]). For this purpose, we developed “Timo’s Adventure,” a on just one of the pathways, whereas one can conclude from computer-based game that consists of 6 mini-games. These the model by Sonuga-Barke et al that it is necessary to gain mini-games aim to assess different cognitive processes, for information on possible deficits in all 3 pathways. Sonuga-Barke example, attention, planning, and working memory; delay and colleagues used several distinctive computerized tasks to aversion, as a measure of motivation; and time perception (see collect information about the 3 pathways. These were, however, method section). The aim of this study was to examine the not connected in a fantasy gaming environment or by a story validity of Timo’s Adventure in assessing strengths and line. In our game, a story line was included in order to immerse weaknesses in the above-mentioned domains of young children, the player in an intrinsic fantasy and possibly improve the aged 4-8 years. Proof of validity was sought by studying group reliability of the diagnosis. To our knowledge no computerized differences in performance on the mini-games. Two types of diagnostic tools exist in which all 3 pathways are included in relevant group comparisons were made. First, the age of the combination with all motivation-enhancing elements discussed child is believed to be a relevant variable and was therefore above (including a story line or fantasy game elements), studied in relation to performances on the mini-games: we although some training tools with a story line exist (for instance, expected younger children (aged <6 years) to perform less well Braingame Brian [22]). In Timo’s Adventure story line, than older children on all mini-games (in line with studies on, distracting factors are included to measure real-life distraction. eg, cognitive development [12] and time perception [13,14]). Previous research found that distractors in a computerized http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al continuous performance test resulted in more distractibility in this study by their medical specialist. In parallel, parents of children with ADHD than in their healthy peers [23]. children enrolled in a special needs program for children with behavioral problems were informed by a letter about the study In summary, the aim of this study was to investigate the clinical and asked to participate, via the children’s school. Informed validity of Timo’s Adventure. consent of 62 children with a diagnosis of ADHD was acquired. A total of 22 children did not meet the inclusion criteria because Methods they had a comorbid DSM-V diagnosis (n=4), because they used medication for attentional problems and hyperactive behavior Participants (stimulants, atomoxetine, tricyclic antidepressants, or clonidine; Normally Developing Children n=6), or because of a combination of these exclusion criteria (n=12). The final dataset of the ADHD group consisted of 40 Parents of all children enrolled in the first 4 grades of 4 Dutch children (30 boys), all with a diagnosis of ADHD according to elementary schools were informed by a letter about the study. DSM-V. These diagnoses were made on the basis of a protocol Informed consent of 102 children was acquired. A total of 4 formulated by Goldman et al [24] that includes (1) extensive children were excluded because they were not native speakers history taking, (2) cognitive testing, (3) general physical and and instructions in Timo’s Adventure were in the Dutch neurological examination of the child, and (4) systematic language. In addition, 2 children were excluded because they assessment of ADHD characteristics by means of structured had a DSM-V diagnosis. The final dataset consisted of 96 questions based on the most recent version of the DSM [16]. children (43 boys), age ranging from 4 to 8 years. An overview Age range of the clinical sample was 6-8 years. An overview of characteristics for this group can be found in Table 1. of characteristics of the ADHD group can be found in Table 1. All children were tested individually in a private room at their Children enrolled in the special needs program for children with school. Approval for testing this sample was given by the Ethical behavioral problems were tested individually in a private room Review Committee of the Faculty of Psychology and at their school. The children who were patients of the Center Neuroscience of Maastricht University, the Netherlands. for Neurological Learning Disabilities were seen for Children With Attention-Deficit/Hyperactivity Disorder neuropsychological testing as part of clinical care. Parents of patients of the outpatient clinic Center for Approval for testing this sample was given by the Medical Neurological Learning Disabilities were asked to participate in Ethical Board of Kempenhaeghe. Table 1. Characteristics of all participants. Characteristics Typically developing ADHD group children Number of participants 96 40 Boys/girls 43/53 30/10 Age range, years 4-8 6-8 Mean age (SD), years 5.85 (1.33) 6.90 (0.74) 94.74 (10.79) 87.92 (13.05) Verbal IQ (WPPSI-III-NL Vocabulary), mean (SD) ADHD: attention-deficit/hyperactivity disorder. WPPSI-III-NL: Dutch version of the Wechsler Preschool and Primary Scale of Intelligence [25]. software cyclic model and the spiral model [28-30]. Besides Materials the designers of Eindhoven University of Technology, users were invited in the design process. Users in this case were Computer-Based Game Timo’s Adventure children (who helped us by explaining what they would like Timo’s Adventure is a single-player game. All tasks are and by making drawings) and psychologists from embedded in a story line: the main character in Timo’s Kempenhaeghe (who participated in the development of the Adventure is Timo, a friendly alien whose rocket has run out functionalities of the game, the visual graphics, and story line of fuel [26]. He asks the child to go on an adventure together that needed to match the age of the children). In the to collect stars that can be used as fuel. To complete all tasks, implementation phase, the functionalities of the game were it takes approximately 20 minutes. The game is categorized as created (by engineers of Eindhoven University of Technology a serious game: a game designed for specific purposes beyond in close collaboration with the psychologists from entertainment [27]. The game has a first-person view to simulate Kempenhaeghe). During the evaluation phase (consisting of a the feeling of presence, to make the child feel like he or she is user test with a paper prototype technique and an inside the game world. computer-based prototype), the game was played and evaluated Development of the game was divided into 3 stages: design, by children and by psychologists who were not part of the implementation, and evaluation, as based on the iterative http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al development team. This feedback was used to improve the game he shows the child pictures of the ingredients. The child needs and remove small bugs [31]. to remember these ingredients and select them in the same order as presented by Timo. The first sandwich starts with 2 The computer-based game consists of 6 different tasks (ie, 6 ingredients, adding up to 5 ingredients in the last sandwich. mini-games), each of which measures different neurocognitive This task is a measurement of the capacity of the visual working functions and thus gives information about possible deficits. memory. According to Craeynest [32], the capacity of the The 6 tasks measure aspects of executive functions, time working memory develops from remembering 2 targets when perception, and reward mechanisms to represent the 3 pathways the child is 2.5 years old, to 3 targets when 3 years old, and 5 of Sonuga-Barke’s model. All tasks were developed because targets when the child is 7 years old. Martinussen et al [33] of the need to measure the specific function and modeled after found that the capacity of the working memory in children with the theoretical background of the triple-pathway model. ADHD is markedly lower. In this task, the child can get a total Pathway 1: Executive Functions score of 5 points: one for each correct sandwich. All irrelevant mouse clicks are measured and thus give In the third task, the mini-game Monkey (inhibition), a monkey information on impulsivity and hyperactive behavior of the has thrown banana peels on the road (see Figure 3). To cross child. Furthermore, the following mini-games are included to the road, the child needs to swipe the banana peels and clear assess impairments in this pathway: Dressing up (planning), the road. However, if the monkey sees the child swiping banana Sandwich (working memory), Monkey (inhibition), and Magic peels, it will undo the child’s actions. The monkey is playing Land (simple reaction time). hide-and-seek and appears suddenly. The child needs to wait for the moment the monkey disappears. This is a go or no-go The first task, the mini-game Dressing up (planning), is set in task that gives information on the response inhibition of the the bedroom of the child. Timo tells the child that he or she child: is the child capable of inhibiting his or her response until needs to get dressed before the adventure can start. Several the monkey hides? Children with ADHD have deficits in this garments are spread throughout the room, which can be selected response inhibition and they can be inclined to react impulsively by clicking on them (see Figure 1). After clicking, the garment [19]. The number of failures (ie, when the monkey sees the moves toward the reflection of the child in the mirror and the action of the child) is the outcome variable of this task; the child gets dressed. This task gives information on the planning higher this score, the worse the inhibition skills. and organization skills of the child. The order in which the child selects the clothes is assessed to see whether the child is capable In the mini-game Magic Land (simple reaction time), stars shoot of planning his or her actions in the right order. The child can upward from magic holes (see Figure 4). The child needs to get a total score of 2 points: 1 point for being completely dressed collect these stars. If the child does not react within 2 seconds, and 1 point for using an executable and correct manner. the stars will disappear. The task ends after 50 stars shoot upward. Outcome variables in this task are the number of The second task, the mini-game Sandwich (working memory), collected stars (with a maximum of 50) and average reaction is set in the kitchen (see Figure 2). Timo tells the child that it time for collected stars. Slower and more variable reaction times is necessary to eat something before starting the adventure, and have been found to be a characteristic of ADHD [34]. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 1. Screenshot of the mini-game Dressing up (planning). Figure 2. Screenshot of the mini-game Sandwich (working memory). http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 3. Screenshot of the mini-game Monkey (inhibition). Figure 4. Screenshot of the mini-game Magic Land (simple reaction time). http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 5. Screenshot of the mini-game Rocket (reward mechanisms). a time interval of 10 seconds. A produced interval between 9 Pathway 2: Reward Mechanisms and 11 seconds results in a perfect balloon. When the produced In the task mini-game Rocket (delay aversion; see Figure 5), interval is smaller than 9 seconds the balloon falls into the water, the child gets a choice between an immediate but small reward and a produced interval larger than 11 seconds results in a (ending of the task) or a delayed but bigger reward (a flight in balloon that flies away. The child can make a maximum of 3 the rocket, after 2 minutes of waiting). The child can end the perfect balloons, or the task will end after 3 minutes with a task at any moment. Impulsive behavior occurs when responding perfect balloon (regardless of what the interval is). Barkley et produces more immediate, relatively smaller rewards at the cost al [36] suggested that the estimation of temporal intervals is of delayed, larger rewards [35]. Outcome variables are whether atypical in children with ADHD. The number of correct balloons the child chooses a small or big reward and how long (in is an outcome variable. Furthermore, the average production seconds) the child waited. interval for the first 3 balloons is measured by subtracting 10 seconds from each of the first 3 balloons, transforming these Pathway 3: Time Perception scores to absolute scores, adding these scores, and then dividing The task mini-game Balloon (time production) is set at a river them by 3. The higher this score, the less precise the mean with a broken bridge (see Figure 6). To cross the river, the child produced intervals are. needs to inflate a balloon with the balloon machine by producing http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 6. Screenshot of the mini-game Balloon (time perception). samples t test revealed that children in the ADHD group and Vocabulary children in the normally developing group were equal in terms This subscale of the Dutch version of the Wechsler Preschool of Vocabulary scores (ADHD group: mean 87.92, SD 13.05; and Primary Scale of Intelligence (WPPSI-III-NL [25]) was control group: mean 94.74, SD 10.79; t =1.92, P=.11), therefore used to estimate verbal intelligence [37]. In this task, the child it was not necessary to correct for IQ in further analyses. was asked to give definitions of words such as “umbrella” or Finally, it was analyzed on which variables children with ADHD “shoe.” The total score can be transformed to IQ scores, with scored significantly different scores from normally developing 100 as the mean (SD 15). children. Again, all children aged <6 years in the sample of Statistics normally developing children were excluded from these analyses, in order to match with the age of the children in the Data were analyzed with IBM SPSS Statistics version 21.0.0.0. ADHD group. Three types of analyses were used. For the All outliers (scores with z>3.29) within the concerned group continuous variables, general linear model univariate analyses (ie, normally developing children and the ADHD group) were were used. In four variables, the assumption of homogeneity in replaced by the mean + 3 times its standard deviation as advised variances was violated; therefore, nonparametric t tests by Field [38]. Means and standard deviations of all variables (Mann-Whitney) were used in these variables. In the categorical were calculated. variables, Pearson chi-square analyses were performed. Potential age-related differences on Timo’s Adventure within the sample of normally developing children were examined by Results conducting Pearson correlation (for scale outcome measurements; eg, number of irrelevant mouse clicks, time used Descriptive Statistics of All Variables to complete the task) and Spearman correlation analyses (for Means and standard deviations for all variables of both groups the ordinal outcome measurements; eg, correct or incorrect, did are reported together with the analyses of group differences. the child choose the large or small reward). Age was used as a All reported results are after correction for outliers. continuous variable in these analyses. Developmental Effects Second, a discriminant analysis was performed to investigate An overview of all correlation analyses between age and to which level the variables of the game can discriminate variables of Timo’s Adventure can be found in Table 2. between children belonging to the ADHD group and the normally developing children. Sensitivity and specificity were Pathway 1: Executive Functions measured. All children aged <6 years in the sample of normally Pearson correlation analyses with age as a continuous variable developing children were excluded from these analyses, in order showed significant correlations on 2 tasks: in the inhibition task to match with the age of the children in the ADHD group, (Monkey), older children had significantly fewer inhibition resulting in a control group of 57 children. An independent http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al failures (r=−.33, P=.001); and in the reaction time task (Magic Pathway 3: Time Perception Land), older children collected significantly more stars (r=.60, In the time production task (Balloon), older children produced P<.001) and were faster in collecting these stars (r=−.49, significantly more correct balloons (ρ= .35, P<.001) and had P<.001). more precise time productions than younger children (r=−.25, P=.01). Pathway 2: Reward Mechanisms No significant correlations were found in this pathway, indicating that age does not influence reward mechanisms. Table 2. Correlation between results on Timo’s Adventure and age for normally developing children (N=96). Variables P value Age Pathway 1: executive functions Dressing up, total score ρ=−.08 .46 Dressing up, clicks r=−.05 .66 Sandwich, total score ρ=.21 .05 Sandwich, clicks r=.09 .71 Balloon, clicks r=.14 .18 Monkey, failures r=−.33 .001 Monkey, clicks r=.08 .45 Magic Land, number of collected stars r=.60 <.001 Magic Land, average time for collected stars r=−.49 <.001 Magic Land, clicks r=.16 .14 Pathway 2: reward mechanisms Rocket, reward ρ=.05 .65 Rocket, time waited r=.07 .50 Pathway 3: time perception Balloon, correct ρ=.35 <.001 Balloon, average time for attempts r=−.25 .01 r: Pearson correlation; ρ: Spearman correlation. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 3. Structure matrix in discriminant analysis. Output variable Pooled within-group correlations between discriminating variables and standardized canonical discriminant functions Magic Land, number of clicks −.82 Magic Land, number of collected stars .40 Balloon, number of clicks .34 Monkey, number of failures .31 Dressing up, total score .29 Rocket, time waited −.27 Balloon, number of correct balloons −.18 Sandwich, total score .17 Sandwich, number of clicks .14 Magic Land, average reaction time for collected stars .11 Rocket, small (=0) or large (=1) reward −.10 Monkey, number of clicks .09 Balloon, average time taken to inflate balloons −.02 Dressing up, number of clicks .01 Group Differences on Individual Variables Differences Between Children With ADHD and Because the combination of all variables was useful in Healthy Controls discriminating between children with ADHD and normally Discriminant Analysis developing children, the specific variables for which children with ADHD had a different result from normally developing All variables were included in a discriminant analysis to children were examined. All significant differences between investigate whether Timo’s Adventure can discriminate between children with ADHD and healthy controls are reported. the children with ADHD and the healthy controls. A significant difference between the groups was found: Wilks Λ=.51, Pathway 1: Executive Functions χ =50.8, P<.001. The structure matrix (see Table 3) revealed Results of this pathway can be found in Table 4. There was a that especially the number of mouse clicks in several tasks and significant association between the group (ADHD or control) the mini-games on reaction time (Magic Land), inhibition 2 and the score on the Dressing up task (planning; χ =11.4, (Monkey), and planning (Dressing up) were potential predictors. P=.003, V=.35), indicating that typically developing children The classification results showed that 72% (41/57) children in had better scores than children with ADHD on a planning task. the control group were correctly classified, and 88% (35/40) of In the Magic Land task (simple reaction time), children with the children in the ADHD group were correctly classified as ADHD used more mouse clicks in collecting stars than children having ADHD. Overall, 78% (76/97) of the children were in the control group (U=389.50, P<.001, r=−.55). correctly classified as being in the ADHD group or in the control group. Sensitivity of Timo’s Adventure was 0.89 and specificity was 0.69. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 10 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 4. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 1, executive functions. Variable Group Mean SD Statistic P value Effect size Dressing up, total score (0-2) 2 a Controls 1.18 0.56 .003 χ =11.41 V =.35 0.93 0.81 ADHD Dressing up, number of clicks (minimum for a satisfying result is 4 clicks) c d Controls 17.59 14.32 .09 U =895.00 r =−.17 ADHD 26.14 22.61 Sandwich, total score (0-5) f 2 e Controls 0.89 1.10 .50 F =0.47   =.01 1,95 p ADHD 1.05 0.97 Sandwich, number of clicks (minimum for 5 correct sandwiches is 22 clicks) Controls 87.49 51.12 U=823.00 .25 r=−.12 ADHD 92.57 53.48 Balloon, number of clicks (minimum for 3 correct balloons is 6) Controls 108.52 93.91 U=945.50 .24 r=−.12 ADHD 206.87 227.87 Monkey, number of failures (minimum is 0) Controls 0.77 1.33 F =2.97 .09 1,96   =.03 ADHD 1.26 1.48 Monkey, number of clicks (minimum to complete the task is 6) Controls 29.72 2.20 F =0.82 .37 1,96   =.01 ADHD 33.83 1.62 Magic Land, number of collected stars (maximum is 50) Controls 37.61 10.87 F =2.92 .09 1,95   =0.03 ADHD 41.41 8.33 Magic Land, average reaction time for collected stars Controls 2.25 0.55 F =0.02 .88 1,95   =0.00 ADHD 2.22 0.48 Magic Land, number of clicks Controls 76.27 19.52 U=389.50 <.001 r=−.55 ADHD 155.65 89.68 V: Cramer’s V. ADHD: attention-deficit/hyperactivity disorder. U: Mann-Whitney test. r: Pearson correlation coefficient. e 2   : partial variance explained. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 11 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 5. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 2, reward mechanisms. Variable Group Mean SD Statistic P value Effect size Rocket, small (=0) or large (=1) reward 2 a Controls 0.55 0.50 .01 χ =7.3 V =.28 0.28 0.46 ADHD Rocket, time waited (minimum is 0 seconds, maximum is 120 seconds) 2 d Controls 80.98 50.05 F =5.52 .02 1,92   =.06 ADHD 56.26 50.13 V: Cramer’s V. ADHD: attention-deficit/hyperactivity disorder. c 2   : partial variance explained. for the delayed reward than the children in the ADHD group Pathway 2: Reward Mechanisms (F =5.52, P=.02,   =.06). 1,92 p Results of this pathway can be found in Table 5. In the Rocket task, a significant association between the group (ADHD or Pathway 3: Time Perception control) and whether or not a child chose the delayed reward Results of this pathway can be found in Table 6. No significant was found (χ =7.3, P=.01, V=.28). Also, the total time that the 1 differences between children with ADHD and normally child waited before he or she ended the task was significantly developing children were found in the time production task. different: children in the control group were able to wait longer Table 6. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 3, time perception. Variable Group Mean SD Statistic P value Effect size Balloon, number of correct balloons (minimum is 0, maximum is 3) 2 a Controls 2.25 1.09 F =0.07 .80 1,96   =.00 2.21 1.06 ADHD Balloon, average time taken to inflate balloons Controls 4.33 4.05 F =1.69 .20 1,96   =.02 ADHD 3.38 2.01 b 2   : partial variance explained. ADHD: attention-deficit/hyperactivity disorder. model by Sonuga-Barke and colleagues. To our knowledge, Discussion this is the first computerized tool in which all 3 pathways are assessed. The aim of this study was to investigate the clinical Principal Findings validity of Timo’s Adventure. Recently, the development and use of computerized tasks in The first proof of validity was found in the developmental measuring, for example, neurocognitive abilities is increasing effects of the game. In a population of 96 normally developing and results in promising effects in the field of interventions. For children between 4 and 8 years old, we found significant instance, children with ADHD benefit from game-based training correlations with age in 2 of the 3 pathways. The older the child, tools on executive functions such as Braingame Brian and the faster he or she is in completing the tasks. Furthermore, it Cogmed, as was reported in a review by Peijnenborgh et al [39]. was found that older children are better in inhibiting their Important elements of these training tools are believed to be the response on a go or no-go task. This is in line with previous use of fantasy, a story line, adaptation of the degree of difficulty, research (eg, [40,41]). Also, in a reaction time task, older and the use of immediate rewards. However, most studies focus children have better responses on alertness and have better on training tools, whereas in our research a diagnostic tool was reactions to visually presented stimuli after a visual warning studied. This computerized diagnostic tool, named Timo’s signal. Again, this is in line with previous research (eg, [42]). Adventure, was developed for young children (between 4 and Furthermore, age-related differences were found in the third 8 years old) to investigate the presence of the 3 distinct patterns pathway (ie, time perception), indicating that the older the child, of possible deficits in ADHD as described in the triple-pathway http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 12 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al the better he or she is in producing a predetermined time differences were found in the third pathway of Sonuga-Barke’s interval. This is in line with research by, for example, Friedman model (time perception). Although timing deficits are known and Laycock [43] and Pouthas and Jacquet [13] stating that in children with ADHD, it is not uncommon that time production development of time perception skills increases sharply at an tasks do not result in significant effects [49]. Further research early age (ie, before the age of 7 years) and refines in the last is necessary to gain more information on this aspect of possible grades of elementary school. Interestingly, no developmental ADHD-related deficits. effects were found in the second pathway (reward mechanisms), Finally, we found that user experiences were positive: when indicating that age does not influence the choice between asked afterward, 81% of the children said they liked the game immediate or delayed rewards. This might be caused by the fact very much, and an extra 14% of the children said that they liked that this aspect might be fully developed before the age of 4 the game. years, as the findings by Mischel et al [44] suggest. One limitation of this study is that information on reliability Further proof for clinical validity was found in satisfying results cannot be reported at this moment. Because Timo’s Adventure on the discriminant analysis, indicating that Timo’s Adventure consists of several independent functions, analysis of Cronbach was correct in most classifications. Sensitivity and specificity alpha would automatically result in low consistency between of the measurement were satisfying. Our results are similar to, the items. It would be interesting to test children several times, and sometimes even more promising than, other diagnostic to collect data for test-retest reliability analyses. Further research measurements. For instance, Williams and colleagues [45] could is necessary to examine the reliability of this instrument. classify 68% of the children with ADHD correctly when using Another interesting question for future research might be to IntegNeuro. However, only 2 of the 3 pathways are included in investigate possible effects of use of medication. In our analysis IntegNeuro, and it is not suitable for young children. all children with ADHD who were taking medication (18 in When looking more closely at the individual variables that help total) were excluded, but it might be possible that medication to differentiate between children with and without ADHD, we influences only one (or a combination) of the pathways. Finally, found several significant differences between both groups. In it would be interesting to determine which (or combination of) the first pathway (executive functions), we found that children tasks and corresponding outcome measurements are especially with ADHD have significantly more irrelevant mouse clicks on sensitive for the diagnosis of ADHD. It is possible that a total the Magic Land task (reaction time) than healthy peers. This score and normative data can be measured, which can be used indicates impulsiveness, motor restlessness, and hyperactive to determine a profile of ADHD symptoms. Future research is behavior, as is also suggested by Hervey and colleagues [46]. necessary to determine such a profile or total score. Also, children with ADHD had lower planning skills than Conclusions healthy controls had, what might be expected because planning This is the first time that all 3 pathways of Sonuga-Barke’s and organization are affected in children with ADHD [47]. In model are included in one diagnostic computerized tool with a several tasks we found results that were encouraging but not context of rewards and story line. In clinical care, diagnostic statistically significant (eg, children with ADHD tend to have instruments on time perception and reward mechanisms are more inhibition mistakes in the Monkey task and collect more scarce, but it is necessary to gain information on these aspects stars in the Magic Land than typically developing children). to complete an analysis of strengths and weaknesses of the child. This seems promising for the future: maybe, with some Proof for validity of Timo’s Adventure was found in adjustments to the tasks, sensitivity and specificity can even developmental effects and group differences between normally increase. developing children and children with ADHD, and Timo’s Evidence was also found for differences between the ADHD Adventure was satisfying accurately when classifying to which group and the controls in the second pathway (reward group (ie, the ADHD group or the healthy controls) the child mechanisms): children with ADHD chose the large but delayed belonged. This suggests that Timo’s Adventure can be of added reward less often than the control group and did not wait as long value in the diagnosis of ADHD because it helps in formulating as the control group before deciding to end the task. This is in a profile of strengths and weaknesses. Further research is line with previous research, which states that children with necessary to confirm these findings and to examine potential ADHD have an aversion for delay [48]. No significant effects of medication. Acknowledgments The authors are grateful for the collaboration with Pongpanote Gongsook, PhD, Christian Sallustro, PDEng, and Jun Hu, PhD, from the Eindhoven University of Technology (the Netherlands) in the development of Timo’s Adventure. Conflicts of Interest None declared. References 1. Swaab H, Bouma A, Hendriksen J, König C. Klinische Kinderneuropsychologie. Amsterdam: Boom; 2016. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 13 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al 2. Assef EC, Capovilla AG, Capovilla FC. Computerized stroop test to assess selective attention in children with attention deficit hyperactivity disorder. Span J Psychol 2007 May;10(1):33-40. [Medline: 17549876] 3. Messina F, Tiedemann KB, de Andrade ER, Primi R. Assessment of working memory in children with attention-deficit/hyperactivity disorder. J Atten Disord 2006 Aug;10(1):28-35. [doi: 10.1177/108705284299] [Medline: 16840590] 4. Luciana M, Nelson CA. Assessment of neuropsychological function through use of the Cambridge Neuropsychological Testing Automated Battery: performance in 4- to 12-year-old children. Dev Neuropsychol 2002;22(3):595-624. [doi: 10.1207/S15326942DN2203_3] [Medline: 12661972] 5. Bioulac S, Arfi L, Bouvard MP. Attention deficit/hyperactivity disorder and video games: a comparative study of hyperactive and control children. Eur Psychiatry 2008 Mar;23(2):134-141. [doi: 10.1016/j.eurpsy.2007.11.002] [Medline: 18206354] 6. Wouters P, van Nimwegen C, van Oostendorp H, van der Spek ED. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology 2013;105(2):249-265. [doi: 10.1037/a0031311] 7. Gorini A, Capideville CS, De LG, Mantovani F, Riva G. The role of immersion and narrative in mediated presence: the virtual hospital experience. Cyberpsychol Behav Soc Netw 2011 Mar;14(3):99-105. [doi: 10.1089/cyber.2010.0100] [Medline: 20649451] 8. Habgood MPJ, Ainsworth SE. Motivating Children to Learn Effectively: Exploring the Value of Intrinsic Integration in Educational Games. Journal of the Learning Sciences 2011 Apr 19;20(2):169-206. [doi: 10.1080/10508406.2010.508029] 9. Van der Spek ED, Sidorenkova T, Porskamp P, Rauterberg M. The effect of familiar and fantasy aesthetics on learning experience of serious games. In: Pisan Y, Sgouros NM, Marsh T, editors. Entertainment Computing – ICEC 2014. Berlin, Heidelberg: Springer; 2014:9783662452110-9783662452138. 10. Gongsook P, Peijnenborgh J, Sallustro C, Van der Spek E, Hu J, Bellotti F, et al. In: De Gloria A, editor. A diagnostic tool on time perception of children with ADHD. Paris: Springer; 2014:9783319121567-9783319121405. 11. McCarney R, Warner J, Iliffe S, van HR, Griffin M, Fisher P. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol 2007;7:30 [FREE Full text] [doi: 10.1186/1471-2288-7-30] [Medline: 17608932] 12. Diamond A. The early development of executive functions. In: Bialystok E, Craik FIM, editors. Lifespan Cognition - Mechanisms of Change. New York: Oxford University Press; 2006:9780195169539. 13. Pouthas V, Jacquet AY. A developmental study of timing behavior in 4 1/2- and 7-year-old children. J Exp Child Psychol 1987 Apr;43(2):282-299. [Medline: 3585248] 14. Friedman WJ, Lyon TD. Development of temporal-reconstructive abilities. Child Dev 2005;76(6):1202-1216. [doi: 10.1111/j.1467-8624.2005.00845.x] [Medline: 16274435] 15. Swanson J, Sergeant J, Taylor E, Sonuga-Barke E, Jensen P, Cantwell D. Attention-deficit hyperactivity disorder and hyperkinetic disorder. The Lancet 1998 Feb;351(9100):429-433. [doi: 10.1016/S0140-6736(97)11450-7] [Medline: 9482319] 16. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th edition. Washington, DC: Author; 2013:A. 17. González-Garrido AA, Gómez-Velázquez FR, Zarabozo D, López-Elizalde R, Ontiveros A, Madera-Carrillo H, et al. Time reproduction disturbances in ADHD children: an ERP study. Int J Neurosci 2008 Jan;118(1):119-135. [doi: 10.1080/00207450601042177] [Medline: 18041610] 18. Sonuga-Barke E, Bitsakou P, Thompson M. Beyond the dual pathway model: evidence for the dissociation of timing, inhibitory, and delay-related impairments in attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry 2010 Apr;49(4):345-355. [Medline: 20410727] 19. Barkley RA. Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Psychol Bull 1997 Jan;121(1):65-94. [Medline: 9000892] 20. Hurks PP, Hendriksen JG. Retrospective and prospective time deficits in childhood ADHD: The effects of task modality, duration, and symptom dimensions. Child Neuropsychol 2011;17(1):34-50. [doi: 10.1080/09297049.2010.514403] [Medline: 20936546] 21. Bauermeister JJ, Barkley RA, Martinez JV, Cumba E, Ramirez RR, Reina G, et al. Time estimation and performance on reproduction tasks in subtypes of children with attention deficit hyperactivity disorder. J Clin Child Adolesc Psychol 2005 Mar;34(1):151-162. [doi: 10.1207/s15374424jccp3401_14] [Medline: 15677289] 22. Prins P, Ten Brink EL, Dovis S, Ponsioen AJGB, Geurts H, De Groot H, et al. Braingame Brian: Een Executieve Functie training met game-elementen voor kinderen met cognitieve controleproblemen. Amsterdam: Lucertis & Shosho; 2010. 23. Cassuto H, Ben-Simon A, Berger I. Using environmental distractors in the diagnosis of ADHD. Front Hum Neurosci 2013;7:805 [FREE Full text] [doi: 10.3389/fnhum.2013.00805] [Medline: 24319423] 24. Goldman LS. Diagnosis and Treatment of Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. JAMA 1998 Apr 08;279(14):1100. [doi: 10.1001/jama.279.14.1100] 25. Wechsler D. WPPSI-III-NL Nederlandse bewerking: Technische Handleiding 2nd edition. Amsterdam: Netherlands: Pearson Assessment and Information BV; 2010. 26. Sallustro C. Timo&Friends: A diagnostic tool on the time perception of children with ADHD. Eindhoven: University of Technology; 2013:A. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 14 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al 27. Bellotti F, Berta R, De Gloria A. Designing Effective Serious Games: Opportunities and Challenges for Research. Int. J. Emerg. Technol. Learn 2010 Nov 24;5(SI3). [doi: 10.3991/ijet.v5s3.1500] 28. Boehm BW. A spiral model of software development and enhancement. Computer 1988 May;21(5):61-72. [doi: 10.1109/2.59] 29. Rauterberg M, Strohm O, Kirsch C. Benefits of user-oriented software development based on an iterative cyclic process model for simultaneous engineering. International Journal of Industrial Ergonomics 1995 Oct;16(4-6):391-409. [doi: 10.1016/0169-8141(95)00021-8] 30. Rauterberg M. An iterative-cyclic software process model. 1992 Presented at: Proceedings Fourth International Conference on Software Engineering and Knowledge Engineering; 1992; Zurich, Switzerland p. 600-607. [doi: 10.1109/SEKE.1992.227899] 31. Gongsook P. Interactive diagnostic game for time perception. Eindhoven: Technische Universiteit; 2015:978-990. 32. Craeynest P. Psychologie van de levensloop. Leuven: Acco; 2010. 33. Martinussen R, Hayden J, Hogg-Johnson S, Tannock R. A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry 2005 Apr;44(4):377-384. [doi: 10.1097/01.chi.0000153228.72591.73] [Medline: 15782085] 34. Gooch D, Snowling MJ, Hulme C. Reaction Time Variability in Children With ADHD Symptoms and/or Dyslexia. Developmental Neuropsychology 2012 Jul;37(5):453-472. [doi: 10.1080/87565641.2011.650809] [Medline: 22799763] 35. Grey I, Healy O, Leader G, Hayes D. Using a Time Timer to increase appropriate waiting behavior in a child with developmental disabilities. Res Dev Disabil 2009;30(2):359-366. [doi: 10.1016/j.ridd.2008.07.001] [Medline: 18926663] 36. Barkley RA, Koplowitz S, Anderson T, McMurray MB. Sense of time in children with ADHD: effects of duration, distraction, and stimulant medication. J Int Neuropsychol Soc 1997 Jul;3(4):359-369. [Medline: 9260445] 37. Alloway T. Can interactive working memory training improve learning? Journal of Interactive Learning Research 2002;23(3):197-207. 38. Field A. Discovering Statistics Using IBM SPSS Statistics 3rd edition. London: Sage Publications Ltd; 2009. 39. Peijnenborgh JC, Hurks PM, Aldenkamp AP, Vles JS, Hendriksen JG. Efficacy of working memory training in children and adolescents with learning disabilities: A review study and meta-analysis. Neuropsychol Rehabil 2015 Apr 17:1-28. [doi: 10.1080/09602011.2015.1026356] [Medline: 25886202] 40. Gathercole SE, Pickering SJ, Ambridge B, Wearing H. The structure of working memory from 4 to 15 years of age. Dev Psychol 2004 Mar;40(2):177-190. [doi: 10.1037/0012-1649.40.2.177] [Medline: 14979759] 41. Davidson MC, Amso D, Anderson LC, Diamond A. Development of cognitive control and executive functions from 4 to 13 years: evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia 2006;44(11):2037-2078 [FREE Full text] [doi: 10.1016/j.neuropsychologia.2006.02.006] [Medline: 16580701] 42. Thomas KM, Nelson CA. Serial reaction time learning in preschool- and school-age children. J Exp Child Psychol 2001 Aug;79(4):364-387. [doi: 10.1006/jecp.2000.2613] [Medline: 11511129] 43. Friedman WJ, Laycock F. Children's Analog and Digital Clock Knowledge. Child Development 1989 Apr;60(2):357. [doi: 10.2307/1130982] 44. Mischel W, Shoda Y, Rodriguez M. Delay of gratification in children. Science 1989 May 26;244(4907):933-938. [doi: 10.1126/science.2658056] [Medline: 2658056] 45. Williams LM, Hermens DF, Thein T, Clark CR, Cooper NJ, Clarke SD, et al. Using brain-based cognitive measures to support clinical decisions in ADHD. Pediatr Neurol 2010 Feb;42(2):118-126. [doi: 10.1016/j.pediatrneurol.2009.08.010] [Medline: 20117748] 46. Hervey AS, Epstein JN, Curry JF, Tonev S, Eugene AL, Keith CC, et al. Reaction time distribution analysis of neuropsychological performance in an ADHD sample. Child Neuropsychol 2006 Apr;12(2):125-140. [doi: 10.1080/09297040500499081] [Medline: 16754533] 47. Gau SS, Chiang H. Association between early attention-deficit/hyperactivity symptoms and current verbal and visuo-spatial short-term memory. Res Dev Disabil 2013 Jan;34(1):710-720. [doi: 10.1016/j.ridd.2012.10.005] [Medline: 23137723] 48. Bitsakou P, Psychogiou L, Thompson M, Sonuga-Barke EJ. Delay Aversion in Attention Deficit/Hyperactivity Disorder: an empirical investigation of the broader phenotype. Neuropsychologia 2009 Jan;47(2):446-456. [doi: 10.1016/j.neuropsychologia.2008.09.015] [Medline: 18929587] 49. Zakay D. The evasive art of subjective time measurement: Some methodological dilemmas. In: Block RA, editor. Cognitive models of psychological time. Hillsdale: Lawrence Erlbaum Associates; 1990:59-84. Abbreviations ADHD: attention-deficit/hyperactivity disorder DSM-V: Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) WPPSI-III-NL: Wechsler Preschool and Primary Scale of Intelligence, Dutch version http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 15 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Edited by M Alcañiz; submitted 23.05.16; peer-reviewed by JA Gil Gomez; comments to author 13.07.16; accepted 19.07.16; published 22.09.16 Please cite as: Peijnenborgh JCAW, Hurks PPM, Aldenkamp AP, van der Spek ED, Rauterberg GWM, Vles JSH, Hendriksen JGM A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4-8 Years JMIR Serious Games 2016;4(2):e15 URL: http://games.jmir.org/2016/2/e15/ doi: 10.2196/games.5997 PMID: 27658428 ©Janneke CAW Peijnenborgh, Petra PM Hurks, Albert P Aldenkamp, Erik D van der Spek, Matthias GWM Rauterberg, Johan SH Vles, Jos GM Hendriksen. Originally published in JMIR Serious Games (http://games.jmir.org), 22.09.2016. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 16 (page number not for citation purposes) XSL FO RenderX http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JMIR Serious Games JMIR Publications

A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4-8 Years

Download PDF
16 pages

Loading next page...
 
/lp/jmir-publications/a-study-on-the-validity-of-a-computer-based-game-to-assess-cognitive-Lk7SGMu1UV

References (54)

Publisher
JMIR Publications
Copyright
Copyright © The Author(s). Licensed under Creative Commons Attribution cc-by 4.0
ISSN
2291-9279
DOI
10.2196/games.5997
Publisher site
See Article on Publisher Site

Abstract

Background: A computer-based game, named Timo’s Adventure, was developed to assess specific cognitive functions (eg, attention, planning, and working memory), time perception, and reward mechanisms in young school-aged children. The game consists of 6 mini-games embedded in a story line and includes fantasy elements to enhance motivation. Objective: The aim of this study was to investigate the validity of Timo’s Adventure in normally developing children and in children with attention-deficit/hyperactivity disorder (ADHD). Methods: A total of 96 normally developing children aged 4-8 years and 40 children with ADHD were assessed using the game. Clinical validity was investigated by examining the effects of age on performances within the normally developing children, as well as performance differences between the healthy controls and the ADHD group. Results: Our analyses in the normally developing children showed developmental effects; that is, older children made fewer inhibition mistakes (r=−.33, P=.001), had faster (and therefore better) reaction times (r=−.49, P<.001), and were able to produce time intervals more accurately than younger children (ρ=.35, P<.001). Discriminant analysis showed that Timo’s Adventure was accurate in most classifications whether a child belonged to the ADHD group or the normally developing group: 78% (76/97) of the children were correctly classified as having ADHD or as being in the normally developing group. The classification results showed that 72% (41/57) children in the control group were correctly classified, and 88% (35/40) of the children in the ADHD group were correctly classified as having ADHD. Sensitivity (0.89) and specificity (0.69) of Timo’s Adventure were satisfying. Conclusions: Computer-based games seem to be a valid tool to assess specific strengths and weaknesses in young children with ADHD. (JMIR Serious Games 2016;4(2):e15) doi: 10.2196/games.5997 KEYWORDS experimental games; ADHD; children; neuropsychological test http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al For this purpose, age-related differences in a group of normally Introduction developing children (N=96) were examined. Assessment of children’s cognitive strengths and weaknesses Next, the scores on the mini-games of normally developing is an important focus of clinical child neuropsychological children were compared with those of children with research and clinical care worldwide [1]. Cognitive abilities are attention-deficit/hyperactivity disorder (ADHD; N=40). quantified traditionally by use of, for example, paper-and-pencil Attention-deficit/hyperactivity disorder is a developmental performance tests and more recently with computer-assisted disorder that is associated with academic difficulties and social tools [2,3]. Performances on traditional cognitive tests are disadvantage [15]. According to the Diagnostic and Statistical believed to be influenced significantly by noncognitive Manual of Mental Disorders (Fifth Edition; DSM-V) [16], 2 functions, such as motivation and perseverance [4]. Therefore, main areas of impairment in children with ADHD exist: a lower score on, for instance, a working memory test might inattention (eg, difficulty in maintaining attention during a task indicate a memory problem but also, for example, a decreased or problems in dividing attention) and hyperactivity and motivation. This overall performance score therefore only impulsive behavior (eg, acting out before thinking about the limitedly reflects the underlying “cause” in case of a decreased consequences). Previous research has found that cognitive performance, which makes this score difficult to interpret. In difficulties, more specifically in the domain of working memory order to test one’s cognitive abilities more purely, it would be and attention, occur in children with ADHD [17]. However, preferable to (1) optimize motivation in the test situation and according to the triple-pathway model by Sonuga-Barke et al (2) assess motivation in a separate test additionally— [18], not all children with ADHD have cognitive weaknesses. unfortunately, tests measuring motivation in children are In this model 3 distinct patterns of ADHD deficits are relatively scarcely used in the clinics. distinguished. The first pathway is related to cognitive functions and is called the inhibitory-based executive dysfunction. This On the basis of the literature, it is known that introducing pathway views ADHD as a disorder of dysregulation of thought immediate (vs delayed) rewards and the adaptation of item and action associated with diminished inhibitory control (ie, difficulty levels to the child’s abilities are likely to increase executive functions). In the second pathway, ADHD is explained motivation in children and help the child to stay focused on the as a motivational style associated with fundamental alterations tasks that he or she needs to do [5]. Also, introducing a context in reward mechanisms. Children with ADHD are assumed to (eg, by introducing cognitive tests in the context of a story) may prefer small immediate rewards over large delayed rewards, increase motivation, although data on the effects of including which results in inattentive, overactive, and impulsive behaviors a story line on motivation or engagement in games are so far [19]. The third pathway states that deficits in time perception, inconclusive [6]. The introduction of a story line improves the for instance, deficits in distinguishing between two time child’s feeling of being part of a gaming environment [7]. In intervals, producing time intervals, and estimating time, are addition, the use of intrinsic fantasy elements has been found another component of ADHD. Indeed, time perception deficits to improve motivation to conduct a specific task [8,9]. have been reported for children with ADHD [20]. However, the The above-mentioned increase in popularity of results are not consistent; that is, some authors report no computer-assisted assessment tools is partly caused by the fact ADHD-related deficits [21]. All 3 pathways are believed to that it is relatively easy to implement these immediate rewards, have their own neural substrate [18]. Sonuga-Barke and to adapt difficulty levels, to implement a story, and to use colleagues [18] found in children with ADHD aged between 6 intrinsic fantasy elements. This leads to a situation in which the and 17 years that delay aversion, poor executive functions, and child does not have the feeling of being assessed but instead poor time perception are core, but unrelated and independent, thinks that he or she is playing a game [10]. This is especially characteristics of ADHD. A person with ADHD can have important in tests that are designed for children because it is deficits in one of the pathways or a combination of pathways. known that children can behave differently when they know Neuropsychological measurements (which are used to examine they are being studied (also known as the Hawthorne effect possible deficits in one of the pathways) usually focus primarily [11]). For this purpose, we developed “Timo’s Adventure,” a on just one of the pathways, whereas one can conclude from computer-based game that consists of 6 mini-games. These the model by Sonuga-Barke et al that it is necessary to gain mini-games aim to assess different cognitive processes, for information on possible deficits in all 3 pathways. Sonuga-Barke example, attention, planning, and working memory; delay and colleagues used several distinctive computerized tasks to aversion, as a measure of motivation; and time perception (see collect information about the 3 pathways. These were, however, method section). The aim of this study was to examine the not connected in a fantasy gaming environment or by a story validity of Timo’s Adventure in assessing strengths and line. In our game, a story line was included in order to immerse weaknesses in the above-mentioned domains of young children, the player in an intrinsic fantasy and possibly improve the aged 4-8 years. Proof of validity was sought by studying group reliability of the diagnosis. To our knowledge no computerized differences in performance on the mini-games. Two types of diagnostic tools exist in which all 3 pathways are included in relevant group comparisons were made. First, the age of the combination with all motivation-enhancing elements discussed child is believed to be a relevant variable and was therefore above (including a story line or fantasy game elements), studied in relation to performances on the mini-games: we although some training tools with a story line exist (for instance, expected younger children (aged <6 years) to perform less well Braingame Brian [22]). In Timo’s Adventure story line, than older children on all mini-games (in line with studies on, distracting factors are included to measure real-life distraction. eg, cognitive development [12] and time perception [13,14]). Previous research found that distractors in a computerized http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al continuous performance test resulted in more distractibility in this study by their medical specialist. In parallel, parents of children with ADHD than in their healthy peers [23]. children enrolled in a special needs program for children with behavioral problems were informed by a letter about the study In summary, the aim of this study was to investigate the clinical and asked to participate, via the children’s school. Informed validity of Timo’s Adventure. consent of 62 children with a diagnosis of ADHD was acquired. A total of 22 children did not meet the inclusion criteria because Methods they had a comorbid DSM-V diagnosis (n=4), because they used medication for attentional problems and hyperactive behavior Participants (stimulants, atomoxetine, tricyclic antidepressants, or clonidine; Normally Developing Children n=6), or because of a combination of these exclusion criteria (n=12). The final dataset of the ADHD group consisted of 40 Parents of all children enrolled in the first 4 grades of 4 Dutch children (30 boys), all with a diagnosis of ADHD according to elementary schools were informed by a letter about the study. DSM-V. These diagnoses were made on the basis of a protocol Informed consent of 102 children was acquired. A total of 4 formulated by Goldman et al [24] that includes (1) extensive children were excluded because they were not native speakers history taking, (2) cognitive testing, (3) general physical and and instructions in Timo’s Adventure were in the Dutch neurological examination of the child, and (4) systematic language. In addition, 2 children were excluded because they assessment of ADHD characteristics by means of structured had a DSM-V diagnosis. The final dataset consisted of 96 questions based on the most recent version of the DSM [16]. children (43 boys), age ranging from 4 to 8 years. An overview Age range of the clinical sample was 6-8 years. An overview of characteristics for this group can be found in Table 1. of characteristics of the ADHD group can be found in Table 1. All children were tested individually in a private room at their Children enrolled in the special needs program for children with school. Approval for testing this sample was given by the Ethical behavioral problems were tested individually in a private room Review Committee of the Faculty of Psychology and at their school. The children who were patients of the Center Neuroscience of Maastricht University, the Netherlands. for Neurological Learning Disabilities were seen for Children With Attention-Deficit/Hyperactivity Disorder neuropsychological testing as part of clinical care. Parents of patients of the outpatient clinic Center for Approval for testing this sample was given by the Medical Neurological Learning Disabilities were asked to participate in Ethical Board of Kempenhaeghe. Table 1. Characteristics of all participants. Characteristics Typically developing ADHD group children Number of participants 96 40 Boys/girls 43/53 30/10 Age range, years 4-8 6-8 Mean age (SD), years 5.85 (1.33) 6.90 (0.74) 94.74 (10.79) 87.92 (13.05) Verbal IQ (WPPSI-III-NL Vocabulary), mean (SD) ADHD: attention-deficit/hyperactivity disorder. WPPSI-III-NL: Dutch version of the Wechsler Preschool and Primary Scale of Intelligence [25]. software cyclic model and the spiral model [28-30]. Besides Materials the designers of Eindhoven University of Technology, users were invited in the design process. Users in this case were Computer-Based Game Timo’s Adventure children (who helped us by explaining what they would like Timo’s Adventure is a single-player game. All tasks are and by making drawings) and psychologists from embedded in a story line: the main character in Timo’s Kempenhaeghe (who participated in the development of the Adventure is Timo, a friendly alien whose rocket has run out functionalities of the game, the visual graphics, and story line of fuel [26]. He asks the child to go on an adventure together that needed to match the age of the children). In the to collect stars that can be used as fuel. To complete all tasks, implementation phase, the functionalities of the game were it takes approximately 20 minutes. The game is categorized as created (by engineers of Eindhoven University of Technology a serious game: a game designed for specific purposes beyond in close collaboration with the psychologists from entertainment [27]. The game has a first-person view to simulate Kempenhaeghe). During the evaluation phase (consisting of a the feeling of presence, to make the child feel like he or she is user test with a paper prototype technique and an inside the game world. computer-based prototype), the game was played and evaluated Development of the game was divided into 3 stages: design, by children and by psychologists who were not part of the implementation, and evaluation, as based on the iterative http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al development team. This feedback was used to improve the game he shows the child pictures of the ingredients. The child needs and remove small bugs [31]. to remember these ingredients and select them in the same order as presented by Timo. The first sandwich starts with 2 The computer-based game consists of 6 different tasks (ie, 6 ingredients, adding up to 5 ingredients in the last sandwich. mini-games), each of which measures different neurocognitive This task is a measurement of the capacity of the visual working functions and thus gives information about possible deficits. memory. According to Craeynest [32], the capacity of the The 6 tasks measure aspects of executive functions, time working memory develops from remembering 2 targets when perception, and reward mechanisms to represent the 3 pathways the child is 2.5 years old, to 3 targets when 3 years old, and 5 of Sonuga-Barke’s model. All tasks were developed because targets when the child is 7 years old. Martinussen et al [33] of the need to measure the specific function and modeled after found that the capacity of the working memory in children with the theoretical background of the triple-pathway model. ADHD is markedly lower. In this task, the child can get a total Pathway 1: Executive Functions score of 5 points: one for each correct sandwich. All irrelevant mouse clicks are measured and thus give In the third task, the mini-game Monkey (inhibition), a monkey information on impulsivity and hyperactive behavior of the has thrown banana peels on the road (see Figure 3). To cross child. Furthermore, the following mini-games are included to the road, the child needs to swipe the banana peels and clear assess impairments in this pathway: Dressing up (planning), the road. However, if the monkey sees the child swiping banana Sandwich (working memory), Monkey (inhibition), and Magic peels, it will undo the child’s actions. The monkey is playing Land (simple reaction time). hide-and-seek and appears suddenly. The child needs to wait for the moment the monkey disappears. This is a go or no-go The first task, the mini-game Dressing up (planning), is set in task that gives information on the response inhibition of the the bedroom of the child. Timo tells the child that he or she child: is the child capable of inhibiting his or her response until needs to get dressed before the adventure can start. Several the monkey hides? Children with ADHD have deficits in this garments are spread throughout the room, which can be selected response inhibition and they can be inclined to react impulsively by clicking on them (see Figure 1). After clicking, the garment [19]. The number of failures (ie, when the monkey sees the moves toward the reflection of the child in the mirror and the action of the child) is the outcome variable of this task; the child gets dressed. This task gives information on the planning higher this score, the worse the inhibition skills. and organization skills of the child. The order in which the child selects the clothes is assessed to see whether the child is capable In the mini-game Magic Land (simple reaction time), stars shoot of planning his or her actions in the right order. The child can upward from magic holes (see Figure 4). The child needs to get a total score of 2 points: 1 point for being completely dressed collect these stars. If the child does not react within 2 seconds, and 1 point for using an executable and correct manner. the stars will disappear. The task ends after 50 stars shoot upward. Outcome variables in this task are the number of The second task, the mini-game Sandwich (working memory), collected stars (with a maximum of 50) and average reaction is set in the kitchen (see Figure 2). Timo tells the child that it time for collected stars. Slower and more variable reaction times is necessary to eat something before starting the adventure, and have been found to be a characteristic of ADHD [34]. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 1. Screenshot of the mini-game Dressing up (planning). Figure 2. Screenshot of the mini-game Sandwich (working memory). http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 3. Screenshot of the mini-game Monkey (inhibition). Figure 4. Screenshot of the mini-game Magic Land (simple reaction time). http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 5. Screenshot of the mini-game Rocket (reward mechanisms). a time interval of 10 seconds. A produced interval between 9 Pathway 2: Reward Mechanisms and 11 seconds results in a perfect balloon. When the produced In the task mini-game Rocket (delay aversion; see Figure 5), interval is smaller than 9 seconds the balloon falls into the water, the child gets a choice between an immediate but small reward and a produced interval larger than 11 seconds results in a (ending of the task) or a delayed but bigger reward (a flight in balloon that flies away. The child can make a maximum of 3 the rocket, after 2 minutes of waiting). The child can end the perfect balloons, or the task will end after 3 minutes with a task at any moment. Impulsive behavior occurs when responding perfect balloon (regardless of what the interval is). Barkley et produces more immediate, relatively smaller rewards at the cost al [36] suggested that the estimation of temporal intervals is of delayed, larger rewards [35]. Outcome variables are whether atypical in children with ADHD. The number of correct balloons the child chooses a small or big reward and how long (in is an outcome variable. Furthermore, the average production seconds) the child waited. interval for the first 3 balloons is measured by subtracting 10 seconds from each of the first 3 balloons, transforming these Pathway 3: Time Perception scores to absolute scores, adding these scores, and then dividing The task mini-game Balloon (time production) is set at a river them by 3. The higher this score, the less precise the mean with a broken bridge (see Figure 6). To cross the river, the child produced intervals are. needs to inflate a balloon with the balloon machine by producing http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Figure 6. Screenshot of the mini-game Balloon (time perception). samples t test revealed that children in the ADHD group and Vocabulary children in the normally developing group were equal in terms This subscale of the Dutch version of the Wechsler Preschool of Vocabulary scores (ADHD group: mean 87.92, SD 13.05; and Primary Scale of Intelligence (WPPSI-III-NL [25]) was control group: mean 94.74, SD 10.79; t =1.92, P=.11), therefore used to estimate verbal intelligence [37]. In this task, the child it was not necessary to correct for IQ in further analyses. was asked to give definitions of words such as “umbrella” or Finally, it was analyzed on which variables children with ADHD “shoe.” The total score can be transformed to IQ scores, with scored significantly different scores from normally developing 100 as the mean (SD 15). children. Again, all children aged <6 years in the sample of Statistics normally developing children were excluded from these analyses, in order to match with the age of the children in the Data were analyzed with IBM SPSS Statistics version 21.0.0.0. ADHD group. Three types of analyses were used. For the All outliers (scores with z>3.29) within the concerned group continuous variables, general linear model univariate analyses (ie, normally developing children and the ADHD group) were were used. In four variables, the assumption of homogeneity in replaced by the mean + 3 times its standard deviation as advised variances was violated; therefore, nonparametric t tests by Field [38]. Means and standard deviations of all variables (Mann-Whitney) were used in these variables. In the categorical were calculated. variables, Pearson chi-square analyses were performed. Potential age-related differences on Timo’s Adventure within the sample of normally developing children were examined by Results conducting Pearson correlation (for scale outcome measurements; eg, number of irrelevant mouse clicks, time used Descriptive Statistics of All Variables to complete the task) and Spearman correlation analyses (for Means and standard deviations for all variables of both groups the ordinal outcome measurements; eg, correct or incorrect, did are reported together with the analyses of group differences. the child choose the large or small reward). Age was used as a All reported results are after correction for outliers. continuous variable in these analyses. Developmental Effects Second, a discriminant analysis was performed to investigate An overview of all correlation analyses between age and to which level the variables of the game can discriminate variables of Timo’s Adventure can be found in Table 2. between children belonging to the ADHD group and the normally developing children. Sensitivity and specificity were Pathway 1: Executive Functions measured. All children aged <6 years in the sample of normally Pearson correlation analyses with age as a continuous variable developing children were excluded from these analyses, in order showed significant correlations on 2 tasks: in the inhibition task to match with the age of the children in the ADHD group, (Monkey), older children had significantly fewer inhibition resulting in a control group of 57 children. An independent http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al failures (r=−.33, P=.001); and in the reaction time task (Magic Pathway 3: Time Perception Land), older children collected significantly more stars (r=.60, In the time production task (Balloon), older children produced P<.001) and were faster in collecting these stars (r=−.49, significantly more correct balloons (ρ= .35, P<.001) and had P<.001). more precise time productions than younger children (r=−.25, P=.01). Pathway 2: Reward Mechanisms No significant correlations were found in this pathway, indicating that age does not influence reward mechanisms. Table 2. Correlation between results on Timo’s Adventure and age for normally developing children (N=96). Variables P value Age Pathway 1: executive functions Dressing up, total score ρ=−.08 .46 Dressing up, clicks r=−.05 .66 Sandwich, total score ρ=.21 .05 Sandwich, clicks r=.09 .71 Balloon, clicks r=.14 .18 Monkey, failures r=−.33 .001 Monkey, clicks r=.08 .45 Magic Land, number of collected stars r=.60 <.001 Magic Land, average time for collected stars r=−.49 <.001 Magic Land, clicks r=.16 .14 Pathway 2: reward mechanisms Rocket, reward ρ=.05 .65 Rocket, time waited r=.07 .50 Pathway 3: time perception Balloon, correct ρ=.35 <.001 Balloon, average time for attempts r=−.25 .01 r: Pearson correlation; ρ: Spearman correlation. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 3. Structure matrix in discriminant analysis. Output variable Pooled within-group correlations between discriminating variables and standardized canonical discriminant functions Magic Land, number of clicks −.82 Magic Land, number of collected stars .40 Balloon, number of clicks .34 Monkey, number of failures .31 Dressing up, total score .29 Rocket, time waited −.27 Balloon, number of correct balloons −.18 Sandwich, total score .17 Sandwich, number of clicks .14 Magic Land, average reaction time for collected stars .11 Rocket, small (=0) or large (=1) reward −.10 Monkey, number of clicks .09 Balloon, average time taken to inflate balloons −.02 Dressing up, number of clicks .01 Group Differences on Individual Variables Differences Between Children With ADHD and Because the combination of all variables was useful in Healthy Controls discriminating between children with ADHD and normally Discriminant Analysis developing children, the specific variables for which children with ADHD had a different result from normally developing All variables were included in a discriminant analysis to children were examined. All significant differences between investigate whether Timo’s Adventure can discriminate between children with ADHD and healthy controls are reported. the children with ADHD and the healthy controls. A significant difference between the groups was found: Wilks Λ=.51, Pathway 1: Executive Functions χ =50.8, P<.001. The structure matrix (see Table 3) revealed Results of this pathway can be found in Table 4. There was a that especially the number of mouse clicks in several tasks and significant association between the group (ADHD or control) the mini-games on reaction time (Magic Land), inhibition 2 and the score on the Dressing up task (planning; χ =11.4, (Monkey), and planning (Dressing up) were potential predictors. P=.003, V=.35), indicating that typically developing children The classification results showed that 72% (41/57) children in had better scores than children with ADHD on a planning task. the control group were correctly classified, and 88% (35/40) of In the Magic Land task (simple reaction time), children with the children in the ADHD group were correctly classified as ADHD used more mouse clicks in collecting stars than children having ADHD. Overall, 78% (76/97) of the children were in the control group (U=389.50, P<.001, r=−.55). correctly classified as being in the ADHD group or in the control group. Sensitivity of Timo’s Adventure was 0.89 and specificity was 0.69. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 10 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 4. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 1, executive functions. Variable Group Mean SD Statistic P value Effect size Dressing up, total score (0-2) 2 a Controls 1.18 0.56 .003 χ =11.41 V =.35 0.93 0.81 ADHD Dressing up, number of clicks (minimum for a satisfying result is 4 clicks) c d Controls 17.59 14.32 .09 U =895.00 r =−.17 ADHD 26.14 22.61 Sandwich, total score (0-5) f 2 e Controls 0.89 1.10 .50 F =0.47   =.01 1,95 p ADHD 1.05 0.97 Sandwich, number of clicks (minimum for 5 correct sandwiches is 22 clicks) Controls 87.49 51.12 U=823.00 .25 r=−.12 ADHD 92.57 53.48 Balloon, number of clicks (minimum for 3 correct balloons is 6) Controls 108.52 93.91 U=945.50 .24 r=−.12 ADHD 206.87 227.87 Monkey, number of failures (minimum is 0) Controls 0.77 1.33 F =2.97 .09 1,96   =.03 ADHD 1.26 1.48 Monkey, number of clicks (minimum to complete the task is 6) Controls 29.72 2.20 F =0.82 .37 1,96   =.01 ADHD 33.83 1.62 Magic Land, number of collected stars (maximum is 50) Controls 37.61 10.87 F =2.92 .09 1,95   =0.03 ADHD 41.41 8.33 Magic Land, average reaction time for collected stars Controls 2.25 0.55 F =0.02 .88 1,95   =0.00 ADHD 2.22 0.48 Magic Land, number of clicks Controls 76.27 19.52 U=389.50 <.001 r=−.55 ADHD 155.65 89.68 V: Cramer’s V. ADHD: attention-deficit/hyperactivity disorder. U: Mann-Whitney test. r: Pearson correlation coefficient. e 2   : partial variance explained. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 11 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Table 5. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 2, reward mechanisms. Variable Group Mean SD Statistic P value Effect size Rocket, small (=0) or large (=1) reward 2 a Controls 0.55 0.50 .01 χ =7.3 V =.28 0.28 0.46 ADHD Rocket, time waited (minimum is 0 seconds, maximum is 120 seconds) 2 d Controls 80.98 50.05 F =5.52 .02 1,92   =.06 ADHD 56.26 50.13 V: Cramer’s V. ADHD: attention-deficit/hyperactivity disorder. c 2   : partial variance explained. for the delayed reward than the children in the ADHD group Pathway 2: Reward Mechanisms (F =5.52, P=.02,   =.06). 1,92 p Results of this pathway can be found in Table 5. In the Rocket task, a significant association between the group (ADHD or Pathway 3: Time Perception control) and whether or not a child chose the delayed reward Results of this pathway can be found in Table 6. No significant was found (χ =7.3, P=.01, V=.28). Also, the total time that the 1 differences between children with ADHD and normally child waited before he or she ended the task was significantly developing children were found in the time production task. different: children in the control group were able to wait longer Table 6. Results on Timo’s Adventure for normally developing children in the control group (N=56) and children in the attention-deficit/hyperactivity disorder group (N=40) in pathway 3, time perception. Variable Group Mean SD Statistic P value Effect size Balloon, number of correct balloons (minimum is 0, maximum is 3) 2 a Controls 2.25 1.09 F =0.07 .80 1,96   =.00 2.21 1.06 ADHD Balloon, average time taken to inflate balloons Controls 4.33 4.05 F =1.69 .20 1,96   =.02 ADHD 3.38 2.01 b 2   : partial variance explained. ADHD: attention-deficit/hyperactivity disorder. model by Sonuga-Barke and colleagues. To our knowledge, Discussion this is the first computerized tool in which all 3 pathways are assessed. The aim of this study was to investigate the clinical Principal Findings validity of Timo’s Adventure. Recently, the development and use of computerized tasks in The first proof of validity was found in the developmental measuring, for example, neurocognitive abilities is increasing effects of the game. In a population of 96 normally developing and results in promising effects in the field of interventions. For children between 4 and 8 years old, we found significant instance, children with ADHD benefit from game-based training correlations with age in 2 of the 3 pathways. The older the child, tools on executive functions such as Braingame Brian and the faster he or she is in completing the tasks. Furthermore, it Cogmed, as was reported in a review by Peijnenborgh et al [39]. was found that older children are better in inhibiting their Important elements of these training tools are believed to be the response on a go or no-go task. This is in line with previous use of fantasy, a story line, adaptation of the degree of difficulty, research (eg, [40,41]). Also, in a reaction time task, older and the use of immediate rewards. However, most studies focus children have better responses on alertness and have better on training tools, whereas in our research a diagnostic tool was reactions to visually presented stimuli after a visual warning studied. This computerized diagnostic tool, named Timo’s signal. Again, this is in line with previous research (eg, [42]). Adventure, was developed for young children (between 4 and Furthermore, age-related differences were found in the third 8 years old) to investigate the presence of the 3 distinct patterns pathway (ie, time perception), indicating that the older the child, of possible deficits in ADHD as described in the triple-pathway http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 12 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al the better he or she is in producing a predetermined time differences were found in the third pathway of Sonuga-Barke’s interval. This is in line with research by, for example, Friedman model (time perception). Although timing deficits are known and Laycock [43] and Pouthas and Jacquet [13] stating that in children with ADHD, it is not uncommon that time production development of time perception skills increases sharply at an tasks do not result in significant effects [49]. Further research early age (ie, before the age of 7 years) and refines in the last is necessary to gain more information on this aspect of possible grades of elementary school. Interestingly, no developmental ADHD-related deficits. effects were found in the second pathway (reward mechanisms), Finally, we found that user experiences were positive: when indicating that age does not influence the choice between asked afterward, 81% of the children said they liked the game immediate or delayed rewards. This might be caused by the fact very much, and an extra 14% of the children said that they liked that this aspect might be fully developed before the age of 4 the game. years, as the findings by Mischel et al [44] suggest. One limitation of this study is that information on reliability Further proof for clinical validity was found in satisfying results cannot be reported at this moment. Because Timo’s Adventure on the discriminant analysis, indicating that Timo’s Adventure consists of several independent functions, analysis of Cronbach was correct in most classifications. Sensitivity and specificity alpha would automatically result in low consistency between of the measurement were satisfying. Our results are similar to, the items. It would be interesting to test children several times, and sometimes even more promising than, other diagnostic to collect data for test-retest reliability analyses. Further research measurements. For instance, Williams and colleagues [45] could is necessary to examine the reliability of this instrument. classify 68% of the children with ADHD correctly when using Another interesting question for future research might be to IntegNeuro. However, only 2 of the 3 pathways are included in investigate possible effects of use of medication. In our analysis IntegNeuro, and it is not suitable for young children. all children with ADHD who were taking medication (18 in When looking more closely at the individual variables that help total) were excluded, but it might be possible that medication to differentiate between children with and without ADHD, we influences only one (or a combination) of the pathways. Finally, found several significant differences between both groups. In it would be interesting to determine which (or combination of) the first pathway (executive functions), we found that children tasks and corresponding outcome measurements are especially with ADHD have significantly more irrelevant mouse clicks on sensitive for the diagnosis of ADHD. It is possible that a total the Magic Land task (reaction time) than healthy peers. This score and normative data can be measured, which can be used indicates impulsiveness, motor restlessness, and hyperactive to determine a profile of ADHD symptoms. Future research is behavior, as is also suggested by Hervey and colleagues [46]. necessary to determine such a profile or total score. Also, children with ADHD had lower planning skills than Conclusions healthy controls had, what might be expected because planning This is the first time that all 3 pathways of Sonuga-Barke’s and organization are affected in children with ADHD [47]. In model are included in one diagnostic computerized tool with a several tasks we found results that were encouraging but not context of rewards and story line. In clinical care, diagnostic statistically significant (eg, children with ADHD tend to have instruments on time perception and reward mechanisms are more inhibition mistakes in the Monkey task and collect more scarce, but it is necessary to gain information on these aspects stars in the Magic Land than typically developing children). to complete an analysis of strengths and weaknesses of the child. This seems promising for the future: maybe, with some Proof for validity of Timo’s Adventure was found in adjustments to the tasks, sensitivity and specificity can even developmental effects and group differences between normally increase. developing children and children with ADHD, and Timo’s Evidence was also found for differences between the ADHD Adventure was satisfying accurately when classifying to which group and the controls in the second pathway (reward group (ie, the ADHD group or the healthy controls) the child mechanisms): children with ADHD chose the large but delayed belonged. This suggests that Timo’s Adventure can be of added reward less often than the control group and did not wait as long value in the diagnosis of ADHD because it helps in formulating as the control group before deciding to end the task. This is in a profile of strengths and weaknesses. Further research is line with previous research, which states that children with necessary to confirm these findings and to examine potential ADHD have an aversion for delay [48]. No significant effects of medication. Acknowledgments The authors are grateful for the collaboration with Pongpanote Gongsook, PhD, Christian Sallustro, PDEng, and Jun Hu, PhD, from the Eindhoven University of Technology (the Netherlands) in the development of Timo’s Adventure. Conflicts of Interest None declared. References 1. Swaab H, Bouma A, Hendriksen J, König C. Klinische Kinderneuropsychologie. Amsterdam: Boom; 2016. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 13 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al 2. Assef EC, Capovilla AG, Capovilla FC. Computerized stroop test to assess selective attention in children with attention deficit hyperactivity disorder. Span J Psychol 2007 May;10(1):33-40. [Medline: 17549876] 3. Messina F, Tiedemann KB, de Andrade ER, Primi R. Assessment of working memory in children with attention-deficit/hyperactivity disorder. J Atten Disord 2006 Aug;10(1):28-35. [doi: 10.1177/108705284299] [Medline: 16840590] 4. Luciana M, Nelson CA. Assessment of neuropsychological function through use of the Cambridge Neuropsychological Testing Automated Battery: performance in 4- to 12-year-old children. Dev Neuropsychol 2002;22(3):595-624. [doi: 10.1207/S15326942DN2203_3] [Medline: 12661972] 5. Bioulac S, Arfi L, Bouvard MP. Attention deficit/hyperactivity disorder and video games: a comparative study of hyperactive and control children. Eur Psychiatry 2008 Mar;23(2):134-141. [doi: 10.1016/j.eurpsy.2007.11.002] [Medline: 18206354] 6. Wouters P, van Nimwegen C, van Oostendorp H, van der Spek ED. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology 2013;105(2):249-265. [doi: 10.1037/a0031311] 7. Gorini A, Capideville CS, De LG, Mantovani F, Riva G. The role of immersion and narrative in mediated presence: the virtual hospital experience. Cyberpsychol Behav Soc Netw 2011 Mar;14(3):99-105. [doi: 10.1089/cyber.2010.0100] [Medline: 20649451] 8. Habgood MPJ, Ainsworth SE. Motivating Children to Learn Effectively: Exploring the Value of Intrinsic Integration in Educational Games. Journal of the Learning Sciences 2011 Apr 19;20(2):169-206. [doi: 10.1080/10508406.2010.508029] 9. Van der Spek ED, Sidorenkova T, Porskamp P, Rauterberg M. The effect of familiar and fantasy aesthetics on learning experience of serious games. In: Pisan Y, Sgouros NM, Marsh T, editors. Entertainment Computing – ICEC 2014. Berlin, Heidelberg: Springer; 2014:9783662452110-9783662452138. 10. Gongsook P, Peijnenborgh J, Sallustro C, Van der Spek E, Hu J, Bellotti F, et al. In: De Gloria A, editor. A diagnostic tool on time perception of children with ADHD. Paris: Springer; 2014:9783319121567-9783319121405. 11. McCarney R, Warner J, Iliffe S, van HR, Griffin M, Fisher P. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol 2007;7:30 [FREE Full text] [doi: 10.1186/1471-2288-7-30] [Medline: 17608932] 12. Diamond A. The early development of executive functions. In: Bialystok E, Craik FIM, editors. Lifespan Cognition - Mechanisms of Change. New York: Oxford University Press; 2006:9780195169539. 13. Pouthas V, Jacquet AY. A developmental study of timing behavior in 4 1/2- and 7-year-old children. J Exp Child Psychol 1987 Apr;43(2):282-299. [Medline: 3585248] 14. Friedman WJ, Lyon TD. Development of temporal-reconstructive abilities. Child Dev 2005;76(6):1202-1216. [doi: 10.1111/j.1467-8624.2005.00845.x] [Medline: 16274435] 15. Swanson J, Sergeant J, Taylor E, Sonuga-Barke E, Jensen P, Cantwell D. Attention-deficit hyperactivity disorder and hyperkinetic disorder. The Lancet 1998 Feb;351(9100):429-433. [doi: 10.1016/S0140-6736(97)11450-7] [Medline: 9482319] 16. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th edition. Washington, DC: Author; 2013:A. 17. González-Garrido AA, Gómez-Velázquez FR, Zarabozo D, López-Elizalde R, Ontiveros A, Madera-Carrillo H, et al. Time reproduction disturbances in ADHD children: an ERP study. Int J Neurosci 2008 Jan;118(1):119-135. [doi: 10.1080/00207450601042177] [Medline: 18041610] 18. Sonuga-Barke E, Bitsakou P, Thompson M. Beyond the dual pathway model: evidence for the dissociation of timing, inhibitory, and delay-related impairments in attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry 2010 Apr;49(4):345-355. [Medline: 20410727] 19. Barkley RA. Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Psychol Bull 1997 Jan;121(1):65-94. [Medline: 9000892] 20. Hurks PP, Hendriksen JG. Retrospective and prospective time deficits in childhood ADHD: The effects of task modality, duration, and symptom dimensions. Child Neuropsychol 2011;17(1):34-50. [doi: 10.1080/09297049.2010.514403] [Medline: 20936546] 21. Bauermeister JJ, Barkley RA, Martinez JV, Cumba E, Ramirez RR, Reina G, et al. Time estimation and performance on reproduction tasks in subtypes of children with attention deficit hyperactivity disorder. J Clin Child Adolesc Psychol 2005 Mar;34(1):151-162. [doi: 10.1207/s15374424jccp3401_14] [Medline: 15677289] 22. Prins P, Ten Brink EL, Dovis S, Ponsioen AJGB, Geurts H, De Groot H, et al. Braingame Brian: Een Executieve Functie training met game-elementen voor kinderen met cognitieve controleproblemen. Amsterdam: Lucertis & Shosho; 2010. 23. Cassuto H, Ben-Simon A, Berger I. Using environmental distractors in the diagnosis of ADHD. Front Hum Neurosci 2013;7:805 [FREE Full text] [doi: 10.3389/fnhum.2013.00805] [Medline: 24319423] 24. Goldman LS. Diagnosis and Treatment of Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. JAMA 1998 Apr 08;279(14):1100. [doi: 10.1001/jama.279.14.1100] 25. Wechsler D. WPPSI-III-NL Nederlandse bewerking: Technische Handleiding 2nd edition. Amsterdam: Netherlands: Pearson Assessment and Information BV; 2010. 26. Sallustro C. Timo&Friends: A diagnostic tool on the time perception of children with ADHD. Eindhoven: University of Technology; 2013:A. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 14 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al 27. Bellotti F, Berta R, De Gloria A. Designing Effective Serious Games: Opportunities and Challenges for Research. Int. J. Emerg. Technol. Learn 2010 Nov 24;5(SI3). [doi: 10.3991/ijet.v5s3.1500] 28. Boehm BW. A spiral model of software development and enhancement. Computer 1988 May;21(5):61-72. [doi: 10.1109/2.59] 29. Rauterberg M, Strohm O, Kirsch C. Benefits of user-oriented software development based on an iterative cyclic process model for simultaneous engineering. International Journal of Industrial Ergonomics 1995 Oct;16(4-6):391-409. [doi: 10.1016/0169-8141(95)00021-8] 30. Rauterberg M. An iterative-cyclic software process model. 1992 Presented at: Proceedings Fourth International Conference on Software Engineering and Knowledge Engineering; 1992; Zurich, Switzerland p. 600-607. [doi: 10.1109/SEKE.1992.227899] 31. Gongsook P. Interactive diagnostic game for time perception. Eindhoven: Technische Universiteit; 2015:978-990. 32. Craeynest P. Psychologie van de levensloop. Leuven: Acco; 2010. 33. Martinussen R, Hayden J, Hogg-Johnson S, Tannock R. A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry 2005 Apr;44(4):377-384. [doi: 10.1097/01.chi.0000153228.72591.73] [Medline: 15782085] 34. Gooch D, Snowling MJ, Hulme C. Reaction Time Variability in Children With ADHD Symptoms and/or Dyslexia. Developmental Neuropsychology 2012 Jul;37(5):453-472. [doi: 10.1080/87565641.2011.650809] [Medline: 22799763] 35. Grey I, Healy O, Leader G, Hayes D. Using a Time Timer to increase appropriate waiting behavior in a child with developmental disabilities. Res Dev Disabil 2009;30(2):359-366. [doi: 10.1016/j.ridd.2008.07.001] [Medline: 18926663] 36. Barkley RA, Koplowitz S, Anderson T, McMurray MB. Sense of time in children with ADHD: effects of duration, distraction, and stimulant medication. J Int Neuropsychol Soc 1997 Jul;3(4):359-369. [Medline: 9260445] 37. Alloway T. Can interactive working memory training improve learning? Journal of Interactive Learning Research 2002;23(3):197-207. 38. Field A. Discovering Statistics Using IBM SPSS Statistics 3rd edition. London: Sage Publications Ltd; 2009. 39. Peijnenborgh JC, Hurks PM, Aldenkamp AP, Vles JS, Hendriksen JG. Efficacy of working memory training in children and adolescents with learning disabilities: A review study and meta-analysis. Neuropsychol Rehabil 2015 Apr 17:1-28. [doi: 10.1080/09602011.2015.1026356] [Medline: 25886202] 40. Gathercole SE, Pickering SJ, Ambridge B, Wearing H. The structure of working memory from 4 to 15 years of age. Dev Psychol 2004 Mar;40(2):177-190. [doi: 10.1037/0012-1649.40.2.177] [Medline: 14979759] 41. Davidson MC, Amso D, Anderson LC, Diamond A. Development of cognitive control and executive functions from 4 to 13 years: evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia 2006;44(11):2037-2078 [FREE Full text] [doi: 10.1016/j.neuropsychologia.2006.02.006] [Medline: 16580701] 42. Thomas KM, Nelson CA. Serial reaction time learning in preschool- and school-age children. J Exp Child Psychol 2001 Aug;79(4):364-387. [doi: 10.1006/jecp.2000.2613] [Medline: 11511129] 43. Friedman WJ, Laycock F. Children's Analog and Digital Clock Knowledge. Child Development 1989 Apr;60(2):357. [doi: 10.2307/1130982] 44. Mischel W, Shoda Y, Rodriguez M. Delay of gratification in children. Science 1989 May 26;244(4907):933-938. [doi: 10.1126/science.2658056] [Medline: 2658056] 45. Williams LM, Hermens DF, Thein T, Clark CR, Cooper NJ, Clarke SD, et al. Using brain-based cognitive measures to support clinical decisions in ADHD. Pediatr Neurol 2010 Feb;42(2):118-126. [doi: 10.1016/j.pediatrneurol.2009.08.010] [Medline: 20117748] 46. Hervey AS, Epstein JN, Curry JF, Tonev S, Eugene AL, Keith CC, et al. Reaction time distribution analysis of neuropsychological performance in an ADHD sample. Child Neuropsychol 2006 Apr;12(2):125-140. [doi: 10.1080/09297040500499081] [Medline: 16754533] 47. Gau SS, Chiang H. Association between early attention-deficit/hyperactivity symptoms and current verbal and visuo-spatial short-term memory. Res Dev Disabil 2013 Jan;34(1):710-720. [doi: 10.1016/j.ridd.2012.10.005] [Medline: 23137723] 48. Bitsakou P, Psychogiou L, Thompson M, Sonuga-Barke EJ. Delay Aversion in Attention Deficit/Hyperactivity Disorder: an empirical investigation of the broader phenotype. Neuropsychologia 2009 Jan;47(2):446-456. [doi: 10.1016/j.neuropsychologia.2008.09.015] [Medline: 18929587] 49. Zakay D. The evasive art of subjective time measurement: Some methodological dilemmas. In: Block RA, editor. Cognitive models of psychological time. Hillsdale: Lawrence Erlbaum Associates; 1990:59-84. Abbreviations ADHD: attention-deficit/hyperactivity disorder DSM-V: Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) WPPSI-III-NL: Wechsler Preschool and Primary Scale of Intelligence, Dutch version http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 15 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Peijnenborgh et al Edited by M Alcañiz; submitted 23.05.16; peer-reviewed by JA Gil Gomez; comments to author 13.07.16; accepted 19.07.16; published 22.09.16 Please cite as: Peijnenborgh JCAW, Hurks PPM, Aldenkamp AP, van der Spek ED, Rauterberg GWM, Vles JSH, Hendriksen JGM A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4-8 Years JMIR Serious Games 2016;4(2):e15 URL: http://games.jmir.org/2016/2/e15/ doi: 10.2196/games.5997 PMID: 27658428 ©Janneke CAW Peijnenborgh, Petra PM Hurks, Albert P Aldenkamp, Erik D van der Spek, Matthias GWM Rauterberg, Johan SH Vles, Jos GM Hendriksen. Originally published in JMIR Serious Games (http://games.jmir.org), 22.09.2016. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included. http://games.jmir.org/2016/2/e15/ JMIR Serious Games 2016 | vol. 4 | iss. 2 | e15 | p. 16 (page number not for citation purposes) XSL FO RenderX

Journal

JMIR Serious Games – JMIR Publications

Published: Sep 22, 2016

Keywords: experimental games; ADHD; children; neuropsychological test

There are no references for this article.