Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Joaquim Santos; Mário Vairinhos; Luis M T Jesus

doi:10.2196/13861

Santos, Joaquim; Vairinhos, Mário; Jesus, Luis M T

2019-12-05 00:00:00

Background: A prototype of a tangible user interface (TUI) for a fishing game, which is intended to be used by children with speech sound disorders (SSD), speech and language therapists (SLTs), and kindergarten teachers and assistants (KTAs) and parents alike, has been developed and tested. Objective: The aim of this study was to answer the following question: How can TUIs be used as a tool to help in interventions for children with SSD? Methods: To obtain feedback and to ensure that the prototype was being developed according to the needs of the identified target users, an exploratory test was prepared and carried out. During this test using an ethnographic approach, an observation grid, a semistructured questionnaire, and interviews were used to gather data. A total of 4 different types of stakeholders (sample size of 10) tested the prototype: 2 SLTs, 2 KTAs, and 6 children. Results: The analysis of quantitative and qualitative data revealed that the prototype addresses the existing needs of SLTs and KTAs, and it revealed that 5 out of 6 (83%) children enjoyed the activity. Results also revealed a high replay value, with all children saying they would play more. Conclusions: Serious games and tangible interaction for learning and problem solving serve both teachers and children, as children enjoy playing, and, through a playful approach, learning is facilitated. A clear pattern was observed: Children enjoyed playing, and numerous valid indicators showed the transposition of the traditional game into the TUI artefact was successful. The game is varied and rich enough to be attractive and fun. There is a clear need and interest in similar objects from SLTs and educators. However, the process should be even more iterative, with a multidisciplinary team, and all end users should be able to participate as co-designers. (JMIR Serious Games 2019;7(4):e13861) doi: 10.2196/13861 KEYWORDS children; tangible artefact; speech sound disorders; exploratory test speech and language therapy [2]. However, owing to budget Introduction cuts, schools have fewer professionals to intervene; therefore, the role of parents and kindergarten teachers and assistants The prevalence of speech sound disorders (SSD) in the United (KTAs) is particularly important [3], as the significant effects Kingdom is estimated to range from 2% to 25% in children that SSD can have later in life are well documented [4-6]. aged 5 to 7 years [1]. In Portugal, where the fieldwork reported in this paper took place, it is estimated that thousands of Speech and language therapists use physical media (games and preschool aged children (8%-11% of the total population) need assorted toys) to stimulate speech and help children to overcome http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al SSD [7]. A physical material has intrinsic qualities, such as Tangible Artefacts weight, form, smell, or texture, and these can be used to Beyond conventional interaction paradigms, such as graphical stimulate speech production, thus supporting SLT interventions. user interfaces or command line interfaces, there are several A digital app, especially if tactile or mobile based, has the appeal interaction paradigms (eg, natural interaction, ubiquitous of being a well-known device by many children, and this computing, pervasive computing, mixed realities, or wearable produces a sense of engagement through the use of sound, computing) that can incorporate the activity context in an animation, and color. A multimodal approach, although in need effortless interaction approach. The role that tangible user of further research, appears to be effective in several fields of interfaces (TUIs) can play in education and health, the concept speech therapy [8,9], as well as other areas [10]. What is of interaction and how it differs from adults to children, and described in this paper is the creation of a hybrid artifact, some psychological aspects that affect how children learn are capable of combining physical and digital media—multimodal, briefly discussed in this paper. being used by speech and language therapists (SLTs), KTAs, Tangible User Interfaces parents and children in a one-on-one session or as group activity, with both children with SSD and typically developing children. TUIs seek to move away from the generic combination of screen, mouse, and keyboard interaction and attempt to In this paper, the theoretical background will be discussed, as transform the world itself into an interface [14]. They can be well as previous relevant projects and their contributions to the defined as interfaces that support users’ direct interaction with development of the current prototype; the methodology and the the digital world or digital device by use of real-world objects fishing game tangible interface (FGTI) prototype, its functional or tools [15]. They use physical forms designed and improved design, and technical requirements; the prototype development over millennia to fit a specific task [16], facilitating the user’s phase (ie, the parts that make the prototype and its iterations); discernibility and direct manipulation of the interface through detailed results of the exploratory test. Conclusions and future the user's peripheral senses (eg, touch or vision), because of its work are presented in the final section. The aim of this paper is physical embodiment [14,16]. The user can focus his or her to determine how can tangible user interfaces (TUIs) be used attention and consciousness on the task and not on the interface as a tool to help in interventions for children with SSD? [17]. According to Norman [18], people develop throughout Speech Sound Disorders their lives a process of uninterrupted adaptation to and with the environment and an understanding of how to act in the physical SSD take the form of gaps in children’s speech sound systems, world. It is from this seemingly innate understanding from which which can cause difficulties in producing or understanding the concept of affordances stems. Affordances, according to phonemes [7]. Children with SSDs also exhibit speech patterns the original definition by Gibson [19]—particularly the and structures that should not be present in typically developing affordances of the environment—are what the environment children of their age [7]. A child might use, on a regular basis, offers the animal, whether for its welfare or unease. The what is designated by SLTs as a phonological process, for affordances theory lies at the center of the conceptual model of example, final consonant deletion (ie, the child omits a TUIs, as the incorporation of digital technology into objects of consonant in the final position of the syllable or final position the physical world will make the interface more familiar and in the word) [11]: The Portuguese word <porco> (in English, easier to understand from the user’s viewpoint. TUIs can be <pig>) is produced as <poco>. approached as rigid discrete interfaces that use certain objects Role of Parents and Kindergarten Teachers and or shapes with which the user would interact and which have a Assistants’ Roles perceived meaning with a finite set of objects and possible interactions [16], or they can be perceived as a more “organic” The current recommended speech and language therapy practices and material malleable, taking advantage of new digital and point to a family-centered intervention, promoting not only the physical materials that can seamlessly pair sensing and display parents’ involvement in the sessions and in homework activities capabilities. These interfaces have the potential to break the but also in planning a session and setting goals. Family-centered boundaries of predetermined interactions [16]. guidelines integrate the whole family as a client, positive family and professional relationships, parental decision making, and Tangible Artefacts in Education and Health the empowerment and enablement of families [12]. Tangible artefacts have long been used in interactive games in KTAs are of great importance to child development because of therapeutic contexts, especially in fields of cerebral palsy or the time they spend with children and the nature of their poststroke recovery [20,21]. They also allow one to assess relationship. They are part of a child’s innermost circle [10] and several physiological parameters, without any stress associated can help in the detection and reporting of possible cases of SSD, with a visit to a doctor’s office, relieving an anxiety felt by as well as in the implementation of specific activities with a many children and some adults alike [21]. child, as long as proper training, support, and tools are provided In education, both the needs of the teachers and the needs and to them by SLTs. KTAs are well aware of the cognitive and curricula of the students have to be fully understood and satisfied social impacts of SSD in children and the negative attitudes [22]. Serious games and tangible interaction for learning and people tend to have toward them [13]. However, a caregiver problem solving serve both teachers and children. Children must attend the needs of several children, and in Portugal, enjoy playing, and, through a playful approach, learning is activities have to be group based and have to benefit all. facilitated [23]. Tangible artefacts by nature invite collaboration, http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al allowing several users to interact with the artefact and perceived as such. However, some aspects of it can be further themselves [23], thus increasing productivity levels [21], improved, for example, a leaderboard that, when visible to all particularly as TUIs provide an interface that is space players, stimulates healthy competition. Gamification can be multiplexed instead of the time-multiplexed interaction, as we defined as making use of elements typically found in a game can typically find on conventional digital interfaces that rely in a nongaming context to transform every day, uninteresting on mouse and keyboard. The way in which the user interacts tasks, into engaging ones, while increasing user activity and has to be driven through affordances, mappings, and game logic retention [31-33]. to ensure reliability and take full advantage of the potential of Typical game elements with extensive use of gamification and the artifact [21,24]. with interest in the physical and digital part of the FGTI prototype include the use of points (and point systems) and the Designing Interaction for and With Children existence of levels. According to Zichermann and Cunningham Designing for interaction is all about how to design for people, [31], points are a vital element, and they should always be their needs, emotions, and intellect, making it imperative to be present at any gamified activity, if not in a visible way, at least highly aware of what to expect from those who will interact visible to the activity designer only, so he or she can assess how with the final product [25]. With the shift toward participatory the users interact with the activity and design appropriate and ethnographic methods, those designing interactive apps or outcomes [34]. When visible, points allow the user to know objects have to fully understand how and why people use how close he or she is to his or her goal, and points can thus be technological innovation [22,25]. highly motivational [34]. Levels, as the name implies, mark The children (aged 3-7 years) targeted with the artefact created something, in this case they mark in-game progress, and they during this project are preliterate. With short attention spans, allow players to be aware of where they are, over time, in they have difficulty conceiving abstractions, and their fine motor relation to the game experience. Levels should be logical in skills are not yet fully developed [22,26]. Nonetheless, designing terms of level progression, and they should be easy to add to in a way that can be perceived as too childish can be felt to be [34]. By further hiding away the test or activity behind a boring or disrespectful by the children [22], as they are acutely game-like approach, the stakeholders might feel more relaxed aware of their capabilities [27]. A workaround is to embrace and willing to participate [32]. designing with children as co-designers, evaluators or subjects, Sum of All Parts or a combination of these. This approach has its own drawbacks, requiring that adults and children must work together, but in An effective TUI, usable in SSD intervention, should allow the end, this method assures that the design meets the needs and some degree of simulation and storytelling, as well as the specificities of children [22]. construction of mental models of knowledge. It should also provide some form of social interaction with the artifact and How Do We Learn? the other players, all in a playful atmosphere [29]. A TUI can Children search for multimodal stimulation, which consequently provide natural interaction without emphasizing any cognitive encourages their physical and cognitive development. They are effort—a child does not need to learn or understand a set of naturally motivated to explore what is around them by engaging rules or settings. The perceived focus is on the action executed with their environment, their medium and substance, which and what it can represent. A TUI can help gamify speech and consequently affords certain immediate perceived venues of language therapy intervention with a child; it can provide an action, manufacture, and manipulation [19], reinforcing learning alternative means to promote children’s speech production; it through the dynamics of play. can help parents by being a “fun” homework exercise to do together and can help KTAs by being an activity that can be Cultural and social contexts influence how children construct developed in a group of normal children and children with SSDs. the world and their knowledge [28]. Social experience is a The prototype’s intended use is at the clinic, the kindergarten, critical factor in mental development [29]—interpersonal or at home. However, there are challenges to overcome. The connections and social interaction provide the means for a child prototype must offer more than a traditional game. It has to be to access experiences that they can then integrate into their view lightweight and easy to transport while remaining durable to of how things work [28]. Play helps a child to separate the withstand daily use by children. The software component should meaning of an object from the actual object, and from a child’s incorporate options to intervene in several SSD and remain point of view, it is not just a game, it is a serious thing, which interesting to play with, while sending data to a log that SLTs they consider as work [28]. Adults, who make up a large part can consult later. of our stakeholders, also benefit from something playful and fun. As Donald Norman says in Emotional Design: Why We Related Work Love or Hate Everyday Things [30] (page 103), “Beauty, fun, Some examples of good practices or cases of success can be and pleasure all work together to produce enjoyment, a state of found in the literature [35-39], but not one is an exact fit in positive affect.” These positive emotions, as the author says, terms of technological requirements, target population, or have many benefits and are pivotal in our ability to learn. intervention area of the current project. A total of 3 projects were considered relevant for the conceptualization of FGTI: Gamification first, the table-to-tablet (T2T) intervention materials, designed The prototype presented in this paper is a conversion from a to be a reliable and valid solution [40] to be used by Portuguese game, which, at first glance, does not need extra elements to be SLTs when treating children with SSD. It has a physical and a http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al digital version, and SLTs can use them interchangeably, but It allows the introduction of extra gamification elements. one does not communicate with the other. Second, the It affords extra motivation by presenting a game in a new format LinguaBytes materials, from the Netherlands [41], comprises a that is flexible and allows for uniqueness in each play, full set of exercises and varied activities that are mediated by potentiating children’s preferences toward digital games [42], tangible artefacts. The aim of LinguaBytes was to be a tangible while retaining the physical traits. language learning system for toddlers with some form of motor disability. Third, Jabberstamp [42], developed by a team at the It can also ease the postintervention process. SLTs do not need MIT Media Lab (Tangible Media Group), is a tool that allows to record any data from the session, as a “log” file will be children to add sound to their drawings, collages, or paintings, created for them, with all the relevant information needed enabling them to communicate more effectively before (player’s names, age, intervention time, and what were the developing or mastering writing skills. answers of all children). This file can (ideally) be accessed through a Web-based software or emailed to the SLTs. Relevance and Motivation of the Fishing Game The prototype, at its core, is the traditional fishing game that Tangible Interface Prototype so many know. Conceptually speaking, the users can expect to It can be argued that the traditional fishing game is already find the same organization, functionalities, and set of rules. As engaging and in use in intervention by SLTs, similar to other such, previous experiences with a traditional fishing game will traditional games, such as Bingo or wooden blocks, with the allow players to seamlessly use the prototype with just a very alphabet written on them or animals [43,44]. However, the TUI quick explanation of some components and their functionalities. artefact can present additional advantages: The prototype aimed to be innovative and solve a real-world • It allows the customization of sprites (both the avatar issue, involving different participants, with diverse roles, as can representing the player and the fish). This customization be seen in Figure 1. stimulates user engagement and makes each game unique. • It facilitates the process of preparing the session (SLTs, Several design iterations result from this richness in feedback, parents, or KTAs).: The system will set up almost different uses, and perspectives, because of an encouraged everything. participatory culture (consumer/player active in coproduction) [45]. This constant iteration and evaluation [46] are a trademark It eases the burden of certain game-related tasks, such as keeping of design-based research (DBR). DBR is capable of producing and updating a score or showing who is winning on a 2 different and nonexclusive outcomes [47]: theoretical (this leaderboard. In addition, the software can introduce extra paper) and practical (the FGTI). challenges, bonuses, and “power-ups.” Figure 1. Types of users, their permissions and possible actions within the prototype. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Data Gathering Methods The technique used was direct observation, although the Design-Based Research instruments for data gathering were a form (qualitative data) and a semistructured questionnaire (which allowed the collection DBR allows the researcher to be involved in a way that he or of both qualitative and quantitative data). In the creation of the she may glimpse unexpected uses or interactions with the form as an element to annotate the observations, great care was prototype, causing a need to alter it or re-assess the target users, taken in not only dividing the observable actions but also in what they do and their needs. To reach the test phase, the transforming and categorizing the observable world into physical prototype went through several revisions, always interpretable and observable data. analyzed by SLTs and other project participants, and reworked accordingly. The software part of the prototype (both the game Exploratory Test: Observation Form and Open and the web app) was equally revised and improved. To gather Questions data (qualitative and quantitative) from the designated users, in To carry out an effective, direct, and nonintervening observation, a real-life scenario, an ethnographic approach was used, and a especially of an activity involving children and a certain amount user exploratory test was conducted regarding prototype use. of play, a simple and easy-to-complete form was created. A set These data were collected through observation and a of open and closed questions, using a visual Likert like scale (a questionnaire. Smileyometer as shown in Figure 2) [48], was also prepared to Sample Definition be used at the end of the test, with the target users (children). The sample for the exploratory test comprised 10 expert users, Owing to the variety in the sample and to the constraints an with ages ranging from 4 to 55 years. The parents’ informed observer may face, a form was prepared to address all scenarios consent was procured, in agreement with the World Medical in a single 2-sided A4 page. It was up to the observer to know Association’s Declaration of Helsinki regarding human what he or she was observing and where to annotate it. Both experimentation. Parents also received a document, briefly the form and questionnaire used a unique ID for the person explaining the test. In addition, ethical permission was obtained observed interacting with the prototype, the type of user he or from an independent ethics committee (Comissão de Ética da she was, age, duration of the session, and date. Unidade Investigação em Ciências da Saúde – Enfermagem da The form was divided into dimensions or broad areas with Escola Superior de Enfermagem de Coimbra, Coimbra, clearly defined parameters, to mark as observed (“yes” or “no”), Portugal), process number P159-05/2013. as well as an area reserved to take some quick notes. Those The sample can be further clustered into 3 subgroups as follows: dimensions were as follows: • Children: A total of 6 children, with ages ranging from 4 • Game/Prototype Usability: Parameters regarding the ability to 6 years. This group had 2 boys and 4 girls, all speakers to identify the game, its objectives, and components. with normal development, – no SSD. • Game/Prototype (Physical) Characteristics: Parameters • Speech and language therapists: A total of 2 SLTs, with revolved around the materials, colors (or lack of), ages ranging from 42 to 54 years. Both are also lecturers robustness, and feedback from the game. at the School of Health Sciences (ESSUA), University of • Gamification: Parameters regarding the desire to play more, Aveiro, Portugal. if players know when their turn to play is and their score. • Kindergarten teachers and assistants: One kindergarten The questionnaire was also set on an A4 page, and the observer teacher, aged 55 years, and 1 one kindergarten teaching would choose what to ask and to whom. It was divided into 2 assistant, aged 51 years. parts, 1 part aimed at the children and 1 part aimed at the SLTs This sample allowed testing of a variety of situations, namely and KTAs. The part aimed at the children had 2 questions, with speech and language therapy intervention, children’s use of the 2 Smileyometer scales—1 with 5 smiles representing values 1 activity as a game (group activity), and KTAs with children. to 5 for the question “Have you enjoyed playing this game?” The only missing element(s) from the expected users were The other used 3 smiles, representing values 1 to 3 for the parents. question “Would you play this game again?” The remaining questions were open questions. The last part also had a set of open questions, used to conduct an informal guided interview. Figure 2. One of the Smileyometers used. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al The first scenario tested was the SLT intervention on a child, Location and Setting Up using the activity. The second was the kindergarten teaching The test took place in a kindergarten near the University of assistant and a child and their interaction with each other and Aveiro in Portugal. This setting allowed having, in the same the activity. The third scenario was a group activity—2 children place, 3 of the 5 intended users: the children, the kindergarten playing the activity, talking with each other about what the other teachers and the kindergarten teaching assistants. The fourth caught, points won, and related subjects. The fourth and final intended users, the SLTs, joined the group at the kindergarten. scenario tested was a child and the kindergarten teacher playing This is also coherent with some SLTs’ intervention and how both interacted. The only untested scenario was that locations—kindergartens and schools—allowing the observer of parent and child playing the activity because of time to be in the intervention environment instead of a laboratory. constraints. This limitation is discussed at the end of this paper. The test was held in the kindergarten library, as it is a quiet and Fishing Game Tangible Interface spacious room with plenty of natural light. The prototype was prepared; its contents were set up, and the fish basket was The design and functionality of the FGTI is addressed in this connected to the laptop. A set of printed game rules was subsection, as well as a description of its functionalities and available, and the SLTs and KTAs were encouraged to read needs, as related to SSD. The hardware and software used to them. A brief explanation was given of how the prototype build the prototype and game are briefly described, concluding worked, the role of the components, and functions of the buttons. with a synopsis of the ideas that led to the finished FGTI This explanation was considered to be similar to the instructions prototype. one would receive when buying such equipment. The observer Functional Design prepared the observation forms and the set of questions to ask The Fishing Game (or Pond), is a dexterity game, and the rules the users. (which may vary from publisher to publisher or can be Four Possible Use Cases Tested determined by the players) are as follows: all sea creatures and The prototype’s hardware and software limitations constrained treasure chest, seen in Figure 3 in both physical and digital its use to 2 participants at a time. Owing to the number of counterparts, go into the pond. The players take turns in participants on the exploratory test, more than one expected use attempting to catch a fish with the (magnetic) fishing pole. scenario was tested. Figure 3. Digital and physical activity assets. Each player has a set number of attempts to try to catch a fish, The TUI prototype rules are broadly the same as described without looking into the pond, to better emulate a real fishing above, with the following exceptions: activity at a lake or ocean. The number of allowable attempts • All sea creatures go into the wooden trunk that serves as is agreed upon among players at the outset. Each sea creature both a carrying space and the “board” game area. has a number printed on the back. For older players, the value • Sea creatures will have a certain range of values. For of the sea creatures is added up to determine the winner. For example, a codfish value can go from 5 to 15 points, younger children, the number of caught sea creatures determines mimicking the fact that the fish can be a small or a bigger the winner. codfish. How much each sea creature is worth will be http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al calculated randomly within a range of values, during the creatures, and fishing rod) feedback, expressing the action, game. points received, or player turn. In Figure 4, an infographic with the relevant functional The activity screens are shown in Figure 5. The stakeholders components of the FGTI is shown. The solid lines represent are greeted by the name of the activity (top left) and have a interaction, whereas the dashed line represents the visual and chance to select the level or have it chosen by the SLTs (top auditory cues the player gets from the laptop. The dashed circle right). Immediately below these images, there is a representation and line with the fish represent what is inside the box without of the visual differences between levels (calm and sunny sea perspective skew. for level 1 and a stormy sea for level 2). The sound effects also add to this atmosphere. The remaining images exemplify the The stakeholders interact with the fishing rod, which in turn in-game screens, the different catches, and associated messages. interacts (catches) with the fish (see Figure 4). The stakeholders interact with the captured fish by placing them in the fish basket The FGTI prototype, with all its components, the activity and pressing the necessary buttons to execute the activity software, and the Web-based app, seen below in Figure 6, were (differentiating the word and image) and change player’s developed to be as close as possible to the traditional game and turn—the sound cue is repeated at given intervals until the to use the rules described above. In the case of the website or stakeholder acts. The fish basket interacts with the laptop by app, it was developed to act as a possible replacement for the sending and receiving data according to the fish radio frequency setting up of both the game or activity and the clinical file of identification (RFID) data or activity moment. The users have the child. constant digital (via screen) or real (via the fish basket, sea Figure 4. Relevant functional components of the fishing game tangible interface. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Figure 5. The different screens of the activity. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Figure 6. Web companion screen. Figure 7 shows the complete prototype, assembled and ready Technical Requirements to be tested. It was planned to be easy to transport, install, and The FGTI was designed to be self-contained, low cost, and easy use “as is.” The only requirement is the presence of an electrical to replicate, with embedded physical computing capabilities. outlet. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Figure 7. The prototype ready to go. the block diagram in Figure 8. RFID tags glued on the sea Hardware System Architecture creatures were used, and an RFID reader (RDM6300 125 KHz) Owing to time and budget constraints, the liquid crystal display communicated with an Arduino via a transmission pin, with a (LCD) screen was not placed in the wooden trunk lid. In maximum effective (reading) distance of up to 50 mm, taking addition, a Raspberry Pi approach was discarded for the same less than 100 ms to decode the tag or card. An external antenna reasons, and a laptop (as the prototype “control center”) and its was placed around the fishing basket slit to read the sea screen (for display purposes) were used instead, as depicted in creatures’ RFID tags. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 10 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Figure 8. Technological and software requirements diagram. was heavily used. Bootstrap is a front-end library that allows Software one to quickly build or prototype responsive (ie, which can The development of this prototype (activity and website or app) adapt to any screen dimension) websites or apps [50]. The involved several programming languages and libraries, and website or app companion can be transformed into a mobile various pieces of software (or scripts) were created. app installer package, with software like Apache Cordova (available at cordova.apache.org). Cordova wraps the code into The activity was coded using a mixture of HTML, cascading a native container that can access the mobile device’s native style sheets (CSS), and JavaScript, with the Phaser Framework functions (such as the accelerometer) and several different (available at phaser.io.). Phaser is a JavaScript framework for platforms, enabling Web-based software to deploy in any device 2D game development in mobile and desktop environments, (platform agnostic) [51]. The software on the Arduino was coded also ideal for the prototype goals [49]. JavaScript Object using the Arduino programming language. A specific library Notation (JSON) was used to store several values and properties was also used to integrate the RDM6300 RFID Reader. Figure attributed to each sea creature, for example, name, value (range 8 presents a diagram of the technological and software of), or the sound it represents. To make the activity more requirements, as well as the parts comprising the prototype. scalable and readier to deal with real-time use and requests, Node.JS was also used. Node.JS can use and run a JavaScript Prototype Development server side (traditionally an area for other languages, such as The prototype comprised the following parts: wooden trunk, Hypertext Preprocessor or PHP). With Node.JS and its node sea creatures, fishing poles, fishing basket, and speakers. Some package manager, several packages were installed to allow elements of the prototype, because of their importance, role, or bidirectional communication between the activity and the particular challenge, required more than one revision. Arduino with the server, via WebSockets (available at socket.io) plus node-serialport (available at serialport.io), or to render the Wooden Trunk HTML code in the browser through Node (express.js; available The design of the wooden trunk prototype, similar to all the at expressjs.com). This installation implied the creation of a elements in the fish game, was based on a maritime theme; simple server-side and client-side set of files to handle requests therefore, the idea was that it should mimic an old treasure chest. and responses. The website or app companion used almost all That design was reflected not only in the shape (no round top of the previously mentioned languages and technologies, with lid) but also in the choice of ironmongery and straps. It encloses some exceptions: The Phaser framework was not used; Node.JS all the components used in the activity, except for a laptop. This was used, but with different packages installed; HTML and CSS played a larger role; Bootstrap (available at getbootstrap.com) http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 11 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al laptop is needed to power the Arduino Nano and electronics were created. They were then laser cut, and the result is shown and to display the digital representation of the fishing activity. in Figure 3. Sea Creatures and Treasure Chest Fishing Basket In this prototype, sea creatures have an extra dimension (they The fishing basket, shown in Figure 9, was the prototype are not just an object with a value assigned to it or being counted element that took the longest to build and develop, mainly as one more fish caught by the player) because of their RFID because of the fact that it is the most complete and the one tag. This tag gives users access to as much information as closest to the initial idea of how it should be and behave. Owing needed, included in the JSON file, read during the activity. Sea to its importance and central stage role, it was the driver of creatures and the treasure chest were revised once. Similar to several design, technical and material changes. A consideration the treasure chest, during the build and actual use, some notes was always taken as a decisive factor: the end result had to be were taken on possible improvements. A total of 9 different sea cheap and easy for any hobbyist to develop. creatures (6 fish, 2 crabs, and 1 octopus) and a treasure chest Figure 9. The final iteration of the fish basket, from prototype (top-left corner) to the final result. Results Speaker Rocks and Fishing Poles The speakers were initially conceived as being encased in the Overview wooden trunk lid, close to the LCD. With a reduction in size of In this section, the results obtained in the exploratory test are the wooden trunk, the position of the speakers had to be presented and appraised. Some items were not possible to reconsidered. To keep the maritime theme, the idea was to turn observe, or it was not possible to overhear comments about the speakers into a rock-like element, similar to those found them among the users. However, they were kept in this section around piers. (see tables in the following subsections) because of 2 main The fishing poles were revised twice (ie, they were built during reasons: first, being an exploratory test, done to ascertain the the initial construction of the wooden trunk and the wooden pertinence of further studies [52], the authors felt that its absence trunk revision.02). Despite the several ideas resulting from the would skew the test results, and second, as some unobservable initial brainstorming sessions, time-related constraints led to a items were expected to elicit some sort of feedback (eg, the lack pragmatic and simple approach. The ideas for a more advanced of color on the figures), and as such, the absence of feedback fishing pole were collected to be used in a future iteration of was considered relevant. the game. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 12 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al the subsections below (in table format), we will present the Observation Form feedback collected from every area and parameter. The observation form was divided into 3 dimensions: prototype or activity usability, prototype or activity characteristics, and Prototype Usability gamification. It was used for direct observation, without any This dimension explored parameters regarding the users’ ability intervention during the activity. The observer strived to annotate to identify the game, its objectives, and components. Results all the interactions and relevant exchanges of commentaries. In are shown in Table 1. Table 1. Prototype usability observation parameters. Observation parameters Observation results Was the player able to identify and A total of 2 out of the 6 children tested immediately identified the game. The others were not vocal enough recognize the game and its objec- to demonstrate whether if they knew what they were playing, and the SLTs (who remained present for tives? the whole duration of the test) or the KTAs identified the game for them. They all knew the “classic” objectives and assumed that the novel elements (treasure chest, octopus, and crabs) had a similar value and role as the fish. Only 1 player and 1 Speech and Language Therapist noticed that the unusual elements had different score behaviors. A player even said that she liked this game more than the classic that she had at home: “The one I have at home is a blanket and we can stand on it and fish with our hands. But this one is more fun!” – [CT, aged 6 years]. When asked why, she replied that this had a larger variety of sea creatures and some stole points (this child caught an octopus and noticed that it had halved her score). She went on to say to the Kindergarten Teacher that she should buy this game for their classroom. Was the player able to identify the All participants were able to correctly identify the game components and even discriminated the sea creatures game components (sea creatures, (saying that one was an octopus, another a crab, etc). fishing poles, and wooden trunk)? Was the player able to identify the As none of the children were able to provide a viable answer, the SLTs and the KTAs role -played with game elements and its function them and helped them when they first caught a fish or when they needed to change players or choose a (buttons and slit)? different set of words (minimal -pairs). From that moment on, the game elements and its functions were a learned behavior. The test that involved 2 children playing against each other was also mediated by an SLT; therefore, the same explanations/roleplays were present. Were the game elements timings Unfortunately, it was not possible to register this parameter. The children were having such fun while using the correct? prototype and the SLTs and KTAs were so involved with them that none seemed to notice (or care about) the timings needed for a fish to be recognized or for the word to appear. Further testing is required to understand whether the timings are correct. One cannot assume that the stakeholders will be this engaged all the time and must instead assume that the novelty of the situation and the fact that it was a “one-off” test made stakeholders unaware of the timings. Did the player know when it was his Regarding the entire sample, 9 out of 10 knew when his or her turn to play occurred. However, this does not or her turn to play? mean that they took the correct steps to play or pass their turn. They all knew that after player X would be their turn. But player X would usually forget to press the button to change player, or the next player would fail to re- alize the player number mismatch and correct the situation by pressing the button. This would result in points being given to the wrong player. SLTs: speech and language therapists. KTAs: kindergarten teachers and assistants. KTAs, which made them aware of the uses and whys of many Prototype (Physical) Characteristics of the physical characteristics. Parameters in this dimension are those regarding the game Gamification components, their physical characteristics, and whether they served their intended purpose. Many of these parameters (see In this dimension, the parameters observed were those regarding Table 2) were observable only in the children. This limitation the desire to play more, whether the player knows if or when is was because of the brief explanation given to the SLTs and his or her turn to play and his or her score. Results are presented in Table 3. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 13 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al Table 2. Prototype physical characteristics observation parameters. Observation parameters Observation results Were the game and its elements adequate? No comments from the users were registered. Was the color of the game elements adequate? There were no comments about the (lack of) color of the physical components of the activity, and little attention was paid to the on-screen elements. Do the materials used to build the prototype invite the handling No comments were registered. The observer marked the parameter to ask in the a b of it? guided conversation with the SLTs and KTAs . Was the prototype considered robust? No comments were registered. The observer marked the parameter to ask in the guided conversation with the SLTs and KTAs. Was the feedback throughout the activity efficient? A total of 2 out of the 6 children waited for the feedback (eg, the audio feedback of the word after inserting the sea creatures into the slit) or were aware of it. Did the physical constraints serve their purpose? This was observable in 3 out of the 6 children. It was more apparent regarding the slit and its use. Was the mapping of the buttons and their actions consistent and A total of 4 out of the 6 children correctly and consistently used the buttons when correctly perceived? Were they used during the activity? they were supposed to and to the desired end. SLTs: speech and language therapists. KTAs: kindergarten teachers and assistants. Table 3. Prototype gamification observation parameters. Observation parameters Observation results Were the participants able and willing to play or participate until the end? All the participants were involved until the end, showing great interest and willingness. Some (children included) even wanted to know details about the study. Were the participants willing to play more? A total of 4 out of 6 children asked whether they could play more. Were the participants aware of their score, at any given time? A total of 3 children knew their score, 1 child was not aware of it, and for the other 2 children, this was unknown. Despite knowing the score and being quite attentive to the value of each captured sea creature, the children did not seem to have a clear notion of who was winning. They looked at their points on the screen and would say, “I have x points!” but nothing more. When the activity ended, children would often ask who had won. This was said while pointing to each smile in turn. The child Open Questions Questionnaire was then asked to choose the smile that depicted his or her The open questions questionnaire was divided into 2 sections: feeling toward the game he or she had just played. 1 section for children and another section for SLTs and KTAs. A total of 5 out of 6 children (83%) picked the happiest smile, Below, we discuss the results and answers obtained in both which had a value of 5, and just 1 child chose the third smile, sections. which had a value of 3. That is, more children enjoyed playing Children the activity than not. Children were asked 4 questions, 2 of which using The second question concerned what feature the stakeholders Smileyometers [48], with 2 scales: 1 scale had 5 smiles; enjoyed the most. Despite this being an open question (ie, with therefore, values would range from 1 to 5, whereas the other a broad range of possible answers), most of the children scale had 3 smiles, with values ranging from 1 to 3. Children mentioned similar aspects as the most enjoyable. A total of 4 being our target users, their feedback was very important; children mentioned that they enjoyed capturing the sea creatures. therefore, the observer was very attentive to what they said One child remarked that doing so was like going fishing (actual about the activity. The first question was whether the participant fishing) with her father. A total of 3 children enjoyed pressing had enjoyed playing the activity. The Smileyometer scale was the buttons, and 2 children enjoyed placing the sea creatures in explained to the children in the following manner: the slit. Some additional comments included that they had enjoyed the fishing pole itself, the crab, the octopus, the chance You see, the first smile is sad. He has not enjoyed to play the game, and the variety of sea creatures, as contrasted playing this game. The second smile is –well,...I really with the original game using “just” fish. do not care about it. The third smile liked it a bit. He could play more...or not! The fourth smile is happy, The third question probed what feature the stakeholders enjoyed he enjoyed playing the game. The fifth smile is really the least. This was also an open question. A total of 3 children really happy because he enjoyed it a lot. did not identify anything they disliked. The remaining 3 children reported different aspects: 1 child disliked catching the crab as http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 14 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al it stole points; another disliked pressing the buttons, explaining, 6 to 8 sea creatures per player. With larger number of sea “...you press the button and that’s it!” The third child disliked creatures, they immediately added that the agents of distraction watching the game on the PC, preferring instead to play it (the crab and octopus) numbers should also increase to maintain physically. The gameplay was not fully understood by the proportion and level of challenge. A sort of “superclass” sea stakeholders. Many were not aware that catching a crab or an creature, capable of shifting the entire game for all players, was octopus decreases a player’s points. Similarly, many were not suggested. The way the auditory stimulus was working was also aware that answering correctly (pressing the buttons) would remarked as faulty. Children should pick the sea creature, insert double their score. To overcome this issue, a better visual it, and listen to the word (in loop if needed), and only when this representation of the elements, with both auditory and physical action is completed, should they see the image. What was cues, must be implemented on the existing FGTI components happening was that the image would come up and allow the or components yet to be developed. children to have a visual (not auditory) cue. This feature will be revised in future versions of the prototype. The fourth question regarded whether the stakeholder would play again. This was a question made while showing a The third question enquired as to how they would expand the Smileyometer with 3 values. The scale was explained to the prototype if they could. The kindergarten teacher mentioned children in the following manner: that she works with bilingual children; furthermore, as a second language is taught at a young age, she then suggested the Ok we have more smiles. You see, the first smile is prototype should be multilingual. The SLTs would like the sad. He does not want to play again. The second smile prototype to have more uses and not be just the game of fishing. is again indifferent; he really does not care if he plays They suggested to give names to the sea creatures to make or not. The third smile is so happy he cannot wait to children repeat those names, allowing sound production play again. stimulation. Another approach to extend the game’s functionality All children picked the happiest smile (ie, all children wanted would be to use more game scenarios that the adult could choose to play again). from. Some sea creatures with some word values associated with them could be tossed into the pond for children to capture. Speech and Language Therapists and Kindergarten When captured and inserted into the slit, the word would be Teachers unveiled, and the children would be encouraged to use it to In relation to the SLTs and KTAs, the open questions were construct a story. introduced in an informal talk that occurred after the testing of The fourth question was a preamble to the fifth and sixth all the children and after them having had a chance to try out questions, depending on the answer. The SLTs and KTAs were the activity themselves. This talk took the form of a fluid asked whether they would like to see more games made into conversation, with moments that resembled a brainstorm session. TUI. If they answered “yes,” they would go on to question The observer mainly listened, offering his comments 5—“If yes, which games?”—if not, they would go to question occasionally. number 6: “If not, why?” They all answered, “yes.” The first question was whether they would change anything in The fifth question was which games they would like to see made the concept or prototype. Some said they would not change into a TUI artifact. The answers were as follows: Tic-Tac-Toe, anything, that it was interesting to see this transformation of a Snakes and Ladders, Bingo, and an indirect identification game classic game and that it was nice to have the auditory feedback similar to What am I? These can all be possible stand-alone when something was inserted into the slit. Others had more, development avenues or add-ons to the existing TUI artifact, sometimes divergent, opinions. An SLT said that given adding extra elements and adjusting others, along with the children’s (and their own) difficulty remembering to press the software binding them all. button to change players, this function could be automated. This suggestion prompted an immediate response from the other Questions from the observation form, which were asked during SLT, who said that some children had mentioned that they the conversation, as it was deemed more productive, were about enjoyed pressing the buttons; therefore, this change in button the robustness of the prototype and the adequacy of materials press should be maintained. What this action needed was more and how appealing it was to be used. Both KTAs and SLTs feedback or to be made more visible, she suggested. alike said the prototype was robust enough if the activity was used under (adult) supervision. Left unattended, it was robust, The second question concerned whether they would change but accidents do happen. Regarding the kind of material and its anything in the game or activity. The Kindergarten Teacher said appeal, they all agreed that wood is used quite often and that that she would not change anything in the game, as it is so children are used to seeing it in other toys and enjoy using them. similar to the “original” game that no explanations are needed. They mentioned that some parts could use other materials to However, she felt that the way the score was being handled give extra-sensory input that may foster immersion in the lacked more and better feedback for the player. This feedback activity. would, in her opinion, increase the competition and game aspect of the activity among the users. Speech and Language Therapists and the App The SLTs both enjoyed the game. They would add more sea Both SLTs were shown the partially developed hybrid creatures, so you can have more auditory stimuli, as at the app—hybrid apps are web (browser based) apps, developed moment, there are only 3 sea creatures per player: they suggested using HTML, CSS, and JavaScript, which are then wrapped in http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 15 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al a layer that allows interaction with smartphone and tablet the subsections below, we will present the feedback, ideas, and hardware and software, independent of code or language. This areas for improvement, collected from every area and parameter. app will allow one to prepare sessions remotely and provide the Prototype Usability visible front of a back-office area where SLTs can keep and The feedback and perceived recognition, as well as the ease generate logs or visual representations (ie, graphics and other with which the players started playing the game, are valid visual forms to represent information) from sessions and target indicators that the transposition of the traditional game into the users. SLTs can log in to access a roster of target users or add FGTI artifact was successful. The added value/interest of the a new target user. They can choose an area of intervention, a extra sea creatures (octopus and crab) and treasure chest should set of exercises, what words or facilitator sounds to use, and be noted. However, the fact that the score was different for each other functionalities. This Web-based app can even be used by object (as well as the on-screen object scale that changed KTAs or parents, which will be able to access and set different according to the points earned) was not readily seen, and only parts of the application. Their feedback was collected and saved 2 out of 10 participants noticed it. Better audio and visual for a future implementation. feedback are needed in the next iteration to ensure this is an understood behavior, which can transform the gameplay Discussion experience. The game components seem to be well designed Although it is true that in speech and language therapy, the and help in identifying the game and the activity (fishing) on sound, visual cues from the SLT, and speech production trials which the game is based. Some sea creatures can be further are essential, the main purpose of this game was not to replace developed, as the stakeholders engaged in conversations about an SLT but rather to provide a tool, capable of being used by fish variety (eg, is it a whitefish or a cod?). Further testing with SLTs, parents, or kindergarten staff, with a focus on hearing all the stakeholders is required to understand whether the discrimination instead of articulation. It was felt that if a child affordances and design work and whether, within minutes, the struggles to produce a sound, the adult using the game can offer stakeholders are ready to use the activity to its fullest. guidance, albeit without the know-how of an SLT. This Furthermore, testing is required to understand whether the particular game focused on a single minimal-pairs exercise, timings are correct. One cannot assume that the stakeholders which comprises differentiating similar sounding words. Those will always be highly engaged in each play-through of the game, words or “sound cues” were played until the user decided to assuming that the novelty of the situation and the fact that it press button A or B (associated with an image that corresponded was a “one-off” test made stakeholders unaware of the timings. to one of the sound cues). The SLT present in the room, if any, Knowing one’s place in a queue, which in this case was his or can choose to say the word to show the articulation, if needed. her turn to play, does not seem dependent (especially in an activity with just 2 players) on any sort of visual or auditory By using an ethnographic approach and a single observer, feedback. It is something that the players do and know. present but not participating with any form of help besides the However, interacting with a button to signal this change is not setting up, the impact of the observer presence was kept to a an expected behavior, and as such, despite knowing (SLTs and minimum. However, a known risk is that the observer is not KTAs) or being told (children) and despite the markings above entirely unbiased and his/her presence will always have an the button, this was generally overlooked and was a source of impact. In addition, the inability to generalize from the data distress. gathered, the sample size and the analytical transparency (the observer was also responsible for coding the data) are points Prototype Physical Characteristics that will have to be addressed in future research, by the use of Further testing is required to understand whether the dimensions complementary tools to gather data. Despite these negative are correct and to determine the importance of color. As the points, the richness of the data surmised in location allows for digital part was running on the laptop screen, off-angle relative a better insight on the users’ reaction and relation with to the wooden trunk and the main activity area, it may have technology and their natural environments [53,54]. been perceived as a “secondary” thing to look at. Tighter integration of the display (ie, with the LCD on the wooden trunk The analysis of quantitative and qualitative data revealed that lid) may lead to a better understanding and ability to test this the prototype addresses the existing needs of SLTs and KTAs parameter. Additional testing and modifications are needed to and that 5 out of 6 (83%) children enjoyed the activity. Results improve on feedback, as per user suggestions and observation. also revealed a high replay value, with all children saying they would play more. In the following section, we discuss the On at least 2 occasions, sea creatures were pushed into the slot feedback collected and summarize some ideas and areas for so fast that the RFID reader was unable to detect them; therefore, improvement. this issue also has to be considered. Observation Form Several suggestions were given by the SLTs and KTAs to The observation form was divided into 3 dimensions: prototype slightly alter the phrases and sounds to give more feedback or activity usability, prototype or activity characteristics, and regarding, for example, the score. A possible reason for half the gamification. It was used for direct observation, without any children being aware of the physical constraints, such as the slit intervention during the activity. The observer strived to annotate and its use, may relate to their turn in playing the activity. The all the interactions and relevant exchanges of commentaries. In first one to play used the constraint (and the SLTs/KTAs help) to know what to do with the sea creatures and the slit, whereas http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 16 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al the second did it because she or he had watched the first one do Suggestions and Improvements on the Basis of it. Feedback Gathered During the Exploratory Test During the entirety of the exploratory test, the observer was The mapping of the buttons and their actions were generally able to collect much input and feedback, some direct, as in the well understood and used. There was some “natural resistance” final conversation with SLTs and KTAs, as well as some against using the button to change players; the users seemed to indirect, overheard during the activity testing. Those suggesting expect it to be automatic. The 2 children who did not seem to improvements and feedback are addressed in this section. be capable of using the buttons did play the activity and enjoyed Keeping with the maritime theme, a pirate object/figure could it. The other player or the SLTs/KTAs helped them with the be created. This object would be able to steal all points from buttons, and they started using them after a short while. all players (in practice, resetting the entire score and doing Gamification “tabula rasa” of all progress) and sail away (meaning the pirate All stakeholders enjoyed taking part in the activity. Although would be thrown again into the wooden trunk). This new some of this (especially regarding children) can be dismissed element would be used to counterbalance the treasure chest or looked at, considering the novelty factor, it is encouraging element and would increase the challenge factor. In addition, to get this feedback. the score could be displayed inside a starfish and could have some sort of animation to increase the visual feedback. A total More than half stated their desire to play more. The remainder of 2 sets of scores could be used. One, to reward the better may not have expressed this desire for a number of reasons: for fisherman, would be based on a number of sea creatures caught example, the test being run just before the Christmas holidays and how much they are worth. The other score would be to and children being busy taking part in dance or music rehearsals reward the player who answered correctly more times (in for the school’s Christmas show; therefore, they wanted to go practice who did better at word discrimination, in the case of back to those activities. the minimal pairs). This reward would make being attentive The stakeholders were aware of their score, or they at least knew and answering correctly worth more to a player. The number where to look at, on screen. However, most of them were unable of points for each catch should be more evident with the use of to say who had won and by how many points. auditory cues. In the game’s present iteration, a voice can be heard saying, “You caught a fish. It’s small.” The voice should Open Questions Questionnaire say, “You caught a fish. It’s small. It’s worth X points!” The open questions questionnaire was divided into 2 sections: Similarly, the voice should also say, after some interaction from children and SLTs and KTAs. Below, we discuss the answers the player, (eg, a successful catch but unable to discriminate obtained in both sections. the word, catching a crab) the number of points the player has. Furthermore, what the voice says and what is written on screen Children have slight differences that should be addressed. For example, The majority of children enjoyed playing the activity. Even the the voice says (in Portuguese), “Tenta de novo!” (Try again) child who answered differently did not choose a negative smile and what appears written is, “Tenta outra vez!” One SLT felt or value; therefore, even she would play. that it would be interesting and would help with the immersion to see a representation of the fishing pole on screen, to have an A clear pattern could be observed: Children enjoyed playing idea of its (relative) position. the game. The game is varied and rich enough to be attractive and fun; however, the digital part of it went unnoticed. This Conclusions and Future Work oversight may arise from the previously mentioned off-angle The feedback, the perceived recognition, and the ease with screen. Further testing with a new FGTI revision (of all its which children started playing the game are valid indicators components) is required. that the transposition of the traditional game into the TUI artifact All children wanted to play again. Even considering a factor was successful. The added interest in the extra sea creatures and like “novelty,” the results are encouraging and very positive. treasure chest should be noted. However, the fact that the score What remains unclear and depends on further and more differed for each object (as well as the on-screen object scale exhaustive testing is whether this willingness to play would be that changed according to the points earned) was not readily sustained after a number of games played. seen, and only 2 of 10 participants noticed it. Better audio and visual feedback is needed in the next iteration to ensure that Speech and Language Therapist and Kindergarten this is an understood behavior, which can transform the Teachers gameplay experience. Several suggestions were given by the A need and desire for tools, such as the FGTI prototype, were SLTs and KTAs to slightly alter the phrases and sounds, to give clearly observed. The prototype was seen as almost ready for more feedback regarding, for example, the score. Regarding the serious field and clinical testing after some small bugs and SLTs and KTAs, a need and desire for tools, such as the FGTI, feedback issues were solved. Both SLTs and KTAs said it was were clearly observed. The FGTI and what can be produced something that they could see themselves using with their with the concept behind it can be extremely modular and children. versatile. However, certain aspects should be improved to have an even better product. The TUI game of “Pond,” with the interest caused and ease of use, appears to be a suitable tool for the classroom or as an SLT set of intervention tools; however, http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 17 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al there should be more tests with users, which would produce SLTs, KTAs, and parents as co-designers would allow the more iterations of all aspects of the prototype (physical and development of an end product that is suitable to respond to software). Owing to time constraints, an important group of each user’s needs and desires. More diverse means of gathering stakeholders, the parents, was not tested, but they must be data and analyzing data should be employed to minimize the involved, observed, and questioned in future work. They share, known flaws of ethnography. The FGTI’s perceived value and with the child, a very important space (home), and they spend the positive impact it may have as a tool for intervention for special time and form unique bonds; therefore, they possess children with SSD should be further expanded. The monitoring unheard information, not available to any of the other part of the game, capable of producing usable reports for the stakeholders. A multidisciplinary team that included children, SLTs, will be implemented in future revisions. Conflicts of Interest None declared. References 1. Law J, Boyle J, Harris F, Harkness A, Nye C. Prevalence and natural history of primary speech and language delay: findings from a systematic review of the literature. Int J Lang Commun Disord 2000;35(2):165-188. [doi: 10.1080/136828200247133] [Medline: 10912250] 2. Castro A, Caetano D. History of language impairment/developmental language disorder in Portugal. In: Law J, McKean C, Murphy CA, Thordardottir E, editors. Managing Children with Developmental Language Disorder: Theory and Practice Across Europe and Beyond. Abingdon: Routledge; 2019:374-386. 3. Jesus L, Martinez J, Valente A, Costa M. Speech and language therapy service delivery: overcoming limited provision for children. Public Health 2017 Oct;151:39-50. [doi: 10.1016/j.puhe.2017.06.013] [Medline: 28710926] 4. Lancaster G. Developing Speech and Language Skills: A Resource Book for Teachers, Teaching Assistants, and Speech and Language Therapists. London: Routledge; 2008. 5. McCormack J, McLeod S, McAllister L, Harrison LJ. A systematic review of the association between childhood speech impairment and participation across the lifespan. Int J Speech Lang Pathol 2009;11(2):155-170. [doi: 10.1080/17549500802676859] 6. ASHA. Budget Cuts: Maintaining Speech-Language Pathology and Audiology Services in Schools URL: https://www. asha.org/SLP/schools/Budget-Cuts-Schools/ [accessed 2018-11-28] [WebCite Cache ID 74GS2SgSZ] 7. Bowen C. Children's Speech Sound Disorders. Second Edition. Chichester: John Wiley & Sons; 2015. 8. Williams JH, Massaro DW, Peel NJ, Bosseler A, Suddendorf T. Visual-auditory integration during speech imitation in autism. Res Dev Disabil 2004;25(6):559-575. [doi: 10.1016/j.ridd.2004.01.008] [Medline: 15541632] 9. Rose M, Sussmilch G. The effects of semantic and gesture treatments on verb retrieval and verb use in aphasia. Aphasiology 2008 Jul;22(7-8):691-706. [doi: 10.1080/02687030701800800] 10. Sathappan AV, Luber BM, Lisanby SH. The Dynamic Duo: Combining noninvasive brain stimulation with cognitive interventions. Prog Neuropsychopharmacol Biol Psychiatry 2019 Mar 8;89:347-360. [doi: 10.1016/j.pnpbp.2018.10.006] [Medline: 30312634] 11. Jesus L, Lousada M, Domingues D, Hall A, Tomé D. Phonological processes in Portuguese children with speech sound disorders. Poznan Stud Contemp Linguist 2015;51(1):75-88. [doi: 10.1515/psicl-2015-0003] 12. Pappas NW, McAllister L, McLeod S. Parental beliefs and experiences regarding involvement in intervention for their child with speech sound disorder. Child Lang Teach Ther 2016;32(2):223-239. [doi: 10.1177/0265659015615925] 13. McLeod S, Daniel G, Barr J. 'When he's around his brothers … he's not so quiet': the private and public worlds of school-aged children with speech sound disorder. J Commun Disord 2013;46(1):70-83. [doi: 10.1016/j.jcomdis.2012.08.006] [Medline: 22995337] 14. Ishii H, Ullmer B. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. In: Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. 1997 Presented at: CHI'97; March 22-27, 1997; Atlanta, USA p. 234-241. [doi: 10.1145/258549.258715] 15. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality. IEEE Comput Grap Appl 2001;21(6):34-47. [doi: 10.1109/38.963459] 16. Ishii H. Tangible Bits: Beyond Pixels. In: Proceedings of the 2nd international conference on Tangible and embedded interaction. 2008 Presented at: TEI'08; February 18-20, 2008; Bonn, Germany p. xv-xxv. [doi: 10.1145/1347390.1347392] 17. Ishii H. Bottles: a transparent interface as a tribute to Mark Weiser. IEICE Trans Inf Syst 2004;E87-D(6):1299-1311 [FREE Full text] 18. Norman D. The Design of Everyday Things. New York: Basic Books; 2016. 19. Gibson JJ. The Ecological Approach to Visual Perception. New York: Psychology Press; 1986. 20. Li Y, Fontijn W, Markopoulos P. Tangible tabletop game supporting therapy of children with cerebral palsy. In: Markopoulos P, Ruyter B, IJsselsteijn W, Rowland D, editors. Fun and Games. Berlin: Springer; 2008:182-193. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 18 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al 21. Vonach E, Ternek M, Gerstweiler G, Kaufmann H. Design of a Health Monitoring Toy for Children. In: Proceedings of the The 15th International Conference on Interaction Design and Children. 2016 Presented at: IDC'16; June 21-24, 2016; Manchester, UK p. 58-67. [doi: 10.1145/2930674.2930694] 22. Bruckman A, Bandlow A, Forte A. Hci for kids. In: Sears A, Jacko J, editors. The Human-Computer Interaction Handbook. Boca Raton: CRC; 2007:793-809. 23. Almukadi W, Boy A. Enhancing collaboration facilitating children's learning using TUIs: a human-centered design approach. In: Zaphiris P, Ioannou A, editors. Learning Collaboration Technologies. Berlin: Springer; 2016:105-114. 24. Antle A. The CTI Framework: Informing the Design of Tangible Systems for Children. In: Proceedings of the 1st international conference on Tangible and embedded interaction. 2007 Presented at: TEI'07; February 15-17, 2007; Baton Rouge, USA p. 195-202. [doi: 10.1145/1226969.1227010] 25. Verplank B. Bill Verplank. 2009. Interaction Design Sketchbook. URL: http://www.billverplank.com/IxDSketchBook.pdf [accessed 2019-11-05] 26. Thomas JR. Acquisition of motor skills: information processing differences between children and adults. Res Q Exerc Sport 1980 Mar;51(1):158-173. [doi: 10.1080/02701367.1980.10609281] [Medline: 7394282] 27. Sherwin K, Nielsen J. Nielsen Norman Group: UX Training, Consulting, & Research. Children’s UX: Usability Issues in Designing for Young People. URL: https://www.nngroup.com/articles/childrens-websites-usability-issues/ [accessed 2018-11-27] [WebCite Cache ID 74FinIOkY] 28. Madej K. Physical Play and Children's Digital Games. Berlin: Springer; 2016. 29. Almukadi W, Stephane L. Interactive system supporting children pleasurable learning. In: Chung W, Shin C, editors. Advances in Affective and Pleasurable Design. Berlin: Springer; 2017:67-78. 30. Norman DA. Emotional Design: Why We Love (or Hate) Everyday Things. New York: Basic Books; 2005. 31. de Vette F, Tabak M, Dekker-van Weering M, Vollenbroek-Hutten M. Engaging elderly people in telemedicine through gamification. JMIR Serious Games 2015 Dec 18;3(2):e9 [FREE Full text] [doi: 10.2196/games.4561] [Medline: 26685287] 32. Lumsden J, Edwards EA, Lawrence NS, Coyle D, Munafò MR. Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 2016 Jul 15;4(2):e11 [FREE Full text] [doi: 10.2196/games.5888] [Medline: 27421244] 33. Deterding S, Dixon D, Khaled R, Nacke L. From Game Design Elements to Gamefulness: Defining 'Gamification'. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. 2011 Presented at: MindTrek'11; September 28-30, 2011; Tampere, Finland. [doi: 10.1145/2181037.2181040] 34. Zichermann G, Cunningham C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. First Edition. Sebastopol: O'Reilly; 2011. 35. Bakker S, Vorstenbosch D, Hoven E, Hollemans G, Bergman T. Weathergods: Tangible Interaction in a Digital Tabletop Game. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction 2007. 2007 Presented at: TEI'07; February 15-17, 2007; Baton Rouge, USA. [doi: 10.1145/1226969.1227000] 36. Girouard A, Solovey E, Hirshfield L, Ecott S, Shaer O, Jacob R. Smart Blocks: A Tangible Mathematical Manipulative. In: Proceedings of the 1st international conference on Tangible and embedded interaction. 2007 Presented at: TEI'07; February 15-17, 2007; Baton Rouge, USA p. 183-186. [doi: 10.1145/1226969.1227007] 37. Kaltenbrunner M, Bencina R. reacTIVision: A Computer-Vision Framework for Table-based Tangible Interaction. In: Proceedings of the 1st international conference on Tangible and embedded interaction. 2007 Presented at: TEI'07; February 15-17, 2007; Baton Rouge, USA p. 69-74. [doi: 10.1145/1226969.1226983] 38. Petrelli D, Dulake N, Marshall M, Willox M, Caparrelli F, Goldberg R. Prototyping Tangibles: Exploring Form and Interaction. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. 2014 Presented at: TEI'14; February 16-19, 2014; Munich, Germany p. 41-48. [doi: 10.1145/2540930.2540966] 39. Ruhmann L, Otero N, Oakley I. A Tangible Tool for Visual Impaired Users to Learn Geometry. In: Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. 2016 Presented at: TEI'16; February 14-17, 2016; Eindhoven, Netherlands p. 577-583. [doi: 10.1145/2839462.2856536] 40. Jesus L, Martinez J, Santos J, Hall A, Joffe V. Comparing traditional and tablet-based intervention for children with speech sound disorders: a randomised controlled trial. J Speech Lang Hear Res 2019;62(11):4045-4061. [doi: 10.1044/2019_JSLHR-S-18-0301] [Medline: 31644381] 41. Hengeveld B. Designing LinguaBytes: A Tangible Language Learning System for Non- or Hardly Speaking Toddlers. Eindhoven, The Netherlands: Technische Universiteit Eindhoven; 2011. 42. Raffle H, Vaucelle C, Wang R, Ishii H. Jabberstamp: Embedding sound and voice in traditional drawings. In: Proceedings of the 6th International Conference on Interaction Design and Children. 2007 Presented at: IDC'07; June 6-8, 2007; Aalborg, Denmark. [doi: 10.1145/1297277.1297306] 43. Bowen C. Caroline Bowen Speech-Language Therapy. Controversial Practices in Children's Speech Sound Disorders - Oral Motor Exercises, Dietary Supplements, Auditory Integration Training. URL: https://www.speech-language-therapy.com/ index.php?option=com_content&view=article&id=28:controversy&catid=11&Itemid=101 [accessed 2018-11-28] [WebCite Cache ID 74GP2xnuQ] http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 19 (page number not for citation purposes) XSL FO RenderX JMIR SERIOUS GAMES Santos et al 44. Gillon G, McNeill B. University of Canterbury. 2007. Integrated Phonological Awareness: An intervention program for preschool children with Speech-language impairment. URL: https://www.canterbury.ac.nz/media/documents/ education-and-health/gail-gillon---phonological-awareness-resources/programmes/preschool/ 01-Integrated-Phonological-Awareness-Manual-Sept-07.pdf [accessed 2019-11-05] 45. Bruns A, Jacobs J. Uses of Blogs. New York: Peter Lang; 2006. 46. de Villiers MR, Harpur PA. Design-based Research - The Educational Technology Variant of Design Research: Illustrated by the Design of an M-learning Environment. In: Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference. 2013 Presented at: SAICSIT'13; October 7-9, 2013; East London, South Africa p. 252-261. [doi: 10.1145/2513456.2513471] 47. Barab S, Squire K. Design-based research: putting a stake in the ground. J Learn Sci 2004 Jan;13(1):1-14. [doi: 10.1207/s15327809jls1301_1] 48. Read JC. Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology. Cogn Tech Work 2008;10(2):119-128. [doi: 10.1007/s10111-007-0069-9] 49. Phaser - A fast, fun and free open source HTML5 game. URL: http://phaser.io/ [accessed 2018-11-28] [WebCite Cache ID 74GQXmG9h] 50. Bootstrap. URL: https://getbootstrap.com/ [accessed 2018-11-28] [WebCite Cache ID 74GR7pYWR] 51. Apache Cordova. URL: https://cordova.apache.org/ [accessed 2018-11-28] [WebCite Cache ID 74GRSlXYr] 52. Gray D. Doing Research in the Real World. Second Edition. Los Angeles: SAGE; 2009. 53. Vairinhos M. ARTEFACTOS TANGÍVEIS E ADAPTÁVEIS NO AMBIENTE DOMÉSTICO. Portuguese: Universidade de Aveiro; 2014. 54. Abramson CM, Joslyn J, Rendle KA, Garrett SB, Dohan D. The promises of computational ethnography: improving transparency, replicability, and validity for realist approaches to ethnographic analysis. Ethnography 2018;19(2):254-284. [doi: 10.1177/1466138117725340] Abbreviations CSS: cascading style sheets DBR: design-based research FGTI: fishing game tangible interface JSON: JavaScript Object Notation KTAs: kindergarten teachers and assistants LCD: liquid crystal display RFID: radio frequency identification SLT: speech and language therapist SLTs: speech and language therapists SSD: speech sound disorders TUI: tangible user interface Edited by G Eysenbach; submitted 28.02.19; peer-reviewed by E Zimmerman, H Thabrew; comments to author 14.06.19; revised version received 03.08.19; accepted 24.09.19; published 05.12.19 Please cite as: Santos J, Vairinhos M, Jesus LMT JMIR Serious Games 2019;7(4):e13861 URL: http://games.jmir.org/2019/4/e13861/ doi: 10.2196/13861 PMID: 31804185 ©Joaquim Santos, Mário Vairinhos, Luis M T Jesus. Originally published in JMIR Serious Games (http://games.jmir.org), 05.12.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included. http://games.jmir.org/2019/4/e13861/ JMIR Serious Games 2019 | vol. 7 | iss. 4 | e13861 | p. 20 (page number not for citation purposes) XSL FO RenderX

http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png

JMIR Serious Games JMIR Publications

http://www.deepdyve.com/lp/jmir-publications/treating-children-with-speech-sound-disorders-development-of-a-TUjxEx2kmc

Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Loading next page...

Publisher: JMIR Publications
ISSN: 2291-9279
DOI: 10.2196/13861
Publisher site: See Article on Publisher Site

Abstract

Journal

JMIR Serious Games – JMIR Publications

Published: Dec 5, 2019

Keywords: children; tangible artefact; speech sound disorders; exploratory test

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Treating Children With Speech Sound Disorders: Development of a Tangible Artefact Prototype

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies