Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices

A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and... A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices div.banner_title_bkg div.trangle { border-color: #2F7CAE transparent transparent transparent; opacity:0.8; /*new styles start*/ -ms-filter:"progid:DXImageTransform.Microsoft.Alpha(Opacity=80)" ;filter: alpha(opacity=80); /*new styles end*/ } div.banner_title_bkg_if div.trangle { border-color: transparent transparent #2F7CAE transparent ; opacity:0.8; /*new styles start*/ -ms-filter:"progid:DXImageTransform.Microsoft.Alpha(Opacity=80)" ;filter: alpha(opacity=80); /*new styles end*/ } div.banner_title_bkg div.trangle { width: 316px; }div.banner_title_bkg_if div.trangle { width: 201px; } #banner { background-image: url('http://images.hindawi.com/journals/jhe/jhe.banner.jpg'); background-position: 50% 0;} Hindawi Publishing Corporation Home Journals About Us Journal of Healthcare Engineering Impact Factor 0.754 About this Journal Submit a Manuscript Table of Contents Journal Menu About this Journal · Abstracting and Indexing · Aims and Scope · Article Processing Charges · Articles in Press · Author Guidelines · Bibliographic Information · Citations to this Journal · Contact Information · Editorial Board · Editorial Workflow · Free eTOC Alerts · Publication Ethics · Reviewers Acknowledgment · Submit a Manuscript · Subscription Information · Table of Contents Open Special Issues · Special Issue Guidelines Abstract Full-Text PDF Full-Text HTML Full-Text ePUB Full-Text XML Linked References How to Cite this Article Views 95 Citations 0 ePub 1 PDF 26 Journal of Healthcare Engineering Volume 2016 (2016), Article ID 1048964, 10 pages http://dx.doi.org/10.1155/2016/1048964 Review Article A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices Yiannis Koumpouros Technological Educational Institute of Athens, Department of Informatics, Agiou Spyridonos, Aigaleo, 12243 Athens, Greece Received 27 February 2016; Revised 5 April 2016; Accepted 6 April 2016 Academic Editor: Yinkwee Ng Copyright © 2016 Yiannis Koumpouros. This is an open access article distributed under the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract The objective of the current study is to identify and classify outcome measures currently used for the assessment of rehabilitation or assistive robot devices. We conducted a systematic review of the literature using PubMed, MEDLINE, CIRRIE, and Scopus databases for studies that assessed rehabilitation or assistive robot devices from 1980 through January 2016. In all, 31 articles met all inclusion criteria. Tailor-made questionnaires were the most commonly used tool at 66.7%, while the great majority (93.9%) of the studies used nonvalidated instruments. The study reveals the absence of a standard scale which makes it difficult to compare the results from different researchers. There is a great need, therefore, for a valid and reliable instrument to be available for use by the intended end users for the subjective assessment of robot devices. The study concludes by identifying two scales that have been validated in general assistive technology devices and could support the scope of subjective assessment in rehabilitation or assistive robots (however, with limited coverage) and a new one called PYTHEIA, recently published. The latter intends to close the gap and help researchers and developers to evaluate, assess, and produce products that satisfy the real needs of the end users. 1. Introduction The aging of society along with the lack of caregivers forces innovations to help people in their daily lives. Nowadays, robots come in many forms and can be used in many ways to help people with disabilities. Although much research has been done in the field resulting in several prototypes, few assistive robots exist in common use today. The high costs and the uncertainty regarding the benefits gained are major barriers to their widespread adoption. It is therefore crucial to follow a more multidisciplinary approach during the design phase. For example, an engineering design with only healthy subjects many times leads to a system not appropriate for the target population of persons with a disability. Engineers, therapists, physiatrists, and ergonomics experts as well as the end users (people with disabilities) should be a part of the design team from the beginning of any such effort. To this end, the design team should be able to measure the satisfaction of the end users at any stage of the development phase. Measuring user satisfaction helps to measure the overall quality of a product or service. Tracking user satisfaction during the development phase can help developers and researchers make sure that the changes they are making improve the product/service for users. In customer relationship management, user (or customer) satisfaction is a measure of the degree to which a product or service meets the user’s expectations. Consumer satisfaction is a central concept in many domains (business, research, etc.) and has held a central position in marketing since the 1950s until today, with an increasing interest and importance. The realization of this importance has led to a proliferation of research on consumer satisfaction [ 1 – 4 ]. According to [ 5 ] we can distinguish two different types of user satisfaction: the process-oriented approach (equal to the difference between expected satisfaction and achieved satisfaction) and the outcome-oriented approach (as an attribute extracted from a product or service after its consumption). The evaluation of any technology device requires the objective and subjective assessment of the product/service. An objective assessment is one that needs no professional judgment to give a score correctly, while subjective assessment yields many possible answers of varying quality and requires professional judgment to give a score [ 6 ]. Subjective assessment records the facts presented by the end user that show his/her perception, understanding, and interpretation of what is happening and therefore measures his/her satisfaction. The next step is to quantify them appropriately. The evolution of Information and Communication Technologies (ICT) [ 7 ] along with nanotechnology and other sciences (e.g., medicine and behavioral science) presents a unique potential for innovative technology products and services that help people in their daily lives. The application of such innovations in healthcare has already driven in products that some years ago belonged to the scientific imagination sphere. For example, robotic technology has provided the opportunity to benefit the lives of people with disabilities (i.e., as a manipulator mounted on a desk or wheelchair or mobile base or body worn; as a mobility or communication assistant; etc.). Many commercial products are available nowadays for mobility and manipulation for people with physical disabilities (e.g., the Tek RMD [ 8 ] and the Manus robotic arm [ 9 ]), for telepresence purposes, and so forth [ 10 ], while others are being developed at research institutions worldwide. To this end, researchers are struggling to collect end users’ sentiments towards the technologies developed in order to match the products with the real needs of the end customers. Rehabilitation robotics is a special branch of robotics which focuses on machines that can be used to help people recover from severe physical trauma or assist them in activities of daily living. Rehabilitation robotics has applications in all areas of physical therapy, presenting a wide range of advancements in robotic prosthesis and other domains. Another important sector is that of assistive robotics, which tries to combine and integrate several technologies in order to meet the needs of people with various disabilities. Focusing on the healthcare sector, quality of care and patient satisfaction are major issues [ 11 ]. Thus, the assessment of any service or device from the perspective of the patient is crucial. The current study is concerned with the question of whether any reliable and valid instruments have been developed to assess assistive or rehabilitation robot devices from the user’s perspective. To the best of our knowledge, no such research has previously been carried out or published. 2. Materials and Methods A search of peer-reviewed published literature was conducted in January 2016 for articles related to the subjective assessment of assistive or rehabilitation robot devices. This was performed through the Internet via MEDLINE, PubMed, Scopus, and CIRRIE. The keywords used (employing Boolean phrases) in the searches included the following: satisfaction , assessment , assess , user , subjective , robot , robotic , assistive , rehabilitation , psychometric , test , scale , metrics , evaluation , usability , acceptability , and acceptance . Articles published after 1980 were considered for further studying. We included only studies published in English and research from peer-reviewed journals. The resulting publications were examined in a first step for potential inclusion based on their title and abstract, while we excluded any duplication. In a second step, after reviewing the abstract and title, the remaining publications underwent a full text review. We also manually searched the references lists of these articles for additional relevant sources. The whole process is depicted in Figure 1 . Figure 1: Workflow of the selection process for subjective assessment of rehabilitation and assistive robot devices. The initial query returned 3147 results. After the initial screening (reviewing the titles and abstracts and removing duplicates), the number was reduced to 312 articles. The eligibility criteria for inclusion were as follows: (i) studies involve assistive or rehabilitation robot devices (e.g., socially assistive robots, robot controlled wheelchairs, and exoskeleton devices), (ii) at least one outcome measure was used in the study, (iii) the study should include formative, process, or summative evaluation that assesses the rehabilitation or assistive robot device, (iv) the study was a peer-reviewed article (published in a journal or conference), and (v) articles are written in the English language. Finally, all articles were evaluated in terms of the clarity of the evidence presented. The second phase included a full text review of the selected articles. The main factors for excluding many articles included the following: (i) article was not written in English, (ii) the study provided a descriptive summary of the evaluation without giving more details (e.g., questionnaire/scale used), (iii) the number of participants in the study was very limited (<3 participants) or undefined, (iv) articles/studies were in the following subject areas: biochemistry, genetics and molecular biology, mathematics, chemical engineering, social sciences, materials science, physics and astronomy, psychology, arts and humanities, immunology and microbiology, agricultural and biological sciences, decision sciences, dentistry, energy, environmental science, business management and accounting, chemistry, earth and planetary science, pharmacology-toxicology and pharmaceutics, and undefined, (v) articles were in books and notes, and (vi) they assessed the technology used through quantitative data only (objective evaluation). For data extraction and analysis purposes we followed the Cochrane research methodology [ 12 ]. The processing and organization of the articles was made using the MS Excel and EndNote X6.0.2 programs. 3. Results Even though the initial query returned a large number of articles ( ), only 312 (9.914% of 3147) underwent a full text review. Finally, 31 articles (9.935% of 312) met the inclusion criteria and were included in the review. Table 1 summarizes the subjective measures used in these studies. Table 1: Summary of review results. A wide range of rehabilitation and assistive robot devices are reviewed in Table 1 (e.g., wheelchairs, exoskeleton devices, socially assistive robots, therapeutic systems, telepresence systems, and haptic technologies). The majority of the research presented was conducted in the United States of America (15.2%) and Italy (12.1%), followed by Canada (9.1%), Netherlands (9.1%), Japan (9.1%), France (9.1%), Spain (9.1%), the United Kingdom (6.1%), Germany (6.1%), South Korea (3%), Portugal (3%), Israel (3%), and Austria (3%). In the majority of the studies there were a limited number of patients participating either due to research/pilot testing or due to difficulty finding the right participants fitting in the study. Only one study managed to have 83 patients [ 13 ]. Twenty-two out of the thirty-one studies used patients to evaluate the device, while three recruited college or university students, two recruited healthcare professionals, and the remaining four used healthy subjects. Additionally, two of the studies used healthy subjects as control groups. Figure 2 presents the distribution of the subjective measures used in the studies. Figure 2: Subjective measure used in the studies reviewed (%). The last column of Table 1 presents the number of items (questions) used in each study under the subjective measure used. The question mark “?” in Table 1 , when used, means that there is no evidence (at least to the authors’ knowledge) that the specific subjective measure/scale is valid or reliable. The authors decided to use this instead of marking the specific measure as not valid or not reliable because the articles report the scale as valid. However, after searching the literature, we found no evidence on the validity and reliability of the scale in the language used in the specific study; so, we decided to present it as being in question in terms of its validity and reliability in the language used. Moreover, in the last column of the table there may appear two values separated by a comma. This means that the study used two different subjective measures and each value in the column represents the number of items (questions) used for each one of them. 4. Discussion The review of the literature provided a thorough survey of evaluations for rehabilitation or assistive robot devices concerning the satisfaction and the subjective assessment of the end users. The review began with a great number of articles returned ( ) but was ultimately narrowed to 312 of which 31 were finally included in the review. The majority of the articles failed to meet the inclusion criteria presented because most of the studies evaluated the devices by using only objective measures (e.g., Fugl-Meyer scale [ 14 ], modified Ashworth scale [ 15 ], Barthel index [ 16 ], and NASA-TLX [ 16 ]) and other clinical measurements relevant to the scope and the population of the study. It is really impressive that the great majority (66.7%) of the research included in the review used custom-made subjective measures to evaluate the satisfaction of the end users and assess the rehabilitation or assistive robot device used subjectively [ 17 – 38 ]. However, this percentage does not reveal the real problem faced. Someone should also consider the following facts: (i) Two (2) studies [ 39 , 40 ] used semistructured interviews due to the lack of any valid and reliable subjective measurement instrument. (ii) Two (2) studies [ 13 , 41 ] used the Canadian Occupational Performance Measure (COPM) [ 42 ] which is a semi-interview that enables an open dialogue between the end user and the therapist on issues of importance to the patient. It is designed to identify occupational performance problems faced by the patient. During the interview, the therapist tries to identify daily occupations of importance that the patient wants to do, needs to do, or is expected to do but is unable to accomplish. Other domains are also covered, like self-care, productivity, or leisure. Due to the nature of COPM, as described above, it cannot be adopted in all kinds of purpose rehabilitation or assistive devices. (iii) Four (4) studies [ 30 , 43 – 45 ] used the System Usability Scale (SUS) [ 46 ] which is a “quick and dirty” ten-item scale for administering after usability tests. However, this scale measures only basic issues of the device and does not take into account other very important issues, like the adaptability of the device, the feeling of safety, the social perception of the user when using the device, the individual dimensions of the device, if it fits well in the environment of the end user, and so forth. So, this scale can capture only very basic issues of the subjective assessment needed for robot-based devices. (iv) One (1) study [ 43 ] utilized the Telehealthcare Satisfaction Questionnaire-Wearable Technology (TSQ-WT) [ 47 ]. There has not yet been published (at least to the authors’ knowledge) any article in any language related to the validity and reliability of this questionnaire. Thus, it cannot be considered a reliable and valid instrument for the subjective assessment of a technology device. (v) Two (2) studies [ 48 , 49 ] utilized the Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0) scale [ 50 ]. According to Holz et al., QUEST 2.0 is a standardized satisfaction assessment tool designed for assistive technologies [ 51 ]. Moreover, it has been tested for its validity and reliability in several applications and languages [ 52 , 53 ]. Although this scale is used with assistive devices, del-Ama et al. [ 48 ] used only 7 of the 12 questions incorporated in the original questionnaire in their study. According to the previous analysis, it is apparent that finally 93.9% of the reviewed studies used neither valid nor reliable instruments to assess the robot devices. Only 6.1% of them used a validated measure (the QUEST scale), which can assess only a subset of the desired aspects (as described in the next paragraphs). In the early years of many technical fields, the research community often utilizes a wide range of metrics that are not comparable due to a bias towards application specific measures. The primary difficulty in defining common metrics is the incredibly diverse range of human-robot or robot assisted applications. Thus, although metrics from other fields (HCI, human factors, etc.) can be applied to satisfy specific needs, identifying metrics that can accommodate the entire application space may not be feasible [ 54 ]. Attempts to categorize both objective and subjective metrics have been made. According to the USUS Evaluation Framework for Human-Robot Interaction [ 55 ] the factors usability , social acceptance , user experience , and societal impact are considered the main categories of evaluation factors. Each category is divided into specific metrics, either objectively or subjectively measured: (i) Usability . Effectiveness, efficiency, learnability, flexibility, robustness, and utility. (ii) Social Acceptance . Performance expectancy, effort expectancy, attitude towards using technology, self-efficacy, forms of grouping, attachment, and reciprocity. (iii) User Experience . Embodiment, emotion, human-oriented perception, feeling of security, and coexperience with robots. (iv) Societal Impact . All effects of the introduction of robotic agents’ consequences for the social life of a specific community (taking into account cultural differences) in terms of quality of life, working conditions and employment, and education. Among these, the authors propose that the following could be tested using end-user questionnaires, which means that they could be considered subjective: (i) Utility . It refers to how an interface can be used to reach a certain goal or to perform a certain task. The more the tasks the interface is designed to perform, the more the utility it has. (ii) Performance Expectancy . It is the degree to which an individual believes that using the system will help him or her to attain gains in performance. (iii) Effort Expectancy . It indicates to which extent the user perceives that a system will be easy to use. (iv) Attitude towards Using Technology . It is the sum of all positive or negative feelings and attitudes about solving working tasks supported by a humanoid robot. (v) Self-Efficacy . It relates to a person’s perception of their ability to reach a goal. (vi) Attachment . It is an affection-tie that one person forms between themselves and another person or object—a tie that binds them together in space and endures over time. (vii) Reciprocity . It is the positive or negative response of individuals towards the actions of others. (viii) Embodiment . It describes the relationship between a system and its environment and can be measured by investigating the different perturbatory channels like morphology, which has impact on social expectations. (ix) Emotion . An emotion is an essential part in social interaction; it has to be incorporated in the assessment and design of robots. (x) Feeling of Security . It is important to investigate how to design human-robot interactions in a way that humans experience them to be safe. (xi) Coexperience . Coexperience describes experiences with objects regarding how individuals develop their personal experience based on social interaction with others. (xii) Societal Impact . Societal impact describes all effects the introduction of robotic agents results in for the social life of a specific community—taking into account cultural differences—in terms of quality of life, working conditions, employment, and education. The above categorization is the most full and detailed, including aspects that are rarely taken into account when it comes to evaluating a robotic assistant. Most researchers, however, when evaluating an assistive device/technology tend to use questionnaires that give information on the aforementioned fields. However, as Bartneck et al. mention, due to their naivety and the amount of work necessary to create a valid questionnaire, developers of robots have a tendency to quickly cook up their own questionnaires [ 56 ]. This conduct results in two main problems: firstly, the validity and reliability of these questionnaires have often not been evaluated and, secondly, the absence of standard questionnaires makes it difficult to compare the results from different researchers [ 56 ]. The findings of our review study support this conclusion. It seems that choosing tailored questionnaires is the rule in robotics assessment. However, the existing variety of questionnaires that could be useful for the assessment of rehabilitation or assistive robot devices is narrow, for the reasons described above. As derived from the current review, QUEST 2.0 may be one questionnaire that can be used in the examined field. However, a strategy that would target maximum coverage of the subjective measures spectrum would require the combined use of two (or more) questionnaires, since QUEST 2.0 covers only some subjective aspects (mainly: feeling of security, perceived effectiveness, and ease of use). We should therefore look into the literature for other valid and reliable scales that are used in different sectors and may be applicable to the examined domain. Other valuated and relevantly common used questionnaires we found in the bibliography were the Assistive Technology Device Predisposition Assessment- (ATDPA-) Device Form [ 57 , 58 ] and the Psychosocial Impact of Assistive Devices Scale (PIADS) [ 59 ]. The ATDPA-Device Form is more relevant in context than the PIADS, targeting the evaluation of overall user experience with assistive technology, while PIADS only emphasizes the psychosocial impact of assistive devices, without targeting the evaluation of the actual experience of interacting with a robot device, but rather the impact that this interaction has on quality of life (QoL). Other questionnaires such as the USE-IT [ 60 ] questionnaire were ruled out from the very beginning, since they were not well valuated or not widely used from researchers in the bibliography. It seems therefore that a combination of the QUEST 2.0 questionnaire and the Assistive Technology Device Predisposition Assessment- (ATDPA-) Device Form covers most of the desirable user-experience aspects, with ensured validity and reliability. However, no scales have been identified yet in the literature that could be adopted well and measure the individual functionalities of rehabilitation or assistive robot devices. To this end, the authors developed and are currently examining a new scale called PYTHEIA in order to fill the identified gap. The first results are very satisfactory in terms of their validity and reliability [ 61 ]. More specifically, according to the results from the exploratory factor analysis (EFA) with varimax rotation performed, the PYTHEIA instrument presents a three-factor model (the “Independent Functionalities” factor, the “Fit to Use,” and the “Ease of Use” factors). The overall Cronbach of PYTHEIA was found to be 0.793, indicating sufficient consistency. The ICC was excellent (ICC = 0.992, ), indicating that the PYTHEIA total scores were highly consistent between initial assessment and reassessment. The paired-samples -test between the two instances of administration indicated no statistically significant systematic bias ( ). Pearson’s correlation coefficient indicated stability of participants’ responses over time (Pearson’s , ). Examination of item convergent validity showed that all item intercorrelations for all item pairings were strong or excellent. Pearson’s ranged from 0.946 to 0.996 for the first factor “Independent Functionalities,” from 0.465 to 0.724 for the second “Fit to Use,” and from 0.354 to 0.732 for the third factor “Ease of Use.” This provides evidence that all subscales’ items are related to the same construct. 5. Conclusions In this paper we conducted a systematic review of the literature in order to identify existing scales for the subjective assessment of rehabilitation and assistive robot devices. We found that most of the studies are utilizing either custom-made questionnaires or interviews that are neither valid nor reliable instruments to represent the subjective opinion and perception of the end users. There is therefore a great gap in the subjective assessment of rehabilitation or assistive robot devices. The absence of standard scales/questionnaires for the subjective assessment of robot-based devices makes it difficult to design products that meet exactly the needs of the intended end users, to further improve prototypes, or to compare the results from different researchers. Based on the findings of the review, in order to further improve the subjective assessment of rehabilitation and assistive robot devices it is necessary for each study to (i) select as subjects the appropriate target group based on clear and valid inclusion criteria, (ii) involve a sufficient number of representative subjects, (iii) analyse statistically the collected data, and (iv) select an established methodology in order to enable comparison between results of different studies. Competing Interests The author declares that there are no competing interests regarding the publication of this paper. Acknowledgments The work leading to the presented results has received funding from the European Union under Grant Agreement no. 600796. References B2B International, “Customer Satisfaction Survey: How to Measure Satisfaction,” 2016, https://www.b2binternational.com/publications/customer-satisfaction-survey/ . ESOMAR, “Customer satisfaction studies [Internet]. ESOMAR World Research Codes & Guidelines,” 2005, https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-and-Guidelines_CustomerSatisfaction.pdf . Mckinsey.com, “The three Cs of customer satisfaction: consistency, consistency, consistency,” January 2016, http://www.mckinsey.com/insights/consumer_and_retail/the_three_cs_of_customer_satisfaction_consistency_consistency_consistency . A. Gustafsson, M. D. Johnson, and I. Roos, “The effects of customer satisfaction, relationship commitment dimensions, and triggers on customer retention,” Journal of Marketing , vol. 69, no. 4, pp. 210–218, 2005. View at Publisher · View at Google Scholar · View at Scopus D. de Sá, “Improving user satisfaction in VO through systems usability,” in Encyclopedia of Networked and Virtual Organizations , pp. 694–699, IGI Global, 2008. View at Google Scholar L. Suskie, “What are good assessment practices?” in Assessing Student Learning: A Common Sense Guide , L. Suskie, Ed., Anker, Bolton, Mass, USA, 1st edition, 2004. View at Google Scholar Y. Koumpouros, Information and Communication Technologies & Society , New Technologies, Athens, Greece, 2012. Matia Robotics, “Matia Robotics,” 2016, http://www.matiarobotics.com/ . MIT News, MIT-Manus Robot Aids Physical Therapy of Stroke Victims , 2016, http://news.mit.edu/2000/manus-0607 . Y. Koumpouros, Information and Communication Technologies in Healthcare , Hellenic Academic Libraries Link, Athens, Greece, 2015. C. van Campen, H. Sixma, R. D. Friele, J. J. Kerssens, and L. Peters, “Quality of care and patient satisfaction: a review of measuring instruments,” Medical Care Research and Review , vol. 52, no. 1, pp. 109–133, 1995. View at Publisher · View at Google Scholar · View at Scopus “Cohrane handbook for systematic reviews of interventions 4.2.6,” in The Cochrane Library , J. P. T. Higgins and S. Green, Eds., no. 4, John Wiley & Sons, Chichester, UK, 2006. A. S. Schroeder, R. Von Kries, C. Riedel et al., “Patient-specific determinants of responsiveness to robot-enhanced treadmill therapy in children and adolescents with cerebral palsy,” Developmental Medicine and Child Neurology , vol. 56, no. 12, pp. 1172–1179, 2014. View at Publisher · View at Google Scholar · View at Scopus The Rehabilitation Measures Database, Rehab Measures: Fugl-Meyer Assessment of Motor Recovery after Stroke , 2016, http://www.rehabmeasures.org/lists/rehabmeasures/dispform.aspx?ID=908 . B. C. Craven and A. R. Morris, “Modified ashworth scale reliability for measurement of lower extremity spasticity among patients with SCI,” Spinal Cord , vol. 48, no. 3, pp. 207–213, 2010. View at Publisher · View at Google Scholar · View at Scopus C. Collin, D. T. Wade, S. Davies, and V. Horne, “The Barthel ADL index: a reliability study,” International Disability Studies , vol. 10, no. 2, pp. 61–63, 1988. View at Publisher · View at Google Scholar · View at Scopus H. A. Abdullah, C. Tarry, C. Lambert, S. Barreca, and B. O. Allen, “Results of clinicians using a therapeutic robotic system in an inpatient stroke rehabilitation unit,” Journal of NeuroEngineering and Rehabilitation , vol. 8, no. 1, article 50, 2011. View at Publisher · View at Google Scholar · View at Scopus M. Bortole, A. Venkatakrishnan, F. Zhu et al., “The H2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study,” Journal of NeuroEngineering and Rehabilitation , vol. 12, no. 1, article 54, 2015. View at Publisher · View at Google Scholar · View at Scopus T. L. Chen, T. Bhattacharjee, J. L. McKay et al., “Evaluation by expert dancers of a robot that performs partnered stepping via haptic interaction,” PLoS ONE , vol. 10, no. 5, Article ID e0125179, 2015. View at Publisher · View at Google Scholar · View at Scopus T. L. Chen and C. C. Kemp, “Lead me by the hand,” in Proceeding of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI '10) , pp. 367–374, Osaka, Japan, March 2010. View at Publisher · View at Google Scholar · View at Scopus Y. S. Choi, C. D. Anderson, J. D. Glass, and C. C. Kemp, “Laser pointers and a touch screen: intuitive interfaces for autonomous mobile manipulation for the motor impaired,” in Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '08) , pp. 225–232, October 2008. View at Publisher · View at Google Scholar · View at Scopus P. Coignard, J. P. Departe, O. Remy Neris et al., “ANSO study: evaluation in an indoor environment of a mobile assistance robotic grasping arm,” Annals of Physical and Rehabilitation Medicine , vol. 56, no. 9-10, pp. 621–633, 2013. View at Publisher · View at Google Scholar · View at Scopus G. G. Grindle, H. Wang, H. Jeannis, E. Teodorski, and R. A. Cooper, “Design and user evaluation of a wheelchair mounted robotic assisted transfer device,” BioMed Research International , vol. 2015, Article ID 198476, 9 pages, 2015. View at Publisher · View at Google Scholar · View at Scopus M. Heerink, B. Kröse, V. Evers, and B. Wielinga, “Assessing acceptance of assistive social agent technology by older adults: the Almere model,” International Journal of Social Robotics , vol. 2, no. 4, pp. 361–375, 2010. View at Publisher · View at Google Scholar · View at Scopus M. J. Kim, D. H. Lee, T. Kim et al., “Lower extremity exercise of knee osteoarthritis patients using portable assistive robot (HEXAR-KR40P),” International Journal of Precision Engineering and Manufacturing , vol. 15, no. 12, pp. 2617–2622, 2014. View at Publisher · View at Google Scholar · View at Scopus P. N. Kooren, A. G. Dunning, M. M. H. P. Janssen et al., “Design and pilot validation of A-gear: a novel wearable dynamic arm support,” Journal of NeuroEngineering and Rehabilitation , vol. 12, no. 1, article 83, 2015. View at Publisher · View at Google Scholar · View at Scopus I. Laffont, N. Biard, G. Chalubert et al., “Evaluation of a graphic interface to control a robotic grasping arm: a multicenter study,” Archives of Physical Medicine and Rehabilitation , vol. 90, no. 10, pp. 1740–1748, 2009. View at Publisher · View at Google Scholar · View at Scopus P. Lam, D. Hebert, J. Boger et al., “A haptic-robotic platform for upper-limb reaching stroke therapy: preliminary design and evaluation results,” Journal of NeuroEngineering and Rehabilitation , vol. 5, no. 1, article 15, 2008. View at Publisher · View at Google Scholar · View at Scopus C. Liu, C. T. Ishi, and H. Ishiguro, “Bringing the Scene Back to the Tele-operator: auditory scene manipulation for tele-presence systems,” in Proceedings of the 10th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI '15) , pp. 279–286, Portland, Ore, USA, March 2015. View at Publisher · View at Google Scholar · View at Scopus L. Lopez-Samaniego, B. Garcia-Zapirain, and A. Mendez-Zorrilla, “Memory and accurate processing brain rehabilitation for the elderly: LEGO robot and iPad case study,” Bio-Medical Materials and Engineering , vol. 24, no. 6, pp. 3549–3556, 2014. View at Publisher · View at Google Scholar · View at Scopus S. Mazzoleni, G. Turchetti, I. Palla, F. Posteraro, and P. Dario, “Acceptability of robotic technology in neuro-rehabilitation: preliminary results on chronic stroke patients,” Computer Methods and Programs in Biomedicine , vol. 116, no. 2, pp. 116–122, 2014. View at Publisher · View at Google Scholar · View at Scopus K. Ozaki, H. Kagaya, S. Hirano et al., “Preliminary trial of postural strategy training using a personal transport assistance robot for patients with central nervous system disorder,” Archives of Physical Medicine and Rehabilitation , vol. 94, no. 1, pp. 59–66, 2013. View at Publisher · View at Google Scholar · View at Scopus J. Pineau, A. Atrash, R. Kaplow, and J. Villemure, “On the design and validation of an intelligent powered wheelchair: lessons from the SmartWheeler project,” Advances in Intelligent and Soft Computing , vol. 83, pp. 259–268, 2010. View at Publisher · View at Google Scholar · View at Scopus H. Wang, J. Xu, G. Grindle et al., “Performance evaluation of the personal mobility and manipulation appliance (PerMMA),” Medical Engineering & Physics , vol. 35, no. 11, pp. 1613–1619, 2013. View at Publisher · View at Google Scholar · View at Scopus Y.-H. Wu, J. Wrobel, M. Cornuet, H. Kerhervé, S. Damnée, and A.-S. Rrigaud, “Acceptance of an assistive robot in older adults: a mixed-method study of human-robot interaction over a 1-month period in the Living Lab setting,” Clinical Interventions in Aging , vol. 9, pp. 801–811, 2014. View at Publisher · View at Google Scholar · View at Scopus F. Werner and D. Krainer, “A socially assistive robot to support physical training of older people—an end user acceptance study,” in Proceedings of the 5th International Conference on Social Robotics (ICSR '13) , pp. 562–563, Springer, Bristol, UK, October 2013. I. Treger, S. Faran, and H. Ring, “Robot-assisted therapy for neuromuscular training of sub-acute stroke patients. A feasibility study,” European Journal of Physical and Rehabilitation Medicine , vol. 44, no. 4, pp. 431–435, 2008. View at Google Scholar · View at Scopus A.-M. Hughes, J. Burridge, C. T. Freeman et al., “Stroke participants' perceptions of robotic and electrical stimulation therapy: a new approach,” Disability and Rehabilitation: Assistive Technology , vol. 6, no. 2, pp. 130–138, 2011. View at Publisher · View at Google Scholar · View at Scopus H. Chihara, Y. Takagi, K. Nishino et al., “Factors predicting the effects of hybrid assistive limb robot suit during the acute phase of central nervous system injury,” Neurologia Medico-Chirurgica , vol. 56, no. 1, pp. 33–37, 2016. View at Publisher · View at Google Scholar A. M. von der Pütten, N. C. Krämer, and S. C. Eimler, “Living with a robot companion: empirical study on the interaction with an artificial health advisor,” in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '11) , pp. 327–334, November 2011. View at Publisher · View at Google Scholar · View at Scopus P. Sale, V. Lombardi, and M. Franceschini, “Hand robotics rehabilitation: feasibility and preliminary results of a robotic treatment in patients with hemiparesis,” Stroke Research and Treatment , vol. 2012, Article ID 820931, 5 pages, 2012. View at Publisher · View at Google Scholar · View at Scopus COPM, “Learn to Use the COPM,” 2013, http://www.thecopm.ca/learn/ . E. Ambrosini, S. Ferrante, M. Rossini et al., “Functional and usability assessment of a robotic exoskeleton arm to support activities of daily life,” Robotica , vol. 32, no. 8, pp. 1213–1224, 2014. View at Publisher · View at Google Scholar · View at Scopus F. Amirabdollahian, S. Ates, A. Basteris et al., “Design, development and deployment of a hand/wrist exoskeleton for home-based rehabilitation after stroke—SCRIPT project,” Robotica , vol. 32, no. 8, pp. 1331–1346, 2014. View at Publisher · View at Google Scholar · View at Scopus B. Mónica Faria, S. Vasconcelos, L. Paulo Reis, and N. Lau, “Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study,” Assistive Technology , vol. 25, no. 2, pp. 88–98, 2013. View at Publisher · View at Google Scholar · View at Scopus J. Brooke, “SUS: a “quick and dirty” usability scale,” in Usability Evaluation Industry , P. W. Jordan, B. Thomas, B. A. Weerdmeester, and A. I. McClelland, Eds., Taylor and Francis, London, UK, 1996. View at Google Scholar L. Chiari, R. Van Lummel, C. Becker, K. Pfeiffer, U. Lindemman, and W. Zijlstra, “Classification of the user's needs, characteristics and scenarios—update,” Report from the EU Project (6th Framework Program, IST Contract no. 045622) Sensing and Action to support mobility in Ambient Assisted Living, 2009. A. J. del-Ama, Á. Gil-Agudo, J. L. Pons, and J. C. Moreno, “Hybrid FES-robot cooperative control of ambulatory gait rehabilitation exoskeleton,” Journal of NeuroEngineering and Rehabilitation , vol. 11, no. 1, article 27, 2014. View at Publisher · View at Google Scholar · View at Scopus A. Jardón, Á. M. Gil, A. I. de la Peña, C. A. Monje, and C. Balaguer, “Usability assessment of ASIBOT: a portable robot to aid patients with spinal cord injury,” Disability and Rehabilitation: Assistive Technology , vol. 6, no. 4, pp. 320–330, 2011. View at Publisher · View at Google Scholar · View at Scopus L. Demers, R. Weiss-Lambrou, and B. Ska, Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST Version 2.0): An Outcome Measure for Assistive Technology Devices , Institute for Matching Person & Technology, Webster, NY, USA, 2000. E. M. Holz, J. Höhne, P. Staiger-Sälzer, M. Tangermann, and A. Kübler, “Brain-computer interface controlled gaming: evaluation of usability by severely motor restricted end-users,” Artificial Intelligence in Medicine , vol. 59, no. 2, pp. 111–120, 2013. View at Publisher · View at Google Scholar · View at Scopus L. Demers, M. Monette, Y. Lapierre, D. L. Arnold, and C. Wolfson, “Reliability, validity, and applicability of the quebec user evaluation of satisfaction with assistive technology (QUEST 2.0) for adults with multiple sclerosis,” Disability and Rehabilitation , vol. 24, no. 1–3, pp. 21–30, 2002. View at Publisher · View at Google Scholar · View at Scopus Y. Koumpouros, A. Karavasili, E. Papageorgiou, and P. Siavelis, “Validation of the greek version of the device subscale of the quebec user evaluation of satisfaction with assistive technology 2.0 (QUEST 2.0),” Assistive Technology , 2016. View at Publisher · View at Google Scholar A. Steinfeld, T. Fong, D. Kaber, et al., “Common metrics for human-robot interaction,” in Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI '06) , ACM Proceedings, pp. 33–40, 2006. View at Publisher · View at Google Scholar A. Weiss, R. Bernhaupt, M. Lankes, and M. Tscheligi, “The USUS evaluation framework for human-robot interaction,” in Proceedings of the Symposium on New Frontiers in Human-Robot Interaction (AISB '09) , pp. 158–165, Edinburgh, UK, April 2009. C. Bartneck, D. Kulić, E. Croft, and S. Zoghbi, “Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots,” International Journal of Social Robotics , vol. 1, no. 1, pp. 71–81, 2009. View at Publisher · View at Google Scholar · View at Scopus M. J. Scherer, Matching Person and Technology Process and Accompanying Assessment Instruments , Revised Edition, The Institute for Matching Person & Technology, Webster, NY, USA, 1998. Y. Koumpouros, E. Papageorgiou, A. Karavasili, and D. Alexopoulou, “Translation and validation of the assistive technology device predisposition assessment in Greek in order to assess satisfaction with use of the selected assistive device,” Disability and Rehabilitation: Assistive Technology , 2016. View at Publisher · View at Google Scholar P. Palmer, C. Thursfield, and S. Judge, “An evaluation of the psychosocial impact of assistive devices scale,” in Assistive Technology: From Virtuality to Reality , A. Pruski and H. Knops, Eds., pp. 740–744, IOS Press, Nieuwe Hemweg, The Netherlands, 1st edition, 2005. View at Google Scholar M. B. Michel-Verkerke and A. M. G. M. Hoogeboom, “Evaluation of the USE IT-questionnaire for the evaluation of the adoption of electronic patient records by healthcare professionals,” Methods of Information in Medicine , vol. 52, no. 3, pp. 189–198, 2013. View at Publisher · View at Google Scholar · View at Scopus Y. Koumpouros, E. Papageorgiou, and A. Karavasili, “Development of a new psychometric scale (PYTHEIA) to assess the satisfaction of users with any assistive technology,” in Proceedings of the 7th International Conference on Applied Human Factors and Ergonomics , IEEE, July 2016. (function (i, s, o, g, r, a, m) { i['GoogleAnalyticsObject'] = r; i[r] = i[r] || function () { (i[r].q = i[r].q || []).push(arguments) }, i[r].l = 1 * new Date(); a = s.createElement(o), m = s.getElementsByTagName(o)[0]; a.async = 1; a.src = g; m.parentNode.insertBefore(a, m) })(window, document, 'script', '//www.google-analytics.com/analytics.js', 'ga'); ga('create', 'UA-8578054-2', 'auto'); ga('send', 'pageview'); http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Healthcare Engineering Hindawi Publishing Corporation

A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices

Koumpouros, Yiannis
Journal of Healthcare Engineering , Volume 2016 (2016) – May 4, 2016
Download PDF
11 pages

Loading next page...
 
/lp/hindawi-publishing-corporation/a-systematic-review-on-existing-measures-for-the-subjective-assessment-oS0qNK48tG

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2016 Yiannis Koumpouros.
ISSN
2040-2295
Publisher site
See Article on Publisher Site

Abstract

A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices div.banner_title_bkg div.trangle { border-color: #2F7CAE transparent transparent transparent; opacity:0.8; /*new styles start*/ -ms-filter:"progid:DXImageTransform.Microsoft.Alpha(Opacity=80)" ;filter: alpha(opacity=80); /*new styles end*/ } div.banner_title_bkg_if div.trangle { border-color: transparent transparent #2F7CAE transparent ; opacity:0.8; /*new styles start*/ -ms-filter:"progid:DXImageTransform.Microsoft.Alpha(Opacity=80)" ;filter: alpha(opacity=80); /*new styles end*/ } div.banner_title_bkg div.trangle { width: 316px; }div.banner_title_bkg_if div.trangle { width: 201px; } #banner { background-image: url('http://images.hindawi.com/journals/jhe/jhe.banner.jpg'); background-position: 50% 0;} Hindawi Publishing Corporation Home Journals About Us Journal of Healthcare Engineering Impact Factor 0.754 About this Journal Submit a Manuscript Table of Contents Journal Menu About this Journal · Abstracting and Indexing · Aims and Scope · Article Processing Charges · Articles in Press · Author Guidelines · Bibliographic Information · Citations to this Journal · Contact Information · Editorial Board · Editorial Workflow · Free eTOC Alerts · Publication Ethics · Reviewers Acknowledgment · Submit a Manuscript · Subscription Information · Table of Contents Open Special Issues · Special Issue Guidelines Abstract Full-Text PDF Full-Text HTML Full-Text ePUB Full-Text XML Linked References How to Cite this Article Views 95 Citations 0 ePub 1 PDF 26 Journal of Healthcare Engineering Volume 2016 (2016), Article ID 1048964, 10 pages http://dx.doi.org/10.1155/2016/1048964 Review Article A Systematic Review on Existing Measures for the Subjective Assessment of Rehabilitation and Assistive Robot Devices Yiannis Koumpouros Technological Educational Institute of Athens, Department of Informatics, Agiou Spyridonos, Aigaleo, 12243 Athens, Greece Received 27 February 2016; Revised 5 April 2016; Accepted 6 April 2016 Academic Editor: Yinkwee Ng Copyright © 2016 Yiannis Koumpouros. This is an open access article distributed under the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract The objective of the current study is to identify and classify outcome measures currently used for the assessment of rehabilitation or assistive robot devices. We conducted a systematic review of the literature using PubMed, MEDLINE, CIRRIE, and Scopus databases for studies that assessed rehabilitation or assistive robot devices from 1980 through January 2016. In all, 31 articles met all inclusion criteria. Tailor-made questionnaires were the most commonly used tool at 66.7%, while the great majority (93.9%) of the studies used nonvalidated instruments. The study reveals the absence of a standard scale which makes it difficult to compare the results from different researchers. There is a great need, therefore, for a valid and reliable instrument to be available for use by the intended end users for the subjective assessment of robot devices. The study concludes by identifying two scales that have been validated in general assistive technology devices and could support the scope of subjective assessment in rehabilitation or assistive robots (however, with limited coverage) and a new one called PYTHEIA, recently published. The latter intends to close the gap and help researchers and developers to evaluate, assess, and produce products that satisfy the real needs of the end users. 1. Introduction The aging of society along with the lack of caregivers forces innovations to help people in their daily lives. Nowadays, robots come in many forms and can be used in many ways to help people with disabilities. Although much research has been done in the field resulting in several prototypes, few assistive robots exist in common use today. The high costs and the uncertainty regarding the benefits gained are major barriers to their widespread adoption. It is therefore crucial to follow a more multidisciplinary approach during the design phase. For example, an engineering design with only healthy subjects many times leads to a system not appropriate for the target population of persons with a disability. Engineers, therapists, physiatrists, and ergonomics experts as well as the end users (people with disabilities) should be a part of the design team from the beginning of any such effort. To this end, the design team should be able to measure the satisfaction of the end users at any stage of the development phase. Measuring user satisfaction helps to measure the overall quality of a product or service. Tracking user satisfaction during the development phase can help developers and researchers make sure that the changes they are making improve the product/service for users. In customer relationship management, user (or customer) satisfaction is a measure of the degree to which a product or service meets the user’s expectations. Consumer satisfaction is a central concept in many domains (business, research, etc.) and has held a central position in marketing since the 1950s until today, with an increasing interest and importance. The realization of this importance has led to a proliferation of research on consumer satisfaction [ 1 – 4 ]. According to [ 5 ] we can distinguish two different types of user satisfaction: the process-oriented approach (equal to the difference between expected satisfaction and achieved satisfaction) and the outcome-oriented approach (as an attribute extracted from a product or service after its consumption). The evaluation of any technology device requires the objective and subjective assessment of the product/service. An objective assessment is one that needs no professional judgment to give a score correctly, while subjective assessment yields many possible answers of varying quality and requires professional judgment to give a score [ 6 ]. Subjective assessment records the facts presented by the end user that show his/her perception, understanding, and interpretation of what is happening and therefore measures his/her satisfaction. The next step is to quantify them appropriately. The evolution of Information and Communication Technologies (ICT) [ 7 ] along with nanotechnology and other sciences (e.g., medicine and behavioral science) presents a unique potential for innovative technology products and services that help people in their daily lives. The application of such innovations in healthcare has already driven in products that some years ago belonged to the scientific imagination sphere. For example, robotic technology has provided the opportunity to benefit the lives of people with disabilities (i.e., as a manipulator mounted on a desk or wheelchair or mobile base or body worn; as a mobility or communication assistant; etc.). Many commercial products are available nowadays for mobility and manipulation for people with physical disabilities (e.g., the Tek RMD [ 8 ] and the Manus robotic arm [ 9 ]), for telepresence purposes, and so forth [ 10 ], while others are being developed at research institutions worldwide. To this end, researchers are struggling to collect end users’ sentiments towards the technologies developed in order to match the products with the real needs of the end customers. Rehabilitation robotics is a special branch of robotics which focuses on machines that can be used to help people recover from severe physical trauma or assist them in activities of daily living. Rehabilitation robotics has applications in all areas of physical therapy, presenting a wide range of advancements in robotic prosthesis and other domains. Another important sector is that of assistive robotics, which tries to combine and integrate several technologies in order to meet the needs of people with various disabilities. Focusing on the healthcare sector, quality of care and patient satisfaction are major issues [ 11 ]. Thus, the assessment of any service or device from the perspective of the patient is crucial. The current study is concerned with the question of whether any reliable and valid instruments have been developed to assess assistive or rehabilitation robot devices from the user’s perspective. To the best of our knowledge, no such research has previously been carried out or published. 2. Materials and Methods A search of peer-reviewed published literature was conducted in January 2016 for articles related to the subjective assessment of assistive or rehabilitation robot devices. This was performed through the Internet via MEDLINE, PubMed, Scopus, and CIRRIE. The keywords used (employing Boolean phrases) in the searches included the following: satisfaction , assessment , assess , user , subjective , robot , robotic , assistive , rehabilitation , psychometric , test , scale , metrics , evaluation , usability , acceptability , and acceptance . Articles published after 1980 were considered for further studying. We included only studies published in English and research from peer-reviewed journals. The resulting publications were examined in a first step for potential inclusion based on their title and abstract, while we excluded any duplication. In a second step, after reviewing the abstract and title, the remaining publications underwent a full text review. We also manually searched the references lists of these articles for additional relevant sources. The whole process is depicted in Figure 1 . Figure 1: Workflow of the selection process for subjective assessment of rehabilitation and assistive robot devices. The initial query returned 3147 results. After the initial screening (reviewing the titles and abstracts and removing duplicates), the number was reduced to 312 articles. The eligibility criteria for inclusion were as follows: (i) studies involve assistive or rehabilitation robot devices (e.g., socially assistive robots, robot controlled wheelchairs, and exoskeleton devices), (ii) at least one outcome measure was used in the study, (iii) the study should include formative, process, or summative evaluation that assesses the rehabilitation or assistive robot device, (iv) the study was a peer-reviewed article (published in a journal or conference), and (v) articles are written in the English language. Finally, all articles were evaluated in terms of the clarity of the evidence presented. The second phase included a full text review of the selected articles. The main factors for excluding many articles included the following: (i) article was not written in English, (ii) the study provided a descriptive summary of the evaluation without giving more details (e.g., questionnaire/scale used), (iii) the number of participants in the study was very limited (<3 participants) or undefined, (iv) articles/studies were in the following subject areas: biochemistry, genetics and molecular biology, mathematics, chemical engineering, social sciences, materials science, physics and astronomy, psychology, arts and humanities, immunology and microbiology, agricultural and biological sciences, decision sciences, dentistry, energy, environmental science, business management and accounting, chemistry, earth and planetary science, pharmacology-toxicology and pharmaceutics, and undefined, (v) articles were in books and notes, and (vi) they assessed the technology used through quantitative data only (objective evaluation). For data extraction and analysis purposes we followed the Cochrane research methodology [ 12 ]. The processing and organization of the articles was made using the MS Excel and EndNote X6.0.2 programs. 3. Results Even though the initial query returned a large number of articles ( ), only 312 (9.914% of 3147) underwent a full text review. Finally, 31 articles (9.935% of 312) met the inclusion criteria and were included in the review. Table 1 summarizes the subjective measures used in these studies. Table 1: Summary of review results. A wide range of rehabilitation and assistive robot devices are reviewed in Table 1 (e.g., wheelchairs, exoskeleton devices, socially assistive robots, therapeutic systems, telepresence systems, and haptic technologies). The majority of the research presented was conducted in the United States of America (15.2%) and Italy (12.1%), followed by Canada (9.1%), Netherlands (9.1%), Japan (9.1%), France (9.1%), Spain (9.1%), the United Kingdom (6.1%), Germany (6.1%), South Korea (3%), Portugal (3%), Israel (3%), and Austria (3%). In the majority of the studies there were a limited number of patients participating either due to research/pilot testing or due to difficulty finding the right participants fitting in the study. Only one study managed to have 83 patients [ 13 ]. Twenty-two out of the thirty-one studies used patients to evaluate the device, while three recruited college or university students, two recruited healthcare professionals, and the remaining four used healthy subjects. Additionally, two of the studies used healthy subjects as control groups. Figure 2 presents the distribution of the subjective measures used in the studies. Figure 2: Subjective measure used in the studies reviewed (%). The last column of Table 1 presents the number of items (questions) used in each study under the subjective measure used. The question mark “?” in Table 1 , when used, means that there is no evidence (at least to the authors’ knowledge) that the specific subjective measure/scale is valid or reliable. The authors decided to use this instead of marking the specific measure as not valid or not reliable because the articles report the scale as valid. However, after searching the literature, we found no evidence on the validity and reliability of the scale in the language used in the specific study; so, we decided to present it as being in question in terms of its validity and reliability in the language used. Moreover, in the last column of the table there may appear two values separated by a comma. This means that the study used two different subjective measures and each value in the column represents the number of items (questions) used for each one of them. 4. Discussion The review of the literature provided a thorough survey of evaluations for rehabilitation or assistive robot devices concerning the satisfaction and the subjective assessment of the end users. The review began with a great number of articles returned ( ) but was ultimately narrowed to 312 of which 31 were finally included in the review. The majority of the articles failed to meet the inclusion criteria presented because most of the studies evaluated the devices by using only objective measures (e.g., Fugl-Meyer scale [ 14 ], modified Ashworth scale [ 15 ], Barthel index [ 16 ], and NASA-TLX [ 16 ]) and other clinical measurements relevant to the scope and the population of the study. It is really impressive that the great majority (66.7%) of the research included in the review used custom-made subjective measures to evaluate the satisfaction of the end users and assess the rehabilitation or assistive robot device used subjectively [ 17 – 38 ]. However, this percentage does not reveal the real problem faced. Someone should also consider the following facts: (i) Two (2) studies [ 39 , 40 ] used semistructured interviews due to the lack of any valid and reliable subjective measurement instrument. (ii) Two (2) studies [ 13 , 41 ] used the Canadian Occupational Performance Measure (COPM) [ 42 ] which is a semi-interview that enables an open dialogue between the end user and the therapist on issues of importance to the patient. It is designed to identify occupational performance problems faced by the patient. During the interview, the therapist tries to identify daily occupations of importance that the patient wants to do, needs to do, or is expected to do but is unable to accomplish. Other domains are also covered, like self-care, productivity, or leisure. Due to the nature of COPM, as described above, it cannot be adopted in all kinds of purpose rehabilitation or assistive devices. (iii) Four (4) studies [ 30 , 43 – 45 ] used the System Usability Scale (SUS) [ 46 ] which is a “quick and dirty” ten-item scale for administering after usability tests. However, this scale measures only basic issues of the device and does not take into account other very important issues, like the adaptability of the device, the feeling of safety, the social perception of the user when using the device, the individual dimensions of the device, if it fits well in the environment of the end user, and so forth. So, this scale can capture only very basic issues of the subjective assessment needed for robot-based devices. (iv) One (1) study [ 43 ] utilized the Telehealthcare Satisfaction Questionnaire-Wearable Technology (TSQ-WT) [ 47 ]. There has not yet been published (at least to the authors’ knowledge) any article in any language related to the validity and reliability of this questionnaire. Thus, it cannot be considered a reliable and valid instrument for the subjective assessment of a technology device. (v) Two (2) studies [ 48 , 49 ] utilized the Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0) scale [ 50 ]. According to Holz et al., QUEST 2.0 is a standardized satisfaction assessment tool designed for assistive technologies [ 51 ]. Moreover, it has been tested for its validity and reliability in several applications and languages [ 52 , 53 ]. Although this scale is used with assistive devices, del-Ama et al. [ 48 ] used only 7 of the 12 questions incorporated in the original questionnaire in their study. According to the previous analysis, it is apparent that finally 93.9% of the reviewed studies used neither valid nor reliable instruments to assess the robot devices. Only 6.1% of them used a validated measure (the QUEST scale), which can assess only a subset of the desired aspects (as described in the next paragraphs). In the early years of many technical fields, the research community often utilizes a wide range of metrics that are not comparable due to a bias towards application specific measures. The primary difficulty in defining common metrics is the incredibly diverse range of human-robot or robot assisted applications. Thus, although metrics from other fields (HCI, human factors, etc.) can be applied to satisfy specific needs, identifying metrics that can accommodate the entire application space may not be feasible [ 54 ]. Attempts to categorize both objective and subjective metrics have been made. According to the USUS Evaluation Framework for Human-Robot Interaction [ 55 ] the factors usability , social acceptance , user experience , and societal impact are considered the main categories of evaluation factors. Each category is divided into specific metrics, either objectively or subjectively measured: (i) Usability . Effectiveness, efficiency, learnability, flexibility, robustness, and utility. (ii) Social Acceptance . Performance expectancy, effort expectancy, attitude towards using technology, self-efficacy, forms of grouping, attachment, and reciprocity. (iii) User Experience . Embodiment, emotion, human-oriented perception, feeling of security, and coexperience with robots. (iv) Societal Impact . All effects of the introduction of robotic agents’ consequences for the social life of a specific community (taking into account cultural differences) in terms of quality of life, working conditions and employment, and education. Among these, the authors propose that the following could be tested using end-user questionnaires, which means that they could be considered subjective: (i) Utility . It refers to how an interface can be used to reach a certain goal or to perform a certain task. The more the tasks the interface is designed to perform, the more the utility it has. (ii) Performance Expectancy . It is the degree to which an individual believes that using the system will help him or her to attain gains in performance. (iii) Effort Expectancy . It indicates to which extent the user perceives that a system will be easy to use. (iv) Attitude towards Using Technology . It is the sum of all positive or negative feelings and attitudes about solving working tasks supported by a humanoid robot. (v) Self-Efficacy . It relates to a person’s perception of their ability to reach a goal. (vi) Attachment . It is an affection-tie that one person forms between themselves and another person or object—a tie that binds them together in space and endures over time. (vii) Reciprocity . It is the positive or negative response of individuals towards the actions of others. (viii) Embodiment . It describes the relationship between a system and its environment and can be measured by investigating the different perturbatory channels like morphology, which has impact on social expectations. (ix) Emotion . An emotion is an essential part in social interaction; it has to be incorporated in the assessment and design of robots. (x) Feeling of Security . It is important to investigate how to design human-robot interactions in a way that humans experience them to be safe. (xi) Coexperience . Coexperience describes experiences with objects regarding how individuals develop their personal experience based on social interaction with others. (xii) Societal Impact . Societal impact describes all effects the introduction of robotic agents results in for the social life of a specific community—taking into account cultural differences—in terms of quality of life, working conditions, employment, and education. The above categorization is the most full and detailed, including aspects that are rarely taken into account when it comes to evaluating a robotic assistant. Most researchers, however, when evaluating an assistive device/technology tend to use questionnaires that give information on the aforementioned fields. However, as Bartneck et al. mention, due to their naivety and the amount of work necessary to create a valid questionnaire, developers of robots have a tendency to quickly cook up their own questionnaires [ 56 ]. This conduct results in two main problems: firstly, the validity and reliability of these questionnaires have often not been evaluated and, secondly, the absence of standard questionnaires makes it difficult to compare the results from different researchers [ 56 ]. The findings of our review study support this conclusion. It seems that choosing tailored questionnaires is the rule in robotics assessment. However, the existing variety of questionnaires that could be useful for the assessment of rehabilitation or assistive robot devices is narrow, for the reasons described above. As derived from the current review, QUEST 2.0 may be one questionnaire that can be used in the examined field. However, a strategy that would target maximum coverage of the subjective measures spectrum would require the combined use of two (or more) questionnaires, since QUEST 2.0 covers only some subjective aspects (mainly: feeling of security, perceived effectiveness, and ease of use). We should therefore look into the literature for other valid and reliable scales that are used in different sectors and may be applicable to the examined domain. Other valuated and relevantly common used questionnaires we found in the bibliography were the Assistive Technology Device Predisposition Assessment- (ATDPA-) Device Form [ 57 , 58 ] and the Psychosocial Impact of Assistive Devices Scale (PIADS) [ 59 ]. The ATDPA-Device Form is more relevant in context than the PIADS, targeting the evaluation of overall user experience with assistive technology, while PIADS only emphasizes the psychosocial impact of assistive devices, without targeting the evaluation of the actual experience of interacting with a robot device, but rather the impact that this interaction has on quality of life (QoL). Other questionnaires such as the USE-IT [ 60 ] questionnaire were ruled out from the very beginning, since they were not well valuated or not widely used from researchers in the bibliography. It seems therefore that a combination of the QUEST 2.0 questionnaire and the Assistive Technology Device Predisposition Assessment- (ATDPA-) Device Form covers most of the desirable user-experience aspects, with ensured validity and reliability. However, no scales have been identified yet in the literature that could be adopted well and measure the individual functionalities of rehabilitation or assistive robot devices. To this end, the authors developed and are currently examining a new scale called PYTHEIA in order to fill the identified gap. The first results are very satisfactory in terms of their validity and reliability [ 61 ]. More specifically, according to the results from the exploratory factor analysis (EFA) with varimax rotation performed, the PYTHEIA instrument presents a three-factor model (the “Independent Functionalities” factor, the “Fit to Use,” and the “Ease of Use” factors). The overall Cronbach of PYTHEIA was found to be 0.793, indicating sufficient consistency. The ICC was excellent (ICC = 0.992, ), indicating that the PYTHEIA total scores were highly consistent between initial assessment and reassessment. The paired-samples -test between the two instances of administration indicated no statistically significant systematic bias ( ). Pearson’s correlation coefficient indicated stability of participants’ responses over time (Pearson’s , ). Examination of item convergent validity showed that all item intercorrelations for all item pairings were strong or excellent. Pearson’s ranged from 0.946 to 0.996 for the first factor “Independent Functionalities,” from 0.465 to 0.724 for the second “Fit to Use,” and from 0.354 to 0.732 for the third factor “Ease of Use.” This provides evidence that all subscales’ items are related to the same construct. 5. Conclusions In this paper we conducted a systematic review of the literature in order to identify existing scales for the subjective assessment of rehabilitation and assistive robot devices. We found that most of the studies are utilizing either custom-made questionnaires or interviews that are neither valid nor reliable instruments to represent the subjective opinion and perception of the end users. There is therefore a great gap in the subjective assessment of rehabilitation or assistive robot devices. The absence of standard scales/questionnaires for the subjective assessment of robot-based devices makes it difficult to design products that meet exactly the needs of the intended end users, to further improve prototypes, or to compare the results from different researchers. Based on the findings of the review, in order to further improve the subjective assessment of rehabilitation and assistive robot devices it is necessary for each study to (i) select as subjects the appropriate target group based on clear and valid inclusion criteria, (ii) involve a sufficient number of representative subjects, (iii) analyse statistically the collected data, and (iv) select an established methodology in order to enable comparison between results of different studies. Competing Interests The author declares that there are no competing interests regarding the publication of this paper. Acknowledgments The work leading to the presented results has received funding from the European Union under Grant Agreement no. 600796. References B2B International, “Customer Satisfaction Survey: How to Measure Satisfaction,” 2016, https://www.b2binternational.com/publications/customer-satisfaction-survey/ . ESOMAR, “Customer satisfaction studies [Internet]. ESOMAR World Research Codes & Guidelines,” 2005, https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-and-Guidelines_CustomerSatisfaction.pdf . Mckinsey.com, “The three Cs of customer satisfaction: consistency, consistency, consistency,” January 2016, http://www.mckinsey.com/insights/consumer_and_retail/the_three_cs_of_customer_satisfaction_consistency_consistency_consistency . A. Gustafsson, M. D. Johnson, and I. Roos, “The effects of customer satisfaction, relationship commitment dimensions, and triggers on customer retention,” Journal of Marketing , vol. 69, no. 4, pp. 210–218, 2005. View at Publisher · View at Google Scholar · View at Scopus D. de Sá, “Improving user satisfaction in VO through systems usability,” in Encyclopedia of Networked and Virtual Organizations , pp. 694–699, IGI Global, 2008. View at Google Scholar L. Suskie, “What are good assessment practices?” in Assessing Student Learning: A Common Sense Guide , L. Suskie, Ed., Anker, Bolton, Mass, USA, 1st edition, 2004. View at Google Scholar Y. Koumpouros, Information and Communication Technologies & Society , New Technologies, Athens, Greece, 2012. Matia Robotics, “Matia Robotics,” 2016, http://www.matiarobotics.com/ . MIT News, MIT-Manus Robot Aids Physical Therapy of Stroke Victims , 2016, http://news.mit.edu/2000/manus-0607 . Y. Koumpouros, Information and Communication Technologies in Healthcare , Hellenic Academic Libraries Link, Athens, Greece, 2015. C. van Campen, H. Sixma, R. D. Friele, J. J. Kerssens, and L. Peters, “Quality of care and patient satisfaction: a review of measuring instruments,” Medical Care Research and Review , vol. 52, no. 1, pp. 109–133, 1995. View at Publisher · View at Google Scholar · View at Scopus “Cohrane handbook for systematic reviews of interventions 4.2.6,” in The Cochrane Library , J. P. T. Higgins and S. Green, Eds., no. 4, John Wiley & Sons, Chichester, UK, 2006. A. S. Schroeder, R. Von Kries, C. Riedel et al., “Patient-specific determinants of responsiveness to robot-enhanced treadmill therapy in children and adolescents with cerebral palsy,” Developmental Medicine and Child Neurology , vol. 56, no. 12, pp. 1172–1179, 2014. View at Publisher · View at Google Scholar · View at Scopus The Rehabilitation Measures Database, Rehab Measures: Fugl-Meyer Assessment of Motor Recovery after Stroke , 2016, http://www.rehabmeasures.org/lists/rehabmeasures/dispform.aspx?ID=908 . B. C. Craven and A. R. Morris, “Modified ashworth scale reliability for measurement of lower extremity spasticity among patients with SCI,” Spinal Cord , vol. 48, no. 3, pp. 207–213, 2010. View at Publisher · View at Google Scholar · View at Scopus C. Collin, D. T. Wade, S. Davies, and V. Horne, “The Barthel ADL index: a reliability study,” International Disability Studies , vol. 10, no. 2, pp. 61–63, 1988. View at Publisher · View at Google Scholar · View at Scopus H. A. Abdullah, C. Tarry, C. Lambert, S. Barreca, and B. O. Allen, “Results of clinicians using a therapeutic robotic system in an inpatient stroke rehabilitation unit,” Journal of NeuroEngineering and Rehabilitation , vol. 8, no. 1, article 50, 2011. View at Publisher · View at Google Scholar · View at Scopus M. Bortole, A. Venkatakrishnan, F. Zhu et al., “The H2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study,” Journal of NeuroEngineering and Rehabilitation , vol. 12, no. 1, article 54, 2015. View at Publisher · View at Google Scholar · View at Scopus T. L. Chen, T. Bhattacharjee, J. L. McKay et al., “Evaluation by expert dancers of a robot that performs partnered stepping via haptic interaction,” PLoS ONE , vol. 10, no. 5, Article ID e0125179, 2015. View at Publisher · View at Google Scholar · View at Scopus T. L. Chen and C. C. Kemp, “Lead me by the hand,” in Proceeding of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI '10) , pp. 367–374, Osaka, Japan, March 2010. View at Publisher · View at Google Scholar · View at Scopus Y. S. Choi, C. D. Anderson, J. D. Glass, and C. C. Kemp, “Laser pointers and a touch screen: intuitive interfaces for autonomous mobile manipulation for the motor impaired,” in Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '08) , pp. 225–232, October 2008. View at Publisher · View at Google Scholar · View at Scopus P. Coignard, J. P. Departe, O. Remy Neris et al., “ANSO study: evaluation in an indoor environment of a mobile assistance robotic grasping arm,” Annals of Physical and Rehabilitation Medicine , vol. 56, no. 9-10, pp. 621–633, 2013. View at Publisher · View at Google Scholar · View at Scopus G. G. Grindle, H. Wang, H. Jeannis, E. Teodorski, and R. A. Cooper, “Design and user evaluation of a wheelchair mounted robotic assisted transfer device,” BioMed Research International , vol. 2015, Article ID 198476, 9 pages, 2015. View at Publisher · View at Google Scholar · View at Scopus M. Heerink, B. Kröse, V. Evers, and B. Wielinga, “Assessing acceptance of assistive social agent technology by older adults: the Almere model,” International Journal of Social Robotics , vol. 2, no. 4, pp. 361–375, 2010. View at Publisher · View at Google Scholar · View at Scopus M. J. Kim, D. H. Lee, T. Kim et al., “Lower extremity exercise of knee osteoarthritis patients using portable assistive robot (HEXAR-KR40P),” International Journal of Precision Engineering and Manufacturing , vol. 15, no. 12, pp. 2617–2622, 2014. View at Publisher · View at Google Scholar · View at Scopus P. N. Kooren, A. G. Dunning, M. M. H. P. Janssen et al., “Design and pilot validation of A-gear: a novel wearable dynamic arm support,” Journal of NeuroEngineering and Rehabilitation , vol. 12, no. 1, article 83, 2015. View at Publisher · View at Google Scholar · View at Scopus I. Laffont, N. Biard, G. Chalubert et al., “Evaluation of a graphic interface to control a robotic grasping arm: a multicenter study,” Archives of Physical Medicine and Rehabilitation , vol. 90, no. 10, pp. 1740–1748, 2009. View at Publisher · View at Google Scholar · View at Scopus P. Lam, D. Hebert, J. Boger et al., “A haptic-robotic platform for upper-limb reaching stroke therapy: preliminary design and evaluation results,” Journal of NeuroEngineering and Rehabilitation , vol. 5, no. 1, article 15, 2008. View at Publisher · View at Google Scholar · View at Scopus C. Liu, C. T. Ishi, and H. Ishiguro, “Bringing the Scene Back to the Tele-operator: auditory scene manipulation for tele-presence systems,” in Proceedings of the 10th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI '15) , pp. 279–286, Portland, Ore, USA, March 2015. View at Publisher · View at Google Scholar · View at Scopus L. Lopez-Samaniego, B. Garcia-Zapirain, and A. Mendez-Zorrilla, “Memory and accurate processing brain rehabilitation for the elderly: LEGO robot and iPad case study,” Bio-Medical Materials and Engineering , vol. 24, no. 6, pp. 3549–3556, 2014. View at Publisher · View at Google Scholar · View at Scopus S. Mazzoleni, G. Turchetti, I. Palla, F. Posteraro, and P. Dario, “Acceptability of robotic technology in neuro-rehabilitation: preliminary results on chronic stroke patients,” Computer Methods and Programs in Biomedicine , vol. 116, no. 2, pp. 116–122, 2014. View at Publisher · View at Google Scholar · View at Scopus K. Ozaki, H. Kagaya, S. Hirano et al., “Preliminary trial of postural strategy training using a personal transport assistance robot for patients with central nervous system disorder,” Archives of Physical Medicine and Rehabilitation , vol. 94, no. 1, pp. 59–66, 2013. View at Publisher · View at Google Scholar · View at Scopus J. Pineau, A. Atrash, R. Kaplow, and J. Villemure, “On the design and validation of an intelligent powered wheelchair: lessons from the SmartWheeler project,” Advances in Intelligent and Soft Computing , vol. 83, pp. 259–268, 2010. View at Publisher · View at Google Scholar · View at Scopus H. Wang, J. Xu, G. Grindle et al., “Performance evaluation of the personal mobility and manipulation appliance (PerMMA),” Medical Engineering & Physics , vol. 35, no. 11, pp. 1613–1619, 2013. View at Publisher · View at Google Scholar · View at Scopus Y.-H. Wu, J. Wrobel, M. Cornuet, H. Kerhervé, S. Damnée, and A.-S. Rrigaud, “Acceptance of an assistive robot in older adults: a mixed-method study of human-robot interaction over a 1-month period in the Living Lab setting,” Clinical Interventions in Aging , vol. 9, pp. 801–811, 2014. View at Publisher · View at Google Scholar · View at Scopus F. Werner and D. Krainer, “A socially assistive robot to support physical training of older people—an end user acceptance study,” in Proceedings of the 5th International Conference on Social Robotics (ICSR '13) , pp. 562–563, Springer, Bristol, UK, October 2013. I. Treger, S. Faran, and H. Ring, “Robot-assisted therapy for neuromuscular training of sub-acute stroke patients. A feasibility study,” European Journal of Physical and Rehabilitation Medicine , vol. 44, no. 4, pp. 431–435, 2008. View at Google Scholar · View at Scopus A.-M. Hughes, J. Burridge, C. T. Freeman et al., “Stroke participants' perceptions of robotic and electrical stimulation therapy: a new approach,” Disability and Rehabilitation: Assistive Technology , vol. 6, no. 2, pp. 130–138, 2011. View at Publisher · View at Google Scholar · View at Scopus H. Chihara, Y. Takagi, K. Nishino et al., “Factors predicting the effects of hybrid assistive limb robot suit during the acute phase of central nervous system injury,” Neurologia Medico-Chirurgica , vol. 56, no. 1, pp. 33–37, 2016. View at Publisher · View at Google Scholar A. M. von der Pütten, N. C. Krämer, and S. C. Eimler, “Living with a robot companion: empirical study on the interaction with an artificial health advisor,” in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '11) , pp. 327–334, November 2011. View at Publisher · View at Google Scholar · View at Scopus P. Sale, V. Lombardi, and M. Franceschini, “Hand robotics rehabilitation: feasibility and preliminary results of a robotic treatment in patients with hemiparesis,” Stroke Research and Treatment , vol. 2012, Article ID 820931, 5 pages, 2012. View at Publisher · View at Google Scholar · View at Scopus COPM, “Learn to Use the COPM,” 2013, http://www.thecopm.ca/learn/ . E. Ambrosini, S. Ferrante, M. Rossini et al., “Functional and usability assessment of a robotic exoskeleton arm to support activities of daily life,” Robotica , vol. 32, no. 8, pp. 1213–1224, 2014. View at Publisher · View at Google Scholar · View at Scopus F. Amirabdollahian, S. Ates, A. Basteris et al., “Design, development and deployment of a hand/wrist exoskeleton for home-based rehabilitation after stroke—SCRIPT project,” Robotica , vol. 32, no. 8, pp. 1331–1346, 2014. View at Publisher · View at Google Scholar · View at Scopus B. Mónica Faria, S. Vasconcelos, L. Paulo Reis, and N. Lau, “Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study,” Assistive Technology , vol. 25, no. 2, pp. 88–98, 2013. View at Publisher · View at Google Scholar · View at Scopus J. Brooke, “SUS: a “quick and dirty” usability scale,” in Usability Evaluation Industry , P. W. Jordan, B. Thomas, B. A. Weerdmeester, and A. I. McClelland, Eds., Taylor and Francis, London, UK, 1996. View at Google Scholar L. Chiari, R. Van Lummel, C. Becker, K. Pfeiffer, U. Lindemman, and W. Zijlstra, “Classification of the user's needs, characteristics and scenarios—update,” Report from the EU Project (6th Framework Program, IST Contract no. 045622) Sensing and Action to support mobility in Ambient Assisted Living, 2009. A. J. del-Ama, Á. Gil-Agudo, J. L. Pons, and J. C. Moreno, “Hybrid FES-robot cooperative control of ambulatory gait rehabilitation exoskeleton,” Journal of NeuroEngineering and Rehabilitation , vol. 11, no. 1, article 27, 2014. View at Publisher · View at Google Scholar · View at Scopus A. Jardón, Á. M. Gil, A. I. de la Peña, C. A. Monje, and C. Balaguer, “Usability assessment of ASIBOT: a portable robot to aid patients with spinal cord injury,” Disability and Rehabilitation: Assistive Technology , vol. 6, no. 4, pp. 320–330, 2011. View at Publisher · View at Google Scholar · View at Scopus L. Demers, R. Weiss-Lambrou, and B. Ska, Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST Version 2.0): An Outcome Measure for Assistive Technology Devices , Institute for Matching Person & Technology, Webster, NY, USA, 2000. E. M. Holz, J. Höhne, P. Staiger-Sälzer, M. Tangermann, and A. Kübler, “Brain-computer interface controlled gaming: evaluation of usability by severely motor restricted end-users,” Artificial Intelligence in Medicine , vol. 59, no. 2, pp. 111–120, 2013. View at Publisher · View at Google Scholar · View at Scopus L. Demers, M. Monette, Y. Lapierre, D. L. Arnold, and C. Wolfson, “Reliability, validity, and applicability of the quebec user evaluation of satisfaction with assistive technology (QUEST 2.0) for adults with multiple sclerosis,” Disability and Rehabilitation , vol. 24, no. 1–3, pp. 21–30, 2002. View at Publisher · View at Google Scholar · View at Scopus Y. Koumpouros, A. Karavasili, E. Papageorgiou, and P. Siavelis, “Validation of the greek version of the device subscale of the quebec user evaluation of satisfaction with assistive technology 2.0 (QUEST 2.0),” Assistive Technology , 2016. View at Publisher · View at Google Scholar A. Steinfeld, T. Fong, D. Kaber, et al., “Common metrics for human-robot interaction,” in Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI '06) , ACM Proceedings, pp. 33–40, 2006. View at Publisher · View at Google Scholar A. Weiss, R. Bernhaupt, M. Lankes, and M. Tscheligi, “The USUS evaluation framework for human-robot interaction,” in Proceedings of the Symposium on New Frontiers in Human-Robot Interaction (AISB '09) , pp. 158–165, Edinburgh, UK, April 2009. C. Bartneck, D. Kulić, E. Croft, and S. Zoghbi, “Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots,” International Journal of Social Robotics , vol. 1, no. 1, pp. 71–81, 2009. View at Publisher · View at Google Scholar · View at Scopus M. J. Scherer, Matching Person and Technology Process and Accompanying Assessment Instruments , Revised Edition, The Institute for Matching Person & Technology, Webster, NY, USA, 1998. Y. Koumpouros, E. Papageorgiou, A. Karavasili, and D. Alexopoulou, “Translation and validation of the assistive technology device predisposition assessment in Greek in order to assess satisfaction with use of the selected assistive device,” Disability and Rehabilitation: Assistive Technology , 2016. View at Publisher · View at Google Scholar P. Palmer, C. Thursfield, and S. Judge, “An evaluation of the psychosocial impact of assistive devices scale,” in Assistive Technology: From Virtuality to Reality , A. Pruski and H. Knops, Eds., pp. 740–744, IOS Press, Nieuwe Hemweg, The Netherlands, 1st edition, 2005. View at Google Scholar M. B. Michel-Verkerke and A. M. G. M. Hoogeboom, “Evaluation of the USE IT-questionnaire for the evaluation of the adoption of electronic patient records by healthcare professionals,” Methods of Information in Medicine , vol. 52, no. 3, pp. 189–198, 2013. View at Publisher · View at Google Scholar · View at Scopus Y. Koumpouros, E. Papageorgiou, and A. Karavasili, “Development of a new psychometric scale (PYTHEIA) to assess the satisfaction of users with any assistive technology,” in Proceedings of the 7th International Conference on Applied Human Factors and Ergonomics , IEEE, July 2016. (function (i, s, o, g, r, a, m) { i['GoogleAnalyticsObject'] = r; i[r] = i[r] || function () { (i[r].q = i[r].q || []).push(arguments) }, i[r].l = 1 * new Date(); a = s.createElement(o), m = s.getElementsByTagName(o)[0]; a.async = 1; a.src = g; m.parentNode.insertBefore(a, m) })(window, document, 'script', '//www.google-analytics.com/analytics.js', 'ga'); ga('create', 'UA-8578054-2', 'auto'); ga('send', 'pageview');

Journal

Journal of Healthcare Engineering – Hindawi Publishing Corporation

Published: May 4, 2016

There are no references for this article.