Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence

Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence Artificial intelligence (AI) education is becoming increasingly important worldwide. However, there has been no measuring instrument for diagnosing the students’ current perspective. Thus the aim of this study was to develop an instrument that measures student attitudes toward AI. The instrument was developed by verifying the reliability and validity by 8 computer education PhD using a sample of 305 K-12 students. This scale made students’ attitudes toward AI operational and quantifiable. Accordingly, educators can use it to diagnose the current status of students or verify the effectiveness of new AI education methods. Keywords evaluation methodologies, artificial intelligence in education, student attitude perception, and sentiment toward AI from previous studies Introduction that measured attitudes toward various forms of technology- Artificial intelligence (AI), according to Coppin (2004, p. 4), enhanced learning (Cheung & Vogel, 2013; Dunn & Kennedy, is the ability of machines to adapt to new situations, deal 2019) and attitudes toward science, technology, engineering, with emerging situations, solve problems, answer questions, and mathematics (STEM) education (Cherry et al., 2020; device plans, and perform various other functions that require Cukurova et al., 2020; Gaines-Ross, 2016; Gherheș & Obrad, some level of intelligence typically evident in human beings. 2018; Manikonda & Kambhampati, 2018), none of them spe- At present, AI is becoming a key variable in the fields of the cifically focused on attitude toward AI education. technology, economy, and politics. The Fourth Industrial If students do not develop a positive attitude toward pro- Revolution would most probably be determined by the rela- fessional skills learning, they are less likely to master it tionship between humans and AI (ETRI, 2019). As a result, regardless of the effectiveness of their education (Ajzen, AI education is gaining prominence, and many countries, 1991; Fredrickson, 2001). Thus, measuring attitudes toward including the United States, Singapore, China, Korea, AI can be an important factor in the success or failure of AI Australia, and those in the European Union, are exploring education. Our review of the literature showed that many of ways to effectively integrate AI education into their K-12 the studies on learners’ attitudes toward AI for learning either curriculum (Chiu, 2021; Heintz, 2021; J. Kim & Park, 2019; measured only computational thinking (K. Kim, 2019) or Steinbauer et al., 2021). used unvalidated modified instruments (K. Kim, 2019; Y. Scholars are, therefore, primarily concerned about what Lee, 2019) and self-developed surveys with open-ended needs to be taught in AI education (Ali et al., 2019; E.-K. Lee, 2020; Touretzky et al., 2019). For example, Yoo (2019) Sungkyunkwan University, Seoul, Korea divided AI-related education into 40 items and examined the importance and performance of each item for graduate stu- Woong Suh: co-first author and corresponding author dents. Yoo found that enhancing openness toward learning Seongjin Ahn: co-first author about AI was the primary prerequisite for developing other Corresponding Author: elements of AI education. However, knowledge on openness Woong Suh, Department of Computer Education, Sungkyunkwan to learn AI or attitudes toward learning AI are mostly public University, 50803, Hoam Hall, 25-2, Sungkyunkwan-Ro, Jongno-Gu, Seoul opinions (Ikkatai et al., 2022; Schepman & Rodway, 2020). 110-745, Korea. While we can gain some insights about motivation, Email: nanwoong@g.skku.edu Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 SAGE Open questions (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. As for education, these attitude-behavioral intentions may Shin et al., 2017, 2018; N. M. Shin & Kim, 2007). Most appear as learning intentions as confirmed by studies related importantly, of these studies have not used measurement to mathematics (Huang et al., 2016), science (Khine, 2015), instrument that have been developed to be reliable and vali- and engineering education (Alias et al., 2018). According to dated. The purpose of our study, therefore, is to develop a the TPB (Ajzen, 1991) students’ attitudes toward profes- standardized instrument that measures K-12 student attitudes sional skill learning play an important role in whether they toward AI. Specifically, we would like to identify questions actually acquire that skill, while at the same time positive that can be used to objectively measure learners’ attitudes attitudes toward learning positively influence students’ learn- toward AI. We would also like to further test the validity of ing intentions. Accordingly, it has been found that students’ categorizing student attitudes toward AI by the traditional positive attitudes can improve learning achievements (Alias approach that suggests attitude is made of three compo- et al., 2018; Cukurova et al., 2020) and help curriculum nents—cognitive, affective, and behavioral. Our study, we developers and teachers optimize lessons (Dunlap & Dugger, believe, is significant because it would be the first study that 1991; Yu et al., 2012). This is also related to the idea that presents K-12 perceptions on AI and brings AI education “people’s general attitudes toward AI are likely to play a into operational and quantifiable areas. large role in their acceptance of AI” (Schepman & Rodway, 2020). Therefore, to successfully implement AI education, there is a need to measure and understand students’ attitudes Theoretical Background toward AI. Attitude Attitude Measurements In modern times, attitude is described as a “psychological tendency, expressed by evaluating a particular entity with This study draws from the methodology used in previous some degree of favor or disfavor” (Eagly & Chaiken, 1993). studies to determine the reliability and validity of instru- The traditional approach defines attitude as consisting of ments used for measuring attitudes. References include atti- three complementary and not mutually exclusive compo- tudes toward technology (Ardies et al., 2013; Choi et al, nents (Bagozzi, 1978; Fishbein & Ajzen, 1972; Metsärinne 2009; C. S. Lee 2008; Svenningsson et al., 2018), mathemat- & Kallio, 2016): (1) the emotional component signifies the ics (Hannula et al., 2016), STEM (Benek & Akcay, 2019), positive-negative emotional relations or feelings one has engineering (Choi et al., 2009), and science education (Glynn toward an object or activity; (2) the behavioral component et al., 2009), as well as evaluations of the effects of software indicates the action tendencies one has to respond to an (SW) education on attitude (Park & Shin, 2017). Further, we object in a particular way; and (3) the cognitive component referred to the study by Schepman and Rodway (2020), marks the content of one’s thoughts, such as one’s beliefs where a scale was developed to measure the general attitudes regarding what constitutes a fact. toward AI. The following approaches are useful in investigating attitude as a determinant of the use of IT or other new tech- Materials and Methods nologies: the theory of reasoned action (TRA; Fishbein & Ajzen, 1975), the theory of planned behavior (TPB; Ajzen, Research Procedure 1991), and the technology acceptance model (TAM; Davis et al., 1989). TRA maintains that one’s attitude toward a First, to solve research problems, related literature and pre- particular behavior and the subjective norms that condition vious studies as shown in Tables 1 and 2 were examined, that behavior encourage or discourage performance of that and 52 preliminary questions for measuring attitudes behavior. TPB adds perceived behavioral control to the two toward AI were selected. Second, a 3-point Likert scale factors affecting intention in TRA. The TAM considers per- consisting of the following choices: “essential,” “useful but ceived usefulness, which is “the prospective user’s subjec- not essential,” and “not necessary” (Likert, 1932), was used tive probability that using a specific application system will to examine experts’ opinions about the validity of each increase his or her job performance within an organiza- item. Content validity refers to how well a survey or test tional context” (Davis et al., 1989), and perceived ease of measures the construct that it sets out to measure. The data use, which is “the degree to which the prospective user were examined by using a content validity ratio (CVR) test expects the target system to be free of effort” (Davis et al., (Lawshe, 1975). A total of 26 items that did not satisfy the 1989, p. 985); both affect attitude and behavioral intention. CVR value were removed. Third, the remaining 26 ques- Indeed, many studies have investigated how attitude pre- tions were gathered into a questionnaire that was adminis- dicts and affects behavior (Bohner & Dickel, 2011; Glasman tered to 305 upper-elementary, middle, and high school & Albarracín, 2006; Gorges et al., 2012; Petty et al., 2009; students in Seoul, Korea. Students were asked to respond Steinmetz et al., 2016). to statements using a 5-point Likert scale ranging from Suh and Ahn 3 Table 1. Previous Studies on Attitude Measurement (Attitude Scale). Question Measurement target Country Purpose Item Response option Example item References Student (all levels) Republic of Attitude toward 58 5-point Likert scale I want to know more about C. S. Lee (2008) Korea technology the computer. I will choose a technical career in the future. Undergraduate Republic of Attitude toward 39 5-point Likert scale Engineering is important Choi et al. student Korea engineering for understanding the (2009) direction of society's development. I enjoy reading books about engineering. Student (all levels) Republic of Evaluation of 40 7-point Likert scale I think SW is very important Park and Shin Korea effectiveness of for the development of (2017) SW education our society. I want to keep learning about SW in the future. Student (all levels) USA Attitude toward 25 5-point Likert scale Technology is very Ardies et al. technology important in life. (2013) There should be more education about technology. Undergraduate USA Motivation to 25 5-point Likert scale Learning science is Glynn et al. student learn science interesting. (2009) Knowing science will give me a career advantage. Learning science makes my life more meaningful. I believe I can master science knowledge and skills. Secondary school Turkey Attitude toward 33 5-point Likert scale I am curious about science. Benek and student STEM In the future, I would like to Akcay (2019) have a job in technology. Fifth and ninth Sweden Attitudes toward 14 Yes or No Could you talk to a robot? Serholt et al. grade students social robots Could you ask a robot (2014) for help with your schoolwork? Note. SW = software education. 1 (strongly disagree) to 5 (strongly agree). Lastly, explor- education. As a result of analyzing the sub-elements classi- atory factor analysis (EFA) and confirmatory factor analy- fied in these studies, it was possible to categorize them into sis (CFA) were conducted, and the final questions were nine categories such as learner’s career, experience of inter- selected. Descriptive statistics and distribution of total est in each subject, necessity of learning each subject in scores were prepared for student responses. The study pro- school, perceived usefulness (for the learner), perceived use- cedure is shown in Table 3. fulness (for the society), acceptance of each subject, career related to each subject, each subject for all genders, and per- ceived ease of use of each subject. These tests had a range of Preparation of Test Questions 14 to 58 questions with either true/false and 5- or 7-point Two categories of studies were referenced to develop the Likert scale responses. attitude scale for AI. First, studies related to attitude mea- Second, the category of studies that we referred to were all surement conducted in Korea, the United States, Turkey, and AI-based (Table 2). All these studies used open-ended ques- Sweden were considered (Table 1). The purpose of the stud- tions and were administered to students at different grade lev- ies was to measure attitudes toward SW, STEM, and science els to understand their perceptions about AI and robots. 4 SAGE Open Table 2. Measurement and Evaluation in Previous Studies of AI Education (AI Education Scale). Measurement target Purpose Question form Example item References th 7 grade student Perception of intelligent Open narrative How do you solve conflicts Y. E. Kim and Kim (2019) robots as companions between a robot and an individual? Elementary school Elementary school students’ Open narrative How do you feel when you S. Shin et al. (2018) student images of AI think of AI? High school student High school students’ Open narrative Job risk awareness and Shin et al. (2017) perceptions of AI (emotion ethical risk awareness. and risk perception) Student (all levels) Perceptions of artificial Park and Shin (2017) intelligence technology and Open narrative What is the perception of AI teachers AI technology? Can AI replace teachers? Student (all levels) Perceptions and attitudes Open narrative Image of the robot. N. M. Shin and Kim (2007) toward robots Learning about robots. Learning from robots. Learning with robots. Table 3. Research Procedure. Step Objectives Method Respondent Preparation of test question Develop instrument Specification of research questions - Related literature Review Pilot survey Confirm instrument Expert validity test 8 Professors of Computer Education First main survey Validated instrument Exploratory factor analysis 119 students Second main survey Confirmatory factor analysis 186 students Statistical analysis Summary of responses Statistical analysis 305 students A total of 52 items were set up through the process of requirements. Our final student questionnaire, therefore, referring to the range of these items, integrating common consisted of the remaining 26 items (listed in Appendix 1). items among the questions related to the research problem, and excluding unrelated items. Main Survey for Students In public schools in Korea, AI classes are not part of the Expert Validity Test regular curriculum. Therefore, students with experience in Experts who participated in the preliminary item validity SW education, most relevant to AI education, were sur- evaluation included professors of computer education and veyed. The survey was conducted twice from November doctors who majored in computer education. The panel had 2019 to December 2019, and in December 2021. Both sur- eight members, which is acceptable (Lynn, 1986). The CVR veys were conducted with the same tool, which is presented was verified through a 3-point Likert scale (Lawshe, 1975). in Appendix 2. Dataset 1 was provided by 119 students from The formula for calculating the CVR is as follows. grades 5 to 6, and Dataset 2 was provided by 186 students from grades 6 to 11 (listed in Table 4). All procedures per- N − formed in this study were in accordance with the ethical N : Number of fitted responses,   CVR = standards of the Institutional Review Board Sungkyunkwan   N:Total responses s   University (approval number SKKU2019-11-012). Consent from the relevant guardians was obtained. The CVR score for this panel should be 0.75, which means that more than seven out of eight respondents designate an Exploratory Factor Analysis item as “essential” (Lawshe, 1975). The CVR is mostly adopted to select or exclude individ- EFA is one of a family of multivariate statistical methods ual items testing its validity (Wilson et al., 2012). Among that attempts to identify the smallest number of hypothetical the initial 52 items, 26 items did not satisfy the CVR value constructs that can parsimoniously explain the covariation Suh and Ahn 5 Table 4. Demographics of Respondents in the Main Study. (χ = 2,645.682, df = 325, p = .000); common factors existed and were appropriate. Accordingly, three sub-factors were Grade Boys Girls Total extracted, and factor 1 was named “Behavioral,” factor 2 was Dataset 1 5th 28 35 119 named “Affective,” and factor 3 was named “Cognitive.” 6th 27 29 Cronbach’s alpha of all factors indicated excellent reliability Dataset 2 6th 42 39 186 (Tavakol & Dennick, 2011). The above results and the eigen- 7th 12 10 value and cumulative percentage for each factor for the 10th 16 0 development of the instrument that measures attitudes toward 11th 46 21 AI are shown in Table 5. Total 171 134 305 A CFA was performed using Dataset 2 to confirm the validity of the items. Validity is secured when the standard- ized λ value is .5 or higher, the AVE is .5 or higher, and the observed among a set of measured variables (Watkins, CR is .7 or higher. After performing a CFA on the sub-factors 2018). EFA was conducted on Dataset 1, the result of 119 for each component, standardized, and appropriate λ, AVE, students’ responses, and through this, the validity and rele- and CR values were all secured, as illustrated in Table 6, vance of sub-factors and items were reviewed. Prior to fac- indicating the model used in the study was valid. tor analysis, the Kaiser–Meyer–Olkin (KMO) test was Among the absolute fit indexes that can be recognized, it conducted, and the suitability of factor analysis was identi- is appropriate for the root mean square error of approxima- fied using Bartlett’s test. Subsequently, to understand the tion (RMSEA) to be between 0.05 and 0.1, and the standard- validity of the constituent concept of the 26 selected scale ized root mean square residual (SRMR) to be less than or questions, EFA was performed to extract potential factors equal to 0.5. Furthermore, among the incremental fit indexes, using the varimax method and principal component analy- both the Turker–Lewis index (TLI) and the comparative fit sis. Additionally, Cronbach’s alpha was used to check the index (CFI) are considered to be 0.9 or higher. In the model reliability of the instrument. adopted for this study, RMSEA was 0.075, SRMR was 0.047, TLI was 0.928, and CFI was 0.937. In other words, all the mentioned goodness of fit were satisfied. Confirmatory Factor Analysis To summarize, 3 factors and 26 questions were found to CFA is a type of structural equation modeling that deals spe- be the most appropriate. This measurement instrument was cifically with measurement models-that is, the relationships named “Student Attitude Toward Artificial Intelligence” between observed measures or indicators and latent variables (SATAI). The detailed questionnaires are the same as those of factors (Brown, 2015). In such CFA, validity is secured in Appendix 2. when the standardized λ value is .5 or higher, the average variance extracted (AVE) is .5 or higher, and the construct Discussion reliability (CR) is .7 or higher. Additionally, the absolute fit index, an index that absolutely evaluates the degree of con- SATAI was developed and validated after a literature review formity between the collected data and the research model, using two groups of respondents. In particular, the results was investigated. The incremental fit index, which compares of the EFA for Dataset 1 showed the SATAI scale’s factor the index indicating the accuracy of the structural equation structure. The SATAI scale consists of 26 items comprising model of the study with the model in which correlation 3 components (cognitive, affective, and behavioral factors), between variables was not set, was also investigated. and each question is measured using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The scores for these items can be summed (ranging from 26 Results to 130) to represent a student’s attitude toward AI: a higher In the expert validity test, a group of eight experts verified score indicates a positive attitude toward AI and the likeli- the content validity of 52 items constructed through litera- hood that a student will be an active participant of AI ture research. They evaluated the content validity according education. to a 3-point scale. Consequently, ambiguous expressions For example, descriptive statistics on the responses of a with the phrases “anyone” or “make everything good,” items total of 305 students who participated in this study are as about gender differences in attitude toward AI, and 26 nega- follows. The average of Cognitive Components consisting tive questions, such as AI is difficult, were rejected as they of four questions was 3.57 (SD = 0.986), the average of did not meet the CVR value. Thus, the final survey question- Affective Components consisting of 10 questions was 3.72 naire for students included the remaining 26 question items. (SD = 0.841), and the average of Behavioral Components The result of EFA using Dataset 1 showed a KMO value consisting of 12 questions was 3.16 (SD = 1.068). The mean of 0.937, indicating that the item selection was good. and standard deviation of each item can be confirmed in Additionally, Bartlett’s sphericity was found to be significant Table 7. 6 SAGE Open Table 5. Results of Exploratory Factor Analysis. Factor Item number Commonality F1 F2 F3 Cronbach’s α 16 0.775 0.837 0.266 0.067 .956 17 0.714 0.818 0.155 0.145 15 0.713 0.812 0.222 0.065 18 0.709 0.763 0.262 0.239 22 0.702 0.717 0.242 0.36 23 0.685 0.684 0.284 0.369 21 0.584 0.672 0.204 0.301 20 0.714 0.651 0.358 0.402 26 0.582 0.625 0.241 0.366 24 0.699 0.613 0.402 0.402 19 0.609 0.607 0.398 0.286 25 0.627 0.583 0.195 0.5 7 0.720 0.264 0.804 0.056 .924 10 0.666 0.234 0.76 0.183 6 0.622 0.078 0.702 0.351 8 0.573 0.304 0.65 0.24 13 0.657 0.449 0.642 0.211 9 0.514 0.257 0.637 0.205 11 0.538 0.164 0.604 0.383 5 0.557 0.192 0.588 0.417 14 0.529 0.431 0.559 0.175 12 0.451 0.249 0.527 0.334 1 0.760 0.287 0.283 0.773 .905 2 0.684 0.236 0.307 0.731 3 0.742 0.296 0.36 0.725 4 0.638 0.313 0.288 0.676 Factor domain Behavioral Affective Cognitive Eigenvalue 13.788 2.422 1.563 Percentage of variance 53.031 9.315 6.013 Cumulative percentage 53.031 62.347 68.359 KMO 0.937 Bartlett’s test Approx. χ 2,645.682 df 325 p-Value .000 Note. Extraction method = principal component analysis; rotation method = varimax with Kaiser normalization. Bold text indicates the same category. Figure 1 shows the distribution of the total score of the intelligence have become measurable areas. In other words, questions answered by 305 people. According to this, the the distribution scores obtained through the SATAI can be total distribution of students tends to gradually increase to used to operatively diagnose and compare attitude changes 1.3% at 26 to 40, 6% at 41 to 55, and 30% at 56 to 70, and in students before and after integrating AI into education. then slightly decreases afterwards. Table 7 and Figure 1 In previous studies, open-ended questions of subjective make it possible to confirm the characteristics of AI of the criteria were used to verify the effectiveness of AI education 305 students who responded in this study. That is, “the aver- (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. Shin et al., age of these students” attitude toward AI considering the 2017, 2018; N. M. Shin & Kim, 2007). Other studies used Affective component was the highest at 3.72, with the instruments to measure attitudes toward other subjects, with- Cognitive and the Behavioral Components at 3.57 and 3.16, out validating them (K. Kim, 2019; Y. Lee, 2019). The respectively, and “The distribution of total scores is the high- SATAI can be differentiated from conventional research est in the middle” can be seen. Through these examples, we because it was developed specifically for AI education, thus confirmed that the students’ attitudes toward artificial enabling teachers to quantify students’ attitudes toward AI. Suh and Ahn 7 Table 6. Results of Confirmatory Factor Analysis. Factor SE CR Standardized coefficient AVE Construct reliability Beh._1 <- Behavioral 1 .856 .582630194 .943552573 Beh._2 <- Behavioral 0.923 14.121 .809 Beh._3 <- Behavioral 0.938 15.316 .847 Beh._4 <- Behavioral 0.836 15.199 .843 Beh._5 <- Behavioral 0.857 15.469 .852 Beh._6 <- Behavioral 0.835 16.434 .879 Beh._7 <- Behavioral 0.818 14.041 .806 Beh._8 <- Behavioral 0.836 15.332 .847 Beh._9 <- Behavioral 0.847 15.295 .846 Beh._10 <- Behavioral 0.737 14.297 .815 Beh._11 <- Behavioral 0.83 14.618 .825 Beh._12 <- Behavioral 0.704 11.777 .721 Aff._1 <- Affective 1 .828 .539872315 .921073786 Aff._2 <- Affective 0.899 12.586 .784 Aff._3 <- Affective 0.979 11.72 .746 Aff._4 <- Affective 1.091 13.352 .815 Aff._5 <- Affective 1.062 13.299 .813 Aff._6 <- Affective 0.846 11.596 .74 Aff._7 <- Affective 0.962 10.793 .703 Aff._8 <- Affective 0.774 9.752 .65 Aff._9 <- Affective 0.998 12.878 .796 Aff._10 <- Affective 0.96 10.888 .707 Cog._1 <- Cognitive 1 .907 .709802673 .907188241 Cog._2 <- Cognitive 0.913 17.098 .857 Cog._3 <- Cognitive 0.96 17.92 .875 Cog._4 <- Cognitive 0.98 16.089 .834 Note. SE = standard error; CR = critical ratio; AVE = average variance extracted. Table 7. Descriptive Statistics of the 305 Student Responses. 30% Factor M SD Factor M SD 25% Cog._1 3.60 1.056 Aff._10 3.70 1.145 20% Cog._2 3.65 1.054 Beh._1 2.80 1.373 15% Cog._3 3.66 1.082 Beh._2 2.69 1.349 10% Cog._4 3.37 1.213 Beh._3 2.92 1.374 5% Aff._1 3.94 1.034 Beh._4 3.26 1.258 Aff._2 4.04 0.976 Beh._5 3.36 1.244 0% 26-40 41-55 56-70 71-8586-100 101-115 116-130 Aff._3 3.65 1.163 Beh._6 3.32 1.211 Aff._4 3.58 1.139 Beh._7 3.13 1.245 Aff._5 3.65 1.090 Beh._8 3.19 1.247 Figure 1. Distribution of total scores. Aff._6 3.93 1.025 Beh._9 3.35 1.274 Note. x-axis = total score; y-axis = the percentage of the total number of respondents. Aff._7 3.49 1.159 Beh._10 3.60 1.188 Aff._8 3.56 0.945 Beh._11 3.16 1.261 Aff._9 3.71 1.084 Beh._12 3.17 1.272 encountering a new subject. Thus, measuring their attitudes can provide instructors with ideas for curriculum design that encourage students to have a positive attitude toward AI Conclusions learning (Ajzen, 1991; Khine, 2015). As for education through AI, AI appeared as a new technology to learners, This study is relevant in that it develops a scale that mea- which reinforced the importance of attitude through the sures student attitudes toward AI and tests the validity of TAM. Hence, in our study, a standardized instrument for that scale. First, the importance of attitudes toward AI edu- measuring a learner’s attitude toward AI was developed cation was confirmed through a literature review. Literature with attention to students’ cognitive, affective, and revealed that learners of AI education felt that they were 8 SAGE Open behavioral attitudes. On the one hand, AI education is new, This study has a few limitations. First, the measure devel- so there are no traditional curriculum or teaching methods oped can be used to verify the students’ attitudes toward AI, (E.-K. Lee, 2020; Touretzky et al., 2019). On the other hand, but cannot measure conceptualizations of AI. Therefore, SATAI can help teachers objectively measure students’ atti- future researchers should develop a scale that measures a tude toward AI. Moreover, instructors can use the test scores learner’s concept of AI. Second, although the results of CFA to design, modify, apply, and customize educational pro- on this dataset are strong evidence that the tool does not grams to meet learner needs. These advancements contrib- leverage age-related differences, there may be limitations in ute to the emerging philosophical cornerstones of AI the generalization of the study by surveying 305 elementary, education. SATAI results, we believe, can be applied to middle, and high school students. Future studies should verify the effects of AI-related education for other K-12 stu- include a wider range of participants, including individuals dents outside the population used in this study. in other regions. Appendix 1. Complete List of All Questions. Result of expert Item Domain CVR validity test It is fun to learn about AI. Behavioral Interest in AI 0.875 Confirmed components It is interesting to use AI. 1 Confirmed I want to continue learning about AI. 1 Confirmed I like to use something related to AI. 0.875 Confirmed I am interested in AI-related TV programs or 0.875 Confirmed Internet videos. I will participate in a club relating to AI (if one 0.875 Confirmed exists). I think I can handle AI well. 0.875 Confirmed I am interested in the development of AI. 1 Confirmed I will choose a job in the field of AI. Learner’s career 0.875 Confirmed I want to work in the field of AI. 0.875 Confirmed I want to make something that makes human life 0.875 Confirmed more convenient through AI. I think that there should be more class time Necessity of 0.875 Confirmed devoted to AI in school. learning AI in school I think it is important content to learn about AI in Cognitive 1 Confirmed school. components I think that AI should be taught in school. 1 Confirmed I think every student should learn about AI in 1 Confirmed school. AI class is important. 1 Confirmed AI is related to my life. Affective Perceived 0.875 Confirmed components usefulness (for I think AI makes life more convenient. 0.875 Confirmed the learner) I will use AI to solve problems in daily life. 1 Confirmed AI helps me solve problems in real life. 0.875 Confirmed I will need AI in my life in the future. 0.875 Confirmed AI is worth studying. 1 Confirmed AI is necessary for everyone. Perceived 0.875 Confirmed usefulness (for AI produces more good than bad. 0.875 Confirmed society) AI is very important for developing society. 0.875 Confirmed I think that most jobs in the future will require 0.875 Confirmed knowledge related to AI. AI brightens the future of our country. 0.625 Removed AI could make everything better. 0.625 Removed I could ask AI to complete housework. Behavioral Acceptance of AI 0.625 Removed components I can ride in a car driven by AI. 0.500 Removed I will get help from AI when I have a problem. 0.625 Removed I will receive surgery from an AI doctor. 0.500 Removed I could accept AI as a teacher. 0.375 Removed I can be a good friend with AI. 0.500 Removed (continued) Suh and Ahn 9 Appendix 1. (continued) Result of expert Item Domain CVR validity test Anyone can get a job related to AI. Affective Career related 0.625 Removed components to AI If you have a job related to AI, then your future will 0.625 Removed be bright. I think it will be interesting to work in the AI field. 0.625 Removed Men can do AI-related things better than women. Affective AI for all genders 0.125 Removed components Women can do AI-related things better than men. 0.125 Removed Men know AI better than women. 0.125 Removed Women know AI better than men. 0.125 Removed Men are more suited to AI work than women. 0.125 Removed Women are more suited to AI work than men. 0.125 Removed AI is more difficult for women than men. 0.125 Removed AI is more difficult for men than women. 0.125 Removed Men handle AI better than women. 0.125 Removed Women handle AI better than men. 0.125 Removed You must be smart to learn about AI. Affective Perceived ease of 0.500 Removed components use (reversed) AI is for smart people. 0.375 Removed Studying AI requires talent. 0.250 Removed It is difficult to learn AI. 0.625 Removed Not everyone can learn about AI. 0.500 Removed Appendix 2. Student Attitudes Toward AI (SATAI). Domain No. Item 5 4 3 2 1 Cognitive 1 I think that it is important to learn about AI in school. components 2 AI class is important. 3 I think that lessons about AI should be taught in school. 4 I think every student should learn about AI in school. Affective 5 AI is very important for developing society. components 6 I think AI makes people’s lives more convenient. 7 AI is related to my life. 8 I will use AI to solve problems in daily life. 9 AI helps me solve problems in real life. 10 I will need AI in my life in the future. 11 AI is necessary for everyone. 12 AI produces more good than bad. 13 AI is worth studying. 14 I think that most jobs in the future will require knowledge related to AI. Behavioral 15 I want to work in the field of AI. components 16 I will choose a job in the field of AI. 17 I would participate in a club related to AI if there was one. 18 I like using objects related to AI. 19 It is fun to learn about AI. 20 I want to continue learning about AI. 21 I’m interested in AI-related TV programs or online videos. 22 I want to make something that makes human life more convenient through AI. 23 I am interested in the development of AI. 24 It is interesting to use AI. 25 I think that there should be more class time devoted to AI in school. 26 I think I can handle AI well. Note. The scale uses 5 points for strongly agree, 4 points for agree, 3 points for neutral, 2 points for disagree, and 1 point for strongly disagree. 10 SAGE Open Acknowledgments students. International Journal of Engineering Education, 34(1), 226–235. We would like to thank Editage (www.editage.co.kr) for English Ardies, J., De Maeyer, S., & Gijbels, D. (2013). Reconstructing language editing. the pupils’ attitude towards technology-survey. Design and Technology Education, 18(1), 8–19. Author Contributions Bagozzi, R. P. (1978). The construct validity of the affective, All authors contributed to the study conception and design. Material behavioral, and cognitive components of attitude by analysis preparation, data collection, and analysis were performed by Woong of covariance structures. Multivariate Behavioral Research, Suh. The first draft of the manuscript was written by Seongjin Ahn 13(1), 9–31. https://doi.org/10.1207/s15327906mbr1301_2 and all authors commented on previous versions of the manuscript. Benek, I., & Akcay, B. (2019). Development of STEM attitude scale All authors read and approved the final manuscript. Woong Suh and for secondary school students: Validity and reliability study. Seongjin Ahn are co–first authors. International Journal of Education in Mathematics. International Journal of Education in Mathematics, Science and Technology, 7(1), 32–52. https://doi.org/10.18404/ijemst.509258 Availability of Data and Materials Bohner, G., & Dickel, N. (2011). Attitudes and attitude change. The data that support the findings of this study are available from Annual Review of Psychology, 62, 391–417. https://doi. Sungkyunkwan University, but restrictions apply to the availability org/10.1146/annurev.psych.121208.131609 of these data, which were used under license for the current study, Brown, T. A. (2015), Confirmatory factor analysis for applied and so are not publicly available. Data are however available from research. Guilford Publications. the authors upon reasonable request and with permission of Cheung, R., & Vogel, D. (2013). Predicting user acceptance of Sungkyunkwan University. collaborative technologies: An extension of the technology acceptance model for e-learning. Computers and Education, Ethical Approval 63, 160–175. https://doi.org/10.1016/j.compedu.2012.12.003 All procedures performed in this study were in accordance with Cherry, S., Rohit, S., Ashik, A., Keerthini, M., Aishah, A., Monzon, the ethical standards of the Institutional Review Board (IRB) at L., & Poon, D. S. (2020). Attitudes and perceptions of UK Sungkyunkwan University (approval number SKKU2019-11- medical students towards artificial intelligence and radiology: 012-003). A multicentre survey. Insights into Imaging, 11(1), 14. https:// doi.org/10.1186/s13244-019-0830-7 Chiu, T. K. F. (2021). A holistic approach to the design of artifi- Informed Consent cial intelligence (AI) education for K-12 schools. TechTrends, Written informed consent was obtained from all the guardians of 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1 students who participated in this study. Choi, H.-Y., Park, H.-M., Lee, J.-G., & Ryu, S.-M. (2009). Development of the instrument for assessment the attitude Declaration of Conflicting Interests toward engineering. Journal of Korean Institute of Industrial Education, 34(2), 161–178. The author(s) declared no potential conflicts of interest with respect Coppin, B. (2004), Artificial intelligence illuminated. Jones & to the research, authorship, and/or publication of this article. Bartlett Publishers Learning. Cukurova, M., Luckin, R., & Kent, C. (2020). Impact of an arti- Funding ficial intelligence research frame on the perceived credibility The author(s) received no financial support for the research, author- of educational research evidence. International Journal of ship, and/or publication of this article. Artificial Intelligence in Education, 30(2), 205–235. https:// doi.org/10.1007/s40593-019-00188-w ORCID iD Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User accep- tance of computer technology: A comparison of two theoreti- Woong Suh https://orcid.org/0000-0003-3696-8198 cal models. Management Science, 35(8), 982–1003. https://doi. org/10.1287/mnsc.35.8.982 References Dunlap, D. D., & Dugger, W. E. (1991). Comparing attitudes Ajzen, I. (1991). The theory of planned behavior. Organizational towards technology of third and fourth grade students in Behavior and Human Decision Processes, 50(2), 179–211. Virginia relative to their exposure to technology. PATT 5 https://doi.org/10.1016/0749-5978(91)90020-T Conference Proceedings (Vol. 2, pp. 194–201), Eindhoven. Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. Dunn, T. J., & Kennedy, M. (2019). Technology enhanced learning (2019). Constructionism, ethics, and creativity: Developing in higher education; motivations, engagement and academic primary and middle school artificial intelligence education. achievement. Computers and Education, 137, 104–113. https:// Proceedings of the IJCAI 2019. http://robotic.media.mit. doi.org/10.1016/j.compedu.2019.04.004 edu/wp-content/uploads/sites/7/2019/08/Constructionism__ Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. Ethics__and_Creativity.pdf Harcourt Publishers Brace Jovanovich College Publishers. Alias, M., Lashari, T. A., Akasah, Z. A., & Kesot, M. J. (2018). ETRI. (2019). Seven AI trends by 2020-beyond perception Self-efficacy, attitude, student engagement: Emphasising (ETRI Insight Report). https://ksp.etri.re.kr/ksp/plan-report/ the role of affective learning attributes among engineering doi?id=766 Suh and Ahn 11 Fishbein, M., & Ajzen, I. (1972). Attitudes and opinions. Annual artificial intelligence learning in elementary courses. Journal Review of Psychology, 23(1), 487–544. https://doi.org/10.1146/ of the Korean Association of Information Education, 23(3), annurev.ps.23.020172.002415 229–235. https://doi.org/10.14352/jkaie.2019.23.3.229 Fishbein, M., & Ajzen, I. (Eds.). (1975). Belief, attitude, inten- Kim, K. (2019). An artificial intelligence education program devel- tion, and behavior: An introduction to theory and research. opment and application for elementary teachers. Journal of the Addison-Wesley. Korean Association of Information Education, 23(6), 629–637. Fredrickson, B. L. (2001). The role of positive emotions in posi- https://doi.org/10.14352/jkaie.2019.23.6.629 tive psychology. The broaden-and-build theory of positive Kim, Y.-E., & Kim, H.-C. (2019). A study on middle school stu- emotions. American Psychologist, 56(3), 218–226. https://doi. dents’ perception on intelligent robots as companions. Journal org/10.1037/0003-066X.56.3.218 of Korean Association of Computer Education, 22(4), 35–45. Gaines-Ross, L. (2016, October 24). What do people—not techies, https://doi.org/10.32431/kace.2019.22.4.004 not companies—think about artificial intelligence? Harvard Lawshe, C. H. (1975). A quantitative approach to content valid- Business Review. https://hbr.org/2016/10/what-do-people-not- ity. Personnel Psychology, 28(4), 563–575. https://doi. techies-not-companies-think-about-artificial-intelligence org/10.1111/j.1744-6570.1975.tb01393.x Gherheș, V., & Obrad, C. (2018). Technical and humanities stu- Lee, C.-S. (2008). Development of technology attitude scale for dents’ perspectives on the development and sustainability of Korean pupils. Journal of Korean Practical Arts Education, artificial intelligence (AI). Sustainability, 10(9), 3066. https:// 14(2), 157–174. https://doi.org/10.17055/jpaer.2008.14.2.157 doi.org/10.3390/su10093066 Lee, E.-K. (2020). Analysis of artificial intelligence curriculum Glasman, L. R., & Albarracín, D. (2006). Forming attitudes that of domestic and foreign elementary and secondary school. predict future behavior: A meta-analysis of the attitude-behav- Journal of Computers in Education, 23(1), 37–44. ior relation. Psychological Bulletin, 132(5), 778–822. https:// Lee, Y. (2019). An analysis of the influence of block-type pro- doi.org/10.1037/0033-2909.132.5.778 gramming language-based artificial intelligence education on Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science the learner’s attitude in artificial intelligence. Journal of the motivation questionnaire: Construct validation with nonscience Korean Association of Information Education, 23(2), 189–196. majors. Journal of Research in Science Teaching, 46(2), 127– https://doi.org/10.14352/jkaie.2019.23.2.189 146. https://doi.org/10.1002/tea.20267 Likert, R. (1932). A technique for the measurement of attitudes. Gorges, J., Kandler, C., & Bohner, G. (2012). Internationalization at Archives of Psychology, 22, 5–55. home: Using learning motivation to predict students’ attitudes Lynn, M. R. (1986). Determination and quantification of con- toward teaching in a foreign language. International Journal of tent validity. Nursing Research, 35(6), 382–385. https://doi. Educational Research, 53, 107–116. https://doi.org/10.1016/j. org/10.1097/00006199-198611000-00017 ijer.2012.03.001 Manikonda, L., & Kambhampati, S. (2018). Tweeting AI: Hannula, M. S., Di Martino, P., Pantziara, M., Zhang, Q., Morselli, Perceptions of lay versus expert Twitterati [Conference F., Heyd-Metzuyanim, E., Lutovac, S., Kaasila, R., Middleton, paper]. Twelfth International AAAI Conference on Web and J. A., Jansen, A., & Goldin, G. A. (2016). Attitudes, Beliefs, Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ Motivation and Identity in Mathematics Education: An ICWSM18/paper/viewPaper/17843 Overview of the Field and Future Directions. (ICME-13 Metsärinne, M., & Kallio, M. (2016). How are students’ atti- Topical Surveys). Springer International Publishing tudes related to learning outcomes? International Journal of Hannula, M. S., Di Martino, P., Pantziara, M., Zhang, Q., Morselli, Technology and Design Education, 26(3), 353–371. https://doi. F., Heyd-Metzuyanim, E., Lutovac, S., Kaasila, R., Middleton, org/10.1007/s10798-015-9317-0 J.A., Jansen, A., & Goldin, G. A. (2016). Attitudes, beliefs, Park, J. H., & Shin, N. M. (2017). Students’ perceptions of artifi- motivation, and identity in mathematics education: An over- cial intelligence technology and artificial intelligence teachers. view of the field and future directions (ICME-13 Topical Journal of Korean Teacher Education, 34(2), 169–192. Surveys) (pp. 1–35). Springer. Petty, R. E., Fazio, R. H., & Brinol, P. (Eds.). (2009), Attitudes: Heintz, F. (2021). Three interviews about K-12 AI education in Insights from the new implicit measures. Psychology Press. America, Europe, and Singapore. KI—Künstliche Intelligenz, Schepman, A., & Rodway, P. (2020). Initial validation of the general 35(2), 233–237. https://doi.org/10.1007/s13218-021-00730-w attitudes towards Artificial Intelligence Scale. Computers in Huang, N. N., Chiu, L. J., & Hong, J. C. (2016). Relationship among Human Behavior Reports, 1, 100014. https://doi.org/10.1016/j. students’ problem-solving attitude, perceived value, behavioral chbr.2020.100014 attitude, and intention to participate in a science and technol- Serholt, S., Barendregt, W., Leite, I., Hastie, H., Jones, A., Paiva, ogy contest. International Journal of Science and Mathematics A., Vasalou, A., & Castellano, G. (2014, August 25–29). Education, 14(8), 1419–1435. https://doi.org/10.1007/s10763- Teachers’ views on the use of empathic robotic tutors in the 015-9665-y classroom [Conference paper]. The 23rd IEEE International Ikkatai, Y., Hartwig, T., Takanashi, N., & Yokoyama, H. M. Symposium on Robot and Human Interactive Communication. (2022). Octagon measurement: Public attitudes toward AI eth- https://ieeexplore.ieee.org/document/6926376 ics. International Journal of Human–Computer Interaction, Shin, N. M., & Kim, S. A. (2007). What do robots have to do 1–18. https://doi.org/10.1080/10447318.2021.2009669 with student learning? Journal of Korean Association for Khine, M. S. (Ed.). (2015). Attitude measurements in science edu- Educational Information and Media, 13(3), 79–99. cation: Classic and contemporary approaches. IAP. Shin, S., Ha, M., & Lee, J.-K. (2017). High school students’ percep- Kim, J., & Park, N. (2019). Development of a board game-based tion of artificial intelligence: Focusing on conceptual under- gamification learning model for training on the principles of standing, emotion and risk perception. Korean Association for 12 SAGE Open Learner-Centered Curriculum and Instruction, 17(21), 289– Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. 312. https://doi.org/10.22251/jlcci.2017.17.21.289 (2019). Envisioning AI for K-12: What should every child Shin, S., Ha, M., & Lee, J.-K. (2018). Exploring elementary school know about AI? Proceedings of the AAAI Conference on students’ image of artificial intelligence. Journal of Elementary Artificial Intelligence, Proceedings of the AAAI Conference on Science Education, 37(2), 126–146. https://doi.org/10.15267/ Artificial Intelligence, 33, 9795–9799. https://doi.org/10.1609/ keses.2018.37.2.126 aaai.v33i01.33019795 Steinbauer, G., Kandlhofer, M., Chklovski, T., Heintz, F., & Watkins, M. W. (2018). Exploratory factor analysis: A guide to best Koenig, S. (2021). A differentiated discussion about AI educa- practice. Journal of Black Psychology, 44(3), 219–246. https:// tion K-12. Kunstliche Intelligenz, 35(2), 131–137. https://doi. doi.org/10.1177/0095798418771807 org/10.1007/s13218-021-00724-8 Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalcu- Steinmetz, H., Knappstein, M., Ajzen, I., Schmidt, P., & Kabst, R. lation of the critical values for Lawshe’s content valid- (2016). How effective are behavior change interventions based ity ratio. Measurement and Evaluation in Counseling and on the theory of planned behavior? Zeitschrift für Psychologie, Development, 45(3), 197–210. https://doi.org/10.1177/ 224(3), 216–233. https://doi.org/10.1027/2151-2604/a000255 0748175612440286 Svenningsson, J., Hultén, M., & Hallström, J. (2018). Understanding Yoo, J. (2019). A study on AI education in graduate school through attitude measurement: Exploring meaning and use of the PATT IPA. Journal of the Korean Association of Information short questionnaire. International Journal of Technology and Education, 23(6), 675–687. https://doi.org/10.14352/jkaie. Design Education, 28(1), 67–83. https://doi.org/10.1007/ 2019.23.6.675 s10798-016-9392-x Yu, K.-C., Lin, K.-Y., Han, F.-N., & Hsu, I. Y. (2012). A model Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s of junior high school students’ attitudes toward technology. alpha. International Journal of Medical Education, 2, 53–55. International Journal of Technology and Design Education, https://doi.org/10.5116/ijme.4dfb.8dfd 22(4), 423–436. https://doi.org/10.1007/s10798-011-9154-8 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png SAGE Open SAGE

Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence

Suh, Woong; Ahn, Seongjin
SAGE Open , Volume OnlineFirst: 1 – May 24, 2022
Download PDF
12 pages

Loading next page...
 
/lp/sage/development-and-validation-of-a-scale-measuring-student-attitudes-L0Me0kaRDB

References (100)

Publisher
SAGE
Copyright
© The Author(s) 2022
ISSN
2158-2440
eISSN
2158-2440
DOI
10.1177/21582440221100463
Publisher site
See Article on Publisher Site

Abstract

Artificial intelligence (AI) education is becoming increasingly important worldwide. However, there has been no measuring instrument for diagnosing the students’ current perspective. Thus the aim of this study was to develop an instrument that measures student attitudes toward AI. The instrument was developed by verifying the reliability and validity by 8 computer education PhD using a sample of 305 K-12 students. This scale made students’ attitudes toward AI operational and quantifiable. Accordingly, educators can use it to diagnose the current status of students or verify the effectiveness of new AI education methods. Keywords evaluation methodologies, artificial intelligence in education, student attitude perception, and sentiment toward AI from previous studies Introduction that measured attitudes toward various forms of technology- Artificial intelligence (AI), according to Coppin (2004, p. 4), enhanced learning (Cheung & Vogel, 2013; Dunn & Kennedy, is the ability of machines to adapt to new situations, deal 2019) and attitudes toward science, technology, engineering, with emerging situations, solve problems, answer questions, and mathematics (STEM) education (Cherry et al., 2020; device plans, and perform various other functions that require Cukurova et al., 2020; Gaines-Ross, 2016; Gherheș & Obrad, some level of intelligence typically evident in human beings. 2018; Manikonda & Kambhampati, 2018), none of them spe- At present, AI is becoming a key variable in the fields of the cifically focused on attitude toward AI education. technology, economy, and politics. The Fourth Industrial If students do not develop a positive attitude toward pro- Revolution would most probably be determined by the rela- fessional skills learning, they are less likely to master it tionship between humans and AI (ETRI, 2019). As a result, regardless of the effectiveness of their education (Ajzen, AI education is gaining prominence, and many countries, 1991; Fredrickson, 2001). Thus, measuring attitudes toward including the United States, Singapore, China, Korea, AI can be an important factor in the success or failure of AI Australia, and those in the European Union, are exploring education. Our review of the literature showed that many of ways to effectively integrate AI education into their K-12 the studies on learners’ attitudes toward AI for learning either curriculum (Chiu, 2021; Heintz, 2021; J. Kim & Park, 2019; measured only computational thinking (K. Kim, 2019) or Steinbauer et al., 2021). used unvalidated modified instruments (K. Kim, 2019; Y. Scholars are, therefore, primarily concerned about what Lee, 2019) and self-developed surveys with open-ended needs to be taught in AI education (Ali et al., 2019; E.-K. Lee, 2020; Touretzky et al., 2019). For example, Yoo (2019) Sungkyunkwan University, Seoul, Korea divided AI-related education into 40 items and examined the importance and performance of each item for graduate stu- Woong Suh: co-first author and corresponding author dents. Yoo found that enhancing openness toward learning Seongjin Ahn: co-first author about AI was the primary prerequisite for developing other Corresponding Author: elements of AI education. However, knowledge on openness Woong Suh, Department of Computer Education, Sungkyunkwan to learn AI or attitudes toward learning AI are mostly public University, 50803, Hoam Hall, 25-2, Sungkyunkwan-Ro, Jongno-Gu, Seoul opinions (Ikkatai et al., 2022; Schepman & Rodway, 2020). 110-745, Korea. While we can gain some insights about motivation, Email: nanwoong@g.skku.edu Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 SAGE Open questions (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. As for education, these attitude-behavioral intentions may Shin et al., 2017, 2018; N. M. Shin & Kim, 2007). Most appear as learning intentions as confirmed by studies related importantly, of these studies have not used measurement to mathematics (Huang et al., 2016), science (Khine, 2015), instrument that have been developed to be reliable and vali- and engineering education (Alias et al., 2018). According to dated. The purpose of our study, therefore, is to develop a the TPB (Ajzen, 1991) students’ attitudes toward profes- standardized instrument that measures K-12 student attitudes sional skill learning play an important role in whether they toward AI. Specifically, we would like to identify questions actually acquire that skill, while at the same time positive that can be used to objectively measure learners’ attitudes attitudes toward learning positively influence students’ learn- toward AI. We would also like to further test the validity of ing intentions. Accordingly, it has been found that students’ categorizing student attitudes toward AI by the traditional positive attitudes can improve learning achievements (Alias approach that suggests attitude is made of three compo- et al., 2018; Cukurova et al., 2020) and help curriculum nents—cognitive, affective, and behavioral. Our study, we developers and teachers optimize lessons (Dunlap & Dugger, believe, is significant because it would be the first study that 1991; Yu et al., 2012). This is also related to the idea that presents K-12 perceptions on AI and brings AI education “people’s general attitudes toward AI are likely to play a into operational and quantifiable areas. large role in their acceptance of AI” (Schepman & Rodway, 2020). Therefore, to successfully implement AI education, there is a need to measure and understand students’ attitudes Theoretical Background toward AI. Attitude Attitude Measurements In modern times, attitude is described as a “psychological tendency, expressed by evaluating a particular entity with This study draws from the methodology used in previous some degree of favor or disfavor” (Eagly & Chaiken, 1993). studies to determine the reliability and validity of instru- The traditional approach defines attitude as consisting of ments used for measuring attitudes. References include atti- three complementary and not mutually exclusive compo- tudes toward technology (Ardies et al., 2013; Choi et al, nents (Bagozzi, 1978; Fishbein & Ajzen, 1972; Metsärinne 2009; C. S. Lee 2008; Svenningsson et al., 2018), mathemat- & Kallio, 2016): (1) the emotional component signifies the ics (Hannula et al., 2016), STEM (Benek & Akcay, 2019), positive-negative emotional relations or feelings one has engineering (Choi et al., 2009), and science education (Glynn toward an object or activity; (2) the behavioral component et al., 2009), as well as evaluations of the effects of software indicates the action tendencies one has to respond to an (SW) education on attitude (Park & Shin, 2017). Further, we object in a particular way; and (3) the cognitive component referred to the study by Schepman and Rodway (2020), marks the content of one’s thoughts, such as one’s beliefs where a scale was developed to measure the general attitudes regarding what constitutes a fact. toward AI. The following approaches are useful in investigating attitude as a determinant of the use of IT or other new tech- Materials and Methods nologies: the theory of reasoned action (TRA; Fishbein & Ajzen, 1975), the theory of planned behavior (TPB; Ajzen, Research Procedure 1991), and the technology acceptance model (TAM; Davis et al., 1989). TRA maintains that one’s attitude toward a First, to solve research problems, related literature and pre- particular behavior and the subjective norms that condition vious studies as shown in Tables 1 and 2 were examined, that behavior encourage or discourage performance of that and 52 preliminary questions for measuring attitudes behavior. TPB adds perceived behavioral control to the two toward AI were selected. Second, a 3-point Likert scale factors affecting intention in TRA. The TAM considers per- consisting of the following choices: “essential,” “useful but ceived usefulness, which is “the prospective user’s subjec- not essential,” and “not necessary” (Likert, 1932), was used tive probability that using a specific application system will to examine experts’ opinions about the validity of each increase his or her job performance within an organiza- item. Content validity refers to how well a survey or test tional context” (Davis et al., 1989), and perceived ease of measures the construct that it sets out to measure. The data use, which is “the degree to which the prospective user were examined by using a content validity ratio (CVR) test expects the target system to be free of effort” (Davis et al., (Lawshe, 1975). A total of 26 items that did not satisfy the 1989, p. 985); both affect attitude and behavioral intention. CVR value were removed. Third, the remaining 26 ques- Indeed, many studies have investigated how attitude pre- tions were gathered into a questionnaire that was adminis- dicts and affects behavior (Bohner & Dickel, 2011; Glasman tered to 305 upper-elementary, middle, and high school & Albarracín, 2006; Gorges et al., 2012; Petty et al., 2009; students in Seoul, Korea. Students were asked to respond Steinmetz et al., 2016). to statements using a 5-point Likert scale ranging from Suh and Ahn 3 Table 1. Previous Studies on Attitude Measurement (Attitude Scale). Question Measurement target Country Purpose Item Response option Example item References Student (all levels) Republic of Attitude toward 58 5-point Likert scale I want to know more about C. S. Lee (2008) Korea technology the computer. I will choose a technical career in the future. Undergraduate Republic of Attitude toward 39 5-point Likert scale Engineering is important Choi et al. student Korea engineering for understanding the (2009) direction of society's development. I enjoy reading books about engineering. Student (all levels) Republic of Evaluation of 40 7-point Likert scale I think SW is very important Park and Shin Korea effectiveness of for the development of (2017) SW education our society. I want to keep learning about SW in the future. Student (all levels) USA Attitude toward 25 5-point Likert scale Technology is very Ardies et al. technology important in life. (2013) There should be more education about technology. Undergraduate USA Motivation to 25 5-point Likert scale Learning science is Glynn et al. student learn science interesting. (2009) Knowing science will give me a career advantage. Learning science makes my life more meaningful. I believe I can master science knowledge and skills. Secondary school Turkey Attitude toward 33 5-point Likert scale I am curious about science. Benek and student STEM In the future, I would like to Akcay (2019) have a job in technology. Fifth and ninth Sweden Attitudes toward 14 Yes or No Could you talk to a robot? Serholt et al. grade students social robots Could you ask a robot (2014) for help with your schoolwork? Note. SW = software education. 1 (strongly disagree) to 5 (strongly agree). Lastly, explor- education. As a result of analyzing the sub-elements classi- atory factor analysis (EFA) and confirmatory factor analy- fied in these studies, it was possible to categorize them into sis (CFA) were conducted, and the final questions were nine categories such as learner’s career, experience of inter- selected. Descriptive statistics and distribution of total est in each subject, necessity of learning each subject in scores were prepared for student responses. The study pro- school, perceived usefulness (for the learner), perceived use- cedure is shown in Table 3. fulness (for the society), acceptance of each subject, career related to each subject, each subject for all genders, and per- ceived ease of use of each subject. These tests had a range of Preparation of Test Questions 14 to 58 questions with either true/false and 5- or 7-point Two categories of studies were referenced to develop the Likert scale responses. attitude scale for AI. First, studies related to attitude mea- Second, the category of studies that we referred to were all surement conducted in Korea, the United States, Turkey, and AI-based (Table 2). All these studies used open-ended ques- Sweden were considered (Table 1). The purpose of the stud- tions and were administered to students at different grade lev- ies was to measure attitudes toward SW, STEM, and science els to understand their perceptions about AI and robots. 4 SAGE Open Table 2. Measurement and Evaluation in Previous Studies of AI Education (AI Education Scale). Measurement target Purpose Question form Example item References th 7 grade student Perception of intelligent Open narrative How do you solve conflicts Y. E. Kim and Kim (2019) robots as companions between a robot and an individual? Elementary school Elementary school students’ Open narrative How do you feel when you S. Shin et al. (2018) student images of AI think of AI? High school student High school students’ Open narrative Job risk awareness and Shin et al. (2017) perceptions of AI (emotion ethical risk awareness. and risk perception) Student (all levels) Perceptions of artificial Park and Shin (2017) intelligence technology and Open narrative What is the perception of AI teachers AI technology? Can AI replace teachers? Student (all levels) Perceptions and attitudes Open narrative Image of the robot. N. M. Shin and Kim (2007) toward robots Learning about robots. Learning from robots. Learning with robots. Table 3. Research Procedure. Step Objectives Method Respondent Preparation of test question Develop instrument Specification of research questions - Related literature Review Pilot survey Confirm instrument Expert validity test 8 Professors of Computer Education First main survey Validated instrument Exploratory factor analysis 119 students Second main survey Confirmatory factor analysis 186 students Statistical analysis Summary of responses Statistical analysis 305 students A total of 52 items were set up through the process of requirements. Our final student questionnaire, therefore, referring to the range of these items, integrating common consisted of the remaining 26 items (listed in Appendix 1). items among the questions related to the research problem, and excluding unrelated items. Main Survey for Students In public schools in Korea, AI classes are not part of the Expert Validity Test regular curriculum. Therefore, students with experience in Experts who participated in the preliminary item validity SW education, most relevant to AI education, were sur- evaluation included professors of computer education and veyed. The survey was conducted twice from November doctors who majored in computer education. The panel had 2019 to December 2019, and in December 2021. Both sur- eight members, which is acceptable (Lynn, 1986). The CVR veys were conducted with the same tool, which is presented was verified through a 3-point Likert scale (Lawshe, 1975). in Appendix 2. Dataset 1 was provided by 119 students from The formula for calculating the CVR is as follows. grades 5 to 6, and Dataset 2 was provided by 186 students from grades 6 to 11 (listed in Table 4). All procedures per- N − formed in this study were in accordance with the ethical N : Number of fitted responses,   CVR = standards of the Institutional Review Board Sungkyunkwan   N:Total responses s   University (approval number SKKU2019-11-012). Consent from the relevant guardians was obtained. The CVR score for this panel should be 0.75, which means that more than seven out of eight respondents designate an Exploratory Factor Analysis item as “essential” (Lawshe, 1975). The CVR is mostly adopted to select or exclude individ- EFA is one of a family of multivariate statistical methods ual items testing its validity (Wilson et al., 2012). Among that attempts to identify the smallest number of hypothetical the initial 52 items, 26 items did not satisfy the CVR value constructs that can parsimoniously explain the covariation Suh and Ahn 5 Table 4. Demographics of Respondents in the Main Study. (χ = 2,645.682, df = 325, p = .000); common factors existed and were appropriate. Accordingly, three sub-factors were Grade Boys Girls Total extracted, and factor 1 was named “Behavioral,” factor 2 was Dataset 1 5th 28 35 119 named “Affective,” and factor 3 was named “Cognitive.” 6th 27 29 Cronbach’s alpha of all factors indicated excellent reliability Dataset 2 6th 42 39 186 (Tavakol & Dennick, 2011). The above results and the eigen- 7th 12 10 value and cumulative percentage for each factor for the 10th 16 0 development of the instrument that measures attitudes toward 11th 46 21 AI are shown in Table 5. Total 171 134 305 A CFA was performed using Dataset 2 to confirm the validity of the items. Validity is secured when the standard- ized λ value is .5 or higher, the AVE is .5 or higher, and the observed among a set of measured variables (Watkins, CR is .7 or higher. After performing a CFA on the sub-factors 2018). EFA was conducted on Dataset 1, the result of 119 for each component, standardized, and appropriate λ, AVE, students’ responses, and through this, the validity and rele- and CR values were all secured, as illustrated in Table 6, vance of sub-factors and items were reviewed. Prior to fac- indicating the model used in the study was valid. tor analysis, the Kaiser–Meyer–Olkin (KMO) test was Among the absolute fit indexes that can be recognized, it conducted, and the suitability of factor analysis was identi- is appropriate for the root mean square error of approxima- fied using Bartlett’s test. Subsequently, to understand the tion (RMSEA) to be between 0.05 and 0.1, and the standard- validity of the constituent concept of the 26 selected scale ized root mean square residual (SRMR) to be less than or questions, EFA was performed to extract potential factors equal to 0.5. Furthermore, among the incremental fit indexes, using the varimax method and principal component analy- both the Turker–Lewis index (TLI) and the comparative fit sis. Additionally, Cronbach’s alpha was used to check the index (CFI) are considered to be 0.9 or higher. In the model reliability of the instrument. adopted for this study, RMSEA was 0.075, SRMR was 0.047, TLI was 0.928, and CFI was 0.937. In other words, all the mentioned goodness of fit were satisfied. Confirmatory Factor Analysis To summarize, 3 factors and 26 questions were found to CFA is a type of structural equation modeling that deals spe- be the most appropriate. This measurement instrument was cifically with measurement models-that is, the relationships named “Student Attitude Toward Artificial Intelligence” between observed measures or indicators and latent variables (SATAI). The detailed questionnaires are the same as those of factors (Brown, 2015). In such CFA, validity is secured in Appendix 2. when the standardized λ value is .5 or higher, the average variance extracted (AVE) is .5 or higher, and the construct Discussion reliability (CR) is .7 or higher. Additionally, the absolute fit index, an index that absolutely evaluates the degree of con- SATAI was developed and validated after a literature review formity between the collected data and the research model, using two groups of respondents. In particular, the results was investigated. The incremental fit index, which compares of the EFA for Dataset 1 showed the SATAI scale’s factor the index indicating the accuracy of the structural equation structure. The SATAI scale consists of 26 items comprising model of the study with the model in which correlation 3 components (cognitive, affective, and behavioral factors), between variables was not set, was also investigated. and each question is measured using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The scores for these items can be summed (ranging from 26 Results to 130) to represent a student’s attitude toward AI: a higher In the expert validity test, a group of eight experts verified score indicates a positive attitude toward AI and the likeli- the content validity of 52 items constructed through litera- hood that a student will be an active participant of AI ture research. They evaluated the content validity according education. to a 3-point scale. Consequently, ambiguous expressions For example, descriptive statistics on the responses of a with the phrases “anyone” or “make everything good,” items total of 305 students who participated in this study are as about gender differences in attitude toward AI, and 26 nega- follows. The average of Cognitive Components consisting tive questions, such as AI is difficult, were rejected as they of four questions was 3.57 (SD = 0.986), the average of did not meet the CVR value. Thus, the final survey question- Affective Components consisting of 10 questions was 3.72 naire for students included the remaining 26 question items. (SD = 0.841), and the average of Behavioral Components The result of EFA using Dataset 1 showed a KMO value consisting of 12 questions was 3.16 (SD = 1.068). The mean of 0.937, indicating that the item selection was good. and standard deviation of each item can be confirmed in Additionally, Bartlett’s sphericity was found to be significant Table 7. 6 SAGE Open Table 5. Results of Exploratory Factor Analysis. Factor Item number Commonality F1 F2 F3 Cronbach’s α 16 0.775 0.837 0.266 0.067 .956 17 0.714 0.818 0.155 0.145 15 0.713 0.812 0.222 0.065 18 0.709 0.763 0.262 0.239 22 0.702 0.717 0.242 0.36 23 0.685 0.684 0.284 0.369 21 0.584 0.672 0.204 0.301 20 0.714 0.651 0.358 0.402 26 0.582 0.625 0.241 0.366 24 0.699 0.613 0.402 0.402 19 0.609 0.607 0.398 0.286 25 0.627 0.583 0.195 0.5 7 0.720 0.264 0.804 0.056 .924 10 0.666 0.234 0.76 0.183 6 0.622 0.078 0.702 0.351 8 0.573 0.304 0.65 0.24 13 0.657 0.449 0.642 0.211 9 0.514 0.257 0.637 0.205 11 0.538 0.164 0.604 0.383 5 0.557 0.192 0.588 0.417 14 0.529 0.431 0.559 0.175 12 0.451 0.249 0.527 0.334 1 0.760 0.287 0.283 0.773 .905 2 0.684 0.236 0.307 0.731 3 0.742 0.296 0.36 0.725 4 0.638 0.313 0.288 0.676 Factor domain Behavioral Affective Cognitive Eigenvalue 13.788 2.422 1.563 Percentage of variance 53.031 9.315 6.013 Cumulative percentage 53.031 62.347 68.359 KMO 0.937 Bartlett’s test Approx. χ 2,645.682 df 325 p-Value .000 Note. Extraction method = principal component analysis; rotation method = varimax with Kaiser normalization. Bold text indicates the same category. Figure 1 shows the distribution of the total score of the intelligence have become measurable areas. In other words, questions answered by 305 people. According to this, the the distribution scores obtained through the SATAI can be total distribution of students tends to gradually increase to used to operatively diagnose and compare attitude changes 1.3% at 26 to 40, 6% at 41 to 55, and 30% at 56 to 70, and in students before and after integrating AI into education. then slightly decreases afterwards. Table 7 and Figure 1 In previous studies, open-ended questions of subjective make it possible to confirm the characteristics of AI of the criteria were used to verify the effectiveness of AI education 305 students who responded in this study. That is, “the aver- (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. Shin et al., age of these students” attitude toward AI considering the 2017, 2018; N. M. Shin & Kim, 2007). Other studies used Affective component was the highest at 3.72, with the instruments to measure attitudes toward other subjects, with- Cognitive and the Behavioral Components at 3.57 and 3.16, out validating them (K. Kim, 2019; Y. Lee, 2019). The respectively, and “The distribution of total scores is the high- SATAI can be differentiated from conventional research est in the middle” can be seen. Through these examples, we because it was developed specifically for AI education, thus confirmed that the students’ attitudes toward artificial enabling teachers to quantify students’ attitudes toward AI. Suh and Ahn 7 Table 6. Results of Confirmatory Factor Analysis. Factor SE CR Standardized coefficient AVE Construct reliability Beh._1 <- Behavioral 1 .856 .582630194 .943552573 Beh._2 <- Behavioral 0.923 14.121 .809 Beh._3 <- Behavioral 0.938 15.316 .847 Beh._4 <- Behavioral 0.836 15.199 .843 Beh._5 <- Behavioral 0.857 15.469 .852 Beh._6 <- Behavioral 0.835 16.434 .879 Beh._7 <- Behavioral 0.818 14.041 .806 Beh._8 <- Behavioral 0.836 15.332 .847 Beh._9 <- Behavioral 0.847 15.295 .846 Beh._10 <- Behavioral 0.737 14.297 .815 Beh._11 <- Behavioral 0.83 14.618 .825 Beh._12 <- Behavioral 0.704 11.777 .721 Aff._1 <- Affective 1 .828 .539872315 .921073786 Aff._2 <- Affective 0.899 12.586 .784 Aff._3 <- Affective 0.979 11.72 .746 Aff._4 <- Affective 1.091 13.352 .815 Aff._5 <- Affective 1.062 13.299 .813 Aff._6 <- Affective 0.846 11.596 .74 Aff._7 <- Affective 0.962 10.793 .703 Aff._8 <- Affective 0.774 9.752 .65 Aff._9 <- Affective 0.998 12.878 .796 Aff._10 <- Affective 0.96 10.888 .707 Cog._1 <- Cognitive 1 .907 .709802673 .907188241 Cog._2 <- Cognitive 0.913 17.098 .857 Cog._3 <- Cognitive 0.96 17.92 .875 Cog._4 <- Cognitive 0.98 16.089 .834 Note. SE = standard error; CR = critical ratio; AVE = average variance extracted. Table 7. Descriptive Statistics of the 305 Student Responses. 30% Factor M SD Factor M SD 25% Cog._1 3.60 1.056 Aff._10 3.70 1.145 20% Cog._2 3.65 1.054 Beh._1 2.80 1.373 15% Cog._3 3.66 1.082 Beh._2 2.69 1.349 10% Cog._4 3.37 1.213 Beh._3 2.92 1.374 5% Aff._1 3.94 1.034 Beh._4 3.26 1.258 Aff._2 4.04 0.976 Beh._5 3.36 1.244 0% 26-40 41-55 56-70 71-8586-100 101-115 116-130 Aff._3 3.65 1.163 Beh._6 3.32 1.211 Aff._4 3.58 1.139 Beh._7 3.13 1.245 Aff._5 3.65 1.090 Beh._8 3.19 1.247 Figure 1. Distribution of total scores. Aff._6 3.93 1.025 Beh._9 3.35 1.274 Note. x-axis = total score; y-axis = the percentage of the total number of respondents. Aff._7 3.49 1.159 Beh._10 3.60 1.188 Aff._8 3.56 0.945 Beh._11 3.16 1.261 Aff._9 3.71 1.084 Beh._12 3.17 1.272 encountering a new subject. Thus, measuring their attitudes can provide instructors with ideas for curriculum design that encourage students to have a positive attitude toward AI Conclusions learning (Ajzen, 1991; Khine, 2015). As for education through AI, AI appeared as a new technology to learners, This study is relevant in that it develops a scale that mea- which reinforced the importance of attitude through the sures student attitudes toward AI and tests the validity of TAM. Hence, in our study, a standardized instrument for that scale. First, the importance of attitudes toward AI edu- measuring a learner’s attitude toward AI was developed cation was confirmed through a literature review. Literature with attention to students’ cognitive, affective, and revealed that learners of AI education felt that they were 8 SAGE Open behavioral attitudes. On the one hand, AI education is new, This study has a few limitations. First, the measure devel- so there are no traditional curriculum or teaching methods oped can be used to verify the students’ attitudes toward AI, (E.-K. Lee, 2020; Touretzky et al., 2019). On the other hand, but cannot measure conceptualizations of AI. Therefore, SATAI can help teachers objectively measure students’ atti- future researchers should develop a scale that measures a tude toward AI. Moreover, instructors can use the test scores learner’s concept of AI. Second, although the results of CFA to design, modify, apply, and customize educational pro- on this dataset are strong evidence that the tool does not grams to meet learner needs. These advancements contrib- leverage age-related differences, there may be limitations in ute to the emerging philosophical cornerstones of AI the generalization of the study by surveying 305 elementary, education. SATAI results, we believe, can be applied to middle, and high school students. Future studies should verify the effects of AI-related education for other K-12 stu- include a wider range of participants, including individuals dents outside the population used in this study. in other regions. Appendix 1. Complete List of All Questions. Result of expert Item Domain CVR validity test It is fun to learn about AI. Behavioral Interest in AI 0.875 Confirmed components It is interesting to use AI. 1 Confirmed I want to continue learning about AI. 1 Confirmed I like to use something related to AI. 0.875 Confirmed I am interested in AI-related TV programs or 0.875 Confirmed Internet videos. I will participate in a club relating to AI (if one 0.875 Confirmed exists). I think I can handle AI well. 0.875 Confirmed I am interested in the development of AI. 1 Confirmed I will choose a job in the field of AI. Learner’s career 0.875 Confirmed I want to work in the field of AI. 0.875 Confirmed I want to make something that makes human life 0.875 Confirmed more convenient through AI. I think that there should be more class time Necessity of 0.875 Confirmed devoted to AI in school. learning AI in school I think it is important content to learn about AI in Cognitive 1 Confirmed school. components I think that AI should be taught in school. 1 Confirmed I think every student should learn about AI in 1 Confirmed school. AI class is important. 1 Confirmed AI is related to my life. Affective Perceived 0.875 Confirmed components usefulness (for I think AI makes life more convenient. 0.875 Confirmed the learner) I will use AI to solve problems in daily life. 1 Confirmed AI helps me solve problems in real life. 0.875 Confirmed I will need AI in my life in the future. 0.875 Confirmed AI is worth studying. 1 Confirmed AI is necessary for everyone. Perceived 0.875 Confirmed usefulness (for AI produces more good than bad. 0.875 Confirmed society) AI is very important for developing society. 0.875 Confirmed I think that most jobs in the future will require 0.875 Confirmed knowledge related to AI. AI brightens the future of our country. 0.625 Removed AI could make everything better. 0.625 Removed I could ask AI to complete housework. Behavioral Acceptance of AI 0.625 Removed components I can ride in a car driven by AI. 0.500 Removed I will get help from AI when I have a problem. 0.625 Removed I will receive surgery from an AI doctor. 0.500 Removed I could accept AI as a teacher. 0.375 Removed I can be a good friend with AI. 0.500 Removed (continued) Suh and Ahn 9 Appendix 1. (continued) Result of expert Item Domain CVR validity test Anyone can get a job related to AI. Affective Career related 0.625 Removed components to AI If you have a job related to AI, then your future will 0.625 Removed be bright. I think it will be interesting to work in the AI field. 0.625 Removed Men can do AI-related things better than women. Affective AI for all genders 0.125 Removed components Women can do AI-related things better than men. 0.125 Removed Men know AI better than women. 0.125 Removed Women know AI better than men. 0.125 Removed Men are more suited to AI work than women. 0.125 Removed Women are more suited to AI work than men. 0.125 Removed AI is more difficult for women than men. 0.125 Removed AI is more difficult for men than women. 0.125 Removed Men handle AI better than women. 0.125 Removed Women handle AI better than men. 0.125 Removed You must be smart to learn about AI. Affective Perceived ease of 0.500 Removed components use (reversed) AI is for smart people. 0.375 Removed Studying AI requires talent. 0.250 Removed It is difficult to learn AI. 0.625 Removed Not everyone can learn about AI. 0.500 Removed Appendix 2. Student Attitudes Toward AI (SATAI). Domain No. Item 5 4 3 2 1 Cognitive 1 I think that it is important to learn about AI in school. components 2 AI class is important. 3 I think that lessons about AI should be taught in school. 4 I think every student should learn about AI in school. Affective 5 AI is very important for developing society. components 6 I think AI makes people’s lives more convenient. 7 AI is related to my life. 8 I will use AI to solve problems in daily life. 9 AI helps me solve problems in real life. 10 I will need AI in my life in the future. 11 AI is necessary for everyone. 12 AI produces more good than bad. 13 AI is worth studying. 14 I think that most jobs in the future will require knowledge related to AI. Behavioral 15 I want to work in the field of AI. components 16 I will choose a job in the field of AI. 17 I would participate in a club related to AI if there was one. 18 I like using objects related to AI. 19 It is fun to learn about AI. 20 I want to continue learning about AI. 21 I’m interested in AI-related TV programs or online videos. 22 I want to make something that makes human life more convenient through AI. 23 I am interested in the development of AI. 24 It is interesting to use AI. 25 I think that there should be more class time devoted to AI in school. 26 I think I can handle AI well. Note. The scale uses 5 points for strongly agree, 4 points for agree, 3 points for neutral, 2 points for disagree, and 1 point for strongly disagree. 10 SAGE Open Acknowledgments students. International Journal of Engineering Education, 34(1), 226–235. We would like to thank Editage (www.editage.co.kr) for English Ardies, J., De Maeyer, S., & Gijbels, D. (2013). Reconstructing language editing. the pupils’ attitude towards technology-survey. Design and Technology Education, 18(1), 8–19. Author Contributions Bagozzi, R. P. (1978). The construct validity of the affective, All authors contributed to the study conception and design. Material behavioral, and cognitive components of attitude by analysis preparation, data collection, and analysis were performed by Woong of covariance structures. Multivariate Behavioral Research, Suh. The first draft of the manuscript was written by Seongjin Ahn 13(1), 9–31. https://doi.org/10.1207/s15327906mbr1301_2 and all authors commented on previous versions of the manuscript. Benek, I., & Akcay, B. (2019). Development of STEM attitude scale All authors read and approved the final manuscript. Woong Suh and for secondary school students: Validity and reliability study. Seongjin Ahn are co–first authors. International Journal of Education in Mathematics. International Journal of Education in Mathematics, Science and Technology, 7(1), 32–52. https://doi.org/10.18404/ijemst.509258 Availability of Data and Materials Bohner, G., & Dickel, N. (2011). Attitudes and attitude change. The data that support the findings of this study are available from Annual Review of Psychology, 62, 391–417. https://doi. Sungkyunkwan University, but restrictions apply to the availability org/10.1146/annurev.psych.121208.131609 of these data, which were used under license for the current study, Brown, T. A. (2015), Confirmatory factor analysis for applied and so are not publicly available. Data are however available from research. Guilford Publications. the authors upon reasonable request and with permission of Cheung, R., & Vogel, D. (2013). Predicting user acceptance of Sungkyunkwan University. collaborative technologies: An extension of the technology acceptance model for e-learning. Computers and Education, Ethical Approval 63, 160–175. https://doi.org/10.1016/j.compedu.2012.12.003 All procedures performed in this study were in accordance with Cherry, S., Rohit, S., Ashik, A., Keerthini, M., Aishah, A., Monzon, the ethical standards of the Institutional Review Board (IRB) at L., & Poon, D. S. (2020). Attitudes and perceptions of UK Sungkyunkwan University (approval number SKKU2019-11- medical students towards artificial intelligence and radiology: 012-003). A multicentre survey. Insights into Imaging, 11(1), 14. https:// doi.org/10.1186/s13244-019-0830-7 Chiu, T. K. F. (2021). A holistic approach to the design of artifi- Informed Consent cial intelligence (AI) education for K-12 schools. TechTrends, Written informed consent was obtained from all the guardians of 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1 students who participated in this study. Choi, H.-Y., Park, H.-M., Lee, J.-G., & Ryu, S.-M. (2009). Development of the instrument for assessment the attitude Declaration of Conflicting Interests toward engineering. Journal of Korean Institute of Industrial Education, 34(2), 161–178. The author(s) declared no potential conflicts of interest with respect Coppin, B. (2004), Artificial intelligence illuminated. Jones & to the research, authorship, and/or publication of this article. Bartlett Publishers Learning. Cukurova, M., Luckin, R., & Kent, C. (2020). Impact of an arti- Funding ficial intelligence research frame on the perceived credibility The author(s) received no financial support for the research, author- of educational research evidence. International Journal of ship, and/or publication of this article. Artificial Intelligence in Education, 30(2), 205–235. https:// doi.org/10.1007/s40593-019-00188-w ORCID iD Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User accep- tance of computer technology: A comparison of two theoreti- Woong Suh https://orcid.org/0000-0003-3696-8198 cal models. Management Science, 35(8), 982–1003. https://doi. org/10.1287/mnsc.35.8.982 References Dunlap, D. D., & Dugger, W. E. (1991). Comparing attitudes Ajzen, I. (1991). The theory of planned behavior. Organizational towards technology of third and fourth grade students in Behavior and Human Decision Processes, 50(2), 179–211. Virginia relative to their exposure to technology. PATT 5 https://doi.org/10.1016/0749-5978(91)90020-T Conference Proceedings (Vol. 2, pp. 194–201), Eindhoven. Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. Dunn, T. J., & Kennedy, M. (2019). Technology enhanced learning (2019). Constructionism, ethics, and creativity: Developing in higher education; motivations, engagement and academic primary and middle school artificial intelligence education. achievement. Computers and Education, 137, 104–113. https:// Proceedings of the IJCAI 2019. http://robotic.media.mit. doi.org/10.1016/j.compedu.2019.04.004 edu/wp-content/uploads/sites/7/2019/08/Constructionism__ Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. Ethics__and_Creativity.pdf Harcourt Publishers Brace Jovanovich College Publishers. Alias, M., Lashari, T. A., Akasah, Z. A., & Kesot, M. J. (2018). ETRI. (2019). Seven AI trends by 2020-beyond perception Self-efficacy, attitude, student engagement: Emphasising (ETRI Insight Report). https://ksp.etri.re.kr/ksp/plan-report/ the role of affective learning attributes among engineering doi?id=766 Suh and Ahn 11 Fishbein, M., & Ajzen, I. (1972). Attitudes and opinions. Annual artificial intelligence learning in elementary courses. Journal Review of Psychology, 23(1), 487–544. https://doi.org/10.1146/ of the Korean Association of Information Education, 23(3), annurev.ps.23.020172.002415 229–235. https://doi.org/10.14352/jkaie.2019.23.3.229 Fishbein, M., & Ajzen, I. (Eds.). (1975). Belief, attitude, inten- Kim, K. (2019). An artificial intelligence education program devel- tion, and behavior: An introduction to theory and research. opment and application for elementary teachers. Journal of the Addison-Wesley. Korean Association of Information Education, 23(6), 629–637. Fredrickson, B. L. (2001). The role of positive emotions in posi- https://doi.org/10.14352/jkaie.2019.23.6.629 tive psychology. The broaden-and-build theory of positive Kim, Y.-E., & Kim, H.-C. (2019). A study on middle school stu- emotions. American Psychologist, 56(3), 218–226. https://doi. dents’ perception on intelligent robots as companions. Journal org/10.1037/0003-066X.56.3.218 of Korean Association of Computer Education, 22(4), 35–45. Gaines-Ross, L. (2016, October 24). What do people—not techies, https://doi.org/10.32431/kace.2019.22.4.004 not companies—think about artificial intelligence? Harvard Lawshe, C. H. (1975). A quantitative approach to content valid- Business Review. https://hbr.org/2016/10/what-do-people-not- ity. Personnel Psychology, 28(4), 563–575. https://doi. techies-not-companies-think-about-artificial-intelligence org/10.1111/j.1744-6570.1975.tb01393.x Gherheș, V., & Obrad, C. (2018). Technical and humanities stu- Lee, C.-S. (2008). Development of technology attitude scale for dents’ perspectives on the development and sustainability of Korean pupils. Journal of Korean Practical Arts Education, artificial intelligence (AI). Sustainability, 10(9), 3066. https:// 14(2), 157–174. https://doi.org/10.17055/jpaer.2008.14.2.157 doi.org/10.3390/su10093066 Lee, E.-K. (2020). Analysis of artificial intelligence curriculum Glasman, L. R., & Albarracín, D. (2006). Forming attitudes that of domestic and foreign elementary and secondary school. predict future behavior: A meta-analysis of the attitude-behav- Journal of Computers in Education, 23(1), 37–44. ior relation. Psychological Bulletin, 132(5), 778–822. https:// Lee, Y. (2019). An analysis of the influence of block-type pro- doi.org/10.1037/0033-2909.132.5.778 gramming language-based artificial intelligence education on Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science the learner’s attitude in artificial intelligence. Journal of the motivation questionnaire: Construct validation with nonscience Korean Association of Information Education, 23(2), 189–196. majors. Journal of Research in Science Teaching, 46(2), 127– https://doi.org/10.14352/jkaie.2019.23.2.189 146. https://doi.org/10.1002/tea.20267 Likert, R. (1932). A technique for the measurement of attitudes. Gorges, J., Kandler, C., & Bohner, G. (2012). Internationalization at Archives of Psychology, 22, 5–55. home: Using learning motivation to predict students’ attitudes Lynn, M. R. (1986). Determination and quantification of con- toward teaching in a foreign language. International Journal of tent validity. Nursing Research, 35(6), 382–385. https://doi. Educational Research, 53, 107–116. https://doi.org/10.1016/j. org/10.1097/00006199-198611000-00017 ijer.2012.03.001 Manikonda, L., & Kambhampati, S. (2018). Tweeting AI: Hannula, M. S., Di Martino, P., Pantziara, M., Zhang, Q., Morselli, Perceptions of lay versus expert Twitterati [Conference F., Heyd-Metzuyanim, E., Lutovac, S., Kaasila, R., Middleton, paper]. Twelfth International AAAI Conference on Web and J. A., Jansen, A., & Goldin, G. A. (2016). Attitudes, Beliefs, Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ Motivation and Identity in Mathematics Education: An ICWSM18/paper/viewPaper/17843 Overview of the Field and Future Directions. (ICME-13 Metsärinne, M., & Kallio, M. (2016). How are students’ atti- Topical Surveys). Springer International Publishing tudes related to learning outcomes? International Journal of Hannula, M. S., Di Martino, P., Pantziara, M., Zhang, Q., Morselli, Technology and Design Education, 26(3), 353–371. https://doi. F., Heyd-Metzuyanim, E., Lutovac, S., Kaasila, R., Middleton, org/10.1007/s10798-015-9317-0 J.A., Jansen, A., & Goldin, G. A. (2016). Attitudes, beliefs, Park, J. H., & Shin, N. M. (2017). Students’ perceptions of artifi- motivation, and identity in mathematics education: An over- cial intelligence technology and artificial intelligence teachers. view of the field and future directions (ICME-13 Topical Journal of Korean Teacher Education, 34(2), 169–192. Surveys) (pp. 1–35). Springer. Petty, R. E., Fazio, R. H., & Brinol, P. (Eds.). (2009), Attitudes: Heintz, F. (2021). Three interviews about K-12 AI education in Insights from the new implicit measures. Psychology Press. America, Europe, and Singapore. KI—Künstliche Intelligenz, Schepman, A., & Rodway, P. (2020). Initial validation of the general 35(2), 233–237. https://doi.org/10.1007/s13218-021-00730-w attitudes towards Artificial Intelligence Scale. Computers in Huang, N. N., Chiu, L. J., & Hong, J. C. (2016). Relationship among Human Behavior Reports, 1, 100014. https://doi.org/10.1016/j. students’ problem-solving attitude, perceived value, behavioral chbr.2020.100014 attitude, and intention to participate in a science and technol- Serholt, S., Barendregt, W., Leite, I., Hastie, H., Jones, A., Paiva, ogy contest. International Journal of Science and Mathematics A., Vasalou, A., & Castellano, G. (2014, August 25–29). Education, 14(8), 1419–1435. https://doi.org/10.1007/s10763- Teachers’ views on the use of empathic robotic tutors in the 015-9665-y classroom [Conference paper]. The 23rd IEEE International Ikkatai, Y., Hartwig, T., Takanashi, N., & Yokoyama, H. M. Symposium on Robot and Human Interactive Communication. (2022). Octagon measurement: Public attitudes toward AI eth- https://ieeexplore.ieee.org/document/6926376 ics. International Journal of Human–Computer Interaction, Shin, N. M., & Kim, S. A. (2007). What do robots have to do 1–18. https://doi.org/10.1080/10447318.2021.2009669 with student learning? Journal of Korean Association for Khine, M. S. (Ed.). (2015). Attitude measurements in science edu- Educational Information and Media, 13(3), 79–99. cation: Classic and contemporary approaches. IAP. Shin, S., Ha, M., & Lee, J.-K. (2017). High school students’ percep- Kim, J., & Park, N. (2019). Development of a board game-based tion of artificial intelligence: Focusing on conceptual under- gamification learning model for training on the principles of standing, emotion and risk perception. Korean Association for 12 SAGE Open Learner-Centered Curriculum and Instruction, 17(21), 289– Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. 312. https://doi.org/10.22251/jlcci.2017.17.21.289 (2019). Envisioning AI for K-12: What should every child Shin, S., Ha, M., & Lee, J.-K. (2018). Exploring elementary school know about AI? Proceedings of the AAAI Conference on students’ image of artificial intelligence. Journal of Elementary Artificial Intelligence, Proceedings of the AAAI Conference on Science Education, 37(2), 126–146. https://doi.org/10.15267/ Artificial Intelligence, 33, 9795–9799. https://doi.org/10.1609/ keses.2018.37.2.126 aaai.v33i01.33019795 Steinbauer, G., Kandlhofer, M., Chklovski, T., Heintz, F., & Watkins, M. W. (2018). Exploratory factor analysis: A guide to best Koenig, S. (2021). A differentiated discussion about AI educa- practice. Journal of Black Psychology, 44(3), 219–246. https:// tion K-12. Kunstliche Intelligenz, 35(2), 131–137. https://doi. doi.org/10.1177/0095798418771807 org/10.1007/s13218-021-00724-8 Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalcu- Steinmetz, H., Knappstein, M., Ajzen, I., Schmidt, P., & Kabst, R. lation of the critical values for Lawshe’s content valid- (2016). How effective are behavior change interventions based ity ratio. Measurement and Evaluation in Counseling and on the theory of planned behavior? Zeitschrift für Psychologie, Development, 45(3), 197–210. https://doi.org/10.1177/ 224(3), 216–233. https://doi.org/10.1027/2151-2604/a000255 0748175612440286 Svenningsson, J., Hultén, M., & Hallström, J. (2018). Understanding Yoo, J. (2019). A study on AI education in graduate school through attitude measurement: Exploring meaning and use of the PATT IPA. Journal of the Korean Association of Information short questionnaire. International Journal of Technology and Education, 23(6), 675–687. https://doi.org/10.14352/jkaie. Design Education, 28(1), 67–83. https://doi.org/10.1007/ 2019.23.6.675 s10798-016-9392-x Yu, K.-C., Lin, K.-Y., Han, F.-N., & Hsu, I. Y. (2012). A model Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s of junior high school students’ attitudes toward technology. alpha. International Journal of Medical Education, 2, 53–55. International Journal of Technology and Design Education, https://doi.org/10.5116/ijme.4dfb.8dfd 22(4), 423–436. https://doi.org/10.1007/s10798-011-9154-8

Journal

SAGE Open – SAGE

Published: May 24, 2022

Keywords: evaluation methodologies; artificial intelligence in education; student attitude

There are no references for this article.