Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Application of the Technology Acceptance Model to OCR-based CAPTCHA systems. Preliminary results

Application of the Technology Acceptance Model to OCR-based CAPTCHA systems. Preliminary results This research investigates the applicability of F. Davis' Technology Acceptance Model (TAM) to OCR-based CAPTCHAs involved in a free e-mail accounts registration process. The results provide preliminary evidence that the model may be a useful tool for research focused on acceptance of CAPTCHAs. The results are especially interesting when it comes to the concept of perceived usefulness and its direct relationship to the concept of attitude toward use. This relation indicates a new area of research for CAPTCHA developers interested in improving usability and accessibility of these systems. KEYWORDS: CAPTCHA, Turing test, technology acceptance model, usability, accessibility 1. Introduction 1.1. Aim of the study The aim of this study is to investigate the applicability of Fred Davis' Technology Acceptance Model (cf. [10]) to OCR-based CAPTCHAs involved in free e-mail accounts registration process. In our opinion the Technology Acceptance Model research might bring new and fruitful ideas into CAPTCHAs' development area, especially when it comes to the user-friendliness aspects of CAPTCHA systems. 1.2. What is a CAPTCHA? CAPTCHA stands for Completely Automated Turing Test To Tell Computers and Humans Apart. The very idea of CAPTCHA systems comes from the seminal paper of Moni Naor [20], while the name was coined by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford (who also developed first CAPTCHAs, cf. www.captcha.net). The main task of a CAPTCHA is to automatically differentiate bots (malicious programs) and human users in on-line services. There are many domains where such systems are needed, like for example, commenting blogs, Internet message boards, sending SMS/MMS messages via web-pages, on-line polls etc. However, the most important domain where providers want to be sure that they are dealing with human users are free e-mail accounts. CAPTCHAs are widely used by free e-mail accounts providers to stop SPAM messages from being sent from fictional accounts registered by bots. Users of Gmail, Yahoo! and many other popular services in this field have to solve a CAPTCHA task while registering an e-mail account. It is worth to stress that CAPTCHAs have become popular not only on large portals but it is also a kind of standard anti-SPAM solution on numerous small (even private) web-sites. The most characteristic properties of CAPTCHAs are (cf. [1], [2], [20]): test instances and their solutions are generated automatically; most humans can pass the test in a very short time; it is difficult to write a computer program that will be successful in passing the test in question; the underlying problem must be difficult for programs used currently as well as those which will be developed in the predictable future. Nowadays, only one CAPTCHA schema is popular, namely the so called OCR-based CAPTCHA (often referred to as `visual CAPTCHAs'). 1 In this type of CAPTCHA tasks consist in identifying the content of a distorted picture, e.g. recognising a text (most popular, used e.g. by Yahoo! or Hotmail), a common animal (Animal-PIX, a CAPTCHA which can be found at www.captcha.net), or a geometric figure. Visual CAPTCHAs exploit the ability of people to read text more reliably than optical character recognition (OCR) or other machine vision systems (cf. [7]). An example of this kind of CAPTCHA (used by Yahoo!) is presented in Figure 1. OCR-based CAPTCHAs are the most popular solution but not the only one. It is worth to mention other interesting approaches used in CAPTCHAs, like semantic based tasks: Egglue Semantic CAPTCHA (http://code.google.com/p/egglue/), SemCAPTCHA (http://semcaptcha.wikidot.com/) and SQUIGL-PIX (http://www.captcha.net); or even more problem-solving oriented ones, like: ESPPIX (http://www.captcha.net). Figure 1. An example of OCR-based CAPTCHA used by Yahoo! Unfortunately, the ability exploited by visual CAPTCHAs seems not to be reliable enough any more. There is a significant number of papers presenting attacks on well known visual CAPTCHA schemes. Let us mention only some of them. Mori and Malik [19] describe an attack on the EZ-Gimpy CAPTCHA used by Yahoo!, which enjoyed a success rate of 92%. In the more difficult case of Gimpy they passed the test 33% of the time. Moy et al. [17] achieved a 99% rate of solved tasks of EZ-Gimpy (and on Gimpy-r ­ a four letter version of GIMPY ­ 78%). Yan and Ahmad in [25] present a segmentation method that enabled them to break the Microsoft CAPTCHA scheme with an overall success rate of 60%. There are works like [25] or [14] where simple techniques were used to break many visual CAPTCHAs. This leads to a situation where CAPTCHA tasks become more and more complicated (motivated by the wish to provide more security). As a result many of tasks offered to users are not even human-readable. To give a simple example. We have performed a simple research where 10 CAPTCHA tasks (retrieved from one of the biggest Polish e-mail account providers) were presented to participants (N=10). Only the correctness of the solution was measured (time was unlimited). Tasks were presented on a web page, and the participants solved them using their home computers. As a result only 3 tasks were solved correctly by all ten participants. As a consequence of growing popularity combined with increasing difficulty of CAPTCHAs, problems of accessibility and usability of this technology become more and more important. There are papers describing research in this field, where some design rules are discussed to make CAPTCHAs more accessible (cf. [15]) and more usable ([27], [5], [6], [13], [24]). What is more, since CAPTCHAs are obligatory elements of the registration process, issues like users' attitudes, feelings and acceptance of this kind of tasks also should become an important element of CAPTCHA design and implementation in a given service. In our opinion there is a need for a tool that will concentrate on user's attitudes and level of acceptance for CAPTCHA systems. What is more, this tool should also allow for identifying factors responsible for users' attitudes. Davis' TAM seems to be a good candidate for such a tool. 1.3. Technology Acceptance Model TAM was proposed by Fred Davis in [10]. The main aim of this model is to provide a tool for assessing Information Technology (IT) acceptance among users. The model (as it is described in [11]) is presented in Figure 2. Figure 2. Technology Acceptance Model It consists of the following components: External Variables (system and/or users characteristics), Perceived Usefulness, Perceived Ease of Use, Attitude Towards Using, Behavioral Intention to Use, and Actual System Use. Perceived Usefulness (PU) is defined as "the degree to which a person believes that using a particular system would enhance his or her job performance" [9, p. 320]. Perceived Ease of Use (PEoU) is understood as "the degree to which a person believes that using a particular system would be free of effort" [10, p. 320]. Attitude Towards Using (ATU) is "the degree of evaluative effect that an individual associates with using the target system in his or her job" [11, p. 476]. Davis developed reliable scales for the model's components (cf. [29]). TAM is widely used to investigate and predict users' acceptance for various IT systems. A good example of TAM application is the research presented in [23]. The main aim of this research was to identify the determinants of intention to use the Firefox browser. This issue is very interesting in the light of the enormous popularity of Firefox among professional users on the one hand, and its belonging to the open source software family, on the other. What is more, as the authors point out, such finding might have theoretical and practical applications for other programs of this kind. Results of the research indicate that the principal determinant of intention to use Firefox is its perceived usefulness. In other words this TAM research allowed to identify that the most important feature of Firefox (from the professionals point of view) is the security of this browser (it is less likely to be targeted by crackers, more resistant to spyware etc. - cf. [23, p. 453]). TAM was also applied for other IT systems, like text editors and e-mail clients [10], knowledge management systems [17], multimedia learning environments [22], or wireless technology [28]. Also many extensions and revisions of TAM are proposed (see for example [29], [16], [21], [9]). 2. Research 2.1. Model and hypotheses For the research we have decided to use the technology acceptance model in the form presented in the previous section. However, one comment is in order. We used the following external variables in our research: age, gender, the number of e-mail accounts one is using and self assessed computer expertise. Our choice of the variables was based on previous TAM research and was aimed at gaining characteristics of the research group related to the CAPTCHA usage. We have not discovered any correlations with the External variables component in our research2. Consequently, we have refrained from using this component in the further model fit analysis (we use it only as the source of information about our group of participants). Our research hypotheses ­ drawn from previous TAM research ­ were: (H1) Positive relationships will be observed between PU and BI. (H2) Positive relationships will be observed between PEoU and ATU. (H3) Positive relationships will be observed between PU and ATU. (H4) Positive relationships will be observed between ATU and BI. (H5) Positive relationships will be observed between BI and AU. (H6) Positive relationships will be observed between PEoU and PU. 2.2. Development of the scale A 21 item survey was prepared for the research. Items and measurement scales were based on the previous TAM research ([12], [23], [29], [16]) and tailored to the specific area of the CAPTCHAs. For all of the TAM components seven-item scales were used. The TAM scales of the questionnaire are Age (r = 0.08, p=0.40), gender (t102=0.75, p=0.45, effect-size r=0.07), the number of e-email accounts one is using (r=0.13, p=0.17), and self assessed computer expertise (r=0.18, p=0.06). shown in Appendix. Four questions concerned the participants (sex, age, number of used e-mail accounts, self assessed computer expertise). The questionnaire was validated in the pilot study during which 112 responses were collected. After the pilot study we have decided to rewrite the negatively phrased items of the questionnaire in order to improve the reliability for questionnaire questions (which is suggested by F. Davis in [10, p. 327]). For discussion on the scale validation see section 3.1 below. 2.2.1. Procedure The questionnaire was powered by a GoogleDocs form and posted on-line with additional explanations about OCR-CAPTCHAs within a web-page prepared for the research (published on the Adam Mickiewicz University server). The link for the page was posted on popular websites (like Facebook) and on portals and forums about IT, psychology and cognitive science, where CAPTCHA issues were discussed. The questionnaire was accessible for 10 weeks. 116 fully filled questionnaires were collected. 2.2.2. Participants Responses were obtained from 116 participants (45 men, 71 women, aged from 17 to 43 (84% of the group were 20-27, average age M=23.8, SD=4.01). Participants were asked to self-assess their computer expertise. Detailed results are presented in Table 1 (scale was 1 ­ beginner to 7 ­ expert). Table 1. Self assessed computer expertise 1 (beginner) 2 3 4 5 6 7 (advanced) N=116 0 0 16 26 35 26 10 0% 0% 13.8% 22.4% 30.2% 22.4% 11.2% The research was focused on CAPTCHAs used by free e-mail accounts providers, therefore we also asked how many e-mail accounts did a subject have. The average result was 3 (3.0+/-1.7 for women and 3.1+/-1.8 for men, from 1­12 e-mail accounts). Men presented a higher (t102 = 2.7, p < 0.01, effect-size r=0.26) self assessed expertise in computer usage (M F = 4.7, SDF = 1.23; MM = 5.3, SDM = 1.08) than women. There was no significant relationship between the level of self assessed level of expertise in computer usage and the number of e-mail accounts in the group of men, and a very weak relationship in the group of women (Spearman's rho = 0.33, p < 0.01). We may say that our average subject was a young person with a high level of computer expertise, actively using e-mail accounts, aware of what CAPTCHA is and with experience in solving CAPTCHA tasks (gained in the process of registration of these e-mail accounts). 3. Results and discussion 3.1. Scale validation The reliability of the whole scale (based on all seventeen questions) is 0.84. The details for each construct are presented in Table 2. For the Actual Use construct, we observed low reliability. Table 2. Cronbach Alpha reliability coefficients (cf. [8]) Construct PU PEoU ATU BI AU scale Reliability 0.42 0.73 0.94 0.73 0.46 0.84 after item removal 0.29; 0.48; 0.41; 0.36; 0.27 0.68; 0.64; 0.76; 0.70; 0.65 0.93; 0.92; 0.92; 0.91 0.65; 0.67; 0.61 0.32; 0.10; 0.15 The reliability was slightly improved by the removal of one item, but it still remained low (in comparison to other constructs). This might be the effect of the used research scenario. Typically, the TAM research is conducted in groups of employees who are presented with a new piece of software and then use it for a period of time. After that time the TAM research is conducted (cf. e.g. [10], [17]). In our research, in order to get more general results, we asked about a system which is not connected to any organisational environment, and which was used by different participants with a different frequency and in different circumstances (there were some common characteristics though, since we have asked about CAPTCHA systems involved in e-mail accounts registration process). The other reason might be a bias in the very idea of this construct, i.e. self-reported usage ­ cf. [28, p. 6]. In our opinion the reliability of the AU construct in CAPTCHA research might be improved in two ways. One is to apply the procedure described above ­ i.e. to present the participants with an instance of a CAPTCHA and invoke them to solve tasks for a while in a defined environment and perform the TAM research afterwards. Another possible solution would be to measure the AU concept in an experimental setting. As for the research presented in this paper we have decided to drop the construct from the further analysis. In our opinion this brings no harm to our scope of interest, since the most interesting part of the TAM model in CAPTCHA context would be PEoU, PU, ATU and BI. As a consequence, we will not consider the 5th hypothesis stated in Section 2.1. 3.2. Model fit Let us remind that the research hypotheses 3 were that positive relationships will be observed between: (H1) Perceived Usefulness and Behavioral Intention to Use; (H2) Perceived Ease of Use and Attitude Towards Use; (H3) Perceived Usefulness and Attitude Towards Use; (H4) Attitude Towards Use and Behavioral Intention to Use; (H6) Perceived Ease of Use and Perceived Usefulness. Analysis results are shown in Figure 3 along with hypotheses numbers. Figure 3. Model fit results (Chi2125 = 228.7, p < .001; CFI = 0.91; NNFI = 0.89; RMSEA = 0.08, 90%CI = 0 - 0.07; SRMR = 0.07) Since a strong Perceived Usefulness and Perceived Ease of Use correlation was observed and confirmation analysis revealed a stronger effect of one of them, we have decided to perform a mediation analysis. We were interested if the relation between Perceived Ease of Use and Attitude Toward Hypothesis 5 was dropped for the reasons pointed out in the previous subsection. Use is mediated by Perceived Usefulness. Sobbel test was carried out and the results confirmed that Perceived Ease of Use influence on Attitude Towards Use is moderated with Perceived Usefulness (p < 0.001) ­ cf. Figure 4. Figure 4. Indirect effect Sobel's test = .29, Z = 3.95, SE = 0.07, p < 0.001 The summary of the research results is presented in Table 3. 3.3. Discussion and further research The results obtained are consistent with previous TAM research. The only exception is (H1), which was not confirmed ­ there was no direct relationship between Perceived Usefulness and Behavioral Intention to Use (such relationship was observed only for Attitude Towards Use and Behavioral Intention to Use). The reason for that might be the research scenario (as described in the Actual System Use concept case, cf. Section 3.1). If the Behavioral Intention to Use is measured long after the introduction of a system under research, there might be a bias in results ­ cf. [16, p. 7]. In our opinion also the properties of CAPTCHA might be important here. The research concerns a class of systems (although with consistent characteristics), not a single software token. Table 3. Summary of research hypothesis findings Hypothesis (H1) Positive relationships will be observed between PU and BI (H2) Positive relationships will be observed between PEoU and ATU (H3) Positive relationships will be observed between PU and ATU (H4) Positive relationships will be observed between ATU and BI (H6) Positive relationships will be observed between PEoU and PU Comments not confirmed confirmed, the effect is mediated by PU confirmed confirmed confirmed One of the most interesting findings is that the influence of Perceived Usefulness influence on Attitude Towards Use is stronger than that of Perceived Ease of Use on Attitude Towards Use. What is more, the influence of Perceived Ease of Use on Attitude Towards Use is mediated by Perceived Usefulness. This suggests that in oder to build positive Attitude Towards Use among users, developers should concentrate not only on making CAPTCHA easier or more human-readable, but also on factors constituting the Perceived Usefulness concept. Let us remind that Perceived Usefulness is defined as: "the degree to which a person believes that using a particular system would enhance his or her job performance" [10, p. 320]. In our research context, this might be seen as the degree to which a person believes that solving a CAPTCHA would enhance his or her performance with the e-mail account (or more broadly with a whole registration process and using this account). Four items were used for this construct: Using CAPTCHAs increases the security of web-pages that I use. Using CAPTCHAs significantly increases the time of registering an e-mail account. When a CAPTCHA task is difficult I tend to resign from registering an e-mail account. CAPTCHAs are responsible for decreasing the number of SPAM messages I receive. The content of these items (especially the first and the fourth one) suggests that one of the key issues in the CAPTCHA context is to make a user aware of the role and significance of this kind of tasks in the process of registration, and further use of the email account. This fact seems to be neglected by researchers investigating CAPTCHAs, as well as by the CAPTCHA developers and e-mail account providers. When we check three very popular e-mail account providers ­ Gmail, Yahoo! and Hotmail, it appears that only on Hotmail a user might find information about what is this CAPTCHA picture for. But even there it is hidden under the `Help' button. The explanation is the following: We ask you to enter the letters and numbers in a picture when we need to make sure that a person, not an automated program, is using Windows Live. The characters are drawn so that it's possible for a person to recognize them, but very difficult for a program to. This helps us prevent automated programs from creating large numbers of accounts and sending spam. (https://signup.live.com/) Neither Gmail, nor Yahoo! provide their users with any kind of explanation. As for Gmail it is particularly strange, since nearly all form input fields are explained or illustrated with an example, but there is no information to come with the CAPTCHA picture. Future research over this finding will concentrate on identifying necessary pieces of information that should be provided with CAPTCHA tasks in order to increase Perceived Usefulness. Another interesting question for further research would be, if it makes a difference when a user knows that solving CAPTCHA tasks results in some extra productivity, like for example in the reCAPTCHA system case. The reCAPTCHA system helps to digitise books, since its tasks consist of words unrecognisable by OCR machines. Human users recognise these words while solving reCAPTCHA. For a detailed description see for example [3]. As it was mentioned in the text, also further works are needed to improve the questionnaire for measuring TAM constructs better in the field of CAPTCHA systems. Acknowledgements The first author was supported by the Foundation for Polish Science (grant number 76.2011). http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Bio-Algorithms and Med-Systems de Gruyter

Application of the Technology Acceptance Model to OCR-based CAPTCHA systems. Preliminary results

Download PDF
14 pages

Loading next page...
 
/lp/de-gruyter/application-of-the-technology-acceptance-model-to-ocr-based-captcha-Q0LhOiNp5k
Publisher
de Gruyter
Copyright
Copyright © 2012 by the
ISSN
1895-9091
eISSN
1896-530X
DOI
10.2478/bams-2012-0015
Publisher site
See Article on Publisher Site

Abstract

This research investigates the applicability of F. Davis' Technology Acceptance Model (TAM) to OCR-based CAPTCHAs involved in a free e-mail accounts registration process. The results provide preliminary evidence that the model may be a useful tool for research focused on acceptance of CAPTCHAs. The results are especially interesting when it comes to the concept of perceived usefulness and its direct relationship to the concept of attitude toward use. This relation indicates a new area of research for CAPTCHA developers interested in improving usability and accessibility of these systems. KEYWORDS: CAPTCHA, Turing test, technology acceptance model, usability, accessibility 1. Introduction 1.1. Aim of the study The aim of this study is to investigate the applicability of Fred Davis' Technology Acceptance Model (cf. [10]) to OCR-based CAPTCHAs involved in free e-mail accounts registration process. In our opinion the Technology Acceptance Model research might bring new and fruitful ideas into CAPTCHAs' development area, especially when it comes to the user-friendliness aspects of CAPTCHA systems. 1.2. What is a CAPTCHA? CAPTCHA stands for Completely Automated Turing Test To Tell Computers and Humans Apart. The very idea of CAPTCHA systems comes from the seminal paper of Moni Naor [20], while the name was coined by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford (who also developed first CAPTCHAs, cf. www.captcha.net). The main task of a CAPTCHA is to automatically differentiate bots (malicious programs) and human users in on-line services. There are many domains where such systems are needed, like for example, commenting blogs, Internet message boards, sending SMS/MMS messages via web-pages, on-line polls etc. However, the most important domain where providers want to be sure that they are dealing with human users are free e-mail accounts. CAPTCHAs are widely used by free e-mail accounts providers to stop SPAM messages from being sent from fictional accounts registered by bots. Users of Gmail, Yahoo! and many other popular services in this field have to solve a CAPTCHA task while registering an e-mail account. It is worth to stress that CAPTCHAs have become popular not only on large portals but it is also a kind of standard anti-SPAM solution on numerous small (even private) web-sites. The most characteristic properties of CAPTCHAs are (cf. [1], [2], [20]): test instances and their solutions are generated automatically; most humans can pass the test in a very short time; it is difficult to write a computer program that will be successful in passing the test in question; the underlying problem must be difficult for programs used currently as well as those which will be developed in the predictable future. Nowadays, only one CAPTCHA schema is popular, namely the so called OCR-based CAPTCHA (often referred to as `visual CAPTCHAs'). 1 In this type of CAPTCHA tasks consist in identifying the content of a distorted picture, e.g. recognising a text (most popular, used e.g. by Yahoo! or Hotmail), a common animal (Animal-PIX, a CAPTCHA which can be found at www.captcha.net), or a geometric figure. Visual CAPTCHAs exploit the ability of people to read text more reliably than optical character recognition (OCR) or other machine vision systems (cf. [7]). An example of this kind of CAPTCHA (used by Yahoo!) is presented in Figure 1. OCR-based CAPTCHAs are the most popular solution but not the only one. It is worth to mention other interesting approaches used in CAPTCHAs, like semantic based tasks: Egglue Semantic CAPTCHA (http://code.google.com/p/egglue/), SemCAPTCHA (http://semcaptcha.wikidot.com/) and SQUIGL-PIX (http://www.captcha.net); or even more problem-solving oriented ones, like: ESPPIX (http://www.captcha.net). Figure 1. An example of OCR-based CAPTCHA used by Yahoo! Unfortunately, the ability exploited by visual CAPTCHAs seems not to be reliable enough any more. There is a significant number of papers presenting attacks on well known visual CAPTCHA schemes. Let us mention only some of them. Mori and Malik [19] describe an attack on the EZ-Gimpy CAPTCHA used by Yahoo!, which enjoyed a success rate of 92%. In the more difficult case of Gimpy they passed the test 33% of the time. Moy et al. [17] achieved a 99% rate of solved tasks of EZ-Gimpy (and on Gimpy-r ­ a four letter version of GIMPY ­ 78%). Yan and Ahmad in [25] present a segmentation method that enabled them to break the Microsoft CAPTCHA scheme with an overall success rate of 60%. There are works like [25] or [14] where simple techniques were used to break many visual CAPTCHAs. This leads to a situation where CAPTCHA tasks become more and more complicated (motivated by the wish to provide more security). As a result many of tasks offered to users are not even human-readable. To give a simple example. We have performed a simple research where 10 CAPTCHA tasks (retrieved from one of the biggest Polish e-mail account providers) were presented to participants (N=10). Only the correctness of the solution was measured (time was unlimited). Tasks were presented on a web page, and the participants solved them using their home computers. As a result only 3 tasks were solved correctly by all ten participants. As a consequence of growing popularity combined with increasing difficulty of CAPTCHAs, problems of accessibility and usability of this technology become more and more important. There are papers describing research in this field, where some design rules are discussed to make CAPTCHAs more accessible (cf. [15]) and more usable ([27], [5], [6], [13], [24]). What is more, since CAPTCHAs are obligatory elements of the registration process, issues like users' attitudes, feelings and acceptance of this kind of tasks also should become an important element of CAPTCHA design and implementation in a given service. In our opinion there is a need for a tool that will concentrate on user's attitudes and level of acceptance for CAPTCHA systems. What is more, this tool should also allow for identifying factors responsible for users' attitudes. Davis' TAM seems to be a good candidate for such a tool. 1.3. Technology Acceptance Model TAM was proposed by Fred Davis in [10]. The main aim of this model is to provide a tool for assessing Information Technology (IT) acceptance among users. The model (as it is described in [11]) is presented in Figure 2. Figure 2. Technology Acceptance Model It consists of the following components: External Variables (system and/or users characteristics), Perceived Usefulness, Perceived Ease of Use, Attitude Towards Using, Behavioral Intention to Use, and Actual System Use. Perceived Usefulness (PU) is defined as "the degree to which a person believes that using a particular system would enhance his or her job performance" [9, p. 320]. Perceived Ease of Use (PEoU) is understood as "the degree to which a person believes that using a particular system would be free of effort" [10, p. 320]. Attitude Towards Using (ATU) is "the degree of evaluative effect that an individual associates with using the target system in his or her job" [11, p. 476]. Davis developed reliable scales for the model's components (cf. [29]). TAM is widely used to investigate and predict users' acceptance for various IT systems. A good example of TAM application is the research presented in [23]. The main aim of this research was to identify the determinants of intention to use the Firefox browser. This issue is very interesting in the light of the enormous popularity of Firefox among professional users on the one hand, and its belonging to the open source software family, on the other. What is more, as the authors point out, such finding might have theoretical and practical applications for other programs of this kind. Results of the research indicate that the principal determinant of intention to use Firefox is its perceived usefulness. In other words this TAM research allowed to identify that the most important feature of Firefox (from the professionals point of view) is the security of this browser (it is less likely to be targeted by crackers, more resistant to spyware etc. - cf. [23, p. 453]). TAM was also applied for other IT systems, like text editors and e-mail clients [10], knowledge management systems [17], multimedia learning environments [22], or wireless technology [28]. Also many extensions and revisions of TAM are proposed (see for example [29], [16], [21], [9]). 2. Research 2.1. Model and hypotheses For the research we have decided to use the technology acceptance model in the form presented in the previous section. However, one comment is in order. We used the following external variables in our research: age, gender, the number of e-mail accounts one is using and self assessed computer expertise. Our choice of the variables was based on previous TAM research and was aimed at gaining characteristics of the research group related to the CAPTCHA usage. We have not discovered any correlations with the External variables component in our research2. Consequently, we have refrained from using this component in the further model fit analysis (we use it only as the source of information about our group of participants). Our research hypotheses ­ drawn from previous TAM research ­ were: (H1) Positive relationships will be observed between PU and BI. (H2) Positive relationships will be observed between PEoU and ATU. (H3) Positive relationships will be observed between PU and ATU. (H4) Positive relationships will be observed between ATU and BI. (H5) Positive relationships will be observed between BI and AU. (H6) Positive relationships will be observed between PEoU and PU. 2.2. Development of the scale A 21 item survey was prepared for the research. Items and measurement scales were based on the previous TAM research ([12], [23], [29], [16]) and tailored to the specific area of the CAPTCHAs. For all of the TAM components seven-item scales were used. The TAM scales of the questionnaire are Age (r = 0.08, p=0.40), gender (t102=0.75, p=0.45, effect-size r=0.07), the number of e-email accounts one is using (r=0.13, p=0.17), and self assessed computer expertise (r=0.18, p=0.06). shown in Appendix. Four questions concerned the participants (sex, age, number of used e-mail accounts, self assessed computer expertise). The questionnaire was validated in the pilot study during which 112 responses were collected. After the pilot study we have decided to rewrite the negatively phrased items of the questionnaire in order to improve the reliability for questionnaire questions (which is suggested by F. Davis in [10, p. 327]). For discussion on the scale validation see section 3.1 below. 2.2.1. Procedure The questionnaire was powered by a GoogleDocs form and posted on-line with additional explanations about OCR-CAPTCHAs within a web-page prepared for the research (published on the Adam Mickiewicz University server). The link for the page was posted on popular websites (like Facebook) and on portals and forums about IT, psychology and cognitive science, where CAPTCHA issues were discussed. The questionnaire was accessible for 10 weeks. 116 fully filled questionnaires were collected. 2.2.2. Participants Responses were obtained from 116 participants (45 men, 71 women, aged from 17 to 43 (84% of the group were 20-27, average age M=23.8, SD=4.01). Participants were asked to self-assess their computer expertise. Detailed results are presented in Table 1 (scale was 1 ­ beginner to 7 ­ expert). Table 1. Self assessed computer expertise 1 (beginner) 2 3 4 5 6 7 (advanced) N=116 0 0 16 26 35 26 10 0% 0% 13.8% 22.4% 30.2% 22.4% 11.2% The research was focused on CAPTCHAs used by free e-mail accounts providers, therefore we also asked how many e-mail accounts did a subject have. The average result was 3 (3.0+/-1.7 for women and 3.1+/-1.8 for men, from 1­12 e-mail accounts). Men presented a higher (t102 = 2.7, p < 0.01, effect-size r=0.26) self assessed expertise in computer usage (M F = 4.7, SDF = 1.23; MM = 5.3, SDM = 1.08) than women. There was no significant relationship between the level of self assessed level of expertise in computer usage and the number of e-mail accounts in the group of men, and a very weak relationship in the group of women (Spearman's rho = 0.33, p < 0.01). We may say that our average subject was a young person with a high level of computer expertise, actively using e-mail accounts, aware of what CAPTCHA is and with experience in solving CAPTCHA tasks (gained in the process of registration of these e-mail accounts). 3. Results and discussion 3.1. Scale validation The reliability of the whole scale (based on all seventeen questions) is 0.84. The details for each construct are presented in Table 2. For the Actual Use construct, we observed low reliability. Table 2. Cronbach Alpha reliability coefficients (cf. [8]) Construct PU PEoU ATU BI AU scale Reliability 0.42 0.73 0.94 0.73 0.46 0.84 after item removal 0.29; 0.48; 0.41; 0.36; 0.27 0.68; 0.64; 0.76; 0.70; 0.65 0.93; 0.92; 0.92; 0.91 0.65; 0.67; 0.61 0.32; 0.10; 0.15 The reliability was slightly improved by the removal of one item, but it still remained low (in comparison to other constructs). This might be the effect of the used research scenario. Typically, the TAM research is conducted in groups of employees who are presented with a new piece of software and then use it for a period of time. After that time the TAM research is conducted (cf. e.g. [10], [17]). In our research, in order to get more general results, we asked about a system which is not connected to any organisational environment, and which was used by different participants with a different frequency and in different circumstances (there were some common characteristics though, since we have asked about CAPTCHA systems involved in e-mail accounts registration process). The other reason might be a bias in the very idea of this construct, i.e. self-reported usage ­ cf. [28, p. 6]. In our opinion the reliability of the AU construct in CAPTCHA research might be improved in two ways. One is to apply the procedure described above ­ i.e. to present the participants with an instance of a CAPTCHA and invoke them to solve tasks for a while in a defined environment and perform the TAM research afterwards. Another possible solution would be to measure the AU concept in an experimental setting. As for the research presented in this paper we have decided to drop the construct from the further analysis. In our opinion this brings no harm to our scope of interest, since the most interesting part of the TAM model in CAPTCHA context would be PEoU, PU, ATU and BI. As a consequence, we will not consider the 5th hypothesis stated in Section 2.1. 3.2. Model fit Let us remind that the research hypotheses 3 were that positive relationships will be observed between: (H1) Perceived Usefulness and Behavioral Intention to Use; (H2) Perceived Ease of Use and Attitude Towards Use; (H3) Perceived Usefulness and Attitude Towards Use; (H4) Attitude Towards Use and Behavioral Intention to Use; (H6) Perceived Ease of Use and Perceived Usefulness. Analysis results are shown in Figure 3 along with hypotheses numbers. Figure 3. Model fit results (Chi2125 = 228.7, p < .001; CFI = 0.91; NNFI = 0.89; RMSEA = 0.08, 90%CI = 0 - 0.07; SRMR = 0.07) Since a strong Perceived Usefulness and Perceived Ease of Use correlation was observed and confirmation analysis revealed a stronger effect of one of them, we have decided to perform a mediation analysis. We were interested if the relation between Perceived Ease of Use and Attitude Toward Hypothesis 5 was dropped for the reasons pointed out in the previous subsection. Use is mediated by Perceived Usefulness. Sobbel test was carried out and the results confirmed that Perceived Ease of Use influence on Attitude Towards Use is moderated with Perceived Usefulness (p < 0.001) ­ cf. Figure 4. Figure 4. Indirect effect Sobel's test = .29, Z = 3.95, SE = 0.07, p < 0.001 The summary of the research results is presented in Table 3. 3.3. Discussion and further research The results obtained are consistent with previous TAM research. The only exception is (H1), which was not confirmed ­ there was no direct relationship between Perceived Usefulness and Behavioral Intention to Use (such relationship was observed only for Attitude Towards Use and Behavioral Intention to Use). The reason for that might be the research scenario (as described in the Actual System Use concept case, cf. Section 3.1). If the Behavioral Intention to Use is measured long after the introduction of a system under research, there might be a bias in results ­ cf. [16, p. 7]. In our opinion also the properties of CAPTCHA might be important here. The research concerns a class of systems (although with consistent characteristics), not a single software token. Table 3. Summary of research hypothesis findings Hypothesis (H1) Positive relationships will be observed between PU and BI (H2) Positive relationships will be observed between PEoU and ATU (H3) Positive relationships will be observed between PU and ATU (H4) Positive relationships will be observed between ATU and BI (H6) Positive relationships will be observed between PEoU and PU Comments not confirmed confirmed, the effect is mediated by PU confirmed confirmed confirmed One of the most interesting findings is that the influence of Perceived Usefulness influence on Attitude Towards Use is stronger than that of Perceived Ease of Use on Attitude Towards Use. What is more, the influence of Perceived Ease of Use on Attitude Towards Use is mediated by Perceived Usefulness. This suggests that in oder to build positive Attitude Towards Use among users, developers should concentrate not only on making CAPTCHA easier or more human-readable, but also on factors constituting the Perceived Usefulness concept. Let us remind that Perceived Usefulness is defined as: "the degree to which a person believes that using a particular system would enhance his or her job performance" [10, p. 320]. In our research context, this might be seen as the degree to which a person believes that solving a CAPTCHA would enhance his or her performance with the e-mail account (or more broadly with a whole registration process and using this account). Four items were used for this construct: Using CAPTCHAs increases the security of web-pages that I use. Using CAPTCHAs significantly increases the time of registering an e-mail account. When a CAPTCHA task is difficult I tend to resign from registering an e-mail account. CAPTCHAs are responsible for decreasing the number of SPAM messages I receive. The content of these items (especially the first and the fourth one) suggests that one of the key issues in the CAPTCHA context is to make a user aware of the role and significance of this kind of tasks in the process of registration, and further use of the email account. This fact seems to be neglected by researchers investigating CAPTCHAs, as well as by the CAPTCHA developers and e-mail account providers. When we check three very popular e-mail account providers ­ Gmail, Yahoo! and Hotmail, it appears that only on Hotmail a user might find information about what is this CAPTCHA picture for. But even there it is hidden under the `Help' button. The explanation is the following: We ask you to enter the letters and numbers in a picture when we need to make sure that a person, not an automated program, is using Windows Live. The characters are drawn so that it's possible for a person to recognize them, but very difficult for a program to. This helps us prevent automated programs from creating large numbers of accounts and sending spam. (https://signup.live.com/) Neither Gmail, nor Yahoo! provide their users with any kind of explanation. As for Gmail it is particularly strange, since nearly all form input fields are explained or illustrated with an example, but there is no information to come with the CAPTCHA picture. Future research over this finding will concentrate on identifying necessary pieces of information that should be provided with CAPTCHA tasks in order to increase Perceived Usefulness. Another interesting question for further research would be, if it makes a difference when a user knows that solving CAPTCHA tasks results in some extra productivity, like for example in the reCAPTCHA system case. The reCAPTCHA system helps to digitise books, since its tasks consist of words unrecognisable by OCR machines. Human users recognise these words while solving reCAPTCHA. For a detailed description see for example [3]. As it was mentioned in the text, also further works are needed to improve the questionnaire for measuring TAM constructs better in the field of CAPTCHA systems. Acknowledgements The first author was supported by the Foundation for Polish Science (grant number 76.2011).

Journal

Bio-Algorithms and Med-Systems – de Gruyter

Published: Jan 1, 2012

There are no references for this article.