Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A comparative study of the effectiveness of sentiment tools and human coding in sarcasm detection

A comparative study of the effectiveness of sentiment tools and human coding in sarcasm detection Sarcasm is often used in everyday speech and writing and is prevalent in online contexts. The purpose of this paper is to investigate the analogy between sarcasm comments from sentiment tools and the human coder.Design/methodology/approachUsing the Verbal Irony Procedure, eight human coders were engaged to analyse comments collected from an online commercial page, and a dissimilarity analysis was conducted with sentiment tools. Three constants were tested, namely, polarity from sentiment tools, polarity rating by human coders; and sarcasm-level ratings by human coders.FindingsResults found an inconsistent ratio between these three constants. Sentiment tools used did not have the capability or reliability to detect the subtle, contextualized meanings of sarcasm statements that human coders could detect. Further research is required to refine the sentiment tools to enhance their sensitivity and capability.Practical implicationsWith these findings, it is recommended that further research and commercialization efforts be directed at improving current sentiment tools – for example, to incorporate sophisticated human sarcasm texts in their analytical systems. Sarcasm exists frequently in media, politics and human forms of communications in society. Therefore, more highly sophisticated sentiment tools with the abilities to detect human sarcasm would be vital in research and industry.Social implicationsThe findings suggest that presently, of the sentiment tools investigated, most are still unable to pick up subtle contexts within the text which can reverse or change the message that the writer intends to send to his/her receiver. Hence, the use of the relevant hashtags (e.g. #sarcasm; #irony) are of fundamental importance in detection tools. This would aid the evaluation of product reviews online for commercial usage.Originality/valueThe value of this study lies in its original, empirical findings on the inconsistencies between sentiment tools and human coders in sarcasm detection. The current study proves these inconsistencies are detected between human and sentiment tools in social media texts and points to the inadequacies of current sentiment tools. With these findings, it is recommended that further research and commercialization efforts be directed at improving current sentiment tools – to incorporate sophisticated human sarcasm texts in their analytical systems. The system can then be used as a reference for psychologists, media analysts, researchers and speech writers to detect cues in the inconsistencies in behaviour and language. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Systems and Information Technology Emerald Publishing

A comparative study of the effectiveness of sentiment tools and human coding in sarcasm detection

Download PDF
17 pages

Loading next page...
 
/lp/emerald-publishing/a-comparative-study-of-the-effectiveness-of-sentiment-tools-and-human-5VZdX4rwVW

References (49)

Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
1328-7265
DOI
10.1108/jsit-12-2017-0120
Publisher site
See Article on Publisher Site

Abstract

Sarcasm is often used in everyday speech and writing and is prevalent in online contexts. The purpose of this paper is to investigate the analogy between sarcasm comments from sentiment tools and the human coder.Design/methodology/approachUsing the Verbal Irony Procedure, eight human coders were engaged to analyse comments collected from an online commercial page, and a dissimilarity analysis was conducted with sentiment tools. Three constants were tested, namely, polarity from sentiment tools, polarity rating by human coders; and sarcasm-level ratings by human coders.FindingsResults found an inconsistent ratio between these three constants. Sentiment tools used did not have the capability or reliability to detect the subtle, contextualized meanings of sarcasm statements that human coders could detect. Further research is required to refine the sentiment tools to enhance their sensitivity and capability.Practical implicationsWith these findings, it is recommended that further research and commercialization efforts be directed at improving current sentiment tools – for example, to incorporate sophisticated human sarcasm texts in their analytical systems. Sarcasm exists frequently in media, politics and human forms of communications in society. Therefore, more highly sophisticated sentiment tools with the abilities to detect human sarcasm would be vital in research and industry.Social implicationsThe findings suggest that presently, of the sentiment tools investigated, most are still unable to pick up subtle contexts within the text which can reverse or change the message that the writer intends to send to his/her receiver. Hence, the use of the relevant hashtags (e.g. #sarcasm; #irony) are of fundamental importance in detection tools. This would aid the evaluation of product reviews online for commercial usage.Originality/valueThe value of this study lies in its original, empirical findings on the inconsistencies between sentiment tools and human coders in sarcasm detection. The current study proves these inconsistencies are detected between human and sentiment tools in social media texts and points to the inadequacies of current sentiment tools. With these findings, it is recommended that further research and commercialization efforts be directed at improving current sentiment tools – to incorporate sophisticated human sarcasm texts in their analytical systems. The system can then be used as a reference for psychologists, media analysts, researchers and speech writers to detect cues in the inconsistencies in behaviour and language.

Journal

Journal of Systems and Information Technology – Emerald Publishing

Published: Nov 14, 2018

Keywords: Detection; Social media; Linguistic; Sarcasm; Verbal irony

There are no references for this article.