Volume 3, issue 2 : Summer 2022. DOI: 10.1037/tmb0000067
The growing interest for emoticons and emojis has recently led to research examining their use and impact on various behaviors. As emoticons and emojis may lead to misinterpretations and misunderstandings between senders and recipients in online communication, it is necessary to examine whether emotions conveyed by these symbols are well recognized by individuals. In this perspective, a systematic review from 2001 to 2021 using the Preferred Reporting of Items for Systematic reviews and Meta-Analyses (PRISMA) method was conducted to determine which emoticons and emojis can help individuals to recognize emotions, and how the recognition of emotions based on emoticons and emojis is studied. A total of 501 articles were screened from three major databases in psychology, and 23 articles met the predefined inclusion criteria. The results suggest that the recognition of emotions should be examined before using emoticons in larger studies. They also revealed that the recognition varied according to the methods used to assess the valence of emoticons or to attribute a specific emotion to them (self-report, free expression, or categorization). Finally, a summary Table of the emotions conveyed by emoticons and emojis is proposed in this review.
Keywords: emoticons, emojis, recognition of emotions, PRISMA, emotion
Action Editor: Danielle S. McNamara was the action editor for this article.
Funding: This work was supported by the French Investment program for the future (Digital innovation for educational excellence action). This research is a part of the ACTIF-eFRAN project (Digital training, research, and animation area).
Disclosures: The authors have no competing interests to declare, and this research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Open Science Disclosures:
The data are available at https://osf.io/qj8nr/
The experimental materials are available at https://osf.io/qj8nr/
Correspondence concerning this article should be addressed to Anthony Cherbonnier, Department of Psychology, University Rennes, LP3C (Psychology Laboratory: Cognition, Behavior, Communication), F-35000 Rennes, France [email protected]
Emoticons and emojis are widely used in digital communication, but their visual nature may lead to misinterpretations within and across platforms (Miller et al., 2016; Tigwell & Flatla, 2016). Consequently, it is necessary to investigate whether emotions conveyed by such symbols are well recognized by individuals. In this perspective, a systematic literature review based on psychological research from 2001 to 2021 was conducted to determine whether emoticons and emojis can help individuals to recognize emotions, and how the recognition of emotions based on emoticons and emojis is studied.
Emoticons and emojis are abundantly present and extremely popular in social media and online communications (Chen et al., 2020; Ljubešić & Fišer, 2016; Lu et al., 2016; Oleszkiewicz, Karwowski, et al., 2017). Two formats of emoticons have been distinguished: (a) typographic emoticons, which are combinations of keyboard symbols, when read sideways, the combination represents a face, such as “:)” to express happiness or “:(” to express sadness; (b) graphic emoticons, which are graphic representations of a face, generally represented by a yellow circle symbolizing the outline of a face in which the eyes and the mouth are reproduced, such as 😄 for expressing happiness. Emojis refer to all the images that are coded in Unicode and which provide “a unique numerical identifier for each character (here the images) whatever the platform” (Unicode, 2017). In total, 3,521 emojis are available, divided into eight main categories representing various concepts such as Smileys & People, Animals & Nature, Foods & Drink, Flags, etc. Each platform or operating system can develop its own graphic charter that can be available on the emojipedia website (Emojipedia, n.d.). Graphic emoticons used in digital environments are coded in Unicode as part of the emoji family in the “Smileys & People” category in which 90 are referenced. In fact, the “Smileys & People” category is the most frequently used in digital environments (Cramer et al., 2016). Among the set of emojis proposed in this category, there is a low diversity used in digital environments (Al Rashdi, 2018; An et al., 2018; Oleszkiewicz, Karwowski, et al., 2017). In addition to these graphic emoticons, other emoticons have been specifically designed for research purposes (Cherbonnier & Michinov, 2021b; Toet et al., 2018).
The field of study of these emotional cues has grown in recent years, more specifically in the context of online communication. Generally defined as tools for transmitting emotions and feelings, they may be considered to have similar functions to the nonverbal cues present in face-to-face communication (Derks et al., 2008; Duan et al., 2018; Rezabek & Cochenour, 1998; Saini et al., 2018). Many studies have shown that emoticons have effects on a wide range of behaviors. For example, the presence of emoticons in emails tends to indicate that the sender has a positive attitude, and their use can modify the emotion the sender wishes to convey to the recipient (Skovholt et al., 2014). Emoticons are also used to provide an emotional tone to a message (Kaye et al., 2016), and to express humor and irony (Prada et al., 2018; Skovholt et al., 2014; Thompsen & Foulger, 1996; Wolf, 2000). In addition, emojis are contextualizing cues used to disambiguate unclear messages, for example, when messages may be interpreted as joking or teasing rather than being taken as serious or expressing hostility (Al Rashdi, 2018). Similarly, the repeated use of the same emoji or a variation of an emoji in the same message can express enthusiasm (Al Rashdi, 2018), and it can give a more cheerful tone to the message (Riordan, 2017). Emoticons seem to make messages more emotionally extreme, with the use of a negative graphic emoticon increasing the negativity of a message, while a positive graphic emoticon increases the positivity of a message (Luor et al., 2010). As nonverbal cues, emoticons can affect the perception of a message by increasing the richness of the information conveyed (Hsieh & Tseng, 2017; Huang et al., 2008), for example, in expressing humor and conviviality without modifying the content of the message (Ernst & Huschens, 2018), or influencing the consideration of advice (Duan et al., 2018).
Overall, studies suggest that emoticons and emojis are strongly related to the expression of emotion in online communications, and they have been essentially conducted to examine their use and impact on human behaviors. Although these nonverbal cues are used to express emotions in online communications, paradoxically, we ignore whether they are well recognized by individuals. As some studies revealed that the visual nature of emoticons and emojis may lead to misinterpretations (Miller et al., 2016), it is important to examine the quality of the recognition of emotions conveyed by emoticons and emojis. Similarly, identifying emotional meanings of emoticons and emojis listed in a database (as in facial expressions of emotions research) would provide a useful instrument for further research in the field. Indeed, many studies have examined how facial expressions in face-to-face communication are used to convey emotions (Ekman, 2003), and the recognition of emotions conveyed by facial expressions (Calvo & Nummenmaa, 2016; Elfenbein & Ambady, 2002; Ko, 2018; Russell, 1994). Several databases have been developed to specify the emotions conveyed by photographs of facial expressions such as FACS (Facial Action Coding System, Ekman et al., 2002), RaFD (Radboud Faces Database, Langner et al., 2010), KDEF (Karolinska Directed Emotional Faces, Goeleven et al., 2008), and FACES (Ebner et al., 2010). By contrast, and as far as we know, the recognition of emotions based on emoticons or emojis referenced on the two main platforms, Unicode or emojipedia, has not been intensively examined in the literature. Although the platforms provide databases allowing access to all available emojis, unlike the facial expression databases mentioned above, emojis are not associated with a specific emotion, and their creation is not detailed since each platform has its own rendering. For each emoji, the database provides a name, a code, and the different graphic charter available. In recent research, inspired by the FACS (Ekman et al., 2002), Fugate and Franco (2021) created an equivalent “emoji FACS” system attributing action units (AUs) to 31 graphic emoticons. They found a difference in AU count across the three main platforms (Apple, Google Android, and Samsung), and between different versions of graphic emoticons on a given platform. In the same vein, the present study aimed to provide another instrument based on a systematic literature review to help researchers to identify the emotional meaning of emoticons and emojis.
In summary, literature reviews have largely focused on the role of emoticons in computer-mediated communication (for review, see Derks et al., 2008; Jibril & Abdullah, 2013) and their function and impact in such communications (for review, see Aldunate et al., 2018; Chen et al., 2020; Dunlap et al., 2016; Tang & Hew, 2019). By contrast, the present study specifically focuses on the recognition of emotions conveyed by emoticons and emojis, where the systematic review of research aims to answer the following questions:
Research Question 1: How is the recognition of emotions from emoticons and emojis studied?
Research Question 2: What emotions are conveyed by emoticons and emojis?
This systematic review was based on the principle of the PRISMA method (Preferred Reporting of Items for Systematic reviews and Meta-Analyses). This method allows reproducibility as well as transparency of the research by relying on a systematic checklist composed of 27 criteria as well as a flow diagram (Moher et al., 2015; Page et al., 2021).
The article search was performed using three major psychology databases: PsycInfo, PsycArticles, and Psychology and Behavior Science Collection. These databases were chosen because they cover our fields of interest (emotion recognition and emoticons), and their similar structure allows us to use the same search function. In order to widen the search as much as possible, several synonyms relating to recognition were used, such as identification, detection, or discrimination. As a result, the following Boolean function was used for the search, which was run on all the content of articles: “(emoticon OR emoji) AND (recognition OR perception OR valence OR identification OR discern OR detect OR descry OR distinguish).” The search was conducted in March 2021.
This function was supplemented by a number of criteria including “peer-reviewed.” First, the three expanders were used to broaden the scope of the search: apply related words (“include synonyms and plurals of the terms”), search within documents (“search for the keywords within the full text of articles, as well as abstract and citation information”), apply equivalent subjects (“utilize mapped vocabulary terms to add precision to unqualified keyword searches”). Then testing, editorial, dissertation, and book reviews were excluded. To be included, articles had to be written in English. To be examined, each article had to correspond to the inclusion and exclusion criteria presented in Table 1.
Inclusion criteria |
---|
1. The research work is related to emoticons or emojis |
2. The research work has included recognition of emotions as a measure |
3. The research work has described a method to study recognition of emotion |
4. Research articles have been published between 2001 and 2021 |
5. Research articles are written in English |
6. Research articles have been published after being submitted to a peer review process |
7. The full version of the publication is available through the subscription of our institution |
Exclusion criteria |
1. The research work is not related to emoticons or emojis |
2. The research work has not included recognition of emotions as a measure |
3. The research work has not described a method to study recognition of emotion |
4. Research articles have not been published between 2001 and 2021 |
5. Research articles are not written in English |
6. Research articles have been published without a peer review process |
7. The full version of the publication is not available through the subscription of our institution |
The article search identified 501 articles, including 18 duplicates. The abstracts of these 483 articles were read resulting in the exclusion of 395 references that did not mention emoticon or emoji (first exclusion criteria). The remaining 88 articles were uploaded in full text. Sixty-nine were deleted because they lacked a study with a detailed protocol on the emotional recognition of the emoticons or emojis used (exclusion criteria 2 and 3). In addition to the 19 articles selected for analysis, four other articles were added to this research. Two articles were published in the field of psychology, and were found in a journal during background reading on the subject and in RSS feeds of journals. The two others were referred to by Cherbonnier and Michinov (2021b). The procedure is presented through a PRISMA flow chart in Figure 1 and the study selection file is available online on our OSF page https://osf.io/qj8nr/ (Cherbonnier & Michinov, 2021a).
The 23 studies analyzing the recognition of emotions based on emoticons and emojis are presented in Appendix A. After analyzing the 23 articles, they could be separated into two categories of studies (see Table 2): (a) complete studies (n = 13), whose objective was to analyze the recognition of emoticons and emojis; (b) pretests (n = 10), in which the recognition of emoticons and emojis was studied prior to their use in a larger study, for example, as emotional feedback, or to assess emotional associations with other products (e.g., food). In each category of study, there were two distinctive evaluation tasks, either a task measuring valence of emoticons and emojis (n = 10), generally using a positive/negative Likert scale, or a task consisting of recognizing specific emotions (n = 13). For the latter, two distinct methods were used: (a) freely expressing emotions conveyed by emoticons or emojis (n = 3), (b) assigning an emotion conveyed by emoticons or emojis from a predetermined list of emotions (n = 10).
Number of Articles According to Their Category (Pretest or Complete Study), Evaluation Tasks (Recognition or Valence of Emotions), and Type of Clues | ||||||
Type of clues | Pretest | Complete study | ||||
---|---|---|---|---|---|---|
Recognition | Valence | Both | Recognition | Valence | Both | |
Typographic emoticons | 1 | 1 | 0 | 0 | 0 | 2 |
Graphic emoticons | 2 | 4 | 0 | 6 | 4 | 16 |
Typographic and graphic emoticons | 0 | 0 | 1 | 2 | 0 | 3 |
Emojis | 0 | 1 | 0 | 0 | 0 | 1 |
Emoticons and emojis | 0 | 0 | 0 | 1 | 0 | 1 |
Total | 3 | 6 | 1 | 9 | 4 | 23 |
The selected articles were quite recent, and 86.9% were published after 2017. This result shows a recent and growing interest in defining the emotions or valence conveyed by emoticons and emojis. Moreover, both categories have been studied similarly over the period.
As shown in Table 2, the results showed that graphic emoticons were the main focus of research. In fact, typographic emoticons have been studied in only two pretests (see Table 3), one about the emotions conveyed (Walther & D’Addario, 2001) and the other about their valence (Aldunate et al., 2018). On the one hand, Walther and D’Addario (2001) studied the emotional recognition of three typographic emoticons “:),” “:(,” “;).” Participants had to attribute one of the three emoticons to each of twelve emotions, for example, happy, honest, and angry. As shown in Table 3, nine emotions were associated more than 84% with a single typographic emoticon. On the other hand, Aldunate et al. (2018) pretested the valence (positive/negative) of 30 typographic emoticons using a Likert scale, and confirmed the choice of the 15 positive and 15 negative emoticons for their study. It consisted of using the pretested emoticons to determine whether they could disambiguate a textual message based on the inference of the sender’s mood.
Summary of Results for Typographical Emoticons | ||
Typographic emoticons | Aldunate et al. (2018) | Walther and D’Addario (2001) |
---|---|---|
:) :-) =) =] =-) :] XD =-D :D :-D 8D 8-D 8-) =D ;-) =( =[ =-( :[ :S =-C :C :-C :-[ :-/ =/ 8-( 8( :( :-( | Positive | Happy (98.3%) |
Honest (84.8%) | ||
Positive | X | |
Positive | Seductive (85.4%) | |
Secretive (88.7%) | ||
Sarcastic (84.1%) | ||
Negative | X | |
Negative | Sadness (98.0%) | |
Anger (88.1%) | ||
Disgust (88.1%) | ||
Fear (85.4%) |
The valence of graphic emoticons has been measured using two different methods: (a) self-report measures on a Likert scale or a Likert-type scale (n = 5) and (b) categorization, for which emoticons are associated with one of the proposed categories (n = 3). Clark-Gordon et al. (2018), Huang et al. (2020), and Luor et al. (2010) pretested the valence of emoticons/emojis to select the best candidate(s) to express a positive emotion, a negative emotion, and a neutral emotion. Gallo et al. (2017) showed that among 50 graphic emoticons only five were unambiguously categorized as positive, four as negative, and only one was considered as neutral (see Table 4). Taken together, only the emoticon 😀 was evaluated as positive in three studies, and the emoticon 😍 in two studies. None of the studied emoticons expressing a negative emotion were common to the four studies.
Summary of Results for the Valence of Graphic Emoticons | ||
Note. X = not present in the study |
In another study, Aluja et al. (2020), used a factorial analysis of the ratings of 30 graphic emoticons selected from three different platforms (and evaluated with a self-report measure), revealed that 15 of them were rated as pleasant and 15 as unpleasant. Moreover, based on an analysis of the valence of 70 graphic emoticons, Rodrigues et al. (2018) showed that 24 emoticons were considered as positive, 25 as negative, and 21 as neutral. In addition, with the exception of positive graphic emoticons, women rated them more negatively than men (Jones et al., 2020). In addition, Wang et al. (2014) identified eight emoticons, four of them represent “liking” and the other four “disliking.” Then, these eight emoticons were assessed the extent to which they represented these feelings, and to select two emoticons. Their objective was to examine the different effects of two types of emotion on the perception of feedback: one expressing like 🤤 and the other dislike 😡.
In an intercultural study, Zhong et al. (2019) examined the valence differences between two cultures (U.S. vs. Chinese) of six graphic emoticons using a self-report scale. With the exception of surprise (😳), results showed that the two cultures rated each emoticon with the same valence. Indeed, 🙂 (smile) and 😊 (blush) were evaluated positively, while 😒 (smirk), ☹️(frown), and 😠 (angry) were evaluated negatively. In addition, the American participants rated the valence of emoticons more extremely than the Chinese.
The emotions conveyed by emoticons and emojis were identified using two methods. The first involved associating an emotion with a given emoticon or emoji; the second consisted in freely expressing the emotion conveyed by an emoticon or emoji. In their studies, Annamalai and Abdul Salam (2017), Franco and Fugate (2020), Jaeger and Ares (2017), and Rodrigues et al. (2018) presented the attributed emotions for all emoticons used as well as the version of a graphic chart. The full results of these four studies are summarized in a table available online on our OSF page, https://osf.io/qj8nr/ (Cherbonnier & Michinov, 2021a). It revealed that most of the emoticons conveyed different emotions, and it is not easy to reach a consensus when we have to associate a single emotion with a single emoticon. However, some emoticons conveyed the same emotion throughout the different studies when different methods were used, such as “angry face” (😠) for anger, or “Smiling face with heart-eyes” (😍) for love. For example, Jaeger and Ares (2017) showed that only 15 graphic emoticons were more than 50% associated with a single emotion, while Rodrigues et al. (2018) showed among 128 graphic emoticons (58 Android and 70 iOS), only two had an interpretation score of 100% 😠 (anger), 🤒 (Illness), 35 had an interpretation score from 75% to 100%; 59 had an interpretation score from 50% to 75%, and 32 had an interpretation score below 50%.
Similar results were also found in two pretests (see Table 5). Fane et al. (2018) showed that five graphic emoticons were associated with several emotions when participants freely expressed them. Moreover, the results of Gantiva et al. (2020) led to 24 of 36 graphic emoticons being retained with a recognition rate greater than 80%, eight each expressing happiness, anger, and a neutral emotion. In another study (Weiß et al., 2020), each of the 13 graphic emoticons, selected among 78, had to be evaluated with 18 emotion terms such as amused, awed, fearful, etc. Results showed that age had an effect on the evaluation of some emoticons: the disgust emoticon (😖) was evaluated as “frustration” by older adults and the fearful emoticon (😧) was evaluated as “surprise” by younger adults.
Summary of Results About Specific Emotions Convey by Graphic Emoticons | ||
Note. X = Graphic emoticons not present in the study. | ||
Four studies have focused on recognizing the emotions conveyed by both graphic and/or typographical emoticons by comparing them to photographs of facial expressions according to different criteria: (a) age and gender (Oleszkiewicz, Frackowiak, et al., 2017), (b) gender and intensity (Cherbonnier & Michinov, 2021b), (c) culture (Takahashi et al., 2017), and (d) color (Ikeda, 2020).
Using a list of the six basic emotions (Ekman, 1992a, 1992b), it was found that 4- to 8-year-old children recognized similarly the emotions conveyed by emoticons from Facebook and photographs of facial expressions (Oleszkiewicz, Frackowiak, et al., 2017). It was also observed that typographic emoticons were significantly less recognized than photographs of facial expressions and graphic emoticons. Additionally, fear (😧) and disgust (😖) were the least recognized emotions for the typographic and graphic emoticons. An in-depth analysis revealed that age had an influence on the recognition of emotions but only among boys, with the older boys having better recognition scores. Moreover, the gender of the participants had an influence only among the youngest (4–6 years), with girls recognizing emotions better than boys.
In another study, Cherbonnier and Michinov (2021b), designed and pretested a series of emoticons to convey six basic emotions: happiness (🙂), sadness (😟), anger (😠), fear (😨), disgust (😖), and surprise (😲).1 Then, they compared the recognition of emotions with other modes of emotional expression, such as facial expressions and emoticons from two main platforms (Facebook and iOS). Results showed that the specifically designed emoticons are recognized better, and more intensely, than photographs of facial expressions (female and male), and also than Facebook and iOS emoticons. No gender effect was observed on recognition of emotions whatever the mode of emotional expression.
In an intercultural study, Takahashi et al. (2017) examined differences between Cameroonians, Tanzanians, and Japanese in the recognition of three emotions (happy, sad, and neutral) expressed either by emoticons in three distinct formats (Western “:-),” Japanese “(^_^),” and graphic ☺️) or by photographs of facial expressions. Their results revealed that the Japanese recognized better the emotions conveyed by emoticons independently of the format, but such a positive effect was not observed on photographs. Moreover, this study suggested that familiarity with digital environments may have a crucial role in the recognition of emotions based on emoticons because Cameroonians and Tanzanians living in cities showed better recognition.
Finally, the background color may have an impact on the recognition of emotions (happiness, anger, and sadness) expressed by photographs of facial expressions and graphic emoticons (Ikeda, 2020). Based on prior research on emotion-color associations, in a first study Ikeda (2020) chose to associate red as the background color for anger, green for happiness, and blue for sadness and, in a second study, blue for sadness and yellow for happiness. The results showed that red (anger) and green (happiness) influenced the recognition of the emotions conveyed by the photographs, but not by the graphic emoticons. On the other hand, yellow (happiness) and blue (sadness) had a greater influence on the recognition of graphic emoticons.
In order to create a system to detect the emotional level of a sentence, Asghar et al. (2017) studied both the valence (positive/negative) of 450 emoticons (graphic and typographic) and the emotions they conveyed. They asked five coders to give a score to each emoticon (−1; −0.5; 0; +0.5; +1) and associate them with one of the eight emotions proposed (fear, anger, happiness, disgust, surprise, sadness, embarrassment, reactive). The results made it possible to create a lexicon of emoticons and to improve the analysis of emotion at the sentence level (the accuracy of analysis improved from 78.96% to 83.45%).
From another perspective, Toet et al. (2018) aimed to design a new tool to assess affective associations with food based on a grid of graphic emoticons called the EmojiGrid. With this objective, 17 graphic emoticons were developed to have a neutral expression in the center of the grid and five others on each axis. A horizontal axis, representing valence, varies from disliking to liking, and a vertical axis representing arousal, varies from low to high intensity. Graphic emoticons were created from a neutral, yellow-colored graphic emoticon on which the mouth, eyes, and eyebrows differed. In order to validate the graphic emoticons and their positions within the grid, a series of three methods was carried out in which the participants had to: (a) evaluate the valence and the arousal of the 17 graphic emoticons on a 5-point Self-Assessment Manikin scale (Bradley & Lang, 1994); (b) classify two sets of five graphic emoticons according to their valence as well as two other sets of five graphic emoticons according to their arousal; (c) place all the graphic emoticons on the grid.
This study aimed to understand how the recognition of emotions from emoticons and emojis has been studied, and to determine what emotions are conveyed by emoticons and emojis. A systematic literature review on this topic is crucial because emoticons and emojis may lead to misinterpretations and misunderstandings between senders and recipients in online communication. We also provide a summary table (https://osf.io/qj8nr/) that can be used as a useful instrument for research in the field, helping researchers and designers to choose emoticons and emojis that convey specific emotions (Cherbonnier & Michinov, 2021a).
Our review of the literature revealed that most of the articles reviewed in this review were published between 2017 and 2020, and very few before 2017, showing that the recognition of emotions from emoticons and emojis has grown in recent years. The recognition of emotions has predominantly examined using graphic emoticons, and much less frequently from typographic emoticons. A possible explanation for focusing on graphic emoticons is that they are standardized, not editable by users unlike typographic emoticons and they are the category of emojis used the most in online environments (Cramer et al., 2016). Moreover, there is competition between the use of emojis and typographic emoticons because people who use emojis (and graphic emoticons) more tend to use typographic emoticons less often (Pavalanathan & Eisenstein, 2016). In fact, emojis (and graphic emoticons) are generally preferred to typographic emoticons (Prada et al., 2018) and the former are more expressive and visual than the latter (Pavalanathan & Eisenstein, 2016; Rodrigues et al., 2018).
An important aspect of the research is to ensure that the emoticons used in the studies convey the desired emotions. This is confirmed as 10 of the 23 articles of the present literature review refer to pretests aiming to identify the recognition of emotions based on emoticons for a subsequent usage in complete studies. Thus, it seems that when researchers want to use emoticons in their research to convey emotions, it is necessary to verify whether the participants have correctly perceived the emotions, as is the case for research on emotional facial expressions.
The literature review also showed that the recognition of emotions was studied using various tasks. Some studies tried to assess the valence of emoticons while others asked participants to attribute a specific emotion to emoticons. In both tasks, different methods were used. The method to assess the valence of emoticons privileged self-report measures using Likert scales, while the method aiming to attribute an emotion to an emoticon mainly used free expression or a choice in a list of emotions. The variety of methods used to attribute a specific emotion to an emoticon is interesting, and it reveals a diversity of studies to recognize emotion. The differences in results between methods (list of emotions vs. free expression of emotions) may be due to the fact that emoticons may convey several emotions. Indeed, in the studies using a list of emotions, an emotion conveyed by a specific emoticon may not be in the list from which the participants must choose. This result is not surprising, and in line with the study by Betz et al. (2019) which recently showed a difference between free-label expression and forced-choice in a list of emotions on 12 “Finch” emojis.2 When using a list, participants tend to attribute the correct emotion more easily than when freely expressing the emotions conveyed by emojis (Betz et al., 2019). Another possible explanation about differences between methods to attributed emotion is that when participants express freely the emotion conveyed by emoticons or emojis, they sometimes do not generate their emotion with a specific word, but instead use a description of a situation, a behavior or word reflecting a nonmental state. Research on the recognition of emotions tries to provide a clearer picture of the emotions that are being conveyed by emoticons. It appears to be relatively easy to assess the valence of emoticons and emojis, but it is more difficult to attribute a specific emotion to them. Indeed, when participants had to freely express the emotions conveyed by emoticons a consensus was rarely reached. In fact, the results showed that for some emoticons, the emotion conveyed was identical across the different studies and platforms as the emoticon “face with open mouth” (😮) to communicate surprise, “angry face” (😠) to communicate anger, or “disappointed face” (😞) to communicate sadness. On the contrary, other emoticons conveyed different emotions such as the “persevering face” (😣) that can express sadness, disgust, anger or fear, or the “grimacing face” (😬) that can be used to communicate fear or anger. The differences in emotions conveyed by such emoticons are not surprising because all emoticons are not necessarily designed to convey a specific emotion, except in Cherbonnier and Michinov (2021b) study. Indeed, no information was given about the design of graphic emoticons proposed by the different platforms, for which a simple description is generally provided. For example, the “relieved face” (😌) was described as “A yellow face with soft, closed eyes, raised eyebrows, and a slight smile” (https://emojipedia.org/relieved-face/). Furthermore, Emojipedia (n.d.) also provides a description of emojis meanings showing that graphic emoticons may convey several emotions.
The recognition of emotions conveyed by emoticons has been studied without any online context, in each study emoticons were presented in isolation, such as a picture or photograph, and participants were asked to assign an emotion according to a method (free expression of emotion or list of emotions). Nevertheless, emoticons are generally used in online communication to express emotions or feeling (Derks et al., 2008; Garrison et al., 2011; Kaye et al., 2017). Thus, the recognition of emoticons and emojis in a specific context may be influenced by the representations that users have constructed from their prior use. Future research would benefit from extending these findings by studying the recognition of emoticons and emojis in a natural context of use, notably in online communications.
Research has begun to take into account some factors influencing the recognition of emotions conveyed by graphic emoticons such as gender or age, but the studies comparing the recognition of emotions conveyed by emoticons to other modes of emotional expression remain relatively scarce to date.
One of the main merits of this literature review has been to create an instrument summarizing the emotions conveyed by emoticons on the main platforms in a table. This instrument, available on our OSF page (https://osf.io/qj8nr/), can be used for future research to help researchers and designers choose emoticons according to the emotions they would like to induce in their study (Cherbonnier & Michinov, 2021a).
Among the main limitations of the present literature review is that the search for articles focused on articles only in psychology databases. Yet, emoticons and emojis have also been studied in other disciplines such as communication, linguistics, sociology, education, and media studies, but their analysis is beyond the scope of the present literature review. It is also possible that the review has omitted the articles in which recognition was studied, but not mentioned in the abstract or keywords.
The literature about the recognition of emotions based on emoticons and emojis may have practical implications in different fields such as e-marketing, communication with individuals with autism spectrum disorders, or communication in videoconferencing. For example, using the “right” emoticon (i.e., conveying an emotion that people recognized correctly) may have strong effects on consumer behaviors, notably on e-business. In this domain, an emoticon can reinforce the usefulness of advice and reduce the intention to book in a hotel (Manganari & Dimara, 2017), or increase the desire to buy a product and encourage a purchase on the web (Saini et al., 2018). Another useful application of emoticons can be helping to communicate with autism spectrum disorders individuals in the classroom or, more broadly, in their social life. As these individuals have some difficulties to communicate with others, and specifically to recognize emotions based on facial expressions, the emoticons may be used as a kind of “emotional prosthesis” helping them to recognize better emotions of their peers and teachers in inclusive classrooms. Finally, another practical use can be found in communicating online using videoconferencing systems. As suggested by Cherbonnier and Michinov (2021b), emoticons may offer the users of videoconferencing systems such as Zoom or Teams, the possibility to convey their emotions without having to activate their webcam showing their faces. Of course, many other applications can be found, but one of the prerequisites is that emotions conveyed by an emoticon are recognized by other people.
This review highlights the growing interest in the emotions conveyed by emoticons and emojis and the way they have been studied in an emerging research field in psychological and behavioral sciences. It also reveals a variety of methods used to capture the emotions (self-report, free expression, or categorization), indicating that a consensus about emotions conveyed by emoticons has not yet been reached. Beyond the recognition of emotions from facial expressions, we hope that the present review may contribute to opening a new avenue of study in an emerging research field concerning the recognition of emotions conveyed by emoticons and emojis.
Included studies | ||||||
Study | Type | Category | Task | Method | Participant | Country |
---|---|---|---|---|---|---|
1. | Typographic | Pretest | Valence | Likert negative/positive | 75 Students | Chile |
2. Aluja et al. (2020) | Graphic | Study | Valence | Likert unpleasant/pleasant | 190 Students | Spain (not mentioned) |
3. Annamalai and Abdul Salam (2017) | Graphic | Study | Emotions | Free expression | 210 Students | Malaysia |
4. Asghar et al. (2017) | Emoticons | Pretest | Emotions and valence Emotions: categories | Valence: score | 5 participants | Pakistan (not mentioned) |
5. Cherbonnier and Michinov (2021b) | Graphic | Study | Emotions | List of 14 emotions | 351 students 606 online participants | France |
6. Clark-Gordon et al. (2018) | Emojis | Pretest | Valence | Likert negative/positive | 28 Students | USA |
7. Fane et al. (2018) | Graphic | Pretest | Emotions | Free expression | 78 children 3–5 years old | Australia |
8. Franco and Fugate (2020) | Study | Emotions | List of 10 emotions | 228 online participants | USA | |
9. Gallo et al. (2017) | Graphic | Study | Valence | Categories (positive, negative and neutral) | 17 children 8–11 years old | USA |
10. Gantiva et al. (2020) | Graphic | Pretest | Emotions | List of 6 emotions and neutral | 30 students | Colombia (not mentioned) |
11. Huang et al. (2020) | Graphic | Pretest | Valence | Likert negative/positive | 30 online participants | USA |
12. Ikeda (2020) | Graphic | Study | Emotions | Choice between 2 emotions | 47 students 35 students | Japan |
13. Jaeger and Ares (2017) | Graphic | Study | Emotions | List of 39 emotions (CATA) | 1,084 online participants | China |
14. Jones et al. (2020) | Graphic | Study | Valence | Likert negative/positive | 299 Students | USA |
15. Luor et al. (2010) | Graphic | Pretest | Valence | Selection (positive, neutral, negative) | 32 employees | Taiwan |
16. Oleszkiewicz, Frackowiak, et al. (2017) | Emoticons | Study | Emotions | List of 6 basic emotions | 68 children (4–8 years old) | Poland |
17. Rodrigues et al. (2018) | Emoticons andemojis | Study | Emotions | Free expression of emotions or senses | 505 online participants | Portugal |
18. Takahashi et al. (2017) | Emoticons | Study | Emotions | Analog scale from joy to sadness | 32 Cameroonians 37 Tanzanians 24 Japanese | Cameroun |
Tanzania | ||||||
Japan | ||||||
19. Toet et al. (2018) | Graphic | Pretest | Valence | (1) Creation (2) Likert (3) Layout (4) Placement | 48 students: (2) 28 (3) 10 (4) 10 | Germany |
20. Walther D’Addario (2001) | Typographic | Pretest | Emotions | Associate with a list of emotions | 226 Students | Not mentioned |
21. Wang et al. (2014) | Graphic | Pretest | Valence | Likert like/dislike | 28 Students | Not mentioned |
22. Weiß et al. (2020) | Graphic | Study | Emotions | Likert | 170 online participants | Germany |
23. Zhong et al. (2019) | Graphic | Study | Valence | Likert negative/positive | 102 Americans 65 Chinese students | USA and China |