Analysis of emotion recognition using facial expressions, speech and multimodal information

Name: Work Video:
Duration: 3 min 30 s
Carlos Busso; Zhigang Deng; Serdar Yıldırım; Murtaza Bulut; Chul Min Lee; Abe Kazemzadeh; Sungbok Lee; Ulrich Neumann; Shrikanth Narayanan
{'id': 'https://openalex.org/W2143350951', 'doi': 'https://doi.org/10.1145/1027933.1027968', 'title': 'Analysis of emotion recognition using facial expressions, speech and multimodal information', 'display_name': 'Analysis of emotion recognition using facial expressions, speech and multimodal information', 'publication_year': 2004, 'publication_date': '2004-10-13', 'ids': {'openalex': 'https://openalex.org/W2143350951', 'doi': 'https://doi.org/10.1145/1027933.1027968', 'mag': '2143350951'}, 'language': 'en', 'primary_location': {'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/1027933.1027968', 'pdf_url': None, 'source': None, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': False, 'oa_status': 'closed', 'oa_url': None, 'any_repository_has_fulltext': False}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5040793194', 'display_name': 'Carlos Busso', 'orcid': 'https://orcid.org/0000-0002-4075-4072'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Carlos Busso', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5107984555', 'display_name': 'Zhigang Deng', 'orcid': 'https://orcid.org/0000-0003-2571-5865'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Zhigang Deng', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5084041766', 'display_name': 'Serdar Yıldırım', 'orcid': 'https://orcid.org/0000-0003-3151-9916'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Serdar Yildirim', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5109250685', 'display_name': 'Murtaza Bulut', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Murtaza Bulut', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5103682078', 'display_name': 'Chul Min Lee', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Chul Min Lee', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5019787390', 'display_name': 'Abe Kazemzadeh', 'orcid': 'https://orcid.org/0000-0002-1851-294X'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Abe Kazemzadeh', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102118359', 'display_name': 'Sungbok Lee', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Sungbok Lee', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5082235583', 'display_name': 'Ulrich Neumann', 'orcid': 'https://orcid.org/0000-0001-8977-7112'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Ulrich Neumann', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5010028928', 'display_name': 'Shrikanth Narayanan', 'orcid': 'https://orcid.org/0000-0002-1052-6204'}, 'institutions': [{'id': 'https://openalex.org/I1174212', 'display_name': 'University of Southern California', 'ror': 'https://ror.org/03taz7m60', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I1174212']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Shrikanth Narayanan', 'raw_affiliation_strings': ['University of Southern California , Los Angeles#TAB#'], 'affiliations': [{'raw_affiliation_string': 'University of Southern California , Los Angeles#TAB#', 'institution_ids': ['https://openalex.org/I1174212']}]}], 'institution_assertions': [], 'countries_distinct_count': 1, 'institutions_distinct_count': 1, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 6.829, 'has_fulltext': True, 'fulltext_origin': 'ngrams', 'cited_by_count': 804, 'citation_normalized_percentile': {'value': 0.982278, 'is_in_top_1_percent': False, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 99, 'max': 100}, 'biblio': {'volume': None, 'issue': None, 'first_page': None, 'last_page': None}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10057', 'display_name': 'Face and Expression Recognition', 'score': 0.9987, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10057', 'display_name': 'Face and Expression Recognition', 'score': 0.9987, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10667', 'display_name': 'Emotion and Mood Recognition', 'score': 0.9985, 'subfield': {'id': 'https://openalex.org/subfields/3205', 'display_name': 'Experimental and Cognitive Psychology'}, 'field': {'id': 'https://openalex.org/fields/32', 'display_name': 'Psychology'}, 'domain': {'id': 'https://openalex.org/domains/2', 'display_name': 'Social Sciences'}}, {'id': 'https://openalex.org/T10860', 'display_name': 'Speech and Audio Processing', 'score': 0.9977, 'subfield': {'id': 'https://openalex.org/subfields/1711', 'display_name': 'Signal Processing'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/sadness', 'display_name': 'Sadness', 'score': 0.8740537}, {'id': 'https://openalex.org/keywords/modalities', 'display_name': 'Modalities', 'score': 0.766163}, {'id': 'https://openalex.org/keywords/robustness', 'display_name': 'Robustness', 'score': 0.6814037}, {'id': 'https://openalex.org/keywords/affective-computing', 'display_name': 'Affective Computing', 'score': 0.42520896}], 'concepts': [{'id': 'https://openalex.org/C2779812673', 'wikidata': 'https://www.wikidata.org/wiki/Q169251', 'display_name': 'Sadness', 'level': 3, 'score': 0.8740537}, {'id': 'https://openalex.org/C2779903281', 'wikidata': 'https://www.wikidata.org/wiki/Q6888026', 'display_name': 'Modalities', 'level': 2, 'score': 0.766163}, {'id': 'https://openalex.org/C195704467', 'wikidata': 'https://www.wikidata.org/wiki/Q327968', 'display_name': 'Facial expression', 'level': 2, 'score': 0.74413395}, {'id': 'https://openalex.org/C28490314', 'wikidata': 'https://www.wikidata.org/wiki/Q189436', 'display_name': 'Speech recognition', 'level': 1, 'score': 0.69651073}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.6929306}, {'id': 'https://openalex.org/C63479239', 'wikidata': 'https://www.wikidata.org/wiki/Q7353546', 'display_name': 'Robustness (evolution)', 'level': 3, 'score': 0.6814037}, {'id': 'https://openalex.org/C2778999518', 'wikidata': 'https://www.wikidata.org/wiki/Q8', 'display_name': 'Happiness', 'level': 2, 'score': 0.6107779}, {'id': 'https://openalex.org/C2779302386', 'wikidata': 'https://www.wikidata.org/wiki/Q79871', 'display_name': 'Anger', 'level': 2, 'score': 0.5738748}, {'id': 'https://openalex.org/C31510193', 'wikidata': 'https://www.wikidata.org/wiki/Q1192553', 'display_name': 'Facial recognition system', 'level': 3, 'score': 0.5073342}, {'id': 'https://openalex.org/C2777438025', 'wikidata': 'https://www.wikidata.org/wiki/Q1339090', 'display_name': 'Emotion recognition', 'level': 2, 'score': 0.47760913}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.46210632}, {'id': 'https://openalex.org/C6438553', 'wikidata': 'https://www.wikidata.org/wiki/Q1185804', 'display_name': 'Affective computing', 'level': 2, 'score': 0.42520896}, {'id': 'https://openalex.org/C52622490', 'wikidata': 'https://www.wikidata.org/wiki/Q1026626', 'display_name': 'Feature extraction', 'level': 2, 'score': 0.3327344}, {'id': 'https://openalex.org/C15744967', 'wikidata': 'https://www.wikidata.org/wiki/Q9418', 'display_name': 'Psychology', 'level': 0, 'score': 0.15926847}, {'id': 'https://openalex.org/C77805123', 'wikidata': 'https://www.wikidata.org/wiki/Q161272', 'display_name': 'Social psychology', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C36289849', 'wikidata': 'https://www.wikidata.org/wiki/Q34749', 'display_name': 'Social science', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C55493867', 'wikidata': 'https://www.wikidata.org/wiki/Q7094', 'display_name': 'Biochemistry', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C185592680', 'wikidata': 'https://www.wikidata.org/wiki/Q2329', 'display_name': 'Chemistry', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C118552586', 'wikidata': 'https://www.wikidata.org/wiki/Q7867', 'display_name': 'Psychiatry', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C144024400', 'wikidata': 'https://www.wikidata.org/wiki/Q21201', 'display_name': 'Sociology', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C104317684', 'wikidata': 'https://www.wikidata.org/wiki/Q7187', 'display_name': 'Gene', 'level': 2, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/1027933.1027968', 'pdf_url': None, 'source': None, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}], 'best_oa_location': None, 'sustainable_development_goals': [{'id': 'https://metadata.un.org/sdg/16', 'display_name': 'Peace, justice, and strong institutions', 'score': 0.69}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 25, 'referenced_works': ['https://openalex.org/W1509274023', 'https://openalex.org/W1528162760', 'https://openalex.org/W1668904664', 'https://openalex.org/W1815942593', 'https://openalex.org/W1923034539', 'https://openalex.org/W2032254851', 'https://openalex.org/W2096711002', 'https://openalex.org/W2102953093', 'https://openalex.org/W2103886445', 'https://openalex.org/W2109269337', 'https://openalex.org/W2115289180', 'https://openalex.org/W2122609807', 'https://openalex.org/W2127986803', 'https://openalex.org/W2139212933', 'https://openalex.org/W2154739180', 'https://openalex.org/W2157543951', 'https://openalex.org/W2160947254', 'https://openalex.org/W2162591825', 'https://openalex.org/W2171746100', 'https://openalex.org/W2171939880', 'https://openalex.org/W2995034616', 'https://openalex.org/W3141774725', 'https://openalex.org/W3193477162', 'https://openalex.org/W4285719527', 'https://openalex.org/W47598355'], 'related_works': ['https://openalex.org/W4365519294', 'https://openalex.org/W3179181153', 'https://openalex.org/W3135436127', 'https://openalex.org/W3031187173', 'https://openalex.org/W2805647211', 'https://openalex.org/W2557139663', 'https://openalex.org/W2481378124', 'https://openalex.org/W2100930849', 'https://openalex.org/W2073249509', 'https://openalex.org/W1980267738'], 'abstract_inverted_index': {'The': [0, 140], 'interaction': [1], 'between': [2], 'human': [3, 20, 34], 'beings': [4], 'and': [5, 17, 52, 59, 71, 97, 115, 174, 184], 'computers': [6, 12], 'will': [7], 'be': [8], 'more': [9], 'natural': [10], 'if': [11], 'are': [13, 180], 'able': [14], 'to': [15, 19, 32, 48, 55, 90], 'perceive': [16], 'respond': [18], 'non-verbal': [21], 'communication': [22], 'such': [23], 'as': [24], 'emotions.': [25], 'Although': [26], 'several': [27], 'approaches': [28, 88], 'have': [29], 'been': [30, 46], 'proposed': [31], 'recognize': [33], 'emotions': [35, 109, 163], 'based': [36, 76, 146, 156], 'on': [37, 78, 123, 147, 157], 'facial': [38, 79, 127, 148], 'expressions': [39, 80], 'or': [40, 81], 'speech,': [41], 'relatively': [42], 'limited': [43], 'work': [44], 'has': [45], 'done': [47], 'fuse': [49, 91], 'these': [50, 92, 177], 'two,': [51], 'other,': [53], 'modalities': [54, 173, 179], 'improve': [56, 192], 'the': [57, 62, 69, 72, 119, 144, 154, 162, 168, 171, 182, 185, 188], 'accuracy': [58], 'robustness': [60, 186], 'of': [61, 74, 121, 170, 187], 'emotion': [63, 189], 'recognition': [64, 190], 'system.': [65], 'This': [66], 'paper': [67], 'analyzes': [68], 'strengths': [70], 'limitations': [73], 'systems': [75], 'only': [77], 'acoustic': [82, 159], 'information.': [83], 'It': [84], 'also': [85, 166], 'discusses': [86], 'two': [87, 93, 172, 178], 'used': [89], 'modalities:': [94], 'decision': [95], 'level': [96, 99], 'feature': [98], 'integration.': [100], 'Using': [101], 'a': [102], 'database': [103], 'recorded': [104], 'from': [105], 'an': [106], 'actress,': [107], 'four': [108], 'were': [110, 129], 'classified:': [111], 'sadness,': [112], 'anger,': [113], 'happiness,': [114], 'neutral': [116], 'state.': [117], 'By': [118], 'use': [120], 'markers': [122], 'her': [124], 'face,': [125], 'detailed': [126], 'motions': [128], 'captured': [130], 'with': [131, 136], 'motion': [132], 'capture,': [133], 'in': [134], 'conjunction': [135], 'simultaneous': [137], 'speech': [138], 'recordings.': [139], 'results': [141], 'reveal': [142], 'that': [143, 175], 'system': [145, 155, 191], 'expression': [149], 'gave': [150], 'better': [151], 'performance': [152, 183], 'than': [153], 'just': [158], 'information': [160], 'for': [161], 'considered.': [164], 'Results': [165], 'show': [167], 'complementarily': [169], 'when': [176], 'fused,': [181], 'measurably.': [193]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W2143350951', 'counts_by_year': [{'year': 2024, 'cited_by_count': 21}, {'year': 2023, 'cited_by_count': 31}, {'year': 2022, 'cited_by_count': 33}, {'year': 2021, 'cited_by_count': 51}, {'year': 2020, 'cited_by_count': 66}, {'year': 2019, 'cited_by_count': 63}, {'year': 2018, 'cited_by_count': 65}, {'year': 2017, 'cited_by_count': 59}, {'year': 2016, 'cited_by_count': 73}, {'year': 2015, 'cited_by_count': 41}, {'year': 2014, 'cited_by_count': 35}, {'year': 2013, 'cited_by_count': 42}, {'year': 2012, 'cited_by_count': 34}], 'updated_date': '2024-12-20T07:56:14.536797', 'created_date': '2016-06-24'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
