Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment

Name: Work Video:
Duration: 3 min 30 s
Description: Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, Yongbin Li. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
Tianshu Yu; Haoyu Gao; Ting-En Lin; Min Yang; Yuchuan Wu; Wentao Ma; Chao Wang; Fei Huang; Yongbin Li
{'id': 'https://openalex.org/W4385571111', 'doi': 'https://doi.org/10.18653/v1/2023.acl-long.438', 'title': 'Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment', 'display_name': 'Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment', 'publication_year': 2023, 'publication_date': '2023-01-01', 'ids': {'openalex': 'https://openalex.org/W4385571111', 'doi': 'https://doi.org/10.18653/v1/2023.acl-long.438'}, 'language': 'en', 'primary_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2023.acl-long.438', 'pdf_url': 'https://aclanthology.org/2023.acl-long.438.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': True, 'oa_status': 'hybrid', 'oa_url': 'https://aclanthology.org/2023.acl-long.438.pdf', 'any_repository_has_fulltext': True}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5101448071', 'display_name': 'Tianshu Yu', 'orcid': 'https://orcid.org/0009-0001-9613-5123'}, 'institutions': [{'id': 'https://openalex.org/I4210165038', 'display_name': 'University of Chinese Academy of Sciences', 'ror': 'https://ror.org/05qbk4x57', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I19820366', 'https://openalex.org/I4210165038']}, {'id': 'https://openalex.org/I126520041', 'display_name': 'University of Science and Technology of China', 'ror': 'https://ror.org/04c4dkn09', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I126520041', 'https://openalex.org/I19820366']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Tianshu Yu', 'raw_affiliation_strings': [' University of Chinese Academy of Sciences', ' University of Science and Technology of China'], 'affiliations': [{'raw_affiliation_string': ' University of Chinese Academy of Sciences', 'institution_ids': ['https://openalex.org/I4210165038']}, {'raw_affiliation_string': ' University of Science and Technology of China', 'institution_ids': ['https://openalex.org/I126520041']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5074425605', 'display_name': 'Haoyu Gao', 'orcid': 'https://orcid.org/0000-0003-4724-0987'}, 'institutions': [{'id': 'https://openalex.org/I126520041', 'display_name': 'University of Science and Technology of China', 'ror': 'https://ror.org/04c4dkn09', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I126520041', 'https://openalex.org/I19820366']}, {'id': 'https://openalex.org/I4210165038', 'display_name': 'University of Chinese Academy of Sciences', 'ror': 'https://ror.org/05qbk4x57', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I19820366', 'https://openalex.org/I4210165038']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Haoyu Gao', 'raw_affiliation_strings': [' University of Chinese Academy of Sciences', ' University of Science and Technology of China'], 'affiliations': [{'raw_affiliation_string': ' University of Science and Technology of China', 'institution_ids': ['https://openalex.org/I126520041']}, {'raw_affiliation_string': ' University of Chinese Academy of Sciences', 'institution_ids': ['https://openalex.org/I4210165038']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5069007219', 'display_name': 'Ting-En Lin', 'orcid': 'https://orcid.org/0000-0003-0650-7286'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Ting-En Lin', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5054067088', 'display_name': 'Min Yang', 'orcid': 'https://orcid.org/0000-0003-3814-2728'}, 'institutions': [{'id': 'https://openalex.org/I19820366', 'display_name': 'Chinese Academy of Sciences', 'ror': 'https://ror.org/034t30j35', 'country_code': 'CN', 'type': 'government', 'lineage': ['https://openalex.org/I19820366']}, {'id': 'https://openalex.org/I4210145761', 'display_name': 'Shenzhen Institutes of Advanced Technology', 'ror': 'https://ror.org/04gh4er46', 'country_code': 'CN', 'type': 'facility', 'lineage': ['https://openalex.org/I19820366', 'https://openalex.org/I4210145761']}], 'countries': ['CN'], 'is_corresponding': True, 'raw_author_name': 'Min Yang', 'raw_affiliation_strings': ['Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences'], 'affiliations': [{'raw_affiliation_string': 'Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences', 'institution_ids': ['https://openalex.org/I19820366', 'https://openalex.org/I4210145761']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102918313', 'display_name': 'Yuchuan Wu', 'orcid': 'https://orcid.org/0009-0000-3657-5859'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Yuchuan Wu', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102447652', 'display_name': 'Wentao Ma', 'orcid': 'https://orcid.org/0000-0001-7293-3756'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Wentao Ma', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100407035', 'display_name': 'Chao Wang', 'orcid': 'https://orcid.org/0000-0002-7427-793X'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Chao Wang', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100620486', 'display_name': 'Fei Huang', 'orcid': 'https://orcid.org/0000-0001-9665-6642'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Fei Huang', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5100644428', 'display_name': 'Yongbin Li', 'orcid': 'https://orcid.org/0009-0008-4504-2163'}, 'institutions': [{'id': 'https://openalex.org/I4210095624', 'display_name': 'Alibaba Group (United States)', 'ror': 'https://ror.org/00rn0m335', 'country_code': 'US', 'type': 'company', 'lineage': ['https://openalex.org/I4210095624', 'https://openalex.org/I45928872']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Yongbin Li', 'raw_affiliation_strings': ['Alibaba Group'], 'affiliations': [{'raw_affiliation_string': 'Alibaba Group', 'institution_ids': ['https://openalex.org/I4210095624']}]}], 'institution_assertions': [], 'countries_distinct_count': 2, 'institutions_distinct_count': 5, 'corresponding_author_ids': ['https://openalex.org/A5054067088'], 'corresponding_institution_ids': ['https://openalex.org/I19820366', 'https://openalex.org/I4210145761'], 'apc_list': None, 'apc_paid': None, 'fwci': 2.114, 'has_fulltext': True, 'fulltext_origin': 'pdf', 'cited_by_count': 7, 'citation_normalized_percentile': {'value': 0.999976, 'is_in_top_1_percent': True, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 93, 'max': 94}, 'biblio': {'volume': None, 'issue': None, 'first_page': None, 'last_page': None}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9179, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9179, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/computational-linguistics', 'display_name': 'Computational linguistics', 'score': 0.4885777}], 'concepts': [{'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.63289857}, {'id': 'https://openalex.org/C173853756', 'wikidata': 'https://www.wikidata.org/wiki/Q86915', 'display_name': 'Dialog box', 'level': 2, 'score': 0.6154423}, {'id': 'https://openalex.org/C71139939', 'wikidata': 'https://www.wikidata.org/wiki/Q910194', 'display_name': 'Modal', 'level': 2, 'score': 0.5373619}, {'id': 'https://openalex.org/C28490314', 'wikidata': 'https://www.wikidata.org/wiki/Q189436', 'display_name': 'Speech recognition', 'level': 1, 'score': 0.53672296}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.5025556}, {'id': 'https://openalex.org/C155092808', 'wikidata': 'https://www.wikidata.org/wiki/Q182557', 'display_name': 'Computational linguistics', 'level': 2, 'score': 0.4885777}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.45582566}, {'id': 'https://openalex.org/C20556612', 'wikidata': 'https://www.wikidata.org/wiki/Q4469374', 'display_name': 'Volume (thermodynamics)', 'level': 2, 'score': 0.45378515}, {'id': 'https://openalex.org/C136764020', 'wikidata': 'https://www.wikidata.org/wiki/Q466', 'display_name': 'World Wide Web', 'level': 1, 'score': 0.12633258}, {'id': 'https://openalex.org/C121332964', 'wikidata': 'https://www.wikidata.org/wiki/Q413', 'display_name': 'Physics', 'level': 0, 'score': 0.07782227}, {'id': 'https://openalex.org/C188027245', 'wikidata': 'https://www.wikidata.org/wiki/Q750446', 'display_name': 'Polymer chemistry', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C185592680', 'wikidata': 'https://www.wikidata.org/wiki/Q2329', 'display_name': 'Chemistry', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C62520636', 'wikidata': 'https://www.wikidata.org/wiki/Q944', 'display_name': 'Quantum mechanics', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2023.acl-long.438', 'pdf_url': 'https://aclanthology.org/2023.acl-long.438.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2023.acl-long.438', 'pdf_url': 'https://aclanthology.org/2023.acl-long.438.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'sustainable_development_goals': [{'score': 0.8, 'display_name': 'Quality education', 'id': 'https://metadata.un.org/sdg/4'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 47, 'referenced_works': ['https://openalex.org/W2146334809', 'https://openalex.org/W2465534249', 'https://openalex.org/W2768282280', 'https://openalex.org/W2883409523', 'https://openalex.org/W2896457183', 'https://openalex.org/W2897182555', 'https://openalex.org/W2899377381', 'https://openalex.org/W2899663614', 'https://openalex.org/W2952409498', 'https://openalex.org/W2954492830', 'https://openalex.org/W2956125353', 'https://openalex.org/W2965373594', 'https://openalex.org/W2966715458', 'https://openalex.org/W2973049979', 'https://openalex.org/W2982223350', 'https://openalex.org/W3021096583', 'https://openalex.org/W3034266838', 'https://openalex.org/W3035451444', 'https://openalex.org/W3036601975', 'https://openalex.org/W3094502228', 'https://openalex.org/W3096109555', 'https://openalex.org/W3107826490', 'https://openalex.org/W3113822960', 'https://openalex.org/W3148001440', 'https://openalex.org/W3161302809', 'https://openalex.org/W3166396011', 'https://openalex.org/W3169320628', 'https://openalex.org/W3207222250', 'https://openalex.org/W3209984917', 'https://openalex.org/W3212743544', 'https://openalex.org/W4221141243', 'https://openalex.org/W4221142780', 'https://openalex.org/W4223622550', 'https://openalex.org/W4224903019', 'https://openalex.org/W4226315417', 'https://openalex.org/W4281701185', 'https://openalex.org/W4282030117', 'https://openalex.org/W4285149123', 'https://openalex.org/W4287065249', 'https://openalex.org/W4297783677', 'https://openalex.org/W4297833904', 'https://openalex.org/W4304080418', 'https://openalex.org/W4372260047', 'https://openalex.org/W4372266552', 'https://openalex.org/W4372278801', 'https://openalex.org/W4385573166', 'https://openalex.org/W4385573848'], 'related_works': ['https://openalex.org/W4298396513', 'https://openalex.org/W4233992201', 'https://openalex.org/W2795961259', 'https://openalex.org/W2417260800', 'https://openalex.org/W2283130723', 'https://openalex.org/W2117933979', 'https://openalex.org/W2104718772', 'https://openalex.org/W2098987383', 'https://openalex.org/W1596203174', 'https://openalex.org/W103938586'], 'abstract_inverted_index': {'Tianshu': [0], 'Yu,': [1], 'Haoyu': [2], 'Gao,': [3], 'Ting-En': [4], 'Lin,': [5], 'Min': [6], 'Yang,': [7], 'Yuchuan': [8], 'Wu,': [9], 'Wentao': [10], 'Ma,': [11], 'Chao': [12], 'Wang,': [13], 'Fei': [14], 'Huang,': [15], 'Yongbin': [16], 'Li.': [17], 'Proceedings': [18], 'of': [19, 24], 'the': [20, 25], '61st': [21], 'Annual': [22], 'Meeting': [23], 'Association': [26], 'for': [27], 'Computational': [28], 'Linguistics': [29], '(Volume': [30], '1:': [31], 'Long': [32], 'Papers).': [33], '2023.': [34]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W4385571111', 'counts_by_year': [{'year': 2024, 'cited_by_count': 6}, {'year': 2023, 'cited_by_count': 1}], 'updated_date': '2025-01-11T19:45:23.880640', 'created_date': '2023-08-05'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
