Implicit Representations of Meaning in Neural Language Models

Name: Work Video:
Duration: 3 min 30 s
Description: Belinda Z. Li, Maxwell Nye, Jacob Andreas. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Belinda Z. Li; Maxwell Nye; Jacob Andreas
{'id': 'https://openalex.org/W3173798466', 'doi': 'https://doi.org/10.18653/v1/2021.acl-long.143', 'title': 'Implicit Representations of Meaning in Neural Language Models', 'display_name': 'Implicit Representations of Meaning in Neural Language Models', 'publication_year': 2021, 'publication_date': '2021-01-01', 'ids': {'openalex': 'https://openalex.org/W3173798466', 'doi': 'https://doi.org/10.18653/v1/2021.acl-long.143', 'mag': '3173798466'}, 'language': 'en', 'primary_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.acl-long.143', 'pdf_url': 'https://aclanthology.org/2021.acl-long.143.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': True, 'oa_status': 'hybrid', 'oa_url': 'https://aclanthology.org/2021.acl-long.143.pdf', 'any_repository_has_fulltext': True}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5015729879', 'display_name': 'Belinda Z. Li', 'orcid': 'https://orcid.org/0000-0002-9961-180X'}, 'institutions': [{'id': 'https://openalex.org/I63966007', 'display_name': 'Massachusetts Institute of Technology', 'ror': 'https://ror.org/042nb2s44', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I63966007']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Belinda Z. Li', 'raw_affiliation_strings': ['Massachusetts Institute of Technology'], 'affiliations': [{'raw_affiliation_string': 'Massachusetts Institute of Technology', 'institution_ids': ['https://openalex.org/I63966007']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5112464301', 'display_name': 'Maxwell Nye', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I63966007', 'display_name': 'Massachusetts Institute of Technology', 'ror': 'https://ror.org/042nb2s44', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I63966007']}], 'countries': ['US'], 'is_corresponding': True, 'raw_author_name': 'Maxwell Nye', 'raw_affiliation_strings': ['Massachusetts Institute of Technology'], 'affiliations': [{'raw_affiliation_string': 'Massachusetts Institute of Technology', 'institution_ids': ['https://openalex.org/I63966007']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5070829652', 'display_name': 'Jacob Andreas', 'orcid': 'https://orcid.org/0000-0002-3141-5845'}, 'institutions': [{'id': 'https://openalex.org/I63966007', 'display_name': 'Massachusetts Institute of Technology', 'ror': 'https://ror.org/042nb2s44', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I63966007']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Jacob Andreas', 'raw_affiliation_strings': ['Massachusetts Institute of Technology'], 'affiliations': [{'raw_affiliation_string': 'Massachusetts Institute of Technology', 'institution_ids': ['https://openalex.org/I63966007']}]}], 'institution_assertions': [], 'countries_distinct_count': 1, 'institutions_distinct_count': 1, 'corresponding_author_ids': ['https://openalex.org/A5112464301'], 'corresponding_institution_ids': ['https://openalex.org/I63966007'], 'apc_list': None, 'apc_paid': None, 'fwci': 6.236, 'has_fulltext': True, 'fulltext_origin': 'pdf', 'cited_by_count': 63, 'citation_normalized_percentile': {'value': 0.832635, 'is_in_top_1_percent': False, 'is_in_top_10_percent': False}, 'cited_by_percentile_year': {'min': 98, 'max': 99}, 'biblio': {'volume': None, 'issue': None, 'first_page': '1813', 'last_page': '1827'}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11714', 'display_name': 'Multimodal Machine Learning Applications', 'score': 0.9938, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/computational-linguistics', 'display_name': 'Computational linguistics', 'score': 0.5392481}, {'id': 'https://openalex.org/keywords/association', 'display_name': 'Association (psychology)', 'score': 0.44600925}], 'concepts': [{'id': 'https://openalex.org/C2780876879', 'wikidata': 'https://www.wikidata.org/wiki/Q3054749', 'display_name': 'Meaning (existential)', 'level': 2, 'score': 0.7823689}, {'id': 'https://openalex.org/C41895202', 'wikidata': 'https://www.wikidata.org/wiki/Q8162', 'display_name': 'Linguistics', 'level': 1, 'score': 0.5832138}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.56960785}, {'id': 'https://openalex.org/C155092808', 'wikidata': 'https://www.wikidata.org/wiki/Q182557', 'display_name': 'Computational linguistics', 'level': 2, 'score': 0.5392481}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.46596578}, {'id': 'https://openalex.org/C50644808', 'wikidata': 'https://www.wikidata.org/wiki/Q192776', 'display_name': 'Artificial neural network', 'level': 2, 'score': 0.4506193}, {'id': 'https://openalex.org/C20556612', 'wikidata': 'https://www.wikidata.org/wiki/Q4469374', 'display_name': 'Volume (thermodynamics)', 'level': 2, 'score': 0.4480463}, {'id': 'https://openalex.org/C142853389', 'wikidata': 'https://www.wikidata.org/wiki/Q744778', 'display_name': 'Association (psychology)', 'level': 2, 'score': 0.44600925}, {'id': 'https://openalex.org/C188147891', 'wikidata': 'https://www.wikidata.org/wiki/Q147638', 'display_name': 'Cognitive science', 'level': 1, 'score': 0.44103745}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.4405266}, {'id': 'https://openalex.org/C195324797', 'wikidata': 'https://www.wikidata.org/wiki/Q33742', 'display_name': 'Natural language', 'level': 2, 'score': 0.4364128}, {'id': 'https://openalex.org/C18555067', 'wikidata': 'https://www.wikidata.org/wiki/Q8375051', 'display_name': 'Joint (building)', 'level': 2, 'score': 0.43383226}, {'id': 'https://openalex.org/C111472728', 'wikidata': 'https://www.wikidata.org/wiki/Q9471', 'display_name': 'Epistemology', 'level': 1, 'score': 0.2675408}, {'id': 'https://openalex.org/C138885662', 'wikidata': 'https://www.wikidata.org/wiki/Q5891', 'display_name': 'Philosophy', 'level': 0, 'score': 0.26157254}, {'id': 'https://openalex.org/C15744967', 'wikidata': 'https://www.wikidata.org/wiki/Q9418', 'display_name': 'Psychology', 'level': 0, 'score': 0.23859835}, {'id': 'https://openalex.org/C127413603', 'wikidata': 'https://www.wikidata.org/wiki/Q11023', 'display_name': 'Engineering', 'level': 0, 'score': 0.1544792}, {'id': 'https://openalex.org/C121332964', 'wikidata': 'https://www.wikidata.org/wiki/Q413', 'display_name': 'Physics', 'level': 0, 'score': 0.09555486}, {'id': 'https://openalex.org/C170154142', 'wikidata': 'https://www.wikidata.org/wiki/Q150737', 'display_name': 'Architectural engineering', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C62520636', 'wikidata': 'https://www.wikidata.org/wiki/Q944', 'display_name': 'Quantum mechanics', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 3, 'locations': [{'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.acl-long.143', 'pdf_url': 'https://aclanthology.org/2021.acl-long.143.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://hdl.handle.net/1721.1/142870', 'pdf_url': 'https://dspace.mit.edu/bitstream/1721.1/142870/2/2021.acl-long.143.pdf', 'source': {'id': 'https://openalex.org/S4306400425', 'display_name': 'DSpace@MIT (Massachusetts Institute of Technology)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I63966007', 'host_organization_name': 'Massachusetts Institute of Technology', 'host_organization_lineage': ['https://openalex.org/I63966007'], 'host_organization_lineage_names': ['Massachusetts Institute of Technology'], 'type': 'repository'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://arxiv.org/abs/2106.00737', 'pdf_url': 'https://arxiv.org/pdf/2106.00737', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.acl-long.143', 'pdf_url': 'https://aclanthology.org/2021.acl-long.143.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'sustainable_development_goals': [{'id': 'https://metadata.un.org/sdg/4', 'score': 0.8, 'display_name': 'Quality education'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 29, 'referenced_works': ['https://openalex.org/W1585104069', 'https://openalex.org/W1927197813', 'https://openalex.org/W2071089353', 'https://openalex.org/W2141599568', 'https://openalex.org/W2149305746', 'https://openalex.org/W2563574619', 'https://openalex.org/W2810346659', 'https://openalex.org/W2906152891', 'https://openalex.org/W2914081143', 'https://openalex.org/W2946359678', 'https://openalex.org/W2946417913', 'https://openalex.org/W2962739339', 'https://openalex.org/W2963341956', 'https://openalex.org/W2963372003', 'https://openalex.org/W2970120757', 'https://openalex.org/W2970476646', 'https://openalex.org/W2970862333', 'https://openalex.org/W2972324944', 'https://openalex.org/W2974273066', 'https://openalex.org/W3027086341', 'https://openalex.org/W3034723486', 'https://openalex.org/W3034999214', 'https://openalex.org/W3082274269', 'https://openalex.org/W3097977265', 'https://openalex.org/W3102659883', 'https://openalex.org/W3103054319', 'https://openalex.org/W3138301265', 'https://openalex.org/W3172224317', 'https://openalex.org/W4238198763'], 'related_works': ['https://openalex.org/W4396737233', 'https://openalex.org/W4309440960', 'https://openalex.org/W2888483922', 'https://openalex.org/W2748574964', 'https://openalex.org/W2389491697', 'https://openalex.org/W2363046693', 'https://openalex.org/W2352440174', 'https://openalex.org/W2348940229', 'https://openalex.org/W2062168445', 'https://openalex.org/W1996130883'], 'abstract_inverted_index': {'Belinda': [0], 'Z.': [1], 'Li,': [2], 'Maxwell': [3], 'Nye,': [4], 'Jacob': [5], 'Andreas.': [6], 'Proceedings': [7], 'of': [8, 13], 'the': [9, 14, 20], '59th': [10], 'Annual': [11], 'Meeting': [12], 'Association': [15], 'for': [16], 'Computational': [17], 'Linguistics': [18], 'and': [19], '11th': [21], 'International': [22], 'Joint': [23], 'Conference': [24], 'on': [25], 'Natural': [26], 'Language': [27], 'Processing': [28], '(Volume': [29], '1:': [30], 'Long': [31], 'Papers).': [32], '2021.': [33]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W3173798466', 'counts_by_year': [{'year': 2024, 'cited_by_count': 9}, {'year': 2023, 'cited_by_count': 38}, {'year': 2022, 'cited_by_count': 13}, {'year': 2021, 'cited_by_count': 2}], 'updated_date': '2025-01-19T02:00:11.756949', 'created_date': '2021-07-05'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
