Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder

Name: Work Video:
Duration: 3 min 30 s
Description: Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, Arnold Overwijk. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.
Shuqi Lu; Di He; Chenyan Xiong; Guolin Ke; Waleed Malik; Zhicheng Dou; Paul N. Bennett; Tie‐Yan Liu; Arnold Overwijk
{'id': 'https://openalex.org/W3212725701', 'doi': 'https://doi.org/10.18653/v1/2021.emnlp-main.220', 'title': 'Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder', 'display_name': 'Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder', 'publication_year': 2021, 'publication_date': '2021-01-01', 'ids': {'openalex': 'https://openalex.org/W3212725701', 'doi': 'https://doi.org/10.18653/v1/2021.emnlp-main.220', 'mag': '3212725701'}, 'language': 'en', 'primary_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.emnlp-main.220', 'pdf_url': 'https://aclanthology.org/2021.emnlp-main.220.pdf', 'source': {'id': 'https://openalex.org/S4363608991', 'display_name': 'Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': True, 'oa_status': 'hybrid', 'oa_url': 'https://aclanthology.org/2021.emnlp-main.220.pdf', 'any_repository_has_fulltext': False}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5101016825', 'display_name': 'Shuqi Lu', 'orcid': 'https://orcid.org/0000-0003-3899-6926'}, 'institutions': [{'id': 'https://openalex.org/I78988378', 'display_name': 'Renmin University of China', 'ror': 'https://ror.org/041pakw92', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I78988378']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Shuqi Lu', 'raw_affiliation_strings': ['Renmin University of China'], 'affiliations': [{'raw_affiliation_string': 'Renmin University of China', 'institution_ids': ['https://openalex.org/I78988378']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100739032', 'display_name': 'Di He', 'orcid': 'https://orcid.org/0000-0002-3449-6935'}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Di He', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102363883', 'display_name': 'Chenyan Xiong', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Chenyan Xiong', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5061200287', 'display_name': 'Guolin Ke', 'orcid': 'https://orcid.org/0000-0002-1227-7221'}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Guolin Ke', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5014094721', 'display_name': 'Waleed Malik', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Waleed Malik', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5010558184', 'display_name': 'Zhicheng Dou', 'orcid': 'https://orcid.org/0000-0002-9781-948X'}, 'institutions': [{'id': 'https://openalex.org/I78988378', 'display_name': 'Renmin University of China', 'ror': 'https://ror.org/041pakw92', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I78988378']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Zhicheng Dou', 'raw_affiliation_strings': ['Renmin University of China'], 'affiliations': [{'raw_affiliation_string': 'Renmin University of China', 'institution_ids': ['https://openalex.org/I78988378']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102869952', 'display_name': 'Paul N. Bennett', 'orcid': 'https://orcid.org/0000-0002-8846-5480'}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Paul Bennett', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5101884287', 'display_name': 'Tie‐Yan Liu', 'orcid': 'https://orcid.org/0000-0002-0476-8020'}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Tie-Yan Liu', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5005972047', 'display_name': 'Arnold Overwijk', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I4210164937', 'display_name': 'Microsoft Research (United Kingdom)', 'ror': 'https://ror.org/05k87vq12', 'country_code': 'GB', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210164937']}], 'countries': ['GB'], 'is_corresponding': False, 'raw_author_name': 'Arnold Overwijk', 'raw_affiliation_strings': ['Microsoft'], 'affiliations': [{'raw_affiliation_string': 'Microsoft', 'institution_ids': ['https://openalex.org/I4210164937']}]}], 'institution_assertions': [], 'countries_distinct_count': 2, 'institutions_distinct_count': 2, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 3.864, 'has_fulltext': True, 'fulltext_origin': 'pdf', 'cited_by_count': 38, 'citation_normalized_percentile': {'value': 0.756396, 'is_in_top_1_percent': False, 'is_in_top_10_percent': False}, 'cited_by_percentile_year': {'min': 97, 'max': 98}, 'biblio': {'volume': None, 'issue': None, 'first_page': None, 'last_page': None}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9995, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T13083', 'display_name': 'Advanced Text Analysis Techniques', 'score': 0.9939, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [], 'concepts': [{'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.69865096}, {'id': 'https://openalex.org/C118505674', 'wikidata': 'https://www.wikidata.org/wiki/Q42586063', 'display_name': 'Encoder', 'level': 2, 'score': 0.66751015}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.51678646}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.41570544}, {'id': 'https://openalex.org/C28490314', 'wikidata': 'https://www.wikidata.org/wiki/Q189436', 'display_name': 'Speech recognition', 'level': 1, 'score': 0.3982853}, {'id': 'https://openalex.org/C31972630', 'wikidata': 'https://www.wikidata.org/wiki/Q844240', 'display_name': 'Computer vision', 'level': 1, 'score': 0.32942355}, {'id': 'https://openalex.org/C111919701', 'wikidata': 'https://www.wikidata.org/wiki/Q9135', 'display_name': 'Operating system', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.emnlp-main.220', 'pdf_url': 'https://aclanthology.org/2021.emnlp-main.220.pdf', 'source': {'id': 'https://openalex.org/S4363608991', 'display_name': 'Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.emnlp-main.220', 'pdf_url': 'https://aclanthology.org/2021.emnlp-main.220.pdf', 'source': {'id': 'https://openalex.org/S4363608991', 'display_name': 'Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'sustainable_development_goals': [{'score': 0.81, 'display_name': 'Quality education', 'id': 'https://metadata.un.org/sdg/4'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 43, 'referenced_works': ['https://openalex.org/W1522301498', 'https://openalex.org/W1566289585', 'https://openalex.org/W1959608418', 'https://openalex.org/W2145094598', 'https://openalex.org/W2593864460', 'https://openalex.org/W2763421725', 'https://openalex.org/W2896457183', 'https://openalex.org/W2899771611', 'https://openalex.org/W2912924812', 'https://openalex.org/W2923014074', 'https://openalex.org/W2933138175', 'https://openalex.org/W2937036051', 'https://openalex.org/W2951434086', 'https://openalex.org/W2951534261', 'https://openalex.org/W2963310665', 'https://openalex.org/W2963341956', 'https://openalex.org/W2963403868', 'https://openalex.org/W2964110616', 'https://openalex.org/W2964121744', 'https://openalex.org/W2965373594', 'https://openalex.org/W2970641574', 'https://openalex.org/W2982096936', 'https://openalex.org/W2983180560', 'https://openalex.org/W2995744795', 'https://openalex.org/W2996035354', 'https://openalex.org/W2997200074', 'https://openalex.org/W2997574889', 'https://openalex.org/W3011411500', 'https://openalex.org/W3024786184', 'https://openalex.org/W3034503922', 'https://openalex.org/W3034696692', 'https://openalex.org/W3037983807', 'https://openalex.org/W3098708719', 'https://openalex.org/W3099700870', 'https://openalex.org/W3118668786', 'https://openalex.org/W3122515622', 'https://openalex.org/W3130740619', 'https://openalex.org/W3157758108', 'https://openalex.org/W3170261818', 'https://openalex.org/W3175111331', 'https://openalex.org/W4287824654', 'https://openalex.org/W4299585995', 'https://openalex.org/W4385245566'], 'related_works': ['https://openalex.org/W3116076068', 'https://openalex.org/W2775347418', 'https://openalex.org/W2772917594', 'https://openalex.org/W2755342338', 'https://openalex.org/W2275988210', 'https://openalex.org/W2229312674', 'https://openalex.org/W2166024367', 'https://openalex.org/W2036807459', 'https://openalex.org/W1969923398', 'https://openalex.org/W1891287906'], 'abstract_inverted_index': {'Shuqi': [0], 'Lu,': [1], 'Di': [2], 'He,': [3], 'Chenyan': [4], 'Xiong,': [5], 'Guolin': [6], 'Ke,': [7], 'Waleed': [8], 'Malik,': [9], 'Zhicheng': [10], 'Dou,': [11], 'Paul': [12], 'Bennett,': [13], 'Tie-Yan': [14], 'Liu,': [15], 'Arnold': [16], 'Overwijk.': [17], 'Proceedings': [18], 'of': [19], 'the': [20], '2021': [21], 'Conference': [22], 'on': [23], 'Empirical': [24], 'Methods': [25], 'in': [26], 'Natural': [27], 'Language': [28], 'Processing.': [29], '2021.': [30]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W3212725701', 'counts_by_year': [{'year': 2024, 'cited_by_count': 4}, {'year': 2023, 'cited_by_count': 21}, {'year': 2022, 'cited_by_count': 12}], 'updated_date': '2024-12-12T08:18:34.227306', 'created_date': '2021-11-22'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
