Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

Name: Work Video:
Duration: 3 min 30 s
Description: Jiangang Bai, Yujing Wang, Yiren Chen, Yaming Yang, Jing Bai, Jing Yu, Yunhai Tong. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.
Jiangang Bai; Yujing Wang; Yiren Chen; Yaming Yang; Jing Bai; Jing Yu; Yunhai Tong
{'id': 'https://openalex.org/W3154449322', 'doi': 'https://doi.org/10.18653/v1/2021.eacl-main.262', 'title': 'Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees', 'display_name': 'Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees', 'publication_year': 2021, 'publication_date': '2021-01-01', 'ids': {'openalex': 'https://openalex.org/W3154449322', 'doi': 'https://doi.org/10.18653/v1/2021.eacl-main.262', 'mag': '3154449322'}, 'language': 'en', 'primary_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.eacl-main.262', 'pdf_url': 'https://aclanthology.org/2021.eacl-main.262.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': True, 'oa_status': 'hybrid', 'oa_url': 'https://aclanthology.org/2021.eacl-main.262.pdf', 'any_repository_has_fulltext': True}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5018600735', 'display_name': 'Jiangang Bai', 'orcid': 'https://orcid.org/0000-0002-2299-2464'}, 'institutions': [{'id': 'https://openalex.org/I20231570', 'display_name': 'Peking University', 'ror': 'https://ror.org/02v51f717', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I20231570']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Jiangang Bai', 'raw_affiliation_strings': ['Peking University, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Peking University, Beijing, China', 'institution_ids': ['https://openalex.org/I20231570']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100776575', 'display_name': 'Yujing Wang', 'orcid': 'https://orcid.org/0000-0002-6334-245X'}, 'institutions': [{'id': 'https://openalex.org/I20231570', 'display_name': 'Peking University', 'ror': 'https://ror.org/02v51f717', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I20231570']}, {'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Yujing Wang', 'raw_affiliation_strings': ['Microsoft Research Asia, Beijing, China', 'Peking University, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Peking University, Beijing, China', 'institution_ids': ['https://openalex.org/I20231570']}, {'raw_affiliation_string': 'Microsoft Research Asia, Beijing, China', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5101786165', 'display_name': 'Yiren Chen', 'orcid': 'https://orcid.org/0000-0003-2538-1105'}, 'institutions': [{'id': 'https://openalex.org/I20231570', 'display_name': 'Peking University', 'ror': 'https://ror.org/02v51f717', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I20231570']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Yiren Chen', 'raw_affiliation_strings': ['Peking University, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Peking University, Beijing, China', 'institution_ids': ['https://openalex.org/I20231570']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5102968006', 'display_name': 'Yaming Yang', 'orcid': 'https://orcid.org/0000-0001-7575-5780'}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Yaming Yang', 'raw_affiliation_strings': ['Microsoft Research Asia, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia, Beijing, China', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5006792341', 'display_name': 'Jing Bai', 'orcid': 'https://orcid.org/0000-0003-4247-6210'}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Jing Bai', 'raw_affiliation_strings': ['Microsoft Research Asia, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia, Beijing, China', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5101672023', 'display_name': 'Jing Yu', 'orcid': 'https://orcid.org/0000-0002-7577-5893'}, 'institutions': [{'id': 'https://openalex.org/I4210156404', 'display_name': 'Institute of Information Engineering', 'ror': 'https://ror.org/04r53se39', 'country_code': 'CN', 'type': 'facility', 'lineage': ['https://openalex.org/I19820366', 'https://openalex.org/I4210156404']}, {'id': 'https://openalex.org/I19820366', 'display_name': 'Chinese Academy of Sciences', 'ror': 'https://ror.org/034t30j35', 'country_code': 'CN', 'type': 'government', 'lineage': ['https://openalex.org/I19820366']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Jing Yu', 'raw_affiliation_strings': ['Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China', 'institution_ids': ['https://openalex.org/I4210156404', 'https://openalex.org/I19820366']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5024097240', 'display_name': 'Yunhai Tong', 'orcid': 'https://orcid.org/0000-0001-8735-2516'}, 'institutions': [{'id': 'https://openalex.org/I20231570', 'display_name': 'Peking University', 'ror': 'https://ror.org/02v51f717', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I20231570']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Yunhai Tong', 'raw_affiliation_strings': ['Peking University, Beijing, China'], 'affiliations': [{'raw_affiliation_string': 'Peking University, Beijing, China', 'institution_ids': ['https://openalex.org/I20231570']}]}], 'institution_assertions': [], 'countries_distinct_count': 1, 'institutions_distinct_count': 4, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 3.661, 'has_fulltext': True, 'fulltext_origin': 'pdf', 'cited_by_count': 38, 'citation_normalized_percentile': {'value': 0.999914, 'is_in_top_1_percent': True, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 97, 'max': 98}, 'biblio': {'volume': None, 'issue': None, 'first_page': None, 'last_page': None}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9998, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 0.9998, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 0.9998, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10260', 'display_name': 'Software Engineering Research', 'score': 0.9812, 'subfield': {'id': 'https://openalex.org/subfields/1710', 'display_name': 'Information Systems'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/computational-linguistics', 'display_name': 'Computational linguistics', 'score': 0.45910218}, {'id': 'https://openalex.org/keywords/chen', 'display_name': 'Chen', 'score': 0.44930226}], 'concepts': [{'id': 'https://openalex.org/C60048249', 'wikidata': 'https://www.wikidata.org/wiki/Q37437', 'display_name': 'Syntax', 'level': 2, 'score': 0.7623646}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.66601115}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.5807984}, {'id': 'https://openalex.org/C58646249', 'wikidata': 'https://www.wikidata.org/wiki/Q127380', 'display_name': 'Abstract syntax tree', 'level': 3, 'score': 0.5415244}, {'id': 'https://openalex.org/C41895202', 'wikidata': 'https://www.wikidata.org/wiki/Q8162', 'display_name': 'Linguistics', 'level': 1, 'score': 0.516153}, {'id': 'https://openalex.org/C199360897', 'wikidata': 'https://www.wikidata.org/wiki/Q9143', 'display_name': 'Programming language', 'level': 1, 'score': 0.46775934}, {'id': 'https://openalex.org/C155092808', 'wikidata': 'https://www.wikidata.org/wiki/Q182557', 'display_name': 'Computational linguistics', 'level': 2, 'score': 0.45910218}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.45514634}, {'id': 'https://openalex.org/C2776085556', 'wikidata': 'https://www.wikidata.org/wiki/Q183361', 'display_name': 'Chen', 'level': 2, 'score': 0.44930226}, {'id': 'https://openalex.org/C66322947', 'wikidata': 'https://www.wikidata.org/wiki/Q11658', 'display_name': 'Transformer', 'level': 3, 'score': 0.43287188}, {'id': 'https://openalex.org/C138885662', 'wikidata': 'https://www.wikidata.org/wiki/Q5891', 'display_name': 'Philosophy', 'level': 0, 'score': 0.18202105}, {'id': 'https://openalex.org/C127413603', 'wikidata': 'https://www.wikidata.org/wiki/Q11023', 'display_name': 'Engineering', 'level': 0, 'score': 0.13633808}, {'id': 'https://openalex.org/C151730666', 'wikidata': 'https://www.wikidata.org/wiki/Q7205', 'display_name': 'Paleontology', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C165801399', 'wikidata': 'https://www.wikidata.org/wiki/Q25428', 'display_name': 'Voltage', 'level': 2, 'score': 0.0}, {'id': 'https://openalex.org/C119599485', 'wikidata': 'https://www.wikidata.org/wiki/Q43035', 'display_name': 'Electrical engineering', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C86803240', 'wikidata': 'https://www.wikidata.org/wiki/Q420', 'display_name': 'Biology', 'level': 0, 'score': 0.0}], 'mesh': [], 'locations_count': 2, 'locations': [{'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.eacl-main.262', 'pdf_url': 'https://aclanthology.org/2021.eacl-main.262.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://arxiv.org/abs/2103.04350', 'pdf_url': 'https://arxiv.org/pdf/2103.04350', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2021.eacl-main.262', 'pdf_url': 'https://aclanthology.org/2021.eacl-main.262.pdf', 'source': None, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'sustainable_development_goals': [{'display_name': 'Quality education', 'score': 0.83, 'id': 'https://metadata.un.org/sdg/4'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 34, 'referenced_works': ['https://openalex.org/W1508977358', 'https://openalex.org/W1840435438', 'https://openalex.org/W2097606805', 'https://openalex.org/W2250539671', 'https://openalex.org/W2250861254', 'https://openalex.org/W2251939518', 'https://openalex.org/W2562220579', 'https://openalex.org/W2798560962', 'https://openalex.org/W2896457183', 'https://openalex.org/W2923014074', 'https://openalex.org/W2940744433', 'https://openalex.org/W2946359678', 'https://openalex.org/W2948947170', 'https://openalex.org/W2962788148', 'https://openalex.org/W2962982474', 'https://openalex.org/W2963310665', 'https://openalex.org/W2963341956', 'https://openalex.org/W2963355447', 'https://openalex.org/W2963403868', 'https://openalex.org/W2963411763', 'https://openalex.org/W2963754491', 'https://openalex.org/W2963846996', 'https://openalex.org/W2964217331', 'https://openalex.org/W2965373594', 'https://openalex.org/W2979748886', 'https://openalex.org/W2987266335', 'https://openalex.org/W2995259046', 'https://openalex.org/W3008282111', 'https://openalex.org/W3082274269', 'https://openalex.org/W3153543512', 'https://openalex.org/W4288089799', 'https://openalex.org/W4289373464', 'https://openalex.org/W4323654151', 'https://openalex.org/W4385245566'], 'related_works': ['https://openalex.org/W2536864162', 'https://openalex.org/W2390421503', 'https://openalex.org/W2387926336', 'https://openalex.org/W2385522099', 'https://openalex.org/W2364428493', 'https://openalex.org/W2354135050', 'https://openalex.org/W2095633838', 'https://openalex.org/W2095381028', 'https://openalex.org/W2077104824', 'https://openalex.org/W1533028283'], 'abstract_inverted_index': {'Jiangang': [0], 'Bai,': [1, 9], 'Yujing': [2], 'Wang,': [3], 'Yiren': [4], 'Chen,': [5], 'Yaming': [6], 'Yang,': [7], 'Jing': [8, 10], 'Yu,': [11], 'Yunhai': [12], 'Tong.': [13], 'Proceedings': [14], 'of': [15, 19, 23], 'the': [16, 20, 24], '16th': [17], 'Conference': [18], 'European': [21], 'Chapter': [22], 'Association': [25], 'for': [26], 'Computational': [27], 'Linguistics:': [28], 'Main': [29], 'Volume.': [30], '2021.': [31]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W3154449322', 'counts_by_year': [{'year': 2024, 'cited_by_count': 10}, {'year': 2023, 'cited_by_count': 19}, {'year': 2022, 'cited_by_count': 8}, {'year': 2021, 'cited_by_count': 1}], 'updated_date': '2025-01-07T08:20:48.063986', 'created_date': '2021-04-26'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
