BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation

Name: Work Video:
Duration: 3 min 30 s
Description: Yuchen Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, Ming Zhou. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
Yuchen Jiang; Tianyu Liu; Shuming Ma; Dongdong Zhang; Jian Yang; Haoyang Huang; Rico Sennrich; Ryan Cotterell; Mrinmaya Sachan; Ming Zhou
{'id': 'https://openalex.org/W4287258879', 'doi': 'https://doi.org/10.18653/v1/2022.naacl-main.111', 'title': 'BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation', 'display_name': 'BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation', 'publication_year': 2022, 'publication_date': '2022-01-01', 'ids': {'openalex': 'https://openalex.org/W4287258879', 'doi': 'https://doi.org/10.18653/v1/2022.naacl-main.111'}, 'language': 'en', 'primary_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2022.naacl-main.111', 'pdf_url': 'https://aclanthology.org/2022.naacl-main.111.pdf', 'source': {'id': 'https://openalex.org/S4363608774', 'display_name': 'Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': True, 'oa_status': 'hybrid', 'oa_url': 'https://aclanthology.org/2022.naacl-main.111.pdf', 'any_repository_has_fulltext': True}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5068475055', 'display_name': 'Yuchen Jiang', 'orcid': 'https://orcid.org/0000-0003-3918-7039'}, 'institutions': [{'id': 'https://openalex.org/I35440088', 'display_name': 'ETH Zurich', 'ror': 'https://ror.org/05a28rw58', 'country_code': 'CH', 'type': 'education', 'lineage': ['https://openalex.org/I2799323385', 'https://openalex.org/I35440088']}], 'countries': ['CH'], 'is_corresponding': False, 'raw_author_name': 'Yuchen Jiang', 'raw_affiliation_strings': ['ETH Zürich'], 'affiliations': [{'raw_affiliation_string': 'ETH Zürich', 'institution_ids': ['https://openalex.org/I35440088']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100379419', 'display_name': 'Tianyu Liu', 'orcid': 'https://orcid.org/0000-0001-8107-8587'}, 'institutions': [{'id': 'https://openalex.org/I35440088', 'display_name': 'ETH Zurich', 'ror': 'https://ror.org/05a28rw58', 'country_code': 'CH', 'type': 'education', 'lineage': ['https://openalex.org/I2799323385', 'https://openalex.org/I35440088']}], 'countries': ['CH'], 'is_corresponding': False, 'raw_author_name': 'Tianyu Liu', 'raw_affiliation_strings': ['ETH Zürich'], 'affiliations': [{'raw_affiliation_string': 'ETH Zürich', 'institution_ids': ['https://openalex.org/I35440088']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5113130010', 'display_name': 'Shuming Ma', 'orcid': 'https://orcid.org/0000-0003-1091-1206'}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Shuming Ma', 'raw_affiliation_strings': ['Microsoft Research Asia'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5091932303', 'display_name': 'Dongdong Zhang', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Dongdong Zhang', 'raw_affiliation_strings': ['Microsoft Research Asia'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5009644404', 'display_name': 'Jian Yang', 'orcid': 'https://orcid.org/0000-0002-4408-1952'}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Jian Yang', 'raw_affiliation_strings': ['Microsoft Research Asia'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5020769158', 'display_name': 'Haoyang Huang', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I4210113369', 'display_name': 'Microsoft Research Asia (China)', 'ror': 'https://ror.org/0300m5276', 'country_code': 'CN', 'type': 'company', 'lineage': ['https://openalex.org/I1290206253', 'https://openalex.org/I4210113369']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Haoyang Huang', 'raw_affiliation_strings': ['Microsoft Research Asia'], 'affiliations': [{'raw_affiliation_string': 'Microsoft Research Asia', 'institution_ids': ['https://openalex.org/I4210113369']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5005771535', 'display_name': 'Rico Sennrich', 'orcid': 'https://orcid.org/0000-0002-1438-4741'}, 'institutions': [{'id': 'https://openalex.org/I202697423', 'display_name': 'University of Zurich', 'ror': 'https://ror.org/02crff812', 'country_code': 'CH', 'type': 'education', 'lineage': ['https://openalex.org/I202697423']}], 'countries': ['CH'], 'is_corresponding': False, 'raw_author_name': 'Rico Sennrich', 'raw_affiliation_strings': ['Universität Zürich'], 'affiliations': [{'raw_affiliation_string': 'Universität Zürich', 'institution_ids': ['https://openalex.org/I202697423']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5061951606', 'display_name': 'Ryan Cotterell', 'orcid': 'https://orcid.org/0000-0003-4080-1833'}, 'institutions': [{'id': 'https://openalex.org/I35440088', 'display_name': 'ETH Zurich', 'ror': 'https://ror.org/05a28rw58', 'country_code': 'CH', 'type': 'education', 'lineage': ['https://openalex.org/I2799323385', 'https://openalex.org/I35440088']}], 'countries': ['CH'], 'is_corresponding': False, 'raw_author_name': 'Ryan Cotterell', 'raw_affiliation_strings': ['ETH Zürich'], 'affiliations': [{'raw_affiliation_string': 'ETH Zürich', 'institution_ids': ['https://openalex.org/I35440088']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5002316432', 'display_name': 'Mrinmaya Sachan', 'orcid': 'https://orcid.org/0000-0001-8787-8681'}, 'institutions': [{'id': 'https://openalex.org/I35440088', 'display_name': 'ETH Zurich', 'ror': 'https://ror.org/05a28rw58', 'country_code': 'CH', 'type': 'education', 'lineage': ['https://openalex.org/I2799323385', 'https://openalex.org/I35440088']}], 'countries': ['CH'], 'is_corresponding': False, 'raw_author_name': 'Mrinmaya Sachan', 'raw_affiliation_strings': ['ETH Zürich'], 'affiliations': [{'raw_affiliation_string': 'ETH Zürich', 'institution_ids': ['https://openalex.org/I35440088']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5100701574', 'display_name': 'Ming Zhou', 'orcid': 'https://orcid.org/0000-0002-5701-1996'}, 'institutions': [], 'countries': [], 'is_corresponding': False, 'raw_author_name': 'Ming Zhou', 'raw_affiliation_strings': ['Langboat.com'], 'affiliations': [{'raw_affiliation_string': 'Langboat.com', 'institution_ids': []}]}], 'institution_assertions': [], 'countries_distinct_count': 2, 'institutions_distinct_count': 3, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 3.694, 'has_fulltext': False, 'cited_by_count': 23, 'citation_normalized_percentile': {'value': 0.999875, 'is_in_top_1_percent': True, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 96, 'max': 97}, 'biblio': {'volume': None, 'issue': None, 'first_page': '1550', 'last_page': '1565'}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10181', 'display_name': 'Natural Language Processing Techniques', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10028', 'display_name': 'Topic Modeling', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11714', 'display_name': 'Multimodal Machine Learning Applications', 'score': 0.9883, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/zhàng', 'display_name': 'Zhàng', 'score': 0.7989143}, {'id': 'https://openalex.org/keywords/computational-linguistics', 'display_name': 'Computational linguistics', 'score': 0.58447313}], 'concepts': [{'id': 'https://openalex.org/C203005215', 'wikidata': 'https://www.wikidata.org/wiki/Q79798', 'display_name': 'Machine translation', 'level': 2, 'score': 0.79951215}, {'id': 'https://openalex.org/C2777045944', 'wikidata': 'https://www.wikidata.org/wiki/Q12170198', 'display_name': 'Zhàng', 'level': 3, 'score': 0.7989143}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.62878233}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.6169426}, {'id': 'https://openalex.org/C176217482', 'wikidata': 'https://www.wikidata.org/wiki/Q860554', 'display_name': 'Metric (unit)', 'level': 2, 'score': 0.6125512}, {'id': 'https://openalex.org/C155092808', 'wikidata': 'https://www.wikidata.org/wiki/Q182557', 'display_name': 'Computational linguistics', 'level': 2, 'score': 0.58447313}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.5520511}, {'id': 'https://openalex.org/C41895202', 'wikidata': 'https://www.wikidata.org/wiki/Q8162', 'display_name': 'Linguistics', 'level': 1, 'score': 0.4041528}, {'id': 'https://openalex.org/C127413603', 'wikidata': 'https://www.wikidata.org/wiki/Q11023', 'display_name': 'Engineering', 'level': 0, 'score': 0.1837771}, {'id': 'https://openalex.org/C95457728', 'wikidata': 'https://www.wikidata.org/wiki/Q309', 'display_name': 'History', 'level': 0, 'score': 0.14613888}, {'id': 'https://openalex.org/C138885662', 'wikidata': 'https://www.wikidata.org/wiki/Q5891', 'display_name': 'Philosophy', 'level': 0, 'score': 0.11429688}, {'id': 'https://openalex.org/C191935318', 'wikidata': 'https://www.wikidata.org/wiki/Q148', 'display_name': 'China', 'level': 2, 'score': 0.05902493}, {'id': 'https://openalex.org/C21547014', 'wikidata': 'https://www.wikidata.org/wiki/Q1423657', 'display_name': 'Operations management', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C166957645', 'wikidata': 'https://www.wikidata.org/wiki/Q23498', 'display_name': 'Archaeology', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 4, 'locations': [{'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2022.naacl-main.111', 'pdf_url': 'https://aclanthology.org/2022.naacl-main.111.pdf', 'source': {'id': 'https://openalex.org/S4363608774', 'display_name': 'Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://hdl.handle.net/20.500.11850/588582', 'pdf_url': 'https://www.research-collection.ethz.ch/bitstream/20.500.11850/588582/2/2022.naacl-main.111.pdf', 'source': {'id': 'https://openalex.org/S4306402302', 'display_name': 'Repository for Publications and Research Data (ETH Zurich)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I35440088', 'host_organization_name': 'ETH Zurich', 'host_organization_lineage': ['https://openalex.org/I35440088'], 'host_organization_lineage_names': ['ETH Zurich'], 'type': 'repository'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://www.zora.uzh.ch/id/eprint/224509/1/2022.naacl_main.111.pdf', 'pdf_url': 'https://www.zora.uzh.ch/id/eprint/224509/1/2022.naacl_main.111.pdf', 'source': {'id': 'https://openalex.org/S4306401281', 'display_name': 'Zurich Open Repository and Archive (University of Zurich)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I202697423', 'host_organization_name': 'University of Zurich', 'host_organization_lineage': ['https://openalex.org/I202697423'], 'host_organization_lineage_names': ['University of Zurich'], 'type': 'repository'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, {'is_oa': True, 'landing_page_url': 'https://arxiv.org/abs/2103.11878', 'pdf_url': 'https://arxiv.org/pdf/2103.11878', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'https://doi.org/10.18653/v1/2022.naacl-main.111', 'pdf_url': 'https://aclanthology.org/2022.naacl-main.111.pdf', 'source': {'id': 'https://openalex.org/S4363608774', 'display_name': 'Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies', 'issn_l': None, 'issn': None, 'is_oa': False, 'is_in_doaj': False, 'is_core': False, 'host_organization': None, 'host_organization_name': None, 'host_organization_lineage': [], 'host_organization_lineage_names': [], 'type': 'conference'}, 'license': 'cc-by', 'license_id': 'https://openalex.org/licenses/cc-by', 'version': 'publishedVersion', 'is_accepted': True, 'is_published': True}, 'sustainable_development_goals': [{'id': 'https://metadata.un.org/sdg/4', 'score': 0.83, 'display_name': 'Quality education'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 45, 'referenced_works': ['https://openalex.org/W1738081185', 'https://openalex.org/W1902237438', 'https://openalex.org/W1906341845', 'https://openalex.org/W1910834847', 'https://openalex.org/W1979402949', 'https://openalex.org/W2101105183', 'https://openalex.org/W2101566153', 'https://openalex.org/W2115030595', 'https://openalex.org/W2143803796', 'https://openalex.org/W2151401338', 'https://openalex.org/W2163038970', 'https://openalex.org/W2166545452', 'https://openalex.org/W22168010', 'https://openalex.org/W2250188786', 'https://openalex.org/W2250234233', 'https://openalex.org/W2250479947', 'https://openalex.org/W2250552440', 'https://openalex.org/W2250645967', 'https://openalex.org/W2331726854', 'https://openalex.org/W2419539795', 'https://openalex.org/W2437005631', 'https://openalex.org/W2508316494', 'https://openalex.org/W2582446770', 'https://openalex.org/W2622101571', 'https://openalex.org/W2729046720', 'https://openalex.org/W2759173152', 'https://openalex.org/W2888159079', 'https://openalex.org/W2902767466', 'https://openalex.org/W2903193068', 'https://openalex.org/W2933138175', 'https://openalex.org/W2950308662', 'https://openalex.org/W2952446148', 'https://openalex.org/W2962712961', 'https://openalex.org/W2963506925', 'https://openalex.org/W2964120396', 'https://openalex.org/W2964130895', 'https://openalex.org/W2964291396', 'https://openalex.org/W2970279348', 'https://openalex.org/W2971248291', 'https://openalex.org/W3106146701', 'https://openalex.org/W3119878165', 'https://openalex.org/W4253787053', 'https://openalex.org/W4300428972', 'https://openalex.org/W4385245566', 'https://openalex.org/W566301461'], 'related_works': ['https://openalex.org/W4386247111', 'https://openalex.org/W4327642362', 'https://openalex.org/W3107474891', 'https://openalex.org/W2775554247', 'https://openalex.org/W2587014613', 'https://openalex.org/W2389678104', 'https://openalex.org/W2375840727', 'https://openalex.org/W2364559953', 'https://openalex.org/W2250213760', 'https://openalex.org/W2110168585'], 'abstract_inverted_index': {'Yuchen': [0], 'Jiang,': [1], 'Tianyu': [2], 'Liu,': [3], 'Shuming': [4], 'Ma,': [5], 'Dongdong': [6], 'Zhang,': [7], 'Jian': [8], 'Yang,': [9], 'Haoyang': [10], 'Huang,': [11], 'Rico': [12], 'Sennrich,': [13], 'Ryan': [14], 'Cotterell,': [15], 'Mrinmaya': [16], 'Sachan,': [17], 'Ming': [18], 'Zhou.': [19], 'Proceedings': [20], 'of': [21, 25, 30], 'the': [22, 26, 31], '2022': [23], 'Conference': [24], 'North': [27], 'American': [28], 'Chapter': [29], 'Association': [32], 'for': [33], 'Computational': [34], 'Linguistics:': [35], 'Human': [36], 'Language': [37], 'Technologies.': [38], '2022.': [39]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W4287258879', 'counts_by_year': [{'year': 2024, 'cited_by_count': 5}, {'year': 2023, 'cited_by_count': 16}, {'year': 2022, 'cited_by_count': 2}], 'updated_date': '2024-12-17T08:49:03.760688', 'created_date': '2022-07-25'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
