Interactive deduplication using active learning

Name: Work Video:
Duration: 3 min 30 s
Sunita Sarawagi; Anuradha Bhamidipaty
{'id': 'https://openalex.org/W2067566391', 'doi': 'https://doi.org/10.1145/775047.775087', 'title': 'Interactive deduplication using active learning', 'display_name': 'Interactive deduplication using active learning', 'publication_year': 2002, 'publication_date': '2002-07-23', 'ids': {'openalex': 'https://openalex.org/W2067566391', 'doi': 'https://doi.org/10.1145/775047.775087', 'mag': '2067566391'}, 'language': 'en', 'primary_location': {'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/775047.775087', 'pdf_url': None, 'source': None, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}, 'type': 'article', 'type_crossref': 'proceedings-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': False, 'oa_status': 'closed', 'oa_url': None, 'any_repository_has_fulltext': False}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5031035935', 'display_name': 'Sunita Sarawagi', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I162827531', 'display_name': 'Indian Institute of Technology Bombay', 'ror': 'https://ror.org/02qyf5152', 'country_code': 'IN', 'type': 'education', 'lineage': ['https://openalex.org/I162827531']}], 'countries': ['IN'], 'is_corresponding': False, 'raw_author_name': 'Sunita Sarawagi', 'raw_affiliation_strings': ['IIT-Bombay'], 'affiliations': [{'raw_affiliation_string': 'IIT-Bombay', 'institution_ids': ['https://openalex.org/I162827531']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5039999123', 'display_name': 'Anuradha Bhamidipaty', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I162827531', 'display_name': 'Indian Institute of Technology Bombay', 'ror': 'https://ror.org/02qyf5152', 'country_code': 'IN', 'type': 'education', 'lineage': ['https://openalex.org/I162827531']}], 'countries': ['IN'], 'is_corresponding': False, 'raw_author_name': 'Anuradha Bhamidipaty', 'raw_affiliation_strings': ['IIT-Bombay'], 'affiliations': [{'raw_affiliation_string': 'IIT-Bombay', 'institution_ids': ['https://openalex.org/I162827531']}]}], 'institution_assertions': [], 'countries_distinct_count': 1, 'institutions_distinct_count': 1, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 17.01, 'has_fulltext': True, 'fulltext_origin': 'ngrams', 'cited_by_count': 694, 'citation_normalized_percentile': {'value': 0.999919, 'is_in_top_1_percent': True, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 99, 'max': 100}, 'biblio': {'volume': None, 'issue': None, 'first_page': '269', 'last_page': '278'}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T11269', 'display_name': 'Algorithms and Data Compression', 'score': 0.9975, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T11269', 'display_name': 'Algorithms and Data Compression', 'score': 0.9975, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11719', 'display_name': 'Data Quality and Management', 'score': 0.9973, 'subfield': {'id': 'https://openalex.org/subfields/1803', 'display_name': 'Management Science and Operations Research'}, 'field': {'id': 'https://openalex.org/fields/18', 'display_name': 'Decision Sciences'}, 'domain': {'id': 'https://openalex.org/domains/2', 'display_name': 'Social Sciences'}}, {'id': 'https://openalex.org/T12072', 'display_name': 'Machine Learning and Algorithms', 'score': 0.9916, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/data-deduplication', 'display_name': 'Data deduplication', 'score': 0.96691704}, {'id': 'https://openalex.org/keywords/training-set', 'display_name': 'Training set', 'score': 0.4693457}], 'concepts': [{'id': 'https://openalex.org/C32587265', 'wikidata': 'https://www.wikidata.org/wiki/Q1182260', 'display_name': 'Data deduplication', 'level': 2, 'score': 0.96691704}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.8703091}, {'id': 'https://openalex.org/C95623464', 'wikidata': 'https://www.wikidata.org/wiki/Q1096149', 'display_name': 'Classifier (UML)', 'level': 2, 'score': 0.5736698}, {'id': 'https://openalex.org/C179518139', 'wikidata': 'https://www.wikidata.org/wiki/Q5140297', 'display_name': 'Coding (social sciences)', 'level': 2, 'score': 0.5255655}, {'id': 'https://openalex.org/C26517878', 'wikidata': 'https://www.wikidata.org/wiki/Q228039', 'display_name': 'Key (lock)', 'level': 2, 'score': 0.52043325}, {'id': 'https://openalex.org/C51632099', 'wikidata': 'https://www.wikidata.org/wiki/Q3985153', 'display_name': 'Training set', 'level': 2, 'score': 0.4693457}, {'id': 'https://openalex.org/C119857082', 'wikidata': 'https://www.wikidata.org/wiki/Q2539', 'display_name': 'Machine learning', 'level': 1, 'score': 0.46129167}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.44552812}, {'id': 'https://openalex.org/C14036430', 'wikidata': 'https://www.wikidata.org/wiki/Q3736076', 'display_name': 'Function (biology)', 'level': 2, 'score': 0.41565263}, {'id': 'https://openalex.org/C124101348', 'wikidata': 'https://www.wikidata.org/wiki/Q172491', 'display_name': 'Data mining', 'level': 1, 'score': 0.39830518}, {'id': 'https://openalex.org/C77088390', 'wikidata': 'https://www.wikidata.org/wiki/Q8513', 'display_name': 'Database', 'level': 1, 'score': 0.18886945}, {'id': 'https://openalex.org/C105795698', 'wikidata': 'https://www.wikidata.org/wiki/Q12483', 'display_name': 'Statistics', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C33923547', 'wikidata': 'https://www.wikidata.org/wiki/Q395', 'display_name': 'Mathematics', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C38652104', 'wikidata': 'https://www.wikidata.org/wiki/Q3510521', 'display_name': 'Computer security', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C78458016', 'wikidata': 'https://www.wikidata.org/wiki/Q840400', 'display_name': 'Evolutionary biology', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C86803240', 'wikidata': 'https://www.wikidata.org/wiki/Q420', 'display_name': 'Biology', 'level': 0, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/775047.775087', 'pdf_url': None, 'source': None, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}], 'best_oa_location': None, 'sustainable_development_goals': [], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 39, 'referenced_works': ['https://openalex.org/W107306860', 'https://openalex.org/W1484228408', 'https://openalex.org/W1485156179', 'https://openalex.org/W1486089539', 'https://openalex.org/W1487199280', 'https://openalex.org/W1504694836', 'https://openalex.org/W1507034068', 'https://openalex.org/W1514707997', 'https://openalex.org/W1514940655', 'https://openalex.org/W1523728576', 'https://openalex.org/W1528361845', 'https://openalex.org/W1545344782', 'https://openalex.org/W1612155886', 'https://openalex.org/W1970945111', 'https://openalex.org/W1977245551', 'https://openalex.org/W2001496424', 'https://openalex.org/W2018810220', 'https://openalex.org/W2029873015', 'https://openalex.org/W2036216970', 'https://openalex.org/W2080021732', 'https://openalex.org/W2081869978', 'https://openalex.org/W2106895292', 'https://openalex.org/W2118017131', 'https://openalex.org/W2125055259', 'https://openalex.org/W2135892731', 'https://openalex.org/W2138745909', 'https://openalex.org/W2139212933', 'https://openalex.org/W2150698190', 'https://openalex.org/W2151023586', 'https://openalex.org/W2154498027', 'https://openalex.org/W2161936973', 'https://openalex.org/W2426031434', 'https://openalex.org/W2913519381', 'https://openalex.org/W3022479311', 'https://openalex.org/W3171280080', 'https://openalex.org/W3193477162', 'https://openalex.org/W3204261337', 'https://openalex.org/W4285719527', 'https://openalex.org/W46452414'], 'related_works': ['https://openalex.org/W83146503', 'https://openalex.org/W4206462905', 'https://openalex.org/W3203938600', 'https://openalex.org/W3144870715', 'https://openalex.org/W2981877337', 'https://openalex.org/W2169074127', 'https://openalex.org/W2165396616', 'https://openalex.org/W2163707935', 'https://openalex.org/W2044488462', 'https://openalex.org/W202723009'], 'abstract_inverted_index': {'Deduplication': [0], 'is': [1, 17, 54, 94], 'a': [2, 19, 25, 57, 77, 118, 124, 165], 'key': [3], 'operation': [4], 'in': [5, 14, 34, 111, 163], 'integrating': [6], 'data': [7, 38, 103], 'from': [8], 'multiple': [9], 'sources.': [10], 'The': [11, 65], 'main': [12], 'challenge': [13], 'this': [15, 68], 'task': [16], 'designing': [18], 'function': [20], 'that': [21, 85, 122, 142, 161], 'can': [22], 'resolve': [23], 'when': [24], 'pair': [26], 'of': [27, 36, 52, 67, 82, 90, 117, 127, 149], 'records': [28, 108], 'refer': [29], 'to': [30, 48, 55, 59, 75, 152, 167], 'the': [31, 50, 88, 147], 'same': [32], 'entity': [33], 'spite': [35], 'various': [37, 102, 158], 'inconsistencies.': [39], 'Most': [40], 'existing': [41], 'systems': [42], 'use': [43], 'hand-coded': [44], 'functions.': [45], 'One': [46], 'way': [47], 'overcome': [49], 'tedium': [51], 'hand-coding': [53], 'train': [56], 'classifier': [58], 'distinguish': [60], 'between': [61, 105], 'duplicates': [62], 'and': [63, 79, 173], 'non-duplicates.': [64], 'success': [66], 'method': [69, 126], 'critically': [70], 'hinges': [71], 'on': [72, 138], 'being': [73], 'able': [74], 'provide': [76, 168], 'covering': [78], 'challenging': [80, 130], 'set': [81], 'training': [83, 131], 'pairs': [84, 132], 'bring': [86], 'out': [87], 'subtlety': [89], 'deduplication': [91, 120], 'function.': [92], 'This': [93], 'non-trivial': [95], 'because': [96], 'it': [97], 'requires': [98], 'manually': [99], 'searching': [100], 'for': [101], 'inconsistencies': [104], 'any': [106], 'two': [107], 'spread': [109], 'apart': [110], 'large': [112], 'lists.We': [113], 'present': [114], 'our': [115], 'design': [116, 159], 'learning-based': [119], 'system': [121, 166], 'uses': [123], 'novel': [125], 'interactively': [128], 'discovering': [129], 'using': [133], 'active': [134, 143], 'learning.': [135], 'Our': [136], 'experiments': [137], 'real-life': [139], 'datasets': [140], 'show': [141], 'learning': [144], 'significantly': [145], 'reduces': [146], 'number': [148], 'instances': [150], 'needed': [151], 'achieve': [153], 'high': [154], 'accuracy.': [155], 'We': [156], 'investigate': [157], 'issues': [160], 'arise': [162], 'building': [164], 'interactive': [169], 'response,': [170], 'fast': [171], 'convergence,': [172], 'interpretable': [174], 'output.': [175]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W2067566391', 'counts_by_year': [{'year': 2024, 'cited_by_count': 10}, {'year': 2023, 'cited_by_count': 14}, {'year': 2022, 'cited_by_count': 16}, {'year': 2021, 'cited_by_count': 31}, {'year': 2020, 'cited_by_count': 26}, {'year': 2019, 'cited_by_count': 30}, {'year': 2018, 'cited_by_count': 49}, {'year': 2017, 'cited_by_count': 30}, {'year': 2016, 'cited_by_count': 38}, {'year': 2015, 'cited_by_count': 31}, {'year': 2014, 'cited_by_count': 52}, {'year': 2013, 'cited_by_count': 28}, {'year': 2012, 'cited_by_count': 44}], 'updated_date': '2025-01-06T13:30:15.029128', 'created_date': '2016-06-24'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
