Learning Aligned Cross-Modal Representations from Weakly Aligned Data

Name: Work Video:
Duration: 3 min 30 s
Lluís Castrejón; Yusuf Aytar; Carl Vondrick; Hamed Pirsiavash; Antonio Torralba
{'id': 'https://openalex.org/W2952745240', 'doi': None, 'title': 'Learning Aligned Cross-Modal Representations from Weakly Aligned Data', 'display_name': 'Learning Aligned Cross-Modal Representations from Weakly Aligned Data', 'publication_year': 2016, 'publication_date': '2016-07-25', 'ids': {'openalex': 'https://openalex.org/W2952745240', 'mag': '2952745240'}, 'language': 'en', 'primary_location': {'is_oa': False, 'landing_page_url': 'http://export.arxiv.org/pdf/1607.07295', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}, 'type': 'preprint', 'type_crossref': 'posted-content', 'indexed_in': [], 'open_access': {'is_oa': False, 'oa_status': 'closed', 'oa_url': None, 'any_repository_has_fulltext': False}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5031988630', 'display_name': 'Lluís Castrejón', 'orcid': None}, 'institutions': [{'id': 'https://openalex.org/I185261750', 'display_name': 'University of Toronto', 'ror': 'https://ror.org/03dbr7087', 'country_code': 'CA', 'type': 'education', 'lineage': ['https://openalex.org/I185261750']}], 'countries': ['CA'], 'is_corresponding': False, 'raw_author_name': 'Lluis Castrejon', 'raw_affiliation_strings': ['University of Toronto'], 'affiliations': [{'raw_affiliation_string': 'University of Toronto', 'institution_ids': ['https://openalex.org/I185261750']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5056410851', 'display_name': 'Yusuf Aytar', 'orcid': None}, 'institutions': [], 'countries': [], 'is_corresponding': False, 'raw_author_name': 'Yusuf Aytar', 'raw_affiliation_strings': ['MIT - CSAIL'], 'affiliations': [{'raw_affiliation_string': 'MIT - CSAIL', 'institution_ids': []}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5103033393', 'display_name': 'Carl Vondrick', 'orcid': 'https://orcid.org/0000-0001-5676-2387'}, 'institutions': [], 'countries': [], 'is_corresponding': False, 'raw_author_name': 'Carl Vondrick', 'raw_affiliation_strings': ['MIT - CSAIL'], 'affiliations': [{'raw_affiliation_string': 'MIT - CSAIL', 'institution_ids': []}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5031860916', 'display_name': 'Hamed Pirsiavash', 'orcid': 'https://orcid.org/0000-0002-2528-3305'}, 'institutions': [{'id': 'https://openalex.org/I79272384', 'display_name': 'University of Maryland, Baltimore County', 'ror': 'https://ror.org/02qskvh78', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I79272384']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Hamed Pirsiavash', 'raw_affiliation_strings': ['University of Maryland,Baltimore County'], 'affiliations': [{'raw_affiliation_string': 'University of Maryland,Baltimore County', 'institution_ids': ['https://openalex.org/I79272384']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5085020955', 'display_name': 'Antonio Torralba', 'orcid': 'https://orcid.org/0000-0003-4915-0256'}, 'institutions': [], 'countries': [], 'is_corresponding': False, 'raw_author_name': 'Antonio Torralba', 'raw_affiliation_strings': ['MIT - CSAIL'], 'affiliations': [{'raw_affiliation_string': 'MIT - CSAIL', 'institution_ids': []}]}], 'institution_assertions': [], 'countries_distinct_count': 2, 'institutions_distinct_count': 2, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': None, 'has_fulltext': False, 'cited_by_count': 1, 'citation_normalized_percentile': {'value': 0.381231, 'is_in_top_1_percent': False, 'is_in_top_10_percent': False}, 'cited_by_percentile_year': {'min': 66, 'max': 73}, 'biblio': {'volume': None, 'issue': None, 'first_page': None, 'last_page': None}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10627', 'display_name': 'Advanced Image and Video Retrieval Techniques', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10627', 'display_name': 'Advanced Image and Video Retrieval Techniques', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11714', 'display_name': 'Multimodal Machine Learning Applications', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11307', 'display_name': 'Domain Adaptation and Few-Shot Learning', 'score': 0.9986, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/modalities', 'display_name': 'Modalities', 'score': 0.8551364}, {'id': 'https://openalex.org/keywords/modality', 'display_name': 'Modality (human–computer interaction)', 'score': 0.82901585}, {'id': 'https://openalex.org/keywords/representation', 'display_name': 'Representation', 'score': 0.81192577}, {'id': 'https://openalex.org/keywords/transfer-of-learning', 'display_name': 'Transfer of learning', 'score': 0.5694912}, {'id': 'https://openalex.org/keywords/feature-learning', 'display_name': 'Feature Learning', 'score': 0.43069023}], 'concepts': [{'id': 'https://openalex.org/C2779903281', 'wikidata': 'https://www.wikidata.org/wiki/Q6888026', 'display_name': 'Modalities', 'level': 2, 'score': 0.8551364}, {'id': 'https://openalex.org/C71139939', 'wikidata': 'https://www.wikidata.org/wiki/Q910194', 'display_name': 'Modal', 'level': 2, 'score': 0.85245335}, {'id': 'https://openalex.org/C2780226545', 'wikidata': 'https://www.wikidata.org/wiki/Q6888030', 'display_name': 'Modality (human–computer interaction)', 'level': 2, 'score': 0.82901585}, {'id': 'https://openalex.org/C2776359362', 'wikidata': 'https://www.wikidata.org/wiki/Q2145286', 'display_name': 'Representation (politics)', 'level': 3, 'score': 0.81192577}, {'id': 'https://openalex.org/C94124525', 'wikidata': 'https://www.wikidata.org/wiki/Q912550', 'display_name': 'Categorization', 'level': 2, 'score': 0.7198151}, {'id': 'https://openalex.org/C81363708', 'wikidata': 'https://www.wikidata.org/wiki/Q17084460', 'display_name': 'Convolutional neural network', 'level': 2, 'score': 0.7033517}, {'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.7010056}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.58289987}, {'id': 'https://openalex.org/C150899416', 'wikidata': 'https://www.wikidata.org/wiki/Q1820378', 'display_name': 'Transfer of learning', 'level': 2, 'score': 0.5694912}, {'id': 'https://openalex.org/C59404180', 'wikidata': 'https://www.wikidata.org/wiki/Q17013334', 'display_name': 'Feature learning', 'level': 2, 'score': 0.43069023}, {'id': 'https://openalex.org/C153180895', 'wikidata': 'https://www.wikidata.org/wiki/Q7148389', 'display_name': 'Pattern recognition (psychology)', 'level': 2, 'score': 0.33294258}, {'id': 'https://openalex.org/C204321447', 'wikidata': 'https://www.wikidata.org/wiki/Q30642', 'display_name': 'Natural language processing', 'level': 1, 'score': 0.32678777}, {'id': 'https://openalex.org/C36289849', 'wikidata': 'https://www.wikidata.org/wiki/Q34749', 'display_name': 'Social science', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C185592680', 'wikidata': 'https://www.wikidata.org/wiki/Q2329', 'display_name': 'Chemistry', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C144024400', 'wikidata': 'https://www.wikidata.org/wiki/Q21201', 'display_name': 'Sociology', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C94625758', 'wikidata': 'https://www.wikidata.org/wiki/Q7163', 'display_name': 'Politics', 'level': 2, 'score': 0.0}, {'id': 'https://openalex.org/C17744445', 'wikidata': 'https://www.wikidata.org/wiki/Q36442', 'display_name': 'Political science', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C188027245', 'wikidata': 'https://www.wikidata.org/wiki/Q750446', 'display_name': 'Polymer chemistry', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C199539241', 'wikidata': 'https://www.wikidata.org/wiki/Q7748', 'display_name': 'Law', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': False, 'landing_page_url': 'http://export.arxiv.org/pdf/1607.07295', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}], 'best_oa_location': None, 'sustainable_development_goals': [{'id': 'https://metadata.un.org/sdg/10', 'score': 0.49, 'display_name': 'Reduced inequalities'}], 'grants': [], 'datasets': [], 'versions': [], 'referenced_works_count': 42, 'referenced_works': ['https://openalex.org/W118823016', 'https://openalex.org/W1514535095', 'https://openalex.org/W1527575280', 'https://openalex.org/W1722318740', 'https://openalex.org/W1732154880', 'https://openalex.org/W1852255964', 'https://openalex.org/W1982185844', 'https://openalex.org/W1982428585', 'https://openalex.org/W1996418862', 'https://openalex.org/W2017814585', 'https://openalex.org/W2031342017', 'https://openalex.org/W2100235303', 'https://openalex.org/W2106277773', 'https://openalex.org/W2115733720', 'https://openalex.org/W2115752676', 'https://openalex.org/W2123024445', 'https://openalex.org/W2125436662', 'https://openalex.org/W2128053425', 'https://openalex.org/W2134270519', 'https://openalex.org/W2134670479', 'https://openalex.org/W2144656844', 'https://openalex.org/W2150295085', 'https://openalex.org/W2153404544', 'https://openalex.org/W2155541015', 'https://openalex.org/W2155797815', 'https://openalex.org/W2155853132', 'https://openalex.org/W2155871590', 'https://openalex.org/W2184188583', 'https://openalex.org/W2196697617', 'https://openalex.org/W2210322478', 'https://openalex.org/W2296073425', 'https://openalex.org/W2950094539', 'https://openalex.org/W2950178297', 'https://openalex.org/W2950276680', 'https://openalex.org/W2951638509', 'https://openalex.org/W2951670162', 'https://openalex.org/W2952320381', 'https://openalex.org/W2952751501', 'https://openalex.org/W2953226914', 'https://openalex.org/W848364640', 'https://openalex.org/W870084106', 'https://openalex.org/W93016980'], 'related_works': ['https://openalex.org/W3201736332', 'https://openalex.org/W3184212829', 'https://openalex.org/W3184053512', 'https://openalex.org/W3154243634', 'https://openalex.org/W3123388024', 'https://openalex.org/W3038842520', 'https://openalex.org/W3034982658', 'https://openalex.org/W3023745006', 'https://openalex.org/W3020457430', 'https://openalex.org/W3005864431', 'https://openalex.org/W3004934977', 'https://openalex.org/W2994713247', 'https://openalex.org/W2962868176', 'https://openalex.org/W2952928463', 'https://openalex.org/W2950033033', 'https://openalex.org/W2808730398', 'https://openalex.org/W2526174222', 'https://openalex.org/W2474574787', 'https://openalex.org/W2292933192', 'https://openalex.org/W2252238675'], 'abstract_inverted_index': {'People': [0], 'can': [1, 41, 92], 'recognize': [2], 'scenes': [3, 44], 'across': [4, 24, 54, 96], 'many': [5], 'different': [6], 'modalities': [7, 97], 'beyond': [8], 'natural': [9], 'images.': [10], 'In': [11], 'this': [12, 28], 'paper,': [13], 'we': [14, 30], 'investigate': [15], 'how': [16], 'to': [17, 66, 113], 'learn': [18, 48], 'cross-modal': [19, 34, 43, 60, 68], 'scene': [20, 35, 90], 'representations': [21, 95], 'that': [22, 73, 79, 88, 104, 111], 'transfer': [23, 61, 94], 'modalities.': [25], 'To': [26], 'study': [27], 'problem,': [29], 'introduce': [31], 'a': [32, 76], 'new': [33], 'dataset.': [36], 'While': [37], 'convolutional': [38, 69], 'neural': [39, 70], 'networks': [40, 71], 'categorize': [42], 'well,': [45], 'they': [46, 74], 'also': [47], 'an': [49], 'intermediate': [50], 'representation': [51, 78, 91, 110], 'not': [52], 'aligned': [53], 'modalities,': [55], 'which': [56], 'is': [57, 80], 'undesirable': [58], 'for': [59, 98], 'applications.': [62], 'We': [63], 'present': [64], 'methods': [65], 'regularize': [67], 'so': [72], 'have': [75], 'shared': [77, 109], 'agnostic': [81], 'of': [82, 119], 'the': [83, 108, 120], 'modality.': [84, 121], 'Our': [85], 'experiments': [86], 'suggest': [87, 103], 'our': [89, 101], 'help': [93], 'retrieval.': [99], 'Moreover,': [100], 'visualizations': [102], 'units': [105], 'emerge': [106], 'in': [107], 'tend': [112], 'activate': [114], 'on': [115], 'consistent': [116], 'concepts': [117], 'independently': [118]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W2952745240', 'counts_by_year': [{'year': 2018, 'cited_by_count': 1}], 'updated_date': '2024-12-14T22:03:28.540609', 'created_date': '2019-06-27'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
