Transformers in Vision: A Survey

Name: Work Video:
Duration: 3 min 30 s
Salman Khan; Muzammal Naseer; Munawar Hayat; Syed Waqas Zamir; Fahad Shahbaz Khan; Mubarak Shah
{'id': 'https://openalex.org/W4206706211', 'doi': 'https://doi.org/10.1145/3505244', 'title': 'Transformers in Vision: A Survey', 'display_name': 'Transformers in Vision: A Survey', 'publication_year': 2022, 'publication_date': '2022-01-06', 'ids': {'openalex': 'https://openalex.org/W4206706211', 'doi': 'https://doi.org/10.1145/3505244'}, 'language': 'en', 'primary_location': {'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/3505244', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S157921468', 'display_name': 'ACM Computing Surveys', 'issn_l': '0360-0300', 'issn': ['0360-0300', '1557-7341'], 'is_oa': False, 'is_in_doaj': False, 'is_core': True, 'host_organization': 'https://openalex.org/P4310319798', 'host_organization_name': 'Association for Computing Machinery', 'host_organization_lineage': ['https://openalex.org/P4310319798'], 'host_organization_lineage_names': ['Association for Computing Machinery'], 'type': 'journal'}, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}, 'type': 'review', 'type_crossref': 'journal-article', 'indexed_in': ['arxiv', 'crossref', 'datacite'], 'open_access': {'is_oa': True, 'oa_status': 'green', 'oa_url': 'http://arxiv.org/pdf/2101.01169', 'any_repository_has_fulltext': True}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5000300751', 'display_name': 'Salman Khan', 'orcid': 'https://orcid.org/0000-0002-9502-1749'}, 'institutions': [{'id': 'https://openalex.org/I4210113480', 'display_name': 'Mohamed bin Zayed University of Artificial Intelligence', 'ror': 'https://ror.org/0258gkt32', 'country_code': 'AE', 'type': 'education', 'lineage': ['https://openalex.org/I4210113480']}, {'id': 'https://openalex.org/I118347636', 'display_name': 'Australian National University', 'ror': 'https://ror.org/019wvm592', 'country_code': 'AU', 'type': 'education', 'lineage': ['https://openalex.org/I118347636']}], 'countries': ['AE', 'AU'], 'is_corresponding': False, 'raw_author_name': 'Salman Khan', 'raw_affiliation_strings': ['MBZUAI, UAE and Australian National University, Canberra, ACT, AU'], 'affiliations': [{'raw_affiliation_string': 'MBZUAI, UAE and Australian National University, Canberra, ACT, AU', 'institution_ids': ['https://openalex.org/I4210113480', 'https://openalex.org/I118347636']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5083105774', 'display_name': 'Muzammal Naseer', 'orcid': 'https://orcid.org/0000-0001-7663-7161'}, 'institutions': [{'id': 'https://openalex.org/I118347636', 'display_name': 'Australian National University', 'ror': 'https://ror.org/019wvm592', 'country_code': 'AU', 'type': 'education', 'lineage': ['https://openalex.org/I118347636']}, {'id': 'https://openalex.org/I4210113480', 'display_name': 'Mohamed bin Zayed University of Artificial Intelligence', 'ror': 'https://ror.org/0258gkt32', 'country_code': 'AE', 'type': 'education', 'lineage': ['https://openalex.org/I4210113480']}], 'countries': ['AE', 'AU'], 'is_corresponding': False, 'raw_author_name': 'Muzammal Naseer', 'raw_affiliation_strings': ['MBZUAI, UAE and Australian National University, Canberra, ACT, AU'], 'affiliations': [{'raw_affiliation_string': 'MBZUAI, UAE and Australian National University, Canberra, ACT, AU', 'institution_ids': ['https://openalex.org/I118347636', 'https://openalex.org/I4210113480']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5041437150', 'display_name': 'Munawar Hayat', 'orcid': 'https://orcid.org/0000-0002-2706-5985'}, 'institutions': [{'id': 'https://openalex.org/I56590836', 'display_name': 'Monash University', 'ror': 'https://ror.org/02bfwt286', 'country_code': 'AU', 'type': 'education', 'lineage': ['https://openalex.org/I56590836']}], 'countries': ['AU'], 'is_corresponding': False, 'raw_author_name': 'Munawar Hayat', 'raw_affiliation_strings': ['Department of DSAI, Faculty of IT, Monash University, Clayton, Victoria, AU'], 'affiliations': [{'raw_affiliation_string': 'Department of DSAI, Faculty of IT, Monash University, Clayton, Victoria, AU', 'institution_ids': ['https://openalex.org/I56590836']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5060899114', 'display_name': 'Syed Waqas Zamir', 'orcid': 'https://orcid.org/0000-0002-7198-0187'}, 'institutions': [{'id': 'https://openalex.org/I4210116052', 'display_name': 'Inception Institute of Artificial Intelligence', 'ror': 'https://ror.org/02664zk40', 'country_code': 'AE', 'type': 'facility', 'lineage': ['https://openalex.org/I4210116052']}], 'countries': ['AE'], 'is_corresponding': False, 'raw_author_name': 'Syed Waqas Zamir', 'raw_affiliation_strings': ['Inception Institute of Artificial Intelligence, Masdar City, Abu Dhabi, UAE'], 'affiliations': [{'raw_affiliation_string': 'Inception Institute of Artificial Intelligence, Masdar City, Abu Dhabi, UAE', 'institution_ids': ['https://openalex.org/I4210116052']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5100760570', 'display_name': 'Fahad Shahbaz Khan', 'orcid': 'https://orcid.org/0000-0002-4263-3143'}, 'institutions': [{'id': 'https://openalex.org/I102134673', 'display_name': 'Linköping University', 'ror': 'https://ror.org/05ynxx418', 'country_code': 'SE', 'type': 'education', 'lineage': ['https://openalex.org/I102134673']}], 'countries': ['SE'], 'is_corresponding': False, 'raw_author_name': 'Fahad Shahbaz Khan', 'raw_affiliation_strings': ['MBZUAI, UAE and CVL, Linköping University, Linköping, Sweden'], 'affiliations': [{'raw_affiliation_string': 'MBZUAI, UAE and CVL, Linköping University, Linköping, Sweden', 'institution_ids': ['https://openalex.org/I102134673']}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5080823547', 'display_name': 'Mubarak Shah', 'orcid': 'https://orcid.org/0000-0001-6172-5572'}, 'institutions': [{'id': 'https://openalex.org/I106165777', 'display_name': 'University of Central Florida', 'ror': 'https://ror.org/036nfer12', 'country_code': 'US', 'type': 'education', 'lineage': ['https://openalex.org/I106165777']}], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Mubarak Shah', 'raw_affiliation_strings': ['CRCV, University of Central Florida, Orlando, FL, USA'], 'affiliations': [{'raw_affiliation_string': 'CRCV, University of Central Florida, Orlando, FL, USA', 'institution_ids': ['https://openalex.org/I106165777']}]}], 'institution_assertions': [], 'countries_distinct_count': 4, 'institutions_distinct_count': 6, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 84.5, 'has_fulltext': True, 'fulltext_origin': 'pdf', 'cited_by_count': 1689, 'citation_normalized_percentile': {'value': 0.996205, 'is_in_top_1_percent': True, 'is_in_top_10_percent': True}, 'cited_by_percentile_year': {'min': 99, 'max': 100}, 'biblio': {'volume': '54', 'issue': '10s', 'first_page': '1', 'last_page': '41'}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T11714', 'display_name': 'Multimodal Machine Learning Applications', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T11714', 'display_name': 'Multimodal Machine Learning Applications', 'score': 1.0, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T11307', 'display_name': 'Domain Adaptation and Few-Shot Learning', 'score': 0.9997, 'subfield': {'id': 'https://openalex.org/subfields/1702', 'display_name': 'Artificial Intelligence'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10036', 'display_name': 'Advanced Neural Network Applications', 'score': 0.9997, 'subfield': {'id': 'https://openalex.org/subfields/1707', 'display_name': 'Computer Vision and Pattern Recognition'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [], 'concepts': [{'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.8297713}, {'id': 'https://openalex.org/C66322947', 'wikidata': 'https://www.wikidata.org/wiki/Q11658', 'display_name': 'Transformer', 'level': 3, 'score': 0.63074374}, {'id': 'https://openalex.org/C89600930', 'wikidata': 'https://www.wikidata.org/wiki/Q1423946', 'display_name': 'Segmentation', 'level': 2, 'score': 0.5198903}, {'id': 'https://openalex.org/C154945302', 'wikidata': 'https://www.wikidata.org/wiki/Q11660', 'display_name': 'Artificial intelligence', 'level': 1, 'score': 0.51221484}, {'id': 'https://openalex.org/C9417928', 'wikidata': 'https://www.wikidata.org/wiki/Q1070689', 'display_name': 'Image processing', 'level': 3, 'score': 0.45489213}, {'id': 'https://openalex.org/C48044578', 'wikidata': 'https://www.wikidata.org/wiki/Q727490', 'display_name': 'Scalability', 'level': 2, 'score': 0.4461907}, {'id': 'https://openalex.org/C31972630', 'wikidata': 'https://www.wikidata.org/wiki/Q844240', 'display_name': 'Computer vision', 'level': 1, 'score': 0.37411743}, {'id': 'https://openalex.org/C77088390', 'wikidata': 'https://www.wikidata.org/wiki/Q8513', 'display_name': 'Database', 'level': 1, 'score': 0.1296753}, {'id': 'https://openalex.org/C121332964', 'wikidata': 'https://www.wikidata.org/wiki/Q413', 'display_name': 'Physics', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C62520636', 'wikidata': 'https://www.wikidata.org/wiki/Q944', 'display_name': 'Quantum mechanics', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C165801399', 'wikidata': 'https://www.wikidata.org/wiki/Q25428', 'display_name': 'Voltage', 'level': 2, 'score': 0.0}, {'id': 'https://openalex.org/C115961682', 'wikidata': 'https://www.wikidata.org/wiki/Q860623', 'display_name': 'Image (mathematics)', 'level': 2, 'score': 0.0}], 'mesh': [], 'locations_count': 4, 'locations': [{'is_oa': False, 'landing_page_url': 'https://doi.org/10.1145/3505244', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S157921468', 'display_name': 'ACM Computing Surveys', 'issn_l': '0360-0300', 'issn': ['0360-0300', '1557-7341'], 'is_oa': False, 'is_in_doaj': False, 'is_core': True, 'host_organization': 'https://openalex.org/P4310319798', 'host_organization_name': 'Association for Computing Machinery', 'host_organization_lineage': ['https://openalex.org/P4310319798'], 'host_organization_lineage_names': ['Association for Computing Machinery'], 'type': 'journal'}, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}, {'is_oa': True, 'landing_page_url': 'http://arxiv.org/abs/2101.01169', 'pdf_url': 'http://arxiv.org/pdf/2101.01169', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}, {'is_oa': True, 'landing_page_url': 'https://arxiv.org/abs/2101.01169', 'pdf_url': 'https://arxiv.org/pdf/2101.01169', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}, {'is_oa': False, 'landing_page_url': 'https://api.datacite.org/dois/10.48550/arxiv.2101.01169', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S4393179698', 'display_name': 'DataCite API', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I4210145204', 'host_organization_name': 'DataCite', 'host_organization_lineage': ['https://openalex.org/I4210145204'], 'host_organization_lineage_names': ['DataCite'], 'type': 'metadata'}, 'license': None, 'license_id': None, 'version': None}], 'best_oa_location': {'is_oa': True, 'landing_page_url': 'http://arxiv.org/abs/2101.01169', 'pdf_url': 'http://arxiv.org/pdf/2101.01169', 'source': {'id': 'https://openalex.org/S4306400194', 'display_name': 'arXiv (Cornell University)', 'issn_l': None, 'issn': None, 'is_oa': True, 'is_in_doaj': False, 'is_core': False, 'host_organization': 'https://openalex.org/I205783295', 'host_organization_name': 'Cornell University', 'host_organization_lineage': ['https://openalex.org/I205783295'], 'host_organization_lineage_names': ['Cornell University'], 'type': 'repository'}, 'license': None, 'license_id': None, 'version': 'submittedVersion', 'is_accepted': False, 'is_published': False}, 'sustainable_development_goals': [{'score': 0.69, 'display_name': 'Quality education', 'id': 'https://metadata.un.org/sdg/4'}], 'grants': [], 'datasets': [], 'versions': ['https://openalex.org/W4206706211'], 'referenced_works_count': 288, 'referenced_works': ['https://openalex.org/W1673923490', 'https://openalex.org/W1861492603', 'https://openalex.org/W1895577753', 'https://openalex.org/W1920022804', 'https://openalex.org/W1933349210', 'https://openalex.org/W1959608418', 'https://openalex.org/W2064675550', 'https://openalex.org/W2097073572', 'https://openalex.org/W2101032778', 'https://openalex.org/W2108598243', 'https://openalex.org/W2109586012', 'https://openalex.org/W2118858186', 'https://openalex.org/W2147800946', 'https://openalex.org/W2171943915', 'https://openalex.org/W2183341477', 'https://openalex.org/W2185175083', 'https://openalex.org/W2194775991', 'https://openalex.org/W2251512949', 'https://openalex.org/W2277195237', 'https://openalex.org/W2321533354', 'https://openalex.org/W2326925005', 'https://openalex.org/W2331128040', 'https://openalex.org/W2337252826', 'https://openalex.org/W2340897893', 'https://openalex.org/W2405756170', 'https://openalex.org/W24089286', 'https://openalex.org/W2423557781', 'https://openalex.org/W2487442924', 'https://openalex.org/W2489434015', 'https://openalex.org/W2549139847', 'https://openalex.org/W2565639579', 'https://openalex.org/W2580080206', 'https://openalex.org/W2593116425', 'https://openalex.org/W2601564443', 'https://openalex.org/W2619947201', 'https://openalex.org/W2737258237', 'https://openalex.org/W2747898905', 'https://openalex.org/W2762941833', 'https://openalex.org/W2781228439', 'https://openalex.org/W2785325870', 'https://openalex.org/W2798040152', 'https://openalex.org/W2799087757', 'https://openalex.org/W2804078698', 'https://openalex.org/W2866634454', 'https://openalex.org/W2885820039', 'https://openalex.org/W2886641317', 'https://openalex.org/W2888101838', 'https://openalex.org/W2890052321', 'https://openalex.org/W2891158090', 'https://openalex.org/W2895240252', 'https://openalex.org/W2895748257', 'https://openalex.org/W2896457183', 'https://openalex.org/W2908802752', 'https://openalex.org/W2919115771', 'https://openalex.org/W2926645869', 'https://openalex.org/W2944006115', 'https://openalex.org/W2944828972', 'https://openalex.org/W2946794439', 'https://openalex.org/W2947821610', 'https://openalex.org/W2948012107', 'https://openalex.org/W2952132648', 'https://openalex.org/W2953273646', 'https://openalex.org/W2954930822', 'https://openalex.org/W2955425717', 'https://openalex.org/W2961193895', 'https://openalex.org/W2962931121', 'https://openalex.org/W2962964995', 'https://openalex.org/W2963037989', 'https://openalex.org/W2963091558', 'https://openalex.org/W2963109634', 'https://openalex.org/W2963115613', 'https://openalex.org/W2963121255', 'https://openalex.org/W2963163163', 'https://openalex.org/W2963372104', 'https://openalex.org/W2963420272', 'https://openalex.org/W2963470893', 'https://openalex.org/W2963530300', 'https://openalex.org/W2963563276', 'https://openalex.org/W2963684088', 'https://openalex.org/W2963807318', 'https://openalex.org/W2963826423', 'https://openalex.org/W2963916161', 'https://openalex.org/W2963925437', 'https://openalex.org/W2963966654', 'https://openalex.org/W2964015378', 'https://openalex.org/W2964037671', 'https://openalex.org/W2964134613', 'https://openalex.org/W2964208960', 'https://openalex.org/W2965373594', 'https://openalex.org/W2966715458', 'https://openalex.org/W2968124245', 'https://openalex.org/W2969262604', 'https://openalex.org/W2969876226', 'https://openalex.org/W2970231061', 'https://openalex.org/W2970869018', 'https://openalex.org/W2971074500', 'https://openalex.org/W2973857456', 'https://openalex.org/W2975357369', 'https://openalex.org/W2976847563', 'https://openalex.org/W2980088508', 'https://openalex.org/W2981413347', 'https://openalex.org/W2981689412', 'https://openalex.org/W2981851019', 'https://openalex.org/W2982723417', 'https://openalex.org/W2983446232', 'https://openalex.org/W2992308087', 'https://openalex.org/W2995253937', 'https://openalex.org/W2997591391', 'https://openalex.org/W2998108143', 'https://openalex.org/W2998356391', 'https://openalex.org/W3000775737', 'https://openalex.org/W3005680577', 'https://openalex.org/W3006983028', 'https://openalex.org/W3009561768', 'https://openalex.org/W3012126539', 'https://openalex.org/W3015468748', 'https://openalex.org/W3016211260', 'https://openalex.org/W3021293129', 'https://openalex.org/W3023371261', 'https://openalex.org/W3033529678', 'https://openalex.org/W3034312118', 'https://openalex.org/W3034429256', 'https://openalex.org/W3034445277', 'https://openalex.org/W3034499084', 'https://openalex.org/W3034500398', 'https://openalex.org/W3034885317', 'https://openalex.org/W3035022492', 'https://openalex.org/W3035060554', 'https://openalex.org/W3035170495', 'https://openalex.org/W3035251378', 'https://openalex.org/W3035422918', 'https://openalex.org/W3035524453', 'https://openalex.org/W3035574324', 'https://openalex.org/W3036139671', 'https://openalex.org/W3040573126', 'https://openalex.org/W3043034704', 'https://openalex.org/W3049455300', 'https://openalex.org/W3090449556', 'https://openalex.org/W3091588028', 'https://openalex.org/W3092462694', 'https://openalex.org/W3094502228', 'https://openalex.org/W3094751268', 'https://openalex.org/W3096533519', 'https://openalex.org/W3096609285', 'https://openalex.org/W3096739052', 'https://openalex.org/W3097065222', 'https://openalex.org/W3098085362', 'https://openalex.org/W3101065397', 'https://openalex.org/W3106250896', 'https://openalex.org/W3108655343', 'https://openalex.org/W3109097593', 'https://openalex.org/W3113320078', 'https://openalex.org/W3118608800', 'https://openalex.org/W3121523901', 'https://openalex.org/W3123615524', 'https://openalex.org/W3126721948', 'https://openalex.org/W3129603602', 'https://openalex.org/W3131500599', 'https://openalex.org/W3132890542', 'https://openalex.org/W3133696297', 'https://openalex.org/W3134582802', 'https://openalex.org/W3136416617', 'https://openalex.org/W3137963805', 'https://openalex.org/W3138516171', 'https://openalex.org/W3139049060', 'https://openalex.org/W3139445856', 'https://openalex.org/W3139633126', 'https://openalex.org/W3145450063', 'https://openalex.org/W3147852756', 'https://openalex.org/W3151130473', 'https://openalex.org/W3153465022', 'https://openalex.org/W3156811085', 'https://openalex.org/W3157528469', 'https://openalex.org/W3158846111', 'https://openalex.org/W3159481202', 'https://openalex.org/W3159619744', 'https://openalex.org/W3159778524', 'https://openalex.org/W3160566314', 'https://openalex.org/W3163747765', 'https://openalex.org/W3164024107', 'https://openalex.org/W3164208409', 'https://openalex.org/W3164540605', 'https://openalex.org/W3166396011', 'https://openalex.org/W3166942762', 'https://openalex.org/W3168101492', 'https://openalex.org/W3168489096', 'https://openalex.org/W3169064633', 'https://openalex.org/W3170778815', 'https://openalex.org/W3170841864', 'https://openalex.org/W3170874841', 'https://openalex.org/W3171087525', 'https://openalex.org/W3171125843', 'https://openalex.org/W3171206729', 'https://openalex.org/W3171516518', 'https://openalex.org/W3171547673', 'https://openalex.org/W3172345956', 'https://openalex.org/W3172801447', 'https://openalex.org/W3172942063', 'https://openalex.org/W3173151551', 'https://openalex.org/W3173220247', 'https://openalex.org/W3173365702', 'https://openalex.org/W3175199633', 'https://openalex.org/W3175466730', 'https://openalex.org/W3175515048', 'https://openalex.org/W3176153963', 'https://openalex.org/W3176196997', 'https://openalex.org/W3176974620', 'https://openalex.org/W3179869055', 'https://openalex.org/W3180355996', 'https://openalex.org/W3190216403', 'https://openalex.org/W3190965961', 'https://openalex.org/W3194042166', 'https://openalex.org/W3199093552', 'https://openalex.org/W3199245537', 'https://openalex.org/W3202742610', 'https://openalex.org/W3204801262', 'https://openalex.org/W3207918547', 'https://openalex.org/W3210279979', 'https://openalex.org/W3211490618', 'https://openalex.org/W4205481600', 'https://openalex.org/W4214490042', 'https://openalex.org/W4214493665', 'https://openalex.org/W4214588794', 'https://openalex.org/W4214612132', 'https://openalex.org/W4214614183', 'https://openalex.org/W4214709605', 'https://openalex.org/W4214732854', 'https://openalex.org/W4214736485', 'https://openalex.org/W4214893857', 'https://openalex.org/W4225672218', 'https://openalex.org/W4287122830', 'https://openalex.org/W4287126759', 'https://openalex.org/W4287183620', 'https://openalex.org/W4287184379', 'https://openalex.org/W4287196978', 'https://openalex.org/W4287200327', 'https://openalex.org/W4287203089', 'https://openalex.org/W4287250198', 'https://openalex.org/W4287251194', 'https://openalex.org/W4287251776', 'https://openalex.org/W4287257983', 'https://openalex.org/W4287283172', 'https://openalex.org/W4287324101', 'https://openalex.org/W4287330514', 'https://openalex.org/W4287330714', 'https://openalex.org/W4287336254', 'https://openalex.org/W4287391717', 'https://openalex.org/W4287551327', 'https://openalex.org/W4287551345', 'https://openalex.org/W4287551398', 'https://openalex.org/W4287646898', 'https://openalex.org/W4287667694', 'https://openalex.org/W4287725215', 'https://openalex.org/W4287748830', 'https://openalex.org/W4287775237', 'https://openalex.org/W4287792756', 'https://openalex.org/W4287803913', 'https://openalex.org/W4288089799', 'https://openalex.org/W4288349206', 'https://openalex.org/W4288375408', 'https://openalex.org/W4289328742', 'https://openalex.org/W4292779060', 'https://openalex.org/W4293680532', 'https://openalex.org/W4295838474', 'https://openalex.org/W4296979096', 'https://openalex.org/W4297606427', 'https://openalex.org/W4297697565', 'https://openalex.org/W4297814571', 'https://openalex.org/W4298395628', 'https://openalex.org/W4299576237', 'https://openalex.org/W4299802238', 'https://openalex.org/W4300687381', 'https://openalex.org/W4300852401', 'https://openalex.org/W4300860215', 'https://openalex.org/W4306884028', 'https://openalex.org/W4309793872', 'https://openalex.org/W4311415873', 'https://openalex.org/W4312812783', 'https://openalex.org/W4313007769', 'https://openalex.org/W4313009245', 'https://openalex.org/W4320013936', 'https://openalex.org/W4323654151', 'https://openalex.org/W4376983087', 'https://openalex.org/W4385245566', 'https://openalex.org/W4394652702', 'https://openalex.org/W4394666973', 'https://openalex.org/W4394671432', 'https://openalex.org/W639708223'], 'related_works': ['https://openalex.org/W97075385', 'https://openalex.org/W4235240664', 'https://openalex.org/W2965083567', 'https://openalex.org/W2889616422', 'https://openalex.org/W2389214306', 'https://openalex.org/W2357523926', 'https://openalex.org/W2095886385', 'https://openalex.org/W2089704382', 'https://openalex.org/W1983399550', 'https://openalex.org/W1838576100'], 'abstract_inverted_index': {'Astounding': [0], 'results': [1], 'from': [2, 52], 'Transformer': [3, 115, 127], 'models': [4, 128], 'on': [5, 108, 238], 'natural': [6], 'language': [7], 'tasks': [8, 113, 166, 179], 'have': [9, 103], 'intrigued': [10], 'the': [11, 70, 126, 130, 143, 215], 'vision': [12, 20, 112, 132, 162, 196], 'community': [13], 'to': [14, 18, 43, 93, 105, 120, 139], 'study': [15], 'their': [16, 23, 61, 230], 'application': [17], 'computer': [19, 131], 'problems.': [21], 'Among': [22], 'salient': [24], 'benefits,': [25], 'Transformers': [26, 55, 74, 146], 'enable': [27], 'modeling': [28], 'long': [29], 'dependencies': [30], 'between': [31], 'input': [32], 'sequence': [33, 40], 'elements': [34], 'and': [35, 63, 83, 89, 98, 151, 174, 185, 202, 204, 211, 218, 229, 242], 'support': [36], 'parallel': [37], 'processing': [38, 76, 87, 189], 'of': [39, 73, 111, 125, 145, 159, 220, 226], 'as': [41, 67], 'compared': [42], 'recurrent': [44], 'networks': [45, 97], 'e.g.,': [46], 'Long': [47], 'short-term': [48], 'memory': [49], '(LSTM).': [50], 'Different': [51], 'convolutional': [53], 'networks,': [54], 'require': [56], 'minimal': [57], 'inductive': [58], 'biases': [59], 'for': [60], 'design': [62, 72, 228], 'are': [64], 'naturally': [65], 'suited': [66], 'set-functions.': [68], 'Furthermore,': [69], 'straightforward': [71], 'allows': [75], 'multiple': [77], 'modalities': [78], '(e.g.,': [79, 167, 180, 190, 197, 207], 'images,': [80], 'videos,': [81], 'text': [82], 'speech)': [84], 'using': [85, 114], 'similar': [86], 'blocks': [88], 'demonstrates': [90], 'excellent': [91], 'scalability': [92], 'very': [94], 'large': [95], 'capacity': [96], 'huge': [99], 'datasets.': [100], 'These': [101], 'strengths': [102], 'led': [104], 'exciting': [106], 'progress': [107], 'a': [109, 122], 'number': [110], 'networks.': [116], 'This': [117], 'survey': [118], 'aims': [119], 'provide': [121, 235], 'comprehensive': [123], 'overview': [124], 'in': [129, 161, 224], 'discipline.': [133], 'We': [134, 154, 213], 'start': [135], 'with': [136], 'an': [137, 236], 'introduction': [138], 'fundamental': [140], 'concepts': [141], 'behind': [142], 'success': [144], 'i.e.,': [147], 'self-attention,': [148], 'large-scale': [149], 'pre-training,': [150], 'bidirectional': [152], 'encoding.': [153], 'then': [155], 'cover': [156], 'extensive': [157], 'applications': [158], 'transformers': [160], 'including': [163], 'popular': [164, 221], 'recognition': [165], 'image': [168, 198, 200], 'classification,': [169], 'object': [170], 'detection,': [171], 'action': [172], 'recognition,': [173, 192], 'segmentation),': [175], 'generative': [176], 'modeling,': [177], 'multi-modal': [178], 'visual-question': [181], 'answering,': [182], 'visual': [183, 186], 'reasoning,': [184], 'grounding),': [187], 'video': [188, 193], 'activity': [191], 'forecasting),': [194], 'low-level': [195], 'super-resolution,': [199], 'enhancement,': [201], 'colorization)': [203], '3D': [205], 'analysis': [206, 237], 'point': [208], 'cloud': [209], 'classification': [210], 'segmentation).': [212], 'compare': [214], 'respective': [216], 'advantages': [217], 'limitations': [219], 'techniques': [222], 'both': [223], 'terms': [225], 'architectural': [227], 'experimental': [231], 'value.': [232], 'Finally,': [233], 'we': [234], 'open': [239], 'research': [240], 'directions': [241], 'possible': [243], 'future': [244], 'works.': [245]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W4206706211', 'counts_by_year': [{'year': 2024, 'cited_by_count': 587}, {'year': 2023, 'cited_by_count': 637}, {'year': 2022, 'cited_by_count': 312}, {'year': 2021, 'cited_by_count': 81}], 'updated_date': '2024-12-15T19:55:16.175686', 'created_date': '2022-01-26'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
