Software-Testing Contests: Observations and Lessons Learned

Name: Work Video:
Duration: 3 min 30 s
Description: While a significant amount of resources can be spent on software testing, the software produced may still suffer from low quality. The authors describe their experience of hosting industrysponsored software-testing contests to help undergraduate and graduate students, as well as practitioners, improve their testing skills.
Xingya Wang; Weisong Sun; Linghuan Hu; Yuan Zhao; W. Eric Wong; Zhenyu Chen
{'id': 'https://openalex.org/W2975922643', 'doi': 'https://doi.org/10.1109/mc.2019.2905533', 'title': 'Software-Testing Contests: Observations and Lessons Learned', 'display_name': 'Software-Testing Contests: Observations and Lessons Learned', 'publication_year': 2019, 'publication_date': '2019-09-24', 'ids': {'openalex': 'https://openalex.org/W2975922643', 'doi': 'https://doi.org/10.1109/mc.2019.2905533', 'mag': '2975922643'}, 'language': 'en', 'primary_location': {'is_oa': False, 'landing_page_url': 'https://doi.org/10.1109/mc.2019.2905533', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S178916657', 'display_name': 'Computer', 'issn_l': '0018-9162', 'issn': ['0018-9162', '1558-0814'], 'is_oa': False, 'is_in_doaj': False, 'is_core': True, 'host_organization': 'https://openalex.org/P4310320439', 'host_organization_name': 'IEEE Computer Society', 'host_organization_lineage': ['https://openalex.org/P4310320439', 'https://openalex.org/P4310319808'], 'host_organization_lineage_names': ['IEEE Computer Society', 'Institute of Electrical and Electronics Engineers'], 'type': 'journal'}, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}, 'type': 'article', 'type_crossref': 'journal-article', 'indexed_in': ['crossref'], 'open_access': {'is_oa': False, 'oa_status': 'closed', 'oa_url': None, 'any_repository_has_fulltext': False}, 'authorships': [{'author_position': 'first', 'author': {'id': 'https://openalex.org/A5019841065', 'display_name': 'Xingya Wang', 'orcid': 'https://orcid.org/0000-0002-7331-4831'}, 'institutions': [{'id': 'https://openalex.org/I881766915', 'display_name': 'Nanjing University', 'ror': 'https://ror.org/01rxvg760', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I881766915']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Xingya Wang', 'raw_affiliation_strings': ['Novel Software Technology, Nanjing University, China'], 'affiliations': [{'raw_affiliation_string': 'Novel Software Technology, Nanjing University, China', 'institution_ids': ['https://openalex.org/I881766915']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5013856385', 'display_name': 'Weisong Sun', 'orcid': 'https://orcid.org/0000-0001-9236-8264'}, 'institutions': [{'id': 'https://openalex.org/I881766915', 'display_name': 'Nanjing University', 'ror': 'https://ror.org/01rxvg760', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I881766915']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Weisong Sun', 'raw_affiliation_strings': ['Software, Nanjing University, China'], 'affiliations': [{'raw_affiliation_string': 'Software, Nanjing University, China', 'institution_ids': ['https://openalex.org/I881766915']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5072167229', 'display_name': 'Linghuan Hu', 'orcid': 'https://orcid.org/0000-0002-7913-4591'}, 'institutions': [], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'Linghuan Hu', 'raw_affiliation_strings': ['Computer Science, University of Texas at Dallas, United States'], 'affiliations': [{'raw_affiliation_string': 'Computer Science, University of Texas at Dallas, United States', 'institution_ids': []}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5070714615', 'display_name': 'Yuan Zhao', 'orcid': 'https://orcid.org/0000-0003-1980-6277'}, 'institutions': [{'id': 'https://openalex.org/I881766915', 'display_name': 'Nanjing University', 'ror': 'https://ror.org/01rxvg760', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I881766915']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Yuan Zhao', 'raw_affiliation_strings': ['Software, Nanjing University, China'], 'affiliations': [{'raw_affiliation_string': 'Software, Nanjing University, China', 'institution_ids': ['https://openalex.org/I881766915']}]}, {'author_position': 'middle', 'author': {'id': 'https://openalex.org/A5066624039', 'display_name': 'W. Eric Wong', 'orcid': 'https://orcid.org/0000-0002-1021-4753'}, 'institutions': [], 'countries': ['US'], 'is_corresponding': False, 'raw_author_name': 'W. Eric Wong', 'raw_affiliation_strings': ['Computer Science, University of Texas at Dallas, United States'], 'affiliations': [{'raw_affiliation_string': 'Computer Science, University of Texas at Dallas, United States', 'institution_ids': []}]}, {'author_position': 'last', 'author': {'id': 'https://openalex.org/A5100422935', 'display_name': 'Zhenyu Chen', 'orcid': 'https://orcid.org/0000-0002-9592-7022'}, 'institutions': [{'id': 'https://openalex.org/I881766915', 'display_name': 'Nanjing University', 'ror': 'https://ror.org/01rxvg760', 'country_code': 'CN', 'type': 'education', 'lineage': ['https://openalex.org/I881766915']}], 'countries': ['CN'], 'is_corresponding': False, 'raw_author_name': 'Zhenyu Chen', 'raw_affiliation_strings': ['Software, Nanjing University, China'], 'affiliations': [{'raw_affiliation_string': 'Software, Nanjing University, China', 'institution_ids': ['https://openalex.org/I881766915']}]}], 'institution_assertions': [], 'countries_distinct_count': 2, 'institutions_distinct_count': 1, 'corresponding_author_ids': [], 'corresponding_institution_ids': [], 'apc_list': None, 'apc_paid': None, 'fwci': 0.731, 'has_fulltext': True, 'fulltext_origin': 'ngrams', 'cited_by_count': 2, 'citation_normalized_percentile': {'value': 0.736533, 'is_in_top_1_percent': False, 'is_in_top_10_percent': False}, 'cited_by_percentile_year': {'min': 69, 'max': 74}, 'biblio': {'volume': '52', 'issue': '10', 'first_page': '61', 'last_page': '69'}, 'is_retracted': False, 'is_paratext': False, 'primary_topic': {'id': 'https://openalex.org/T10743', 'display_name': 'Software Testing and Debugging Techniques', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1712', 'display_name': 'Software'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, 'topics': [{'id': 'https://openalex.org/T10743', 'display_name': 'Software Testing and Debugging Techniques', 'score': 0.9999, 'subfield': {'id': 'https://openalex.org/subfields/1712', 'display_name': 'Software'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T10260', 'display_name': 'Software Engineering Research', 'score': 0.9995, 'subfield': {'id': 'https://openalex.org/subfields/1710', 'display_name': 'Information Systems'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}, {'id': 'https://openalex.org/T12423', 'display_name': 'Software Reliability and Analysis Research', 'score': 0.9972, 'subfield': {'id': 'https://openalex.org/subfields/1712', 'display_name': 'Software'}, 'field': {'id': 'https://openalex.org/fields/17', 'display_name': 'Computer Science'}, 'domain': {'id': 'https://openalex.org/domains/3', 'display_name': 'Physical Sciences'}}], 'keywords': [{'id': 'https://openalex.org/keywords/software-testing', 'display_name': 'Software testing', 'score': 0.68173814}, {'id': 'https://openalex.org/keywords/system-integration-testing', 'display_name': 'System integration testing', 'score': 0.5986701}, {'id': 'https://openalex.org/keywords/software-reliability-testing', 'display_name': 'Software reliability testing', 'score': 0.5616835}, {'id': 'https://openalex.org/keywords/software-quality-analyst', 'display_name': 'Software quality analyst', 'score': 0.49505153}, {'id': 'https://openalex.org/keywords/software-performance-testing', 'display_name': 'Software performance testing', 'score': 0.4662152}, {'id': 'https://openalex.org/keywords/software-peer-review', 'display_name': 'Software peer review', 'score': 0.44749212}], 'concepts': [{'id': 'https://openalex.org/C41008148', 'wikidata': 'https://www.wikidata.org/wiki/Q21198', 'display_name': 'Computer science', 'level': 0, 'score': 0.75922084}, {'id': 'https://openalex.org/C2984328558', 'wikidata': 'https://www.wikidata.org/wiki/Q188522', 'display_name': 'Software testing', 'level': 3, 'score': 0.68173814}, {'id': 'https://openalex.org/C115903868', 'wikidata': 'https://www.wikidata.org/wiki/Q80993', 'display_name': 'Software engineering', 'level': 1, 'score': 0.63870764}, {'id': 'https://openalex.org/C111524372', 'wikidata': 'https://www.wikidata.org/wiki/Q7663718', 'display_name': 'System integration testing', 'level': 5, 'score': 0.5986701}, {'id': 'https://openalex.org/C52928878', 'wikidata': 'https://www.wikidata.org/wiki/Q7554226', 'display_name': 'Software reliability testing', 'level': 5, 'score': 0.5616835}, {'id': 'https://openalex.org/C117447612', 'wikidata': 'https://www.wikidata.org/wiki/Q1412670', 'display_name': 'Software quality', 'level': 4, 'score': 0.54375917}, {'id': 'https://openalex.org/C2777904410', 'wikidata': 'https://www.wikidata.org/wiki/Q7397', 'display_name': 'Software', 'level': 2, 'score': 0.51363033}, {'id': 'https://openalex.org/C188329197', 'wikidata': 'https://www.wikidata.org/wiki/Q6554613', 'display_name': 'Software quality analyst', 'level': 5, 'score': 0.49505153}, {'id': 'https://openalex.org/C178059732', 'wikidata': 'https://www.wikidata.org/wiki/Q1982529', 'display_name': 'Software performance testing', 'level': 5, 'score': 0.4662152}, {'id': 'https://openalex.org/C2779530757', 'wikidata': 'https://www.wikidata.org/wiki/Q1207505', 'display_name': 'Quality (philosophy)', 'level': 2, 'score': 0.46119073}, {'id': 'https://openalex.org/C186846655', 'wikidata': 'https://www.wikidata.org/wiki/Q3398377', 'display_name': 'Software construction', 'level': 4, 'score': 0.45755258}, {'id': 'https://openalex.org/C74579156', 'wikidata': 'https://www.wikidata.org/wiki/Q7554342', 'display_name': 'Software peer review', 'level': 5, 'score': 0.44749212}, {'id': 'https://openalex.org/C48002344', 'wikidata': 'https://www.wikidata.org/wiki/Q2919644', 'display_name': 'Verification and validation', 'level': 2, 'score': 0.43347389}, {'id': 'https://openalex.org/C529173508', 'wikidata': 'https://www.wikidata.org/wiki/Q638608', 'display_name': 'Software development', 'level': 3, 'score': 0.39564738}, {'id': 'https://openalex.org/C111919701', 'wikidata': 'https://www.wikidata.org/wiki/Q9135', 'display_name': 'Operating system', 'level': 1, 'score': 0.11716044}, {'id': 'https://openalex.org/C127413603', 'wikidata': 'https://www.wikidata.org/wiki/Q11023', 'display_name': 'Engineering', 'level': 0, 'score': 0.069550246}, {'id': 'https://openalex.org/C138885662', 'wikidata': 'https://www.wikidata.org/wiki/Q5891', 'display_name': 'Philosophy', 'level': 0, 'score': 0.0}, {'id': 'https://openalex.org/C21547014', 'wikidata': 'https://www.wikidata.org/wiki/Q1423657', 'display_name': 'Operations management', 'level': 1, 'score': 0.0}, {'id': 'https://openalex.org/C111472728', 'wikidata': 'https://www.wikidata.org/wiki/Q9471', 'display_name': 'Epistemology', 'level': 1, 'score': 0.0}], 'mesh': [], 'locations_count': 1, 'locations': [{'is_oa': False, 'landing_page_url': 'https://doi.org/10.1109/mc.2019.2905533', 'pdf_url': None, 'source': {'id': 'https://openalex.org/S178916657', 'display_name': 'Computer', 'issn_l': '0018-9162', 'issn': ['0018-9162', '1558-0814'], 'is_oa': False, 'is_in_doaj': False, 'is_core': True, 'host_organization': 'https://openalex.org/P4310320439', 'host_organization_name': 'IEEE Computer Society', 'host_organization_lineage': ['https://openalex.org/P4310320439', 'https://openalex.org/P4310319808'], 'host_organization_lineage_names': ['IEEE Computer Society', 'Institute of Electrical and Electronics Engineers'], 'type': 'journal'}, 'license': None, 'license_id': None, 'version': None, 'is_accepted': False, 'is_published': False}], 'best_oa_location': None, 'sustainable_development_goals': [{'score': 0.48, 'display_name': 'Industry, innovation and infrastructure', 'id': 'https://metadata.un.org/sdg/9'}], 'grants': [{'funder': 'https://openalex.org/F4320321001', 'funder_display_name': 'National Natural Science Foundation of China', 'award_id': '61690201'}], 'datasets': [], 'versions': [], 'referenced_works_count': 18, 'referenced_works': ['https://openalex.org/W1964962870', 'https://openalex.org/W1971650562', 'https://openalex.org/W1992896674', 'https://openalex.org/W2033398965', 'https://openalex.org/W2083506996', 'https://openalex.org/W2102742419', 'https://openalex.org/W2105896632', 'https://openalex.org/W2106072155', 'https://openalex.org/W2135937266', 'https://openalex.org/W2138201395', 'https://openalex.org/W2156723666', 'https://openalex.org/W2466388701', 'https://openalex.org/W2616524285', 'https://openalex.org/W2625646106', 'https://openalex.org/W2726443204', 'https://openalex.org/W3141924267', 'https://openalex.org/W4233942029', 'https://openalex.org/W4255156621'], 'related_works': ['https://openalex.org/W3209487211', 'https://openalex.org/W3097506197', 'https://openalex.org/W3088351901', 'https://openalex.org/W2800071253', 'https://openalex.org/W2380462405', 'https://openalex.org/W2062249491', 'https://openalex.org/W2054445351', 'https://openalex.org/W1994545401', 'https://openalex.org/W1923185232', 'https://openalex.org/W1857348236'], 'abstract_inverted_index': {'While': [0], 'a': [1], 'significant': [2], 'amount': [3], 'of': [4, 26], 'resources': [5], 'can': [6], 'be': [7], 'spent': [8], 'on': [9], 'software': [10, 13], 'testing,': [11], 'the': [12], 'produced': [14], 'may': [15], 'still': [16], 'suffer': [17], 'from': [18], 'low': [19], 'quality.': [20], 'The': [21], 'authors': [22], 'describe': [23], 'their': [24, 42], 'experience': [25], 'hosting': [27], 'industrysponsored': [28], 'software-testing': [29], 'contests': [30], 'to': [31], 'help': [32], 'undergraduate': [33], 'and': [34], 'graduate': [35], 'students,': [36], 'as': [37, 39], 'well': [38], 'practitioners,': [40], 'improve': [41], 'testing': [43], 'skills.': [44]}, 'cited_by_api_url': 'https://api.openalex.org/works?filter=cites:W2975922643', 'counts_by_year': [{'year': 2022, 'cited_by_count': 1}, {'year': 2020, 'cited_by_count': 1}], 'updated_date': '2024-12-11T17:11:45.259796', 'created_date': '2019-10-03'}
Publication Information

Basic Information

Access and Citation

AI Researcher Chatbot

Primary Location

Authors

Topics

Keywords

Related Works
