Title: Deciding Which is to Build and Assessing Success: Learning to Do It Better
Abstract: ABSTRACT This paper investigates the effects of efforts by firms to improve the quality and frequency of evaluation for information systems project ideas and completed systems. We develop a framework, based on organizational learning theory, for the expected impacts, singly and together, of efforts to increase evaluation and to improve evaluation processes. We investigate whether developing the evaluation process leads to more frequent and more thorough evaluation and whether increased evaluation, improved evaluation, or both jointly affect evaluation quality. We found that improved evaluation led to more frequent evaluation. Neither improved evaluation processes nor increased frequency and thoroughness of evaluation clearly led to higher evaluation quality. Jointly they resulted in decidedly mixed results. We infer from the results that managers want better methods and tools to (1) develop better information for IS evaluation, (2) help them use information better in decision making, and (3) better align IS plans and projects with strategic business plans. INTRODUCTION That information systems today are important to firms is as well known in popular culture as it is in the IS research literature. Firms are increasingly dependent on such systems in order to be able to conduct day-to-day operations. Many firms have sought to use innovative IS to slash costs, establish marketing channels, or add information value to products and services. Others have sought to redesign production processes around products, using databases and electronic communications to coordinate diverse activities across functional and even organizational boundaries (Venkatraman 1994). Managers have sought to redefine the organization by using IS capabilities to lend assistance in management coordination and control activities, thereby allowing organizations to distribute supply chains globally, to outsource activities that aren't core competencies, and to squash organizational hierarchies by eliminating layers of management. Given all of these potential opportunities, managers should be eager to apply IS in the firm. Unfortunately, managers as a group aren't confident about their ability to make the right choices among these many opportunities (Peffers and Saarinen 2002). The record on payoffs from IS investments is mixed to the extent that some researchers have suggested that there is no relationship, per se, between IS investments and adding value to the firm, e.g., (Brynjolfsson 1993; Brynjolfsson and Hitt 1998; Dos Santos, Peffers and Mauer 1993). Furthermore, the record of IS projects that have failed to produce the desired results, were unusable, were never finished, or ran away with the budget, is nothing less than shocking (Keil and Mann 2000). Information systems now account for expenses equal to up to 8% of revenue in some industries and often have wide ranging impacts on the organization, so it is essential that managers have the tools to determine whether potential IS investments are worthwhile, in terms of strategy, feasibility, risk, and financial impact on the firm. Furthermore, managers need to be able to evaluate these systems at a strategic level to determine whether they are aligned with firm objectives, as well as to zoom in to evaluate the functionality of specific features and the effectiveness of whole systems (Dos Santos and Peffers 1993). Finally, it is important to evaluate systems at several points over the project life cycle, including at the time of the investment decision, during development, and at the completion of the project (Farbey, Land and Targett 1992; Hallikainen, Heikkila, Peffers, Saarinen and Wijnhoven 1998), so as to ensure that expected benefits are realized. Sadly, although many attempts have been made over the years to develop effective evaluation methods (Lincoln 1986; Ward 1990; Wen and Sylla 1999), real managers don't often use them. Apparently, most decisions about IS investments and development are made intuitively, without the benefit of using any of the formal methods (Hallikainen, Heikkila, Peffers, Saarinen and Wijnhoven 1998). …
Publication Year: 2005
Publication Date: 2005-01-01
Language: en
Type: article
Access and Citation
Cited By Count: 1
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot