Sometimes identifying whether your project or implementation was a success or failure is not as straightforward as just looking at the metrics involved with project management. It's not always about on time and on budget. And it may not entirely be about customer satisfaction either.
In Henry Lucas' book 'Information Technology for Management” he defines first what an Implementation is and then looks at some of the possible criteria for determining that implementation's relative success or failure factors.
What is project implementation?
Implementation is part of the process of designing a system and is a component of change. We develop a new information system to change existing information processing procedures and often to change the organization itself. Implementation refers to the design team's strategy and actions for seeing that a system is successful and makes a contribution to the organization.
Our definition stresses the long-term nature of implementation. It is part of a process that begins with the very first idea for a system and the changes it will bring. Implementation terminates when the system is successfully integrated with the operations of the organization.
We expect most of the implementation to be concerned with behavioural phenomena since people are expected to change their information processing activities. Implementation becomes more important and difficult as systems design becomes more radical.
If a firm undertakes a major re-engineering project, it should make major changes in tasks to reduce costs and improve productivity in the organization.
Success or Failure?
How do we know that we have successfully implemented a system? Researchers do not agree on an absolute indicator for successful implementation. One appealing approach is a cost/benefit study. In this evaluation, one totals the costs of developing a system and compares them with the dollar benefits resulting from the system.
In theory, this sounds like a good indicator of success, but in practice, it is difficult to provide meaningful estimates. Obtaining the cost side of the ratio is not too much of a problem if adequate records are kept during system development.
However, an evaluation of the benefits of an information system has eluded most analysts. There are a number of categories into which we might classify the benefits or value provided by an application of technology. These categories include:
- Infrastructure
- Required applications
- Applications where technology was the only solution
- Applications providing a direct return
- Applications with indirect returns
- Technology initiatives that are a competitive necessity
- Strategic applications
- Transformational information technology
For only a few of these categories are we likely to be able to demonstrate a direct financial return, which makes it difficult to perform a cost/benefit analysis to determine the 'success” of a system.
As an alternative, we can choose among several indicators of successful implementation for an individual application, depending on the type of system involved. In many instances, use of a system is voluntary. A manager or other user receives a report but does not have to use the information on it or even read the report.
Systems that provide interactive retrieval of information from a database also can often be classified as voluntary. The use of such a system is frequently at the discretion of the user. A manager with a personal computer in his or her office is not required to use it. For the type of system in which use is voluntary, we shall adopt high levels of use as a sign of successful implementation. We can measure use by interviews with users, through questionnaires, or in some instances, by building a monitor into the system to record actual use.
For systems whose use is mandatory, such as a production control system or a computer that provides stock market quotations for a broker, we shall employ the user's evaluation of the system as a measure of success. For example, one can examine user satisfaction, although it will probably be necessary to measure several facets of satisfaction, such as quality of service, timeliness and accuracy of information, and quality of the schedule for operations.
An evaluation might also involve a panel of information processing experts reviewing the design and operation of the system. We should also note that managers might well consider a system to be successful if it accomplishes its objectives. However, to accomplish its objectives, a system must be used. We would also hope that one objective of a system would be extensive use and a high degree of user satisfaction.
Finally, though it is difficult to do, we can try to estimate the impact of a system on individuals and the organization. How has a system affected personal productivity and output quality? Can the organization point to added sales or increased revenues from a competitive application? Can we show that IT has had an impact on performance, either for individuals or the organization?