A Case for Thorough Testing and Management Oversight

Today I was reading a synopsis of a fiasco that happened in the 1990s with Oxford Health Plans – a successful HMO at the time. If ever there was a case for thorough project management to be wrapped around a major system overhaul and detailed and planned testing to be utilized on a project…this one is surely it.

Here’s the story….

Oxford Health Plans is a successful health maintenance organization (HMO) in the New York area. The firm went public in 1991, and its stock price enjoyed steady growth. In 1997, however, problems with a new computer system. led to significant losses, $120 million in the fourth quarter on top of $78 million in the third quarter. When the company announced its second quarterly loss, its stock price was 75 percent lower than its previous high. It was unable to send out monthly bills for many of its customers, and the company could not track payments to hundreds of doctors and hospitals. During the year, uncollected payments from customers rose to $400 million, while Oxford’s unpaid bills to (caregivers) rose to over $650 million.

The problem began when Oxford started planning a system, based on the Oracle database management system, when it had a little over 200,000 members. By the time the system went live three years later, the HMO had 1.5 million members. The company tried to convert to the new system all at once. While the computer system labored under the load, Oxford management continued its aggressive drive to sign up new members. The new system was intolerant of errors that were accepted in the old one. As a result, an account with thousands of participants might have been rejected for an error in any member’s record.

Some customers refused to pay the HMO after not being billed for months so Oxford had to write off over $100 million in uncollectible bills. The HMO’s failure to pay its bills also angered care providers: At one point it owed Columbia University $16 million and Cornell $17 million for medical services. Oxford lost track of its actual medical costs-information a health care provider needs to set reserves and project liabilities.

While organizations have been implementing IT since the 1950s, we still seem to repeat many of the same problems. Oxford is a clear case of a management failure rather than a technology failure.

Summary

Have you ever heard of one of those projects where you ask yourself – or someone else – what were they thinking? Did they even put a project plan together? Did they even setup any test cases and perform any user acceptance testing at all? Who signed off on all of this? Or worse…have you ever heard this asked about one of your projects?!?

Solid project management practices won’t fix everything and make every project successful, but they often will and certainly will help avoid any frequent occurrence of project nightmares like the one mentioned above.

Establishing Objects and Gaining Conceptual Agreement with the Client

I’m taking a little turn from purely project management concepts and thinking more in terms of the IT Consultant in general. Afterall, many of us working as project managers are at any given time part of a professional services organization that thinks of us basically as consultants to THEIR clients.

Establishing objectives is the starting point of any consulting project. It’s impossible to do anything else until and unless you know the desired ending point. Below are some specific questions to discuss with the client in order to elicit some outcome-based business objectives. Take them with you whenever you sit down with a client or potential client as you work through that initial meeting both selling your services and diving deeper into their need and expected results.

• How would conditions ideally improve as a result of this project?
• Ideally, what would you (the client) like to accomplish?
• What would be the difference in the organization if the project is successful?
• How would your customer (the client’s customer) be better served by this project?
• What is the impact you seek on return on investment/equity/sales/assets?
• What is the impact you seek on shareholder value?
• What is the market share/profitability/productivity improvement expected?
• How will you (the client) be evaluated in terms of the results of the project?
• How would your (the client’s) boss recognize the improvement?
• How would employees notice the difference?
• What precise aspects are most troubling to you – what keeps you up at night?
• What are the top three priorities to be accomplished?

In establishing conceptual agreement about the objectives of the project you are about to undertake together, you – as the consultant – are trying to ensure the following:

• The client is not expecting anything that you cannot deliver to them.
• The client is not expecting anything that is unreasonable under the circumstances and is not within the culture and environment that the project will be performed in.
• There will be no misunderstanding later about why additional work wasn’t performed. The limits and goals of the project will be understood and agreed upon.
• The client is maximizing your contribution and talents on the project so that the project is as effective as possible for the client and as lucrative as possible for you.

If you begin with carefully constructed objectives, you can then create a framework within which the project can be launched. From that, the project can progress toward the agreed upon goals and final solution and draw to a close. Boundaries can be derived from clear objectives. While it seems that a never-ending project is lucrative for you, it actually is not. If the end is never reached, then success is possibly never really understood and realized. Setting clear goals and reaching them is what leads to future work with the same client, not a seemingly ongoing project that drains their financial resources.

Dealing with Conflict on the Project

Conflict on a project is almost always a certainty. A project manager who goes says they never have conflict to deal with on their projects just isn’t paying close enough attention to what’s going on. Or they’re in denial. Conflict is going to happen and it’s the PMs responsibility to help team members and customers control and resolve these conflicts. It must happen…the conflict must be dealt with…in order for success to be realized on the project.

In his book “Project Management Nation,” Jason Charvat discusses controlling conflict on the project in his chapter on Project Change Control.

Controlling Conflict on the Project

On almost every project, the potential for conflict arises at some point. This is a natural trend. The project manager should work proactively with all staff to avoid possible conflicts that may arise. In the event of a conflict, the project manager should be aware that talking can only resolve so much. For situations where conflict cannot be resolved through negotiations or arbitration, it is recommended that the identified individuals be separated or be removed from the project.

It is important to understand that project staff react differently to daily situations and that during the project life cycle, these members all experience various emotions such as joy, sadness, jealousy, anger, frustration, and stress—to name but a few. Many conflicts can be reduced or eliminated by constantly communicating the project objectives to the project team members. Some of the most common conflicts are:

• Conflict over project priorities
• Conflict over administrative procedures
• Personality conflicts
• Lack of respect for one another
• Conflict over technical opinions and performance
• Conflict over staffing resources
• Conflict over cost
• Conflict over schedules

When conflicts do arise, there are several methods to try to resolve them;

• Compromise. Parties consent to agree; each side wins or loses a few points.
• Confrontation. Parties work together to find a solution to the problem.
• Forcing. Power is used to direct the solution. One side gets what the other does not.
• Smoothing. This technique plays down the differences between two groups and gives strong attention to the points of agreement.
• Withdrawal. This technique involves one party removing him- or herself from the conflict.

Defining Implementation Success or Failure

Sometimes indentifying whether your project or implementation was a success or failure is not as straightforward as just looking at the metrics involved with project management. It’s not always about on time and on budget. And it may not entirely be about customer satisfaction either.

In Henry Lucas’ book “Information Technology for Management” he defines first what an Implementation is and then looks at some of the possible criteria for determing that implementation’s relative success or failure factors.

What Is Implementation?

Implementation is part of the process of designing a system and is a component of change. We develop a new information system to change existing information processing procedures and often to change the organization itself. Implementation refers to the design team’s strategy and actions for seeing that a system is successful and makes a contribution to the organization.

Our definition stresses the long-term nature of implementation. It is part of a process that begins with the very first idea for a system and the changes it will bring. Implementation terminates when the system is successfully integrated withthe operations of the organization. We expect most of implementation to be concerned with behavioral phenomena since people are expected to change their information processing activities. Implementation becomes more important and difficult as systems design becomes more radical. If a firm undertakes a maj or reengineering project, it should make major changes in tasks to reduce costs and improve productivity in the organization.

Success or Failure

How do we know that we have successfully implemented a system? Researchers do not agree on an absolute indicator for successful implementation. One appealing approach is a cost/benefit study. In this evaluation, one totals the costs of developing a system and compares them with the dollar benefits resulting from the system.

In theory, this sounds like a good indicator of success, but in practice it is difficult to provide meaningful estimates. Obtaining the cost side of the ratio is not too much of a problem if adequate records are kept during system development. However, an evaluation of the benefits of an information system has eluded most analysts. There are a number of categories into which we might classify the benefits or value provided by an application of technology. These categories include:

• Infrastructure
• Required applications
• Applications where technology was the only solution
• Applications providing a direct return
• Applications with indirect returns
• Technology initiatives that are a competitive necessity
• Strategic applications
• Transformational information technology

For only a few of these categories are we likely to be able to demonstrate a direct financial return, which makes it difficult to perform a cost/benefit analysis to determine the “success” of a system.

As an alternative, we can choose among several indicators of successful implementation for an individual application, depending on the type of system involved. In many instances, use of a system is voluntary. A manager or other user receives a report but does not have to use the information on it or even read the report. Systems that provide interactive retrieval of information from a database also can often be classified as voluntary. The use of such a system is frequently at the discretion of the user. A manager with a personal computer in his or her office is not required to use it. For the type of system in which use is voluntary, we shall adopt high levels of use as a sign of successful implementation. We can measure use by interviews with users, through questionnaires, or in some instances, by building a monitor into the system to record actual use.

For systems whose use is mandatory, such as a production control system or a computer that provides stock market quotations for a broker, we shall employ the user’s evaluation of the system as a measure of success. For example, onecan examine user satisfaction, although it will probably be necessary to measure several facets of satisfaction, such as quality of service, timeliness and accuracy of information, and quality of the schedule for operations. An evaluation might also involve a panel of information processing experts reviewing the design and operation of the system. We should also note that managers might well consider a system to be successful if it accomplishes its objectives. However, to accomplish its objectives, a system must be used. We would also hope that one objective of a system would be extensive use and a high degree of user satisfaction.

Finally, though it is difficult to do, we can try to estimate the impact of a system on individuals and the organization. How has a system affected personal productivity and output quality? Can the organization point to added sales or increased revenues from a competitive application? Can we show that IT has had an impact on performance, either for individuals or the organization?

3 More Tips for Working with Virtual Teams

Yesterday I looked at the roles that a clear project vision, excellent communication, motivational strategies and recognising individual differences have on successfully managing a virtual team.  To recap, a virtual team is one where not all the team members are based in the same location: a non-collacted team.  More and more project teams are like this now, as we work in an global marketplace, and with third-party partners.  Here are three more tips for making sure that your virtual team is as successful as possible:

1.   Resolve conflict effectively

Encourage your team to talk to you, and to each other.  One of the problems with virtual teams is that side conversations can take place, and a conflict could be brewing without you even knowing about it.  It’s everyone’s responsibility to handle conflict professionally – and conflict isn’t always a bad thing.  If two members of the team are having a difficult time, you can step in, but only if you know about it.  Therefore, you should encourage an environment where everyone is responsible for proactively identifying and working to resolve conflict.  Issues can be escalated to you where there is the need for more facilitation, but ideally the team can take some ownership for sorting out low-level issues themselves.

Conflict manifests itself in different ways depending on where you are in the project lifecycle.  In the early days, the issue could be lack of cultural understanding or language problems.  Later on, conflict could be caused by differing approaches to solving a technical or creative problem.  The approach to resolving these conflicts is going to be different every time, but you should at least be aware of the possibility that these will happen – and even more alert to conflict than you would be for a collacted team.

2.   Review performance continually

Just because things are going fine today doesn’t mean they will be tomorrow.  You want to be a high-performing team, but that doesn’t happen overnight.  And on a virtual team, it takes even longer.  Think about how you are going to get to a high-performing state – and that means working out what ‘high-performing’ actually looks like.  Chances are it looks like people working autonomously, referring to team members without having to go through you to open the channels of communication, but with regular progress reports and risk and issue reviews – and of course escalations up to you as the project manager.  Discuss what a high-performing team looks like with your team, and then that gives you a starting point on how to get there.

New people joining the team creates an imbalance in skills and experience, so one of your tasks should be to bring new people into the fold as quickly as possible, so plan an induction process as soon as you realise you have a new starter on the way – or someone gives you a clue that they are going to leave!

3.   Handle crises quickly

Projects have crises.  That’s nothing new.  But in a virtual team you have to act quickly to ensure everyone knows what is going on, and how the problem will be addressed.  One of the issues can be ensuring everyone has the same information at the same time, for example, about a corporate strategy change, or a restructure.  With team members in different time zones this can be difficult, and stopping the rumours can be hard.  Virtual team members can feel isolated at the best of times, so you really need to consider getting everyone together for a briefing in these situations, even if it is only on the phone.

Crises and changes can have an impact on the project, and the team will need to work out together how to step up and meet the challenge.  For example, if someone goes off sick and will not be back at work for a while, what impact will this have on the tasks that person will no longer be able to complete?  Who can step in and do those activities, or if no one is available, what impact will that have on the project’s critical path?  Understanding the issues gets you one step closer to being able to minimse the impact on the project delivery.  And of course, make sure you add an entry to the issue log!

These tips are based on my notes from a presentation by Dr Ginger Levin, PMP, PgMP at the PMI Global Congress North America in October 2009, with some of my own thoughts thrown in.  And there are more bits of advice for managing virtual teams in How to Manage in a Flat World, by Susan Bloch and Philip Whitely.