Defining Risk Management – Part 4: Risk Quantification

There’s more detail available than what I’m going to go into here or include here from the book excerpt. Mostly, because this article would end up being far too long. I’ll go into Risk Quantification at a higher level here, and then present further detail in a subsequent article.

This information below on Risk Quantification comes again – for the most part – from the book “The Project Management Question and Answer Book.”

What is Risk Quantification?

Risk quantification is the process of evaluating the risks that have been identified and developing the data that will be needed for making decisions as to what should be done about them. Risk management is done from very early in the project until the very end. For this reason qualitative analysis should be used at some points in the project, and quantitative techniques should be used at other times.

The objective of quantification is to establish a way of arranging the risks in the order of importance. In most projects there will not be enough time or money to take action against every risk that is identified.

The severity of the risk is a practical measure for quantifying risks. Severity is a combination of the risk probability and the risk impact. In its simplest form the risks can be ranked as high and low severity or possibly high, medium, and low. At the other extreme, the probability of the risk can be a percentage or a decimal value between zero and one, and the impact can be estimated in dollars. When the impact in dollars and the probability in decimal are multiplied together, the result is the quantitative expected value of the risk.

Various statistical techniques such as PERT (program evaluation and review technique), statistical sampling, sensitivity analysis, decision tree analysis, financial ratios, Monte Carlo, and critical chain can all be used to evaluate and quantify risks.

Qualitative and Quantitative Analysis

Qualitative risk analysis is appropriate early in the project and is effective in categorizing which risks should or should not be planned for and what corrective action should be taken for them. Qualitative analysis techniques will not give us the precise values for the risk that we would like to have. They are very effective when we have little time to evaluate risks before they actually happen.

Quantitative values may be applied to risks when using qualitative analysis. Values such as very risky, not so risky; high and low; high, medium, and low; high, high medium, medium, medium low, and low are generally used. Qualitative evaluation might also evaluate the risks on a scale of one to ten. These values can be applied to both the probability and the impact of the risk. The impact and probability can then be combined to give similar descriptions to the severity of the risk.

If an evaluation of impact and probability used a scaled evaluation of one to ten, the numbers could be multiplied to get the severity. In this way a probability of 7 with an impact of 9 would give us a severity of 63. This number for severity should give us plenty of information for ranking the risks. Using the high, medium, and low version sometimes creates disagreements about risks that are on the borderline between one value and another. For example, does this risk have an impact of medium or high when it is close to the border between the two values? And what happens when the impact is very high or very low and the probability is the opposite?

While qualitative analysis is less precise than quantitative analysis, evaluating the results is far less expensive in terms of both time and money. The results are good enough to indicate the overall risk of the project and identify the high-priority risks in order to begin taking some corrective action. This kind of information may assist in pricing the project to a client.

Quantitative risk analysis attempts to attach specific numerical values to the risks. The severity can be assessed from these numerical values for impact and probability. Numerical techniques for decision analysis are used for this approach. These techniques include Monte Carlo analysis, PERT, computer simulations, decision tree analysis, critical chain scheduling, statistical estimating techniques, and expected value analysis. Generally we find the use of statistics and probability theory to be useful in quantitative analysis.

Care should always be used in quantitative analysis because using a good quantitative technique with bad data is worse than not using the technique at all. Many people are impressed with statistical models and simulations and never look at the data to see how good they are. It is quite possible to impress people into making the wrong decision based on excellent analysis of bad data. Care should also be exercised in the use of quantitative techniques because the cost of applying the technique and collecting the data can sometimes be more than the cost of the risks the technique helps to quantify.

Estimating the Project: Avoiding Potential Pitfalls

One old quote states “Any project can be estimated accurately (once it’s completed).”  Yes…hindsight is always 20-20.  The trick is getting it as close to right at the beginning as possible.

In the Beginning

Where I’ve come from recently, the PM involvement in the beginning of the project is scarce at best.  That wasn’t the case earlier on with Rockwell Collins – the PM was everything…sales, requirements definition, implementation manager, deployment oversight, follow-on tech support.  It was very interesting and challenging wearing every hat.  But more recently projects have come to me fully estimated with a shell project plan already in place and also in the hands of the customer.  That’s scary from a PM’s perspective.

Why is this scary?  Well, the project has been priced without PM input.  The PM – and preferably the Business Analyst, too – should review the requirements and SOW and provide input into the pricing of the project and certainly into the timeline and overall project plan before it is ever given to the customer.

What to Do Differently

I’ve said this repeatedly, but the PM must be involved in the estimating process.  In order to have a solid estimate and ultimately a solid final price to the customer, the PM and BA need to review the requirements and SOW with the Sales or the Account Manager or whoever is serving at this point as the primary POC for the client.  Together, these three entities should arrive at the price that is presented to the client. 

These days, we should not have a “sales guy” doing all of the business development for an organization. We are doing the client a disservice by not putting someone incredibly knowledgeable about the product and the technology as the primary contact or at least including them upfront in the sales process.

Include Everything Reasonable

This is where Lessons Learned from other projects comes into play.  For example, you may know that the data load for this project looks similar to one on a previous project that ended up taking 3 extra weeks.  Well, the sales person doesn’t know that so it’s not priced into the project already if you weren’t part of the sales process.  What do you do in a case like that?  About the only thing you can do once the deal has been signed is bring it up during Kickoff as a potential risk and then track it till it becomes a reality.  Revisit it often so it never becomes a surprise to the customer and even provide them with a ballpark dollar figure on what the added effort will be and have a change order ready to give them if and when it becomes apparent that it is necessary.  And definitely pass this knowledge along to sales and management so future issues like this can be avoided when pricing engagements for the potential customer.

Summary 

I realize this is a very specific example of an issue that probably should have been priced differently – and as you can probably guess this exact situation happened to me, which is why I was able to call it up so easily.  However, this scenario could be applied to just about anything you’ve learned along the way on other projects and then see as a potential risk on a new assigned engagement.  It’s just unfortunate that the individuals with this hands-on “in the trenches” knowledge of the potential pitfalls and risks that could and should affect pricing are not usually involved in the sales process at all. 

Dollar Store Concept of Project Management

We’re all familiar with stores like the Dollar Store, 99 Cents Store, etc., right?  At least here in the US, they’re all over the place.  You can walk into any one of these stores, grab 12 items and know for sure you’re going to pay $12 + tax.  No surprises.  Power steering fluid…$1, socks…$1, wine glass…$1, pregnancy test…$1 (though I’m not sure I’d trust it).  Everything they carry costs exactly the same…one price fits all.  Everything is weighted the same and has equal importance to the store’s bottomline.

Equal Importance?

What if that same concept could be applied to IT shops and to Project Management?  Would tech support be just as important and application development.  What about operations, system architecture, performance tuning.  From a project management standpoint, should requirements definition be just as important as design?  Should design be just as important as development?  Should development be just as important as testing?  If you placed a dollar value on each, would they be the same?  Do they take the same amount of effort?

Usually, on an IT project, development takes longer than design….however there is often a higher dollar resource or amount associated with design.  Requirements analysis and definition is an upfront task that requires specific focus and, I think we can all agree, if not done properly can result in many problems later on in the engagement included significant re-design and significant development re-work.

Likewise, testing is an extremely critical portion of any engagement.  You can spend thousands of hours developing a solution, but if it is not tested properly by the delivery team and then by the customer, the likelihood of a successful deployment is almost non-existent. 

Assigning Value

I guess what I’m trying to say here, in terms of a successfully run project, development – while it is actually the act of creating the solution – probably needs less oversight than other phases.  If the others are done properly, then development should be straight forward.  A couple of dev shops I work with often give a rough estimate on a project at 30% design and 70% development and testing, but the per hour estimate for design is higher.  Likewise, requirements analysis – possibly the most important part of any project – must have the appropriate hours and effort and skill level applied to it or problems will occur repeatedly throughout the engagement.  If a project is started and it is later realized that not enough effort was put into requirements definition, it is far better to shelve the project for a period of time and spend additional effort on requirements definition.  I wish I had done that on the project I wrote about in The Worst Project I Ever Managed.

Moving away from project phases and onto project management tasks…is everything weighted the same?  All items are important – all tasks that are regularly performed by the PM are ingredients for a successful project.  But are they all equally important?  As a PM, if you were stretched so thin that you could only perform a few key tasks for a project, what would you do?  For me, I’d delegate as much as possible and focus on one formal status meeting or conference call per week, one project status report (including issues/risks) delivery per week, one delivery team call per week, and one budget/forecast update per week.  Other things like adhoc communication, delivery of documents, etc. would have to be left to project team members and reviewed during status review calls.

Summary

In my opinion, not all IT and PM tasks and functions are equal.  They aren’t all really assigned the same dollar value.  When push comes to shove, there are certain tasks that need more emphasis, greater focus and – when done right – provide a higher likelihood of project success.

Collaboration Management

There is a very interesting story on collaboration Management written by Teresa Leung at Computer World. It talks about how Business leverage new technologies and tools to promote collaboration within their organziations.

Collaboration has become a buzzword in the past two years. But businesses have been using tools to help workers collaborate for decades. From pure voice tools to unified messaging and high-end telepresence, the range of choices keeps increasing.

According to IDC, Hong Kong’s collaborative application market is estimated to reach HK$106.69 million in 2008 and achieve a CAGR (compound annual growth rate) of 7.9 percent from 2008 to 2013. The research house added that the Asia Pacific market excluding Japan is expected to hit US$368.13 million in 2008 and reach a CAGR of 9.7 percent in the same period.

Hot picks in Hong Kong

Messaging applications, said Sheila Lam, senior market analyst at IDC Asia/Pacific, remain the most widely adopted collaboration software in Hong Kong. She noted that IDC’s definition of messaging apps includes e-mail, instant messaging and unified messaging applications.

“It may not sound very exciting or edgy,” said Lam. “But collaborative applications, by definition, are tools to enable information workers to work together by sharing information and processes. Therefore the value of these tools relies on their pervasiveness, regardless of company size or industry.”

Among these apps, unified messaging is the most popular among business today because it integrates all means of pervasive and mature communication technologies, including e-mail, fax, and voice, Lam said, adding that these technologies have also been integrated as part of business processes.

But as technologies advance and business processes evolve, the popularity of collaborative tools changes accordingly, according to Lam, adding that CIOs shouldn’t ignore other upcoming collaborative tools.

Videoconferencing is one of the examples. The tech first stole the limelight in 2003 when SARS hit Hong Kong. But the high cost of adoption prohibited video conferencing from going mainstream, Lam said.

As the technology and network infrastructure matures, video conferencing is now available at a more affordable price with higher stability, she noted. The cutback of business travel during the current economic recession is expected to raise popularity of video conferencing among enterprises, she added.

Read the entire article here.

When Project Funding Hits the Wall

Ideally your project moves out of Sales and into the actual engagement stage with expectations properly set and a solid timeline and budget in place.  Ideally…  But we all know that there bumps in the road and incorrectly documented or relayed expectations, requirements, etc. 

Customer requirements and expectations may not be on par with the actual derived budget and that potentially major issue may rear it’s ugly head early on or it could even come up very late in the project causing major issues, work stoppages or project cancellation.  It’s our job to avoid those events at all costs, but sometimes they do happen – and we have to react.

Delivery Team Funding Issues

There are several issues that can cause delivery team funding to hit the wall.  Here are some examples:

• Development bug that takes longer than anticipated to complete
• Need to put more resources on a project to complete critical tasks
• Need to put higher priced resources on a project to work through issues
• Data handling and/or loading taking longer than planned
• Underestimated or misunderstood customer requirements

The key difference between delivery team funding issues and customer team funding issues is that they really have to just be worked through.  What delivery organization – in their right mind – is going to just throw up their hands in the middle of an implementation and say, “we’re done…we’re not going to throw any more money at it?”  I realize it’s happened…I’ve even experienced it myself, but it’s much rarer than the customer giving up and not deciding to spend any more on a project.

In reality, the delivery organization is more likely to eat the budget overrun in hopes of completing a successful implementation and retaining an important customer.  Ultimately the amount of the budget overrun will play a role in the decision, but if the budget is being tracked along the way as it should be, then the delivery team has likely been strategizing for some time to lessen the overall financial drain while still working to keep the project going and the customer at least somewhat satisfied.

Customer Team Funding Issues

The more critical issue is when the funding problem is on the customer side.  This can present itself in two forms:

• Project budget overruns have taken their toll on the customer funding and the delivery organization is not willing to eat more overage – meaning it’s now the customer’s problem
• Internal issues at the customer organization are affecting their ability to fund remaining portions of the project as originally planned

When the issue is customer funding, and if it’s big enough, the project is likely to be cancelled.  However, if it’s only a few thousand dollars, then it may survive.  Negotiations will start between the delivery organization and the customer organization and a deal may be worked out to spread the financial responsibility across both or the delivery organization may take the full load in order to keep the project going.  But if the budget issue is large enough, then there is likely no other course the customer will take than to cancel the project.

Project Restart?

The outside possibility may remain that the customer will restart the project when funding increases or returns.  If that is an option – and it very well may be (at least the delivery organization is likely to retain that hope for quite awhile), then the deliver organization is faced with a new dilemma…how to keep the project team relatively intact if the project may resume soon.

If there is a possibility that the project will resume, then discussions must happen quickly with the customer to determine what timeframe within which that might occur.  If a restart is a possibility, then it may be possible to make strategic assignments to other projects for the key project personnel can be made in order to keep them somewhat available if the project resumes.

Summary

Budget issues may be the single biggest problem that can occur on a project.  If you’re out of money, you’re out of money.  Throwing more resources at it won’t help – it will only make this particular problem even bigger.  And either way, the project is still dead, and one or both organizations are embarrassed, experienced major failure and may have personnel issues to now deal with.

The best that can happen is to learn key lessons from whatever issues caused the budget issues.  If it’s a poorly run project, then retraining or termination of certain personnel may be necessary.  If it’s poor planning or a sales issue, then measures need to be taken to not repeat the same issues again.  If it’s a delivery organization budget issue, it’s critical that the causes are reviewed, documented and understood so that history does not repeat itself.