The Lessons Learned Survey

Another useful tool in the Lessons Learned process is the Lessons Learned Survey.  This is a survey that can be sent to team members during or after a project, to solicit their feedback on how the project was conducted. It applies to any project; and questions can easily be added to focus on additional areas for your project.

A survey like this can be very useful for capturing lessons learned from the project while they’re fresh in people’s mind. The results can be summarized and recommendations passed on to future teams.

It’s usually best to send this survey by email to members of the project team. Let them know that results can be kept anonymous (to encourage people to be frank in their assessments). Send out this survey before any group “lessons learned” meetings. The feedback you receive from the survey can help point to particular areas that should get special exploration in the group lessons learned meeting.

Again, use this if it’s meaningful to your projects – it’s just another template or project tool that I’ve picked up along the way and wanted to share with our readers.  I’ve found it to be helpful in gathering post-project information from my team members on several key projects.

Post-Project Survey

SECTION 1: General Project Issues and Communication

The questions are geared to your particular project but wherever appropriate you can comment about release-level issues too.

Note: add any particular comments you wish….

1. How clearly defined were the objectives for this project?

___Very ____somewhat ___not very ___not at all

2. How clearly defined were the objectives for your work?

___Very ____somewhat ___not very ___not at all

3. How clear were you on your role in the project?

___Very ____somewhat ___not very ___not at all

4. How adequately involved did you feel in project decisions?

___Very ____somewhat ___not very ___not at all

If you did not feel involved, what decisions did you feel left out of?

5. How efficient and effective were project team meetings?

___Very ____somewhat ___not very ___not at all

What would you change?

6. How efficient and effective were technical meetings?

___Very ____somewhat ___not very ___not at all

What would you change?

___Very ____somewhat ___not very ___not at all

8. How adequate has cross-functional participation been?

___Very ____somewhat ___not very ___not at all

What were the problems encountered in the project-functional area relationship, why, and how could they be fixed? What cross-functional participation, if any, was lacking?

9. Do you feel appreciated, recognized and rewarded for your efforts?

___Very ____somewhat ___not very ___not at all

What if anything has been lacking?

10. To what degree is do you feel the entire team was committed to the project schedule?

___Very ____somewhat ___not very ___not at all

What if any issues are there?

11. To what degree have any “people issues” gotten in the way?

___Very ____somewhat ___not very ___not at all

What issues?

12. What communication, organization, structural problems in general were encountered, and how could we have done better in these areas?

SECTION 2: Schedule Estimation Issues

NOTE: This survey isn’t intended to collect exhaustive data on everything right now. We might decide after the post-mortem to work with a sub-group to investigate certain estimation issues, in order to help us improve next time around. This survey just helps us ferret out the rough scope of any issues.

Which of the following estimation issues did you personally have and what was the impact?

___ I was diverted to work on another project full-time.

Project: ________________________________________ Diverted for how long? ________________

Impact on your project work: ____________________________________________________________

___ I overestimated the amount of time I would have each week to work on this project.

The other work that interfered was ________________________________________________________

The amount of time per week it took up was ________________________________________________

Impact: calendar schedule slip of ____days ____weeks ____ months

___ My initial schedule did not include some pieces of technical design or coding work that I subsequently realized I had to do.

Describe briefly: ______________________________________________________________________

Impact (additional hours of work): ________________________________________________________

___ My initial schedule did not take into account certain project “other” work such as attending other people’s design reviews, doing two rounds of my own design reviews, etc.

Describe: ____________________________________________________________________________

Impact: calendar slip to my work of __ days __ weeks __ months

___ My estimates for particular tasks were not accurate.

Describe: type of task, how “off” the estimate was (days, weeks).

Why was it difficult to estimate?

What would help produce better estimates next time?

__ I unexpectedly had to re-do some work.

Describe: (Did something in the system design change that forced you to redesign? Was there a spec misunderstanding? etc.)___

Impact on your schedule: _______________________________________________________________

What could have helped prevent the problem?

Knowing what you know now, how would you do the scheduling/estimating process differently next time to avoid any problems noted above?

SECTION 3: Design, Implementation, Test Processes

1. How effective was our architecture/system design process in phase 2 and 3?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

2. How effective were our functional specs?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

3. How effective were our design (or implementation) specs?

__Very __somewhat ___not very ___not at all

Comments:__________________________________________________________________________

4. How effective were our design reviews?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

5. How effective were our code reviews or hardware reviews?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

6. How well were interfaces defined?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

7. How well were design and interface decisions documented?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

8. How effective has interaction/cooperation between technical “Sub-teams” been?

___Very ____somewhat ___not very ___not at all

Comments:_________________________________________________________________________

9. How useful was your unit testing?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

Did you take unit testing into account in your schedule? ______________________________________

10. How smooth do you feel Integration has been?

__very __somewhat ___not very ___not at all

Comments (why/why not?):___________________________________________________________

11. How comprehensive was integration testing, especially to allow SQA testing to get off to a good start?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

12. How well is the build process working?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

13. To what degree did you have the tools you needed for testing?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

SECTION 4: Perceived Project Life Cycle, Development, or Process Issues

1. Is there any way in which you think our development process hampered this project?

If so, how?

2. What would you change about our development process?

3. What would you like to better understand or see better documented about how to use our process on this type of project?

SECTION 5: Closing

1. What were up to 5 main causes for schedule slips, and how could we avoid those causes in the future?

2. Was the project significantly delayed/hampered by outside dependencies (outside to the project, that is)? Which ones? How can we resolve these issues?

3. What were the main bottlenecks on the process?

4. What were the main sources of frustration in the project?

6. For the next project, how could we improve on the way project was conducted?

PPM Hero

Ok…this one caught me off guard. It came in the way of a nice follow-up comment to my post on “The Attributes of a Successful Project Manager – Part 3” and it came from Gregor Petri from CA (formerly Computer Associates – name changed in 2006).

Here is Gregor’s original comment:

“Brad, a very thorough and serious exploration of the topic. May I point you and your readers to our PPM Hero game for a bit more of a light hearted approach to the topic? http://www.ppmhero.com”

At this point, my curiosity got the best of me and I had to check and see what PPM Hero was. It’s a game that appears to be created by CA in association with Butler Group, the IT end user division of Datamonitor. What it amounts to is a project setting game where you’re answering questions in a pac-man type layout (although the graphics are waaaay more advanced) and trying to stay on budget with your project.

The Scenario

As Gregor states, it’s a lighthearted approach to testing your PM knowledge and trying to correctly identify some of the key elements that drive successful projects. The scenario – at least the one I encountered – is that a PM is knocked out of work and you have to take over their big project. You answer questions as you move your player over the “Q” - successfully answering questions adds money to your project budget.

Your project budget drops with time and for answering questions wrong. The red flashing Project Police that are moving around can also cut your budget. The levels of the game – called ‘floors’ are Information Technology, Business Operations, and finally Executive Offices. And, of course, the questions get harder as you progress through the game. You can challenge friends to beat your score, there’s a leaderboard for all to see, and you can even submit your own questions for consideration in the game.

Give It a Try

We all have major responsibilities on our shoulders most of the time – sometimes our leadership of the projects we manage can really be a make-it-or-break-it type of situation if our client or delivery organization is small enough for us to make that much of a difference to the bottom line. No matter what, being a PM is no easy task and definitely comes with major pressures and stress. So, take a break and try PPM Hero – it’s an interesting sidebar to the work we normally do as PMs. Thanks Gregor for passing this on to us.

Managing Multiple Projects – Stagger the Lifecycles

I was reading a book on managing projects and how to handle multiple projects at once and this concept jumped out at me. Not that we don’t all know it…I think we may just take it for granted.

Re-Stating the Obvious

I’ve mentioned many times that I’ve usually been managing multiple engagements at once – and I believe that most project managers are probably in that situation much of the time. Usually, you have a group of 5-6 active projects that you’re overseeing at any given time. What I have never discussed is the concept of how you survive that – at least not in any detail. Not that I’m going to really break any new ground here…but it was a bit of a revelation (for some reason) to me that having these projects in much different points in their project life cycles is what keeps us sane. I guess I was just taking it for granted.

One of the most frustrating times I’ve had recently was about 2 years ago when I actually had to kickoff two projects within a week of each other (one was the project where my business analyst ended up crying in the hall and during a customer meeting – see “The Worst Project I Ever Managed”).

Avoid Overload

Having your project life cycles staggered is definitely one key to sanity and greatly increases your chances of success. One can not sanely go through life managing 5 projects that are at the same point in their project life cycles at the same time. The demands on the project manager vary by phase – and are weighted more heavily at the front end and usually then again around Testing and Deployment. The lull usually occurs with Design and Development – phases that require much more attention from business analysts and developers than the project manager other than general oversight and reporting.

As happened with me during that one approximate 6 week period nearly two years ago is that I kicked off two projects a week apart and then took both of them into Exploration (that’s where the breakdown occurred). Kickoff and Exploration are both PM-intensive phases and I was stretched thin along with having 4 other less time-intensive projects to manage also. I’ve written about how the situation was righted, but it still took awhile for sanity to return because even though the situation with the customer on the initially troubled project was corrected and expectations were reset, we were all still stretched too thin.

Summary

If you find yourself assigned to projects that are critical, visible, vital, etc. etc. AND they happen to have PM heavy phases overlapping, then do yourself, your customer, your team and your organization a favor and step back to assess the situation. Don’t try to be Superman. If you can’t provide each project the amount of attention that is needed, let your management know. Better to offload a project than to fail at one or more of them. Way better…. There’s nothing wrong with admitting when your plate is too full.

Project Management: How to Successfully Kickoff a Project

In my article titled “The Project Kickoff” I discussed how the project manager should handle a typical project kickoff session with an external customer on a sizeable engagement. In that post I covered what happened in the Kickoff phase – Phase 1 of the overall PM methodology I presented over eight articles and in summary form in “A Quick Guide to Project Management Methodology“.

One can say I use PMI, etc., but that doesn’t really tell the story. PMI provides the common terminology and practices, but what it doesn’t seem to do a great job of is laying out an entire PM methodology…one that you can follow, build a project plan from that means something in the real world, and manage the project and team from. The Quick Guide I laid out does allow that…and here I’d like to talk more about how the project manager successfully kicks off a project.

The Assumptions

For the purposes of this discussion, we’re going to assume that the project we’re talking about is visible, high dollar, important to your organization, and is for an external customer. Internal customers need a kickoff session, but it’s not usually too elaborate nor does it require as much preparation…funny how that works but the external customers often get top priority, the most attention and the best resources…especially if they’re paying decent money for the project. We’ll also assume this is for some sort of enterprise software solution that your organization offers.

Dropped in Your Lap

So a large project has been passed from Sales to the PMO and your PMO Director has dropped it on you. Now what do you do?

AT this point, Sales has completed their part of the process and you likely have a Statement of Work (SOW), rough resource estimates, specific contract dollar amounts, and a rough project schedule or at least milestone dates in front of you. Whatever you want to call it – kickoff meeting, planning meeting, strategy session – you must hold a face-to-face with the customer to go through these items and ensure that, now that Sales is out of the way, everyone is on the same page. Trust me, it’s not a given and depending on how your organization is run, it’s not even very likely. If you’ve read enough of my articles you’ll understand what I mean – remember when I found my business analyst crying in the hallway? If I’ve learned anything, it’s that you should never assume anything. There…that’s one of my BIG lessons learned.

Wrap Your Arms Around It

In order to prepare for the face-to-face, you first have to get your arms around what you’ve been given. Meet with Sales, go through the SOW with a fine-toothed comb, and review the pseudo schedule and budget that Sales likely passed on to you – and then make them both your own. And by that I mean re-create them how you want to manage against them. Beef up the project schedule with what you know must happen…Sales doesn’t have that PM detail in their heads of what happens on typical projects and neither did the customer when they were meeting with Sales. The onus for that is on you.

The Presentation

For the face-to-face, I always suggest a more formalized meeting to review the following in at least some detail:

• Statement of Work
• Draft project schedule (now with your revisions)
• Milestone dates (pull them from the schedule and review separately so that everyone is very aware of the key dates)
• Budget (if this is appropriate given the attendees and your customer’s preference)
• Your PM methodology (let your customer know how you plan to manage the project)
• Your change order process/change management practices
• How you intend to handle risks/issues
• Status calls and status reporting schedules
• Initial list of risks (if you’re prepared for this at this point…otherwise wait till you have your first status call)

Prepare a presentation deck and get it to the customer for review and agreement prior to the meeting. It will make for a much more successful and productive kickoff session as they’ll already have their questions ready for you – or even answered beforehand – thus, ensuring that uncertainty doesn’t extend into the Exploration phase which comes up next.

Who Should Attend

Face it, you’ll be outnumbered no matter what. I suggest this 1-2 day session should be held at the customer site and they’ll probably try to bring everything but the kitchen sink so be careful. Get a list of their proposed attendees and try to work with them to trim it down. I once had more then 30 customer attendees in a kickoff session and it nearly turned into a fiasco. We were able to trim it down for the 2^nd day, but I’d advise – as a lesson learned – to try to do that before going onsite.

At a minimum, here’s who should attend:

Delivery team:

• Project Manager (that’s probably you if you’re reading this)
• Business Analyst (ideal if one has been assigned…and if one isn’t assigned immediately after you acquire the project, scream! You’ll want their technical expertise available at the kickoff meeting)
• PMO Director (this attendee is not critical, but with a visible, high $$ project, they’re going to want to be there or will be forced to be there by the CEO)

Customer:

• Project sponsor
• Relevant SMEs that have or are defining more detailed requirements and business processes and know what the end users need (and may be the end users themselves)
• Customer Project Manager (often this person is more of a hindrance than a help, but if one is assigned, they need to be present)

Beyond that for the customer and you’re probably going to have too many. Remember, the SMEs are NOT one person…that’s probably 10-12 people depending on the size and type of your enterprise implementation.

Summary

Following these steps will usually ensure that you’ve brought the right people and information to the table to go through the details and get the project started off on the right foot. And it’s extremely important to do that as you’re likely to have some expectation resetting to do….Sales sets one expectation, but yours is the more important, and realistic expectation. Make sure you’re well prepared, well represented, and a little bit stubborn.

Five Factors that Can Derail Your Project

Have you ever been running a project and you thought you had everything under control? Everything seems to be running smoothly and the customer is happy…everybody is happy….and then the wheels start to fall off. If you haven’t been there…that’s great. If you have, then you know what I mean.

Not everything is within the project manager’s control when running an engagement. The PM has their hand in most everything and can speak up to affect the outcome on most things not directly within their control, but as far as having total control over everything…not possible. However, they are still responsible.

So here I would like to discuss five factors that I’ve singled out as issues that are not directly within a PM’s control, but they can watch for them and hopefully take evasive action in time to save their project should these things begin to occur.

Starting too Soon

I’ve discussed this one before. The SOW is in hand, the schedule is drafted and the project is kicking off. What may start to become apparent is that either there are not enough requirements in place or well enough defined to really start the project, or the customer doesn’t yet fully understand the solution and how they will use it. The requirements issue may jump out at you quickly – especially during kickoff discussions. The understanding of the solution and how to use it – that one is going to take some discernment and help from your delivery team to recognize.

Watch for the signs because the deeper you get into the engagement the more apparent this is going to be and when you go to implement that final working solution, the customer is going to think you’re talking a different language if the gap in understanding has continued to widen throughout the implementation.

If you recognize it early, push to shelve the project until the customer has had more training, or reviewed requirements further, or discussed the intended solution at greater depths with their end users and business process SMEs. It’s a tough sell, but it will make everyone happier in the long run.

Revolving Resources

This is a scary situation to be in because it can be so damaging to both your reputation as the PM and to the reputation of the delivery organization. And it doesn’t really matter which side has the revolving resources. If it’s your team, then you’re constantly losing experience with the customer and this particular implementation…and that’s bad. If it’s the customer’s team, then you’re constantly losing your allies and having to start over with individuals on where things stand and bringing them up to speed. It’s a frustrating place to be in and that’s why it’s critical to run through the project team functions – on both sides – during kickoff and ensure that you have buy-in from the powers that be that there is the proper commitment to keep the key resources engaged throughout the project.

Indecisive Customer

An indecisive customer is as bad as an ill-prepared customer. If you’re running a phased project and they keep moving implementation phases around, that can affect many things on your project over the course of the engagement. It can affect when you on-board particular expertise and with resources tight, it can throw off your entire resource planning process.

If the customer continually changes their mind on requirements, tasks, deliverables and milestones, the overall affect to your project schedule can be monumental. The risks to track will mount, and the change orders will mount as well. In the end, customers who have paid for a lot of change orders are not usually the happiest customers – even if they are the primary reason for the many change orders.

Manage the customer well and keep their expectations…and their changes in check if at all possible. Set the understanding up front during kickoff that the project schedule is important and should be changed as little as possible – otherwise it is of no use.

Vague budget

Managing the budget to keep both your executive management happy and the customer happy is one of the key responsibilities of the project manager, in my opinion. Losing control of the budget is never a good thing and can easily lead to a project being canceled…which can be a career-killer if it’s the right..or wrong project.

As bad as that is, having a project that lacks a budget or has a vague budget isn’t much better. Trying to manage a budget that doesn’t really exist or is a moving target can lead to great dissatisfaction for the project manager and the customer.

I was leading a large federal project with a ‘vague’ buget of $1.2 million for the implementation. Some processes were taking longer than anticipated – mostly because the customer data was far more detailed and of a larger volume than previously understood through the exploration phase. I was also of the understanding that the budget could be increased to cover this issue. That was not the case and there was a lot of corrective action that had to be taken – even though the customer was well aware of the issues and the budget concerns.

Having a good understanding of the budget you’re managing is critical and gives the project manager much more control in keeping the project in check and keeping customer confidence high.

Flexible Project Plan

Some of what is applicable here was already covered as part of ‘Indecisive Customer’ above. However, there are more concerns with this one and it bears more discussion here.

It is critical at the time of kickoff to review the project plan in draft and discuss milestones. Get the customer’s verbal agreement at that time as to the general milestones and anticipated implementation date. And, within the first 2-3 weeks of the project, finalize this project plan in greater detail and obtain a formal customer signoff, if necessary, to ensure that everyone is on the same page with dates, expectations, etc.

A moving target project plan is nobody’s friend. You never want to be known as the ‘flexible’ project manager. If dates, tasks, and milestones are constantly shifting, then your ability to hold delivery team members and customer team members accountable to dates and tasks falls apart and you’re still left with the target on your forehead.