The Lessons Learned Survey

Another useful tool in the Lessons Learned process is the Lessons Learned Survey.  This is a survey that can be sent to team members during or after a project, to solicit their feedback on how the project was conducted. It applies to any project; and questions can easily be added to focus on additional areas for your project.

A survey like this can be very useful for capturing lessons learned from the project while they’re fresh in people’s mind. The results can be summarized and recommendations passed on to future teams.

It’s usually best to send this survey by email to members of the project team. Let them know that results can be kept anonymous (to encourage people to be frank in their assessments). Send out this survey before any group “lessons learned” meetings. The feedback you receive from the survey can help point to particular areas that should get special exploration in the group lessons learned meeting.

Again, use this if it’s meaningful to your projects – it’s just another template or project tool that I’ve picked up along the way and wanted to share with our readers.  I’ve found it to be helpful in gathering post-project information from my team members on several key projects.

Post-Project Survey

SECTION 1: General Project Issues and Communication

The questions are geared to your particular project but wherever appropriate you can comment about release-level issues too.

Note: add any particular comments you wish….

1. How clearly defined were the objectives for this project?

___Very ____somewhat ___not very ___not at all

2. How clearly defined were the objectives for your work?

___Very ____somewhat ___not very ___not at all

3. How clear were you on your role in the project?

___Very ____somewhat ___not very ___not at all

4. How adequately involved did you feel in project decisions?

___Very ____somewhat ___not very ___not at all

If you did not feel involved, what decisions did you feel left out of?

5. How efficient and effective were project team meetings?

___Very ____somewhat ___not very ___not at all

What would you change?

6. How efficient and effective were technical meetings?

___Very ____somewhat ___not very ___not at all

What would you change?

___Very ____somewhat ___not very ___not at all

8. How adequate has cross-functional participation been?

___Very ____somewhat ___not very ___not at all

What were the problems encountered in the project-functional area relationship, why, and how could they be fixed? What cross-functional participation, if any, was lacking?

9. Do you feel appreciated, recognized and rewarded for your efforts?

___Very ____somewhat ___not very ___not at all

What if anything has been lacking?

10. To what degree is do you feel the entire team was committed to the project schedule?

___Very ____somewhat ___not very ___not at all

What if any issues are there?

11. To what degree have any “people issues” gotten in the way?

___Very ____somewhat ___not very ___not at all

What issues?

12. What communication, organization, structural problems in general were encountered, and how could we have done better in these areas?

SECTION 2: Schedule Estimation Issues

NOTE: This survey isn’t intended to collect exhaustive data on everything right now. We might decide after the post-mortem to work with a sub-group to investigate certain estimation issues, in order to help us improve next time around. This survey just helps us ferret out the rough scope of any issues.

Which of the following estimation issues did you personally have and what was the impact?

___ I was diverted to work on another project full-time.

Project: ________________________________________ Diverted for how long? ________________

Impact on your project work: ____________________________________________________________

___ I overestimated the amount of time I would have each week to work on this project.

The other work that interfered was ________________________________________________________

The amount of time per week it took up was ________________________________________________

Impact: calendar schedule slip of ____days ____weeks ____ months

___ My initial schedule did not include some pieces of technical design or coding work that I subsequently realized I had to do.

Describe briefly: ______________________________________________________________________

Impact (additional hours of work): ________________________________________________________

___ My initial schedule did not take into account certain project “other” work such as attending other people’s design reviews, doing two rounds of my own design reviews, etc.

Describe: ____________________________________________________________________________

Impact: calendar slip to my work of __ days __ weeks __ months

___ My estimates for particular tasks were not accurate.

Describe: type of task, how “off” the estimate was (days, weeks).

Why was it difficult to estimate?

What would help produce better estimates next time?

__ I unexpectedly had to re-do some work.

Describe: (Did something in the system design change that forced you to redesign? Was there a spec misunderstanding? etc.)___

Impact on your schedule: _______________________________________________________________

What could have helped prevent the problem?

Knowing what you know now, how would you do the scheduling/estimating process differently next time to avoid any problems noted above?

SECTION 3: Design, Implementation, Test Processes

1. How effective was our architecture/system design process in phase 2 and 3?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

2. How effective were our functional specs?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

3. How effective were our design (or implementation) specs?

__Very __somewhat ___not very ___not at all

Comments:__________________________________________________________________________

4. How effective were our design reviews?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

5. How effective were our code reviews or hardware reviews?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

6. How well were interfaces defined?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

7. How well were design and interface decisions documented?

__Very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

8. How effective has interaction/cooperation between technical “Sub-teams” been?

___Very ____somewhat ___not very ___not at all

Comments:_________________________________________________________________________

9. How useful was your unit testing?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

Did you take unit testing into account in your schedule? ______________________________________

10. How smooth do you feel Integration has been?

__very __somewhat ___not very ___not at all

Comments (why/why not?):___________________________________________________________

11. How comprehensive was integration testing, especially to allow SQA testing to get off to a good start?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

12. How well is the build process working?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

13. To what degree did you have the tools you needed for testing?

__very __somewhat ___not very ___not at all

Comments:_________________________________________________________________________

SECTION 4: Perceived Project Life Cycle, Development, or Process Issues

1. Is there any way in which you think our development process hampered this project?

If so, how?

2. What would you change about our development process?

3. What would you like to better understand or see better documented about how to use our process on this type of project?

SECTION 5: Closing

1. What were up to 5 main causes for schedule slips, and how could we avoid those causes in the future?

2. Was the project significantly delayed/hampered by outside dependencies (outside to the project, that is)? Which ones? How can we resolve these issues?

3. What were the main bottlenecks on the process?

4. What were the main sources of frustration in the project?

6. For the next project, how could we improve on the way project was conducted?

The Project Manager and Press Briefings

Thankfully, the projects I’ve run have not required that I go before the press and give any kind of a briefing – either pre- or post-project. The closest I’ve come is helping a customer put together written press briefings – these were in the case of US Airways for an enterprise-wide software implementation and Rockwell Collins for the release of their pharmacy website for employees and retirees for publication in a trade journal.

Carl Pritchard presents his take on press briefings and the PM’s role when leading those types of projects. The following text, for the most part, comes from his book entitled “The Project Management Communications Toolkit.” Again, I’m not wholeheartedly endorsing this process or the information contained here, but I think it is solid information nevertheless and would be helpful to project managers who find themselves faced with the need to “meet the press.”

Press Briefings

Few environments are as grueling for a project manager as when he or she must face the media. Press briefings are held to inform members of the media about the status of a project, its environment, or its supporting organization. They are intended to present the project organization (or host organization) in the best possible light. Press briefings are held when a project or its impact is sufficiently significant that public information campaigns using mass media are appropriate. They should be held whenever the project has achieved sufficient recognition that the project organization’s perspective on the effort is deemed to be of public interest. That recognition may be positive or negative in nature, and may be proactive or reactive, depending on the nature of the project organization.

The Subject Matter

The subject matter for a press briefing should be determined well in advance of the briefing to ensure that the correct information is shared and any information that the organization does not want to share is clearly defined for those hosting the briefing. Members of the media are often given “press kits” at such gatherings, highlighting corporate history, general information, past press releases, and any contact persons’ business cards. The organizational spokesperson (sometimes, the project manager) should open with a statement regarding the nature of the project and the issue(s) that brought the project into the public eye. The statement should anticipate any questions, objections, or concerns that may be raised. If broadcast media are present, consideration should be given to phrases, paragraphs, or references that may be presented in 8- to 20-second sections (classic “sound bites”).

A press briefing need not necessarily include question-and-answer periods, but keep in mind that most members of the media will have questions. Although the spokesperson is not compelled to answer these questions, failure to respond is sometimes interpreted as a lack of cooperation or as a sign of deviousness. In situations where off-the-cuff responses may be dangerous, it is wholly appropriate to offer to do supplemental research and respond at a later time. The most effective spokespersons will identify the time when the additional information will be available and how it will be made available. If “no comment” is the appropriate response, alternative means to couch that phrase can be very effective and can leave media representatives with something quotable. Saying “This would not be the time to offer comment on something of that nature,” followed by an iteration of the key point of the briefing affords the presenter the opportunity to emphasize what is important.

Summary

Press briefings are potentially volatile situations, but they are the host organization’s to control. Simple considerations (like morning coffee and comfortable seating arrangements) can go a long way to defuse a potentially hostile audience. Clear rules of conduct and engagement can also minimize the possibility that the session appears to be out of control—and the more that can be done to ensure a positive attitude and a forward-looking perspective, the better.

Closing Out the Project – Part 3

In Part 1 and Part 2, we covered the first six critical questions listed below that should be addressed when closing out any project. In the finale, Part 3, we’ll cover items seven through nine highlighted in bold letters below:

• Have all the project objectives been achieved?
• Is the client satisfied with the overall project?
• Have the necessary post-project support agreements been established?
• What were the major concerns with the project?
• What are the key lessons learned from the IT project?
• What would you do differently?
• Do you feel the solution was cost effective?
• When would it be applicable to enhance or update the delivered solution?
• What is your executive leaderships view of the project outcome?

Do you feel the solution was cost effective?

Here’s your chance to analyze the solution in words in financial terms. And we’re not really talking about budget here, but that’s a big part of it. In hindsight, did the engagement:

• Utilize the best level of resource skills and thus use resources in the most cost effective-way possible.
• Should Phase A really have been implemented first as the customer required, or would it have been a more sound business decision, in your opinion, to implement Phase B first?
• Is the final solution meeting the customer’s needs in the most cost effective manner possible? Would certain enhancements or different requirements have resulted in a more cost effective solution?

The list could be long, but I think you get the picture. Ask yourself the tough questions and imagine this isn’t for the customer to see. In fact, imagine you ARE the customer on this one but also have your additional insight.

I’m not saying you can’t involve the customer on this one – you certainly can – or you can perform it separately with your team and then with the customer and compare results.

When would it be applicable to enhance or update the delivered solution?

You’ve probably had an eye to the future all along and you’ve probably already discussed some key points along the way with the customer – especially if the project was a successful one and the customer satisfaction seems high. That’s what a good project manager does.

Think about ways you can provide new and future services to this customer and certainly keep in contact with them post-implementation on future product capabilities that you feel they will want or can benefit from.

What is your executive leaderships view of the project outcome?

This one is important to your career. No question about it. How does your leadership feel about the project? This likely will come more from leadership’s discussions with the customer than from your discussions with the leadership. And it should.

If it was a visible, critical project, you know that they’ve been in communication with the client along the way and if anything has gone wrong, they’ve heard about it. They’re not as likely to hear about the successes, but if you think the project has gone well, encourage your CEO or other leadership to follow-up with the client and discuss the outcome with them.

Summary

We’ve covered what I consider to be nine key questions to review once your project has been implemented. Most are for you and your team, some should also include the customer. But be sure to perform some sort of post-implementation checklist like this. You’ll benefit from it as a project manager, your team will benefit from it in learning what went right and what went wrong, and your organization will benefit from it – especially if you can share the successes and the lessons learned with others in the organization.

If you have other key points or questions to add, please comment.

Closing Out the Project – Part 2

In Part 1, we started the process of covering some critical questions to ensure all of the project “i’s” are dotted and “t’s” are crossed. The list can be much longer, but I’m choosing to cover 9 basic questions to look at when shutting down your project. We’ve covered the first three – now for Part 2 we’ll cover items four through six highlighted in bold letters below:

• Have all the project objectives been achieved?
• Is the client satisfied with the overall project?
• Have the necessary post-project support agreements been established?
• What were the major concerns with the project?
• What are the key lessons learned from the IT project?
• What would you do differently?
• Do you feel the solution was cost effective?
• When would it be applicable to enhance or update the delivered solution?
• What is your executive leaderships view of the project outcome?

What were the major concerns with the project?

No matter how perfectly a project goes, there are always concerns. You may have been able to mitigate whatever issues came up, but they still happened whether they knocked your project off course or not.

What we want to do here is document what those major issues or risks were. They key questions to ask during this process would be:

• Was the concern anticipated in advance (meaning was it something your team or customer foresaw as an issue or risk and thus you were already looking out for it)?
• Did the concern end up affecting the project adversely in terms of time, money, resources, or progress?
• Did it affect customer satisfaction and if so, in what way?
• How did you react to the concern and was your action the appropriate one in hindsight?
  

What are the key lessons learned from the IT project?

For this activity, it’s most beneficial to sit down with the customer post-deployment and formally discuss and document lessons learned. This is somewhat similar to reviewing the major concerns in the previous question, though this one is best done face-to-face – or by phone – with the customer as their insight will be invaluable…and is critical feedback to have. As I’ve stated before, your customer may have been unhappy with things and you had no idea – and now will be your opportunity to find some of those things out and understand how to better react in the future.

What would you do differently?

Knowing what went right and what went wrong, how the project turned out and how your customer feels about it, now you can look at what you would do differently in the future. There may even be other opportunities with this customer and having a plan for a different course of action on specific issues that arose on the engagement can be an invaluable piece of information going forward.

Re-review the issues and risks faced on the project and asked yourself and your team – did we react appropriately and, if not, what should we have done differently. Chances are, the issues will show up again in another project and you’ll be ahead of the game. If we don’t learn from our mistakes, we are destined to repeat them.

Next

In Part 3 we’ll further discuss these items in terms of the project closeout checklist/review…

• Do you feel the solution was cost effective?
• When would it be applicable to enhance or update the delivered solution?
• What is your executive leaderships view of the project outcome?

Closing Out the Project – Part 1

When you’ve reached the end and deployed the solution, then it’s time to make sure everything has been completed, all paperwork is done, and no stones are left unturned. It’s best to do that with some sort of checklist and I propose using a list similar to this:

• Have all the project objectives been achieved?
• Is the client satisfied with the overall project?
• Have the necessary post-project support agreements been established?
• What were the major concerns with the project?
• What are the key lessons learned from the IT project?
• What would you do differently?
• Do you feel the solution was cost effective?
• When would it be applicable to enhance or update the delivered solution?
• What is your executive leaderships view of the project outcome?

In Part 1, we’ll discuss the first three items above:

• Have all the project objectives been achieved?
• Is the client satisfied with the overall project?
• Have the necessary post-project support agreements been established?

Have all the project objectives been achieved?

This should be fairly easy to evaluate. Look at the project schedule and review all milestones and deliverables. Have they been met? At this point, we’re not considering timeliness or budget, we’re just concerned with did we hand the customer what we said we would. Did we supply a Functional Design Document, did we provide weekly status reports, was training completed successfully and signed off, was user acceptance testing (UAT) fully completed including re-review of all issues and did we get official signoff, etc.

Though it definitely shouldn’t be the first time during the engagement that you do this – it’s also a very good time to go back to your kickoff session notes and see that all points discussed during that meeting have been addressed. Make sure at this point that the customer knows you’re concerned that you fully covered everything for the project – they’ll appreciate it and it definitely helps with customer satisfaction levels and can increase the chances of repeat business from this customer.

Is the client satisfied with the overall project?

You may think you can simply answer this yes or no based on your knowledge of how things went on the project and the communications you had with the customer along the way – but that is not always the case. On some projects, you can end up being very surprised by the customer’s on viewpoint of their satisfaction levels.

I’ve had customers on projects that went extremely smoothly later state that they weren’t happy about how I handled something or weren’t pleased with one of my team members or how we delivered ‘x’ deliverable. And I’ve had other projects were I was certain the customer was just about ready to ask that I leave the project only to find out that they were very happy with me and with the team and how things were going. They were just demanding and somewhat needed. But we were meeting their needs and they were happy. It’s amazing how off your perception can sometimes be concerning your perception of the customer’s satisfaction level and what their satisfaction level truly is.

Have the necessary post-project support agreements been established?

Usually you’ll move from deployment into a post-deployment with some sort of pre-defined plan to both hand-off overall support to your tech support group but also you’ll agree to make your original delivery team available for a 30 or 60-day window to do immediate fixes should problems arise in the functionality of the delivered solution. That’s something that just needs to be worked out with the customer and something that was likely planned for both by Sales with the customer during the sales process and discussed between you and the customer during kickoff.

Just make sure that whatever the plan is, you act on it. If it’s to keep your team available for 60 days should problems arise, then make sure that every team member knows that and that each of their managers know that as well. It can have an affect on their next assignments, but it must be addressed in advance of deployment.

Next

In Part 2 we’ll further discuss these items in terms of the project closeout checklist/review…

• What were the major concerns with the project?
• What are the key lessons learned from the IT project?
• What would you do differently?