When using a Pilot for process improvement within an organization, you are likely to find two diametrically opposed results:

1. The Pilot will be an unadulterated success
2. The follow-on implementation will be an equally impressive failure

I have heard more than one process improvement/Organizational Development consultant promote Piloting as a near silver bullet. Why? Because it is intended as a means of "testing" out an idea before implementing it across the organization. It allows time to build buy-in and, if done well, it produces lessons learned which can increase the chances of success with the full-blown effort.

The problem is paradoxical. The problem is that the intent of the pilot is to succeed at the full-blown, organization-wide implementation, and as we've stated in our book, organization-wide improvement is more likely to fail than succeed.

What is most telling is that the consultants who use pilots have transition plans at-the-ready.

1. Use the pilot team members to spread the improvement throughout the organization
2. Use the pilot team as spokespersons - give presentations, briefings, and train others
3. Hold focus groups to analyze the results of the pilot
4. Develop an Implementation schedule for a phased approach

All of these transition plans assume that the improvement will be implemented across the organization.

A pilot is defined as a test of a process or improvement before implementing it at a larger scale. The expected benefits include:

• Reducing the risk of failure for the full implementation
• Identifying problems with the improvement before going full scale
• Gaining buy-in by showing "proof of concept"
• Providing a chance to tweak the improvement before increased scope

Notice that all of these reasons (expected benefits) support the idea of implementing an improvement across the organization. It's the basic premise of the pilot, and it's inherent problem - pilots are a precursor to organization-wide implementation.

What may make using pilots even worse is that they almost always show benefits. Most pilots succeed! They are manned by the best the organization has to offer, they are provided the best resources and usually all the funding needed. They are set up for success. Everyone wants it to succeed. The team wants to succeed - it will be a direct reflection on the team. The consultant wants it to succeed as an affirmation that the proposed improvement will work. The leadership wants it to succeed because in their minds if it does, that means they chose wisely. No one wants to be the one that can be pointed to as the reason the pilot failed. So, the pilot is likely to succeed and the unfortunate (but logical) result is that the organization will then move to (be ordered to) implement enterprise-wide. And since the pilot succeeded, the process for implementation will normally try to expand on what worked at the pilot level.

1. Resources will be identified early, and provided readily
2. The best will be selected for roles on each implementation team
3. The Consultant will be kept on contract

So what's wrong? A big part of what's wrong is what's missing. The pilot did NOT test the process with less than the best workforce. It did NOT test the process within a culture that may be less than eager to change. Precisely because the pilot was a success, the organization is MORE likely to bypass some of the change management techniques it may have normally attempted. The pilot's success gives the organization a false sense of security that the wider implementation will work.

Change Management techniques which may be skipped:

• Building awareness - it is assumed that the pilot will provide awareness of the change
• Building motivation - rather than build desire in the organization for the change, the pilot is held up as the example of why the change is good. The pilot's success is pitched as the success the organization will experience at the larger level.
• Ensuring the organization has what it needs to make the change a success - knowledge and skills. The pilot misleads the organization into thinking that since the pilot was able to do the change, the rest of the organization will also. Even if the pilot team received some training, chances are, since they were the best of the best, other members of the organization will require different or more thorough training.
• Some plan for institutionalization - a means for ensuring the change "sticks." This step is stressed less, since the thought is that the organization is already "sold" on the benefits because the pilot was a success.

Bottom Line
The underlying problem with using the pilot as a process improvement tool is that it is by its nature a precursor for Organization-wide Change. Also by its nature, it is designed to succeed, and it's this success which in turn sets up the organization for failure. Of course, if the organization is NOT suffering from Organizational Immaturity, the pilot process is not an issue...but if the organization is immature, the pilot can be a very costly mistake.

References:
http://www.isixsigma.com/index.php?option=com_k2&view=item&id=1127:&Itemid=190
http://technet.microsoft.com/en-us/library/cc961315.aspx