Is Service Performance Benchmarking Important? (3 of 4)
by Service 800, Inc.

Most companies today measure customer satisfaction in some way or another. Many are just realizing the importance of measuring customer satisfaction for service events. The most astute companies are going a step beyond just measuring satisfaction by participating in industry benchmarks.

The four-week series includes the following articles:

• Why is Service Performance Benchmarking Important?
  
• What Makes a Good Benchmark?
  
• What are the Facts and Fallacies of Benchmarking?
  
• Is it Possible to Successfully Benchmark with Non-Competitors?

What are Some Facts and Fallacies of Service Performance Benchmarking?

“Benchmark” is a buzzword in today’s high-tech industry. Most providers claim to offer an industry benchmark for comparison purposes, but let buyers beware…not all benchmarks are the same. Here are some facts and fallacies you should be aware of when walking down the benchmark path.

Fallacy – All benchmarks are the same because they give us a base for comparing our service performance.

Fact – Remember, a true benchmark must be consistent, valid, timely, objective, and confidential.

Fallacy – Our industry executive support group designed a questionnaire together so we can cut costs by using in-house personnel to measure service performance. We can then combine our anonymous data to establish a benchmark.

Fact – A company that uses an in-house team to measure service satisfaction isn’t likely to get a clear picture of customer perceptions.

For example, Global Software Developers, Inc. found that customers responded more honestly to a third-party measurement provider because the servicing technician isn’t the person handing them the questionnaire and isn’t involved in the follow-up process. The data is more accurate because employees aren’t able to wash the data and remove the unhappy responses.

Fallacy – You can establish a benchmark by compiling data from interviews with different questions and different rating scales.

Fact – As we discussed last week, a true benchmark uses the same set of questions, asked the same way on behalf of each participant. This gives the feedback validity. Results compiled from different types of questionnaires might not be credible.

If Global Software uses one set of questions and Really Cool Software, Inc. uses a different set of questions, the performance results are not an apples-to-apples comparison. Even if the questions are similar, but worded differently, the results can be skewed.

Fallacy – You can easily combine the results of your telephone and web-based follow-up interviews.

Fact – Different data collection methods, like telephone and web-based, typically yield different results. Differences in the response rates can negate the validity of the data. Web-based questionnaire response rates are generally three times lower than telephone response rates and require a motivated response. While each collection mode has its merits, combining the data isn’t a good idea.

For example, for two years Global Software used telephone interviews to collect service performance feedback. They consistently got a response rate of 60%-70% of customers interviewed. When they briefly switched to electronic questionnaires, response rates dropped to 20%-30%. To combine the data from results that yielded such different response rates would not be representative of the opinions of the customer base. Regardless of the data collection method, results are only valid if response rates are in the same general range.

Fallacy – It does not skew the results if our data confidence levels are different for each participant.

Fact – All benchmark participants must use the same statistically significant confidence level. A confidence level of 95% (+/- 5%) is most widely accepted by the industry. In other words, you must interview a certain number (or sample) of people to be highly confident that you have an accurate sampling of your customer population. This doesn’t mean you need to contact 95% of your customers, just the proper sample size necessary to represent the views of the majority of customers.

For example, Really Cool Software needs confidence that 90% of their customers are satisfied, while Global Software needs a confidence level of 95%. It isn’t wise to use the results from data gathered using different confidence levels as it may skew the results.

Next Week - Is it Possible to Successfully Benchmark with Non-Competitors?

To Contact Us — To discuss this topic, any other Benchmark Benefits article, or to provide topic suggestions, please contact Jan DeMatteo at jan@service800.com.

For More Information — For additional information about the SSPA SoftwareMetric Customer Satisfaction Benchmark, SERVICE 800, or other benchmark programs, visit www.service800.com/benchmarkprograms.asp.

About SERVICE 800
Founded in 1989, SERVICE 800 designs and administrators real-time customer satisfaction measurement programs, helping service organizations follow up with their customers within hours or days of service events. The company utilizes a distinctive follow up telephone interview process along with e-mail, web, and other survey techniques to measure customer satisfaction. With offices in Minneapolis and London, SERVICE 800 has been measuring customer satisfaction for corporations throughout the world for over a decade.