Are We Completely Prepared To Measure Service Event Satisfaction? (article 1 of 3)
by Service 800, Inc.

Over the past weeks, we’ve offered some practical advice on how to properly prepare to measure service event satisfaction. You’re probably thinking, “How much more prepared can we be?” Well, there’s more work to be done.

Measuring customer satisfaction and using the feedback to its fullest is 80% preparation and 20% execution. If you take the time to really plan for maximum ROI from your measurement program, you will reap the benefits.

With so much to be done, from program cost justification to choosing a provider to designing questionnaires and report, it’s easy to overlook some basic, yet essential questions.

The three-week series of articles focuses on:

• How To Ensure Customer Feedback is Statistically Valid and Accurate
  
• What is a Skip List and Why is it Necessary?
  
• How Can We Enhance Our Customer Issue Escalation Process?

Ensure Your Customer Feedback is Statistically Valid and Accurate

Let’s face it…you’re in the support business, not in the statistics business. That said, this article will give you a better idea as to how to obtain valid and accurate results. Of course, SERVICE 800 or your current provider can accurately calculate the appropriate sample size and help you monitor response rates. Still, it’s essential that you understand the importance of sample size and response rates.

Sample size – To gather statistically valid data, you must first determine the proper sample size for your needs. A confidence level of 95% (+/- 5%) is most widely accepted by the industry. Given the total number of service events for the period you’re measuring, your provider should calculate a sample size that yields a 95% (+/- 5%) confidence level. In other words, you must interview a certain number (or sample) of people to be highly confident that you have an accurate sampling of your customer base. Remember, this doesn’t mean you need to contact 95% of your customers, just the proper sample size (or number) of customers to represent the views of the majority of your customers. While calculating a 95% confidence level requires a complicated algorithm, your measurement provider should have the experience to determine the right sample size for your needs.

Random sample – Another factor in gathering accurate feedback is making sure the provider interviews a random sample of customers. Selecting a random sample of the total population ensures that you get a strong cross section of customer opinions, thus a more accurate real-world picture of how your customers view your service delivery capabilities.

Consistency of questions – It’s crucial to use a consistent set of questions and consistent interview techniques, especially when compiling data for a benchmark, such as the SSPA SoftwareMetric Benchmark. Compiling data using different questions or different interview techniques will definitely impact your customer feedback accuracy.

It’s also critical to design clear, concise questions and communicate the questions in the same way to each customer. This ensures that the data you are collecting data using the same criteria for each customer in the sample, hence accurate.

For example, question one on questionnaire A may ask the customer for feedback on responsiveness, while question one on questionnaire B may focus specifically on the time it took to reach a qualified technician. The first question on questionnaire A could be interpreted by the customer as initial responsiveness to the service issue, or how responsive the technician was perceived, or even response time as it pertains to the initial conversation with the technician.

Number of responses – When measuring customer satisfaction, it is imperative that you study the number of responses compared to the total population. For example, ABC, Inc. was overjoyed that 90% of their customers were either very satisfied or satisfied with their service performance. However, their elation turned to disappointment when they learned that the high marks came from less than 100 responses out of a total population of more than 10,000. Clearly, the small number of responses could not accurately represent the views of the majority of their customers.

Different types of data collection methods typically generate various response rates. For example:

• Web survey: 10 - 40% response rate
  
• Mail survey: 2 - 40%
  
• Telephone survey: 40 - 70%

Finally, if you want to analyze data at multiple levels (by region, branch, customer or even technician) it’s important that results be statistically valid at each level, especially if scores affect performance reviews or bonuses.

Talk with your provider today, to determine if you are gathering accurate and valid results, based on the total customer population.

To Contact Us — To discuss this topic, any other Benchmark Benefits article, or to provide topic suggestions, please contact Jan DeMatteo at jan@service800.com.

For More Information — For additional information about the SSPA SoftwareMetric Customer Satisfaction Benchmark, SERVICE 800, or other benchmark programs, visit www.service800.com/benchmarkprograms.asp.

About SERVICE 800
Founded in 1989, SERVICE 800 designs and administrators real-time customer satisfaction measurement programs, helping service organizations follow up with their customers within hours or days of service events. The company utilizes a distinctive follow up telephone interview process along with e-mail, web, and other survey techniques to measure customer satisfaction. With offices in Minneapolis and London, SERVICE 800 has been measuring customer satisfaction for corporations throughout the world for over a decade.