What Drives the SSPA Software Benchmark? (article 3 of 5)
by Service 800, Inc.

The five-week series includes the following articles:

> SSPA Software Benchmark Participant Success
> How Customers Benefit from Benchmarking Efforts
> The Mechanics of Service Performance Benchmarking
> Participants Power the SSPA Software Benchmark
> The 2004 Annual Benchmark User’s Group Meeting

The Mechanics of Service Performance Benchmarking

We’re often asked, “Is the benchmarking process complicated?” Service performance benchmarking is certainly not rocket science, but consistency in the collection of customer feedback among participants is crucial. The steps are simple, yet they provide the reliability necessary for a truly useful benchmark.

Step 1: Create a consistent set of interview questions all benchmark participants can agree upon
Step 2: Agree on a customer feedback collection method
Step 3: Collect the closed call records from each participant in a timely manner
Step 4: Determine the number of interviews necessary for a statistically valid sample and choose a random selection of customers
Step 5: Follow up immediately with customers who have had a service event
Step 6: Establish the benchmark based on an aggregate of the data collected

Let’s look at each of these steps as they relate to the SSPA SoftwareMetric Benchmark.

Create Consistent Questions — The first step is by far the most important to establish a true benchmark. It is critical that the interview questions, the sequence of questions, and the wording of the questions are all consistent. Compiling data using different questions or approaches definitely affects customer feedback accuracy -- You’ll end up comparing apples to oranges.

For instance, the first question on questionnaire A might ask the customer for feedback on responsiveness, while the first question on questionnaire B might focus specifically on the time it took to reach a qualified technician. The first question on questionnaire A could be interpreted by the customer as initial responsiveness to the service issue, or how responsive the technician was perceived, or even response time as it pertains to the initial conversation with the technician. The opening question in Questionnaire B is much more concise.

To establish consistency, SERVICE 800 uses identical questions and follows the same interview format with each participant. Interviewers receive a basic script to follow and they train simultaneously to ensure all interviewers ask the questions in the same manner. Each year, SERVICE 800 hosts a benchmark user’s conference to give participants the opportunity to modify the questions and format, and add custom questions at the end of the interview.

Agree on Feedback Collection Method — A benchmark must use the same data collection method for each participant because different collection methods typically generate various response rates.

> Web interviews: 10 - 40% response rate
> Mail data collection: 2 - 40%
> Telephone interviews: 40 - 70%

If participant #1 wants to use web-based interviews and participant #2 prefers telephone interviews, it isn’t appropriate to use the data in the same benchmark.

Participants also respond to different collection methods for different reasons. For example, a user who is a techno junky might enjoy responding to a web-based feedback form, while a customer who is technology shy might prefer a telephone interview. Still, it’s imperative to use the same data collection method for each benchmark participant.

Collect Event Data — Eligible SSPA members, who choose to participate in the SoftwareMetric Benchmark, work directly with SERVICE 800 to transfer closed service event information to SERVICE 800 on a daily or weekly basis. Benchmark participants transfer data electronically or via a customer-secured FTP site, among other methods.

The method by which participants transfer closed call data doesn’t affect the interview process or benchmark validity. However, the frequency of records transfer can affect results. It’s important that participants send data on a daily or weekly basis so that the service events are fresh in the minds of customers.

Determine Valid Sample Size and Randomly Select Customer Records for Interview — Based on the total sample size, SERVICE 800 calculates the number of interviews required for a statistically valid sample for each participant.

For example, a confidence level of 95% (+/- 5%) is most widely accepted by the industry. Given the total number of service events for the measurement period, SERVICE 800 uses an algorithm to calculate a sample size that yields a 95% (+/- 5%) confidence level. In other words, you must interview a certain number (or sample) of people to be highly confident that you have an accurate representation of your customer population. Remember, this does not mean you need to contact 95% of your customers, just the proper sample size (or number) of customers to represent the views of the majority of your customers.

Once the sample size is determined, SERVICE 800 removes any duplicate items and a computer program randomly selects the appropriate number of customer records to ensure an unbiased sample of feedback is collected.

Conduct Follow Up Interviews Immediately — An experienced staff uses a distinct telephone interview process to collect customer feedback within hours or days of the service event. Customer satisfaction information about ease of requesting service, response time, technical ability, professionalism, product quality, completeness, solution time, customer loyalty, and more is collected.

Compile Aggregate Data for the Benchmark — Each month, SERVICE 800 consolidates the feedback into service benchmarks that represent industry performance and communicates the aggregate benchmark data to participating members. Individual company performance is always confidential.

Next Week — “Participants Power the SSPA Software Benchmark”

To Contact Us — To discuss this topic, any other Benchmark Benefits article, or to provide topic suggestions, please contact Jan DeMatteo at jan@service800.com.

For More Information — For additional information about the SSPA Software Benchmark, SERVICE 800, or other benchmark programs, visit www.service800.com/benchmarkprograms.asp.

About SERVICE 800 — Founded in 1989, SERVICE 800 designs and administrators real-time customer satisfaction measurement programs, helping service organizations follow up with their customers within hours or days of service events. The company utilizes a distinctive follow up telephone interview process along with e-mail, web, and other follow up techniques to measure customer satisfaction. With offices in Minneapolis and London, SERVICE 800 has been measuring customer satisfaction for corporations throughout the world for over a decade.