An ROI (return on investment) analysis is pretty much a requirement for justifying any large purchase – as well as a wonderful discipline to evaluate any important initiative. Unfortunately, a large degree of, shall we say, creativity seems to reign in ROI analyses, to the point that many people instinctively distrust ROI numbers, having been on the receiving end of many that had been tweaked to show stupendous gains only to fizzle out after implementation. It’s a sad situation, especially since it’s not difficult to create an honest ROI analysis that does not stretch reality and can be used not just as a justification tool, but also to assess the success of the program in the long term. Here’s how.

1. Use a realistic baseline

Yes, it’s a lot easier to justify a self-service initiative, say, if you figure that each support case costs $200. But most support organizations don’t spend nearly as much on the average case (although some do – I have clients with $400 averages, but not many like that!)

I’ve found that a good technique to gather realistic costs is to create a baseline independently of any initiatives for which a ROI analysis might be needed. With the help of a financial analyst, create a simple model that shows how much it costs to run support: how many minutes are spent at each level for what cost per minute; how much do different types of cases cost; how much does the phone portion of a case cost, etc. Once you have the baseline, do not massage it in any way to make the savings come out the way you want.

2. Identify likely savings

Make an inventory of likely savings for the initiative you are considering. This is a brainstorming phase so you don’t have to be exact or restrained, but be realistic: although it’s possible that a self-service support initiative may increase product sales, it’s not exactly likely, is it? If it’s difficult to describe exactly how the initiative will create a specific change, give it up. Recipients easily become suspicious of analyses that seem forced, tainting the entire exercise.

If some savings are intuitively right but hard to measure, don’t try to force them into the analysis. Instead, present them as additional, intangible benefits.

3. Associate metrics to the savings

Clearly, the new ACD you are contemplating will increase customer satisfaction. Well, maybe… How will you measure customer satisfaction? How much will the ratings move up? And the most difficult question: how do you translate customer satisfaction into dollars?

For each potential saving you identified in step 2, you need to identify a metric and a dollar amount. For instance, the new ACD will decrease case length by 20 seconds, saving $X in phone costs (from the baseline you will know how much 20 seconds on the phone costs.) Or the new self-service initiative will decrease case load by 5%. Or the new knowledge base will decrease escalations to level 2 by 10%.

Now where do the 20 seconds of the call time, or the 5% case defection, or the 10% decrease in escalations come from? They are all estimates of the savings you hope to achieve, and they must be grounded in some reality. You can use a reasonable guess or you can use benchmarks, keeping in mind that your mileage will vary. In particular, any estimates you get from a vendor for the benefits of a specific tool should be considered to be a high-water mark.

In many instances you can conduct a small-scale test in your own environment to validate the estimate. For instance, if you can show that agents take 30 seconds to check entitlements but the new automated system takes less than 10, that would be a good indication that your 20-second estimate is valid. Or you can ask customers to use the new search tool and the old one and measure their success with both to estimate case deflection. Or you can study the reasons why level 1 agents escalate and establish that a better knowledge base would cut down a good portion of the escalations.

Conducting a realistic test is vital. If only a fraction of the customer base is correctly recorded in the database, your newfangled automatic entitlement checking system will not bring significant savings until the database is cleaned up. If you select your test customers from the faithful who attend the user group meetings, you can expect a sample biased towards the more knowledgeable customers. If you test with in-house agents you will likely get a different result from outsourced agents. This does not mean that you have to conduct massive tests, just that you have to be careful about how you design the experiment.

4. Phase in the savings

One of the telltale signs of an over-enthusiastic ROI analysis is that all the savings show up at once right at the time of the rollout. Not so in real life: savings usually accrue over time and if people are involved you may have a significant rampup time before they show up. The most striking example would be a new case-tracking system. Even if the old one is awful and the new one is perfect (and how likely is that?) the staff will likely fumble around for several days or weeks, undoing old habits and lowering productivity.

Some improvements can be sudden. For instance, a new automatic entitlement checking tool can have an impact very quickly (assuming that the customer database is in good shape.)

5. Include all the costs

Another common flaw of ROI analyses is under-estimating the actual cost. Count everything: license costs for the new tool, additional hardware and software you will need to purchase, implementation fees, training, maintenance costs, etc. It’s rare to see a project that spent less than the original estimate, so pad up a bit.

6. Walk away if the savings are not there

Here’s the painful part: sometimes the savings just are not there. The project is exciting, it seems like a great idea, but the $20,000 it would save just does not add up to the initial investment. It’s very tempting to go back and tweak the numbers to make the bottom line come out “right”, isn’t it? After all, if we can achieve a 5% case deflection who is to say we cannot do 10%?

This is how a lot of analyses lose their ways: tweaking a bit here and a bit there, adding more exotic benefits to add up to the dollars you need to save, and in the end losing track of common sense. Don’t add benefits after the fact, and don’t inflate the size of the benefits. I would even discard very small benefits entirely – how will you measure accurately saving .01% of the budget?

7. Measure after the fact

Many organizations gleefully discard the ROI analysis after the project has been approved, sometimes, it must be said, because the truth was stretched so much that they know the ROI numbers don’t quite add up. If you’re interested in creating better and better ROI analyses, study the outcomes of old ones. Did you rely on baselines that turn out to be misguided? (Fix them!) Did the test results vary significantly from the actual experience (The tests were probably not valid.) Did you rely too much on benchmarks? (Find better ones, or run your own tests.) Did multiple and simultaneous initiatives make it difficult to tell which ones were successful? (At least determine if the aggregate numbers work out.)

Even if initiatives change the principles of ROI construction remain so you can improve your accuracy over time. Good luck!

About Francoise Tourniaire…………………………………………………

Francoise Tourniaire is the founder and principal of FT Works, a consulting firm that helps technology companies create and grow their support operations. She’s the author of The Art of Software Support, Just Enough CRM, and Collective Wisdom: Transforming Support with Knowledge. For more information, visit www.ftworks.com or call 650 559 9826.