A Product Quality Index (PQI) can assist customer support teams that find themselves stretched between the customers’ expectations of high quality, high performance, easy to operate products and the budget-bounded capabilities of a product development group focused on tightly scoped product releases tied to a pre-defined roadmap. This article shows you how PQI can influence product development to listen to the voice of the customer while measuring and comparing success across different product lines.
The Motivation
In the fall of 2005, Openwave was at a crossroads and was still suffering under the burden of multiple merger & acquisitions that still impacted cultural, process and operational metric norms. Openwave was the result of two major and a host of smaller mergers resulting in different product engineering tribes & product management philosophies, different product development methodologies & documentation approaches and different product quality & performance testing maturities.
The support operations team realized that they were facing three core problems in light of the fact that they had just been reorganized underneath the product engineering group:
- How do we best leverage our organizational restructure to build Profitability, Quality, Reliability, and Customer Trust?
- Concern: how to gain a voice with new product engineering centric management team?
- How do we improve the supportability of our product & maintain customer satisfaction with less engineering staff?
- Concern: how to optimize resources & start engineering for ‘supportability’ into new releases?
- How do we reduce quality issues utilizing only eng team data when it only identifies a small segment of the true root causes?
- Concern: how to measure ‘Success’ across organizations & competing product families?
Openwave was able to improve its products and their supportability through the use of one simple metric. Developed by the Support Operations Group, the Product Quality Index is generated quarterly (using case work margin, defect backlog percentage, case RCA info, number of P1s). This produces a single weighted-index number that can compare different product lines/versions. Using pareto analysis from this metric yields actions that can be taken by the support/engineering teams, such as new product requirements, knowledgebase updates, and the creation of value-added services.
This allowed the Openwave Technical Product Support (TPS) group to escape and flatten the major organizational silo that had held it back for a number of years;
- Each product line believed ‘their way’ was the right way
- No way to side by side compare product line success
- No common language or metrics between the different Eng tribes & support
- Lack of focus on bugged backlog resulting in ‘finger pointing’ between groups
- Support didn’t have a “Voice” for Product Design Requirements (PRD) sessions
Bottom line? Openwave lacked a meaningful ‘cultural currency’ between groups.
What is Product Quality Index?
Product Quality Index (PQI) is an analytical methodology that determines individual product quality relative to:
- Casework Margin = Profitability
- Defect Percentage = Quality
- Critical Situations = Reliability
- Root Cause Ownership = Customer Trust
- Casework Margin (Weight = 1.5):
- Casework Margin = (Product Revenue-Product Expenses)/Product Revenue)
- Product Revenue = M&S revenue
- Product Expenses = (New issue count * Product MTTR * Cost per Hour)
- Why this Metric? Profitability
- Focused on margin and efficiencies of GS
- Financial driver specific to each product
- Case % Defect (Weight = 1.0):
- Defect % = SR count in defect status/Total # of SR in Backlog
- Defect status = Open Defect, Open Escalated, Pending Patch (issues escalated to Engineering)
- Why this Metric? Quality
- Compare/contrast product specific quality issues versus remaining product issues
- Drives “one patch / many fix” mentality
- Critical Situations (Weight = 1.0):
- Critical Situations = # of Critical Situations in period/Total New SR
- Critical Situations = Customer Issue Alerts and # of P1 issues
- Why this metric? Reliability
- Highlights deep impact product issues have on customers
- 80/20 Rule – focus on most critical issues
- Root Cause Ownership (Weight = .5):
- OPWV Root Cause Analysis (RCA) = # of OPWV related SR / Total Closed SR
- OPWV related RCA = Product Quality, Product Documentation, or Product Configuration
- Why this Metric? Trust
- How many issues are caused by product quality issues?
- Identifies potential VAS opportunities
How to improve a product’s PQI?
- Margin/Case improvement:
- Increase Revenue
- Decrease Casework Expense
- Fewer New SRs – Self Service
- Lower MTTR
- % Defects/Case:
- Fewer SR in Defect Status
- Faster Patch releases
- Patch Triage
- 1 Patch = Many SR resolution
- Critical Situations:
- Fewer Critical Situations through Proactive/Pre-emptive Support
- Apply value added services to mission critical situations
- Root Cause Ownership:
- Decrease the number of SRs related to OPWV root cause
- Improve Product (PRD & PEP)
Results 6 Quarters later:
- PQI now is used as a primary decision gate analytic in our product development lifecycle
- Clear Process improvements
- Single bug escalation process (down from 9)
- Single Product Requirement Design (PRD) process with Support workflow go/no-go gates
- Spawned Release Quality Index (RQI) within Engineering
- Spawned Deal Quality Index (DQI) within M&S Deals Desk approval processing
- Clear KPI improvements
- 15% reduction of bugged backlog
- 20% reduction of total case load past 5 QTRs
- 26% reduction in minor releases
- 5% step improvement in key CSAT metrics
- 67% of RCA now indicates a non-OPWV product cause
- Clear Cultural improvements - Single macro metric to govern:
- Product develop investment decisions
- Product success at quarterly ops reviews
All Three problem areas were resolved:
- How to gain a voice with new product engineering centric management team?
By leveraging PQI we were able to build a common ‘cultural’ currency around a metric that was both relevant and respected by the engineering team. The fact
that the Support operations team developed and published the metric gave us a ‘voice’ at the table going forward.
- How to optimize resources & start engineering for ‘supportability’ into new releases?
Again by leveraging the pareto analysis of the PQI scores we were able to insert supportability requirements in to next releases and new product design. Because the ROI
of these requirements was based on the hard data from the PQI analysis, we could work smarter and save man-hours on the back side of supporting these new releases.
- How to measure ‘success’ across organizations & competing product families?
Side by side comparisons of product lines and even product versions was something that we simply could never achieve before. Now product managers were being held
accountable with hard data from across the product portfolio rather than just the marketing data from their own. PQI became a standard barrier for new product investment decisions by our Product Development Counsel and is a key ROI attach point for future product decisions moving forward.
About Brad Smith…………………………………………………………
Brad Smith is currently Director of Southeast Region for InQuira the award winning customer interaction platform provider. Prior to his recent move to InQuira he was Customer Support VP at Verisign for their Content, Commerce & Communications groups and the Global VP of Customer Support for Openwave Systems in 2005-2006. Prior to joining Openwave, Brad held various management and change leadership positions with Oracle Global Support, Untied Space Alliance & Lockheed Martin. Brad is currently a SSPA Advisory Board member and a Board member of the Consortium for Service Innovation (CSI).
About James Pendergast………………………………………………..
Jim Pendergast is the Director of Programs within the Global Support Organization at Openwave Systems. He has over 15 years of high tech experience focusing on driving operational excellence, managing process improvement initiatives to stimulate revenue and margin growth, and building programs to increase customer loyalty and satisfaction. Prior to joining Openwave, Jim held senior management positions at Lotus/IBM centrally managing the Support Account Management team in the Americas, and also at General Electric managing shop operations at a core Aircraft Engine facility. Jim is currently a Board Member of the Consortium of Service Innovation (CSI).
