Product Success Metrics

This is a general checklist about what to measure in order for us to say a product, as in a 'software product' succeeds. There could be various assumptions across the industry which finally boils down to these metrics:

Customers new and old

How many new customers are you acquiring per month? How many customers have canceled maintenance or subscriptions per month? This is not a direct metric of the product itself since it has other parameters such as Sales efficiency, Time to market etc. However, measuring this metric is key to product success. Even if customers are still 'in' and do belong to the client list, the degree of engagement is important to be measured.

Quality!

Yes :-) this is an often heard term but product quality means so many things to so many people.

Product quality should be monitored on an ongoing basis to make sure it doesn't become a serious problem. Set up a tracking mechanism with either weekly or monthly statistics with not just the number of defects but quality of the quality process itself. I have seen product heads just track the traceability from requirements to defects and nothing more. A well tracked quality system produces just the right metrics which would exactly show the quality health of the product.

For released products, technical support typically keeps track of open defects. As soon as a product is released, it is common for the number of defects to go up as more people use the product. If the quality is high, it should be a manageable number and should settle down after awhile. For unreleased products, QA and release management should maintain a strict vigilance on the numbers. If the number of open defects is not going down as the release date approaches, the release date is likely to slip. Obviously we want the Severity 1 and Severity 2 defects approaching zero as the release date approaches. Severity four "defects" are typically enhancement requests and are often ignored in terms of product quality. Yet they indicate areas where the product fails to satisfy the customer. Product quality is the best indicator of internal health of the product;

Technical Support

Product Management and Technical Support should work at tandem. Care should be taken that metrics are not tweaked for a lesser benefit. I have seen Customer Support increase no. of calls they attended by logging the same problem in different ways and product management trying to respond to multiple problems with a single statement of direction or FAQ that would portray them as having anticipated that as a known problem.

Technical (or Customer) Support already measures the number of calls per product and the nature of the calls. Product management should analyze these numbers to identify areas of that product that can be improved for a better customer experience. Are there problems with installation of the product? Is the documentation too hard to understand? Is the product too hard to use? By streamlining a process, will you cut down the number of technical calls? Review the number and type of calls for your product to uncover hidden profit leaks.

Product Schedule, scope and slippage

How long does it take from the time you define requirements until you have a finished product in the market? Keep track of the percentage of original requirement specs that get delivered as the final product. How many "out of scope" requirements were included? Do your developers have a clear idea of what problem to solve and for whom? The problem may be poor requirements definition, which you can control, or it may be scope creep which you need to communicate to your management. Report the facts and let management manage.
Pre-Sales and Sales' support

How many non-sales people does it take to support the sales cycle? It would be a great metric to know how much time is being spent by non-sales people on direct sales efforts such as demo support, conference calls with prospective customers, onsite presentations, requirements assessment, and other sales calls. Early in a company's life, people wear many hats and this is a common thing but as a company grows, product selling needs to be 'empowered' with the right sales kits and training. Sales personnel need to continously update themselves as opposed to just having client meetings and playing golf.

When non-sales people participate heavily in the sales process, they are not doing their day job. And the cost is rarely attributed to the sales activity; another hidden profit leak.

Ultimately, we want to build and sell products profitably.

Product revenue/profit

How do you know which products are worth investing in and which should be retired?

Sales should be able to tell you how much revenue is generated per product. In order to calculate profit, each discrete product (not product line) has a profit and loss statement tracked in accounting. Finance usually has these numbers but it is Sales' responsibility to provide the profitablity numbers to the top management.

Market share

What market share you have. By tracking this metric over time (year over year), you can discover whether the market is saturated or ripe for expansion. If the market is growing and yet your market share is shrinking, it indicates that another competitor is growing at a rate faster than you are. Getting this metric is the hardest among all metrics and not all survey agencies who do this provide reliable data for Sales. Many standard business analysts provide reports for the vertical industry itself such as Forrester, Gartner etc but again, each company has to investigate somethng that works best for them. And management must have the guts to accept the facts.
ROI on Marketing

There is a need for a closed loop lead tracking system that allows you to track activity from the marketing program that generated the lead all the way through to a closed deal. Are the right market segments being targeted? Do the positioning and market messages get a buy in with the buyers? Do you have a compelling solution? Is that packaged as a compelling solution?


And finally....

It's not possible to track all metrics at once, but determine the key performance indicators you need to start tracking. Report the baseline, begin tracking on a periodic basis, graph the trends, and drill down to find out what is going on.

Start with the metrics for customers and product quality. These are the first ones to tackle.
Calculate the profitability of your product. If you have a loss leader, can you justify further investments? Do win/loss analysis to find out whether your product is driving business, even though it isn't profitable. If it should be retired, show some leadership and present the facts about why it should be retired and how you would redeploy the resources in a more profitable way. There's noplace for personal attachments here! And no place for vested interests if you know what I mean.