It can be difficult to create qualitative metrics for modern service systems. Here are some thoughts to help your work.
North Patrol is a consulting firm specialized in the design of digital services and information systems. We shape ideas into a vision and service concept, find the best architectural and technological solutions, design a functional user experience, and compete to find the ideal partner for implementation work. We do not sell implementation projects, nor do we sell licenses; we are genuinely on the side of the customer.
What needs to be measured?
- What is essential for creating and maintaining a high-quality service system? What makes your customer to feel respected?
- What is essential for the business? What makes your system sponsors happy?
- What is essential for the internal processes? What makes other systems to run without extra hiccups?
- What is essential for the lifecycle of the system and for maximizing the ability to serve? How can you make your service system to provide high-quality service for your customers for years and years?
The Customer
The customer wants an easy service experience. Suitable metrics are for example the active time spent for the service process, the number of the errors the customer experiences, and the overall customer experience. The customer may suspend the service process for example to take care of another task in between. Therefore, measuring the active time is important. If measuring the active time appears to be too complicated, the KPI can very well be the median of spent times. Recognizing the possible errors, the customer may experience, is important already in the service development phase. If the erroneous situations have different impacts to the service providing, for example if there are barring, interfering, or non-harming errors, it is beneficial to make a scoring system for creating the KPI. The overall customer experience is most often measured with Net Promoter Score question in the end of the service process.
Finance
The sponsor of the system is often interested in tangible metrics, such as the ROI. They can be relevant measures, if the service process is automated for the first time. If instead, the readily automated process is renewed, the ROI may not be very good measure, even though the threat would be that the customers could abandon using the service system. The use of financial metrics requires special care. TCO is a good start. It is an absolute number and doesn’t usually cause dispute. Financial measures can also be led from the internal process improvements. If, for example, your new system provides better quality data to other systems, you might be able to estimate the savings from the decreased error handling time.
Internal processes
Service systems often produce relatively much data for the other systems inside the organization. The quality of the data can be measured by recognizing the key factors. For example, if the service system generates contracts, the number of the contracts produced by the system but manually altered afterwards, could be measured. Even though this sounds tricky, most of the service systems have related qualitative requirements that are set already in the project phase.
Lifecycle and future service capability
The lifecycle and the future service capability of the system is often measured by researching the organization that is developing it. It does not necessarily give the right image of the development ability of a single system. Measuring the number of reported incidents and the time spend for customer support will already get you far. Especially, when the trends can be observed from the data. If it is a modern and modular system, also the number of modules relative to the number of functions can be used as metrics. Distributed software architecture will last time better than monolith architecture. It will bring less headache when integrating the new development with the existing code and enables running the applications on different platforms. Also, the functions can be produced with different programming languages.
What is not needed to be measured?
When selecting the metrics, it is a good idea to emphasize the system stakeholders’ needs. What will not be measured is as important question as what will be measured. The developers of the balanced scorecard, Kaplan and Norton, said: What you measure is what you get. They hardly meant that if you measure everything you will get everything. How does the end customer, system sponsor or internal processes benefit for example from the common metrics of the amount of new software releases annually? Earlier it was important to measure the service production closely. Among others, these metrics included the downtime and availability of the service. They had a direct impact to the service experience, but their weight is growing smaller all the time with the better reliability of the servers, server software and data communication. Especially, what comes to the cloud-based service systems, these metrics may benefit very little the service development.
Motivation for measuring
Why then should anything be measured? Customers will tell anyways if something is wrong! Customer do tell indeed if something goes wrong. Often, however, they tell it by abandoning your system. In that case, it could be too late to try to fix the situation. Collecting the data for a long period of time and recognizing the trends is essential for the service development. Systematically collected data makes it also easier to communicate the needs for changes for example for the system sponsors.
