Primary Goals and Strategic Metrics vs. Operational Metrics

Measurement is key to any successful engineering effort, the key is what metrics you choose to measure and how to interpret the measurement.  In general there are two types of metrics associated with two different types of goals

- Primary (or strategic) goals and related metrics
- Operational Goals and related operational metrics

When goals are defined, It is very important to define which category a goal belongs to so that progress (or lack thereof can be measured accurately).

An example makes the difference between the two of metrics clear. Imagine you want to drive from San Francisco to Los Angeles. Your goal is very clear: getting to L.A. and the metric associated with it is also clear, how far are you from L.A. (or how far have you traveled so far). You drive a car, so you measure your car's engine temperature and fuel level - these are your operational metrics.

You can keep your engine temperature within a reasonable range and maintain proper fuel level, but these are not the goals of your trip. If you are happy simply b/c your engine does not overheat, and you are not concerned with how far from L.A you are, you are not measuring the right metrics.

Distinction between primary goals and operational goals is not always as easy to spot. Imagine you are designing an Order Management service (or any other service) - it is easy - maybe even common - to measure number of calls to the service per hour, day etc. and assuming that the goal is being achieved. However, if you service can only be called by one type of client (b/c it is the only one that can supply an input parameter for example), are you really achieving one of the most important (and common) goal of service design which is re-use of a capability by all consumers? Here, number of calls is an operational metric, it has to be measured, it may be necessary but not sufficient. The measure the primary goal - you need to measure the diversity of consumers (across languages, device, platforms)

For another example assume you deploy a distributed cache service - the goal of any cache service is to improve performance (and scalability) - again,it is easy to measure metrics such as number of calls to the service or even more specific metrics such as "hit rate" -but the primary goal here is to measure - from consumer point of view - performance and scale improvement.

One main reason, operational metrics are often measured, and used for decision making, instead of primary goal or strategic metrics is that they are generally easier to measure. It is certainly easier to measure "number of calls" to a service than whether a service increases its consumer productivity. "How to Measure Anything",  by Douglas Hubbard is a great resource for techniques to measure hard to quantify goals and metrics.