If there’s one reoccurring truth, it’s that good analysis follows process and a multidimensional one at that.
The first dimension is to take a measure and see it in its own context. This the analytical process described below. In short the process for a basic rounded analysis, to get to an overview quickly and confidently. There are pointers to more to more advanced options below, but not intended to distract from getting to that overview. The second key dimension is to look at a range of measures – assessing measures in the context of other measures - but more of that another time.
The starting point is of course a “Question” for which this analysis provides an answer, (as well as new questions). In the most general sense, that question might be “how are we doing”. Simple examples might be profit, crime rates, sales, customer satisfaction and so on. The broad analysis components are the same, although the emphasis may vary.
So assessing a measure in its own context has three main components, and the analysis of this measure splits into five steps.The components are snapshot – we have a number that is important to us (step 1), trend - to see what’s happening to that number over time (step 2) and benchmark – to see how this compares to others (step 3). This provides the material to provide a consolidated assessment (step 4), and the foundation for further analysis (step 5).
Step 1: Snapshot – we have a number that is important to us.
This is about having a relevant, realistic and meaningful measure to analyse. This first step is to have a value on that measure. A
simple step but important to ensure some clear thinking on the fundamental basics. This should be relevant to a specific purpose, and meaningful in that it can be understood with some ease and confidence. This is all about having a starting point, the right measure which is important to us.After all the analysis of the wrong measure is mostly pointless.
Our starting point might even be a single data point, but one that we know is important to unpack. That one data point might often seem sufficient in a paragraph, but graphically starts to look rather insufficient...
Step 2: Trend – what’s happening over time?
So what is this measure doing over time? Is it stable, getting higher or lower, or more specifically getting better or worse? Using the same measurement over time, there should be some understanding and confidence that this is measuring the same thing, that definitions have remained stable for example. Or if not, when and how they changed.
So now we have at the very least a second data point, and ideally more. In short, more data points the more clarity. After all there will become degree of “noise” in the data.
In fact in many circumstances it can be more probable that the data will be different from one time point to the next, because of that “noise”, in short ‘natural variation’. Hence with two or three data points you still might be seeing noise rather than real signals or messages. In fact in this graphic there are places where the measure goes down from one time period to the next, despite the overall upward trend….just using those two data points on their own would point wrongly to a trend in the opposite direction.
[More advanced options: This would involve looking at the trend over different time groupings.Having looked at a monthly trend, it may well be that a yearly or weekly perspective tells different story. See “Trend, what trend?” ]
Step 3: Benchmark – how this measure compares to others?
Having set that snapshot in the context of time, and seen the trend for the data - the next step is to see that snapshot in the context of other data on this measure. Typically seeing how this compares to other organisations for example
Of course comparing when data with other organisations, we typically do not have the same degree of understanding and confidence about the data, as we do with our own data, so we need to tread more carefully. If not comparing apples and pears, we might at least be comparing different varieties of apples, or if we’re lucky, just different apples of the same variety.
From this we can see things if look better or worse than others. So is our snapshot measure typical, higher or lower, or perhaps in the higher or lower extremes.
[More advanced options: This would involve looking at different sorts of benchmarks which might tell a different story. This might include looking at national or regional perspectives, or indeed other sectors. See “Benchmark, what benchmark?”]
Step 4. Consolidate Assessment
At this point it’s key to consolidate what has been learned to get an overall picture – that basic rounded analysis. Tempting to do more detailed trend or benchmark analysis, but this may be at the detriment of getting to a broader picture quicker.
So now we have a sense of how our snapshot compares over time and to others, which we need to bring together. A simple way to do this is to consider the outcomes of both trend and benchmark
Trend – things getting better or getting worse?
Benchmark – good compared to others or poor compared to others.
It’s really helpful to approach this graphically (based on an approach called the Boston Box, originally developed by the Boston Consulting Group). Often there is no clear cut trend or the benchmarking, but it’s still possible to identify where the data sits in this overall framework.
The best place to be is probably improving and already best in best in class. The worst place is probably getting worse and already worse in class. The remaining two categories are a bit more subjective…worst in class but getting better might just have the edge on better than others but getting worse.Step 5. Further Analysis
The process above can bring us to a rounded and big picture assessment quite quickly. There are more detailed analysis opportunities that are possible along the way- and these are discussed separately - but they should not distract from getting to a quick high level overview on these trend and benchmark perspectives. For example, if we knew from our trend that the performance was downward, our response to that would be different depending on our benchmark, whether for example best in class, or worst in class.
This current picture provides a foundation from which to look ahead, and here’s the quick indicative peek…
5a... look at the trend for the comparison to others [benchmark trend]
5b… look at the trajectory for our measure. [projection]
5c… look at the trajectory for the comparison to others [projection of benchmark trend]
