Buffet v Smorgasbord. A ‘buffet’ is a cornerstone of society round these parts. From the most ‘interesting’ of family gatherings through to the most soul draining of organisational morale boosters; there’s always a buffet. Basically a mixture of foodstuffs, something that will more or less please everyone. The range and randomness can vary a lot, but I’m sure you get the gist.

A buffet is largely the same as a Smorgasbord, ‘a wide range of things / a variety’, but with the benefit of it being from the Nordic regions. This makes it sound a bit more exciting and suitable title for what I want to say about measurement. I’ve been wrangling with this for ages.

The main point I want to get across is to treat measurement as a Smorgasbord. There are a variety of things you can measure (it’s a choice). So pick wisely and don’t shy away from trying something different. I’ve tried to capture it in this sketch note, which I’ll try to explain.

So What’s the Problem?

1. One Size Fits Nobody. Each approach to measurement will have been designed for specific purposes. Just because it works there, doesn’t mean it will work elsewhere. I know that sounds bleedin’ obvious, but it is worth stopping to think about it. Here’s something on OMI Loops (Obsessive Measurement Infinity Loops)I wrote which makes a case for stopping and thinking before measuring.
2. Measurement has it’s own impact. Things like Campbell’s Law, Goodhart’s Law and Dave Snowden talk about how measurement changes human behaviour; it does. Not always in predictable ways and sometimes not in a way that is helpful. I’ve tried to capture a bit of that here in: Campbell’s Law and why Outcome Measurement is a Dead Cobra.
3. People like their favourites. Another statement of the bleedin’ obvious, but it’s worth saying. People will stick with what they like, even when it comes to measurement. This can be a problem if the favourite approach they are using isn’t appropriate for the current situation. There’s also a challenge if you are asking someone to try something different. Offering a different approach to measuring something might not seem like a big deal, until you try to change it.
4. Value for Money (VFM). There’s a cost associated with all measurement activities. They might be quite small if it’s some sort of digital transaction but it’s still a cost. The cost of an annual return for a bureaucratic grant award can be huge (a friend tells me). The VFM question needs to be part of deciding on any measurement activity. If nobody is going to use what is collected, or it is ignored because they don’t like it, where’s the value for money in that?
5. Different ways of knowing. Ages ago I was introduced to the idea of ‘the Dangerous Olive of Evidence’, link here. The analogy goes along the lines of; measurement is like an olive. All of the evidence we could use is the actual olive. The hard wooden pip is just part of what is available. In lots of measurement we focus on the equivalent of just the pip. There are many reasons for this, but the result is that we ignore lots of things that could help us understand what is going on – different ways of knowing. From a decision making perspective, surely it’s got to be better to base your decision on a number of sources of information rather than just one?
6. Over simplifying things. It’s just a fact of life that not everything can be simplified. Some things are complex. That’s all I’m going to say. If you want to know more, have a look at my reflections on Cynefin as a Novice Street Performer at a festival in Cardiff (there’s also a link to a serious explanation).

So what do you do about it? I’m not and shouldn’t try to answer that question here. It’s up to you, and the situation you want to measure. But here are three heuristics (general principles) that might help you:

1. Learn as you go.
   • Short feedback loops and continuous learning are more effective than leaving everything to a ‘post mortem’ at the end. It avoids Retrospective Coherence (making sense of things by looking backwards) and the Cheese Fondue approach to post project learning I’ve written about.
2. Know your context.
   • Use something like the Cynefin Framework to help you decide what approach to take.
   • A clear, structured, predictable situation might favour a numeric Key Performance Indicator (KPI) type approach,
   • A complex human situation with many variables and uncertainty might favour an approach using narrative fragments and the stories people tell each other.
3. Seek user acceptance.
   • Find out what ‘works’ for the people you need to influence, and work with that.
   • Don’t force people into things they don’t like and don’t value.
   • It’s all about understanding your ‘guests’. If they like what you do, there’s more of a chance that they might accept your ideas when you offer them something different.

Finally, for all of the above, DO. IT. NOW. and DO. IT. OFTEN. Hope that’s clear.

So, What’s the PONT?

1. Measurement is most useful when it is a mixture of different things; a variety, a Smorgasbord.
2. Diversity and difference help. A range of different sources of information will help you to make a better decision.
3. All measurement costs something. So from a Value For Money point of view, always keep this in mind before measuring anything. A measure that is ignored costs time, resources and emotion and helps nobody.