This is the third in a series of posts (second one here) about some work using SenseMaker to get a better understanding of attitudes to risk. This final post explains how we used SenseMaker in a new way for us, looking at X-Y plots and possible correlations between data sets.  It is very much ‘work in progress’.

Text from the original post is below, with the hopefully familiar ‘What’s the PONT?’ added at the end.

Why involve service users in decision-making?

There is currently a great deal of activity across Public Services in Wales to ‘involve’ citizen and service users. Things like the Future Generations Act Five Ways of Working (involvement) and the Prudent Healthcare Principles (co-production) all play their part in encouraging this behaviour.

With all of this activity it is reasonable to ask the questions, ‘does involvement make a difference to service users?’ and ‘does it bring any benefits?’

How do you ask a difficult question like that?

When we mentioned we were testing the SenseMaker technology and pushing our understanding to the limits, this is it…

We avoided asking the question directly by looking at the data from a slightly different angle.  We extracted data from two separate triangles, illustrated below (Graphics 1 and 2) that featured service users in some way:

• Benefit felt by service users, and
• Who can contribute most knowledge to the decision?

Pushing our understanding, creating an X / Y Plot

What we did next was to create an X / Y Plot which looked the benefit to Service Users and how much knowledge they contribute to the decision.

Our thinking was, if there is a benefit to be gained by involving service users in decision-making, we should see a relationship on the X / Y Plot.  This would be most clearly seen by plotting a straight line through the dots. The basic idea we followed was:

• If a line appears that runs left to right in an upwards direction, there is a positive relationship and service user involvement in decision-making leads to greater benefits for service users. The more involvement, the greater the benefits.
• If a line appears that runs left to right in a downwards direction, the opposite is true. Service user involvement in the decision leads to less benefits for the service users.
• A flat line would indicate no relationship between the two.

What the X / Y Plots looked like

Graphic 3 All Data

• The upward direction of the line suggests that there is a positive relationship.
• The more the Service User knowledge is used to inform the decision-making, the greater the benefits for the service users.

* see caveats at the end

 

 

Graphic 4.  Different Frameworks

• This shows the results when the data is split across the two approaches to risk management we presented:
  • Safe to Fail (light blue)
  • Failure is not an option (dark green)
• There is an obvious difference in the straight lines through the dots.
• The light blue line for the safe to fail framework suggest a positive relationship.
• The dark green line (failure is not an option) suggests that greater involvement of service users leads to fewer benefits for them.

Does the scenario make a difference?

Another approach to examining the data was to look at the scenarios we used. Does the scenario influence how people think about service user involvement in decision-making. Broadly the three scenarios we considered were:

1. Data sharing. A proposal to share data about service users between several organisations.
2. Service user complaints. Improving the approach to complaints handling in a single organisation (formed from the merger of several organisations).
3. Tackling obesity. A society wide issue involving multiple organisations.

Graphic 5. Influence of the Scenario

• The red line for the Tackling obesity data is very different for the ones for Data Sharing and Service User Complaints
• The upward slope of the red line suggests that service user involvement in the decision-making process will have greater positive benefits for them.
• The difference with the other scenarios could be explained by Data sharing and Service user complaints are more focused on arrangement between or within organisations, rather than something like tackling obesity which potentially has closer involvement with service users.

So, does involving service users in decision-making lead to better decisions?

Our analysis here suggest that in some situations the using knowledge from service users does lead to greater benefits for the service users.

This is dependent upon the context in which the people who took part in this experiment were placed and how they were thinking at the time.

There was a positive relationship where they were thinking about a ‘safe to fail’ situation, and a negative relationship in a ‘failure is not an option’ context.

This might suggest that when operating in a ‘failure is not an option’ environment that service user involvement is not given as much weight compared to a safe to fail approach, suggesting that in a ‘failure is not an option’ circumstance service user involvement becomes less meaningful and more tokenistic.

The scenarios we presented showed a considerable difference in how people responded.  For a scenario where there was likely to be a need to interact with service users (tackling obesity), there as a more positive relationship  than scenarios which were more focused on process and activity between and within organisations (data sharing and service user complaints).

*Caveats. Please remember that there are huge caveats around this data and the limited analysis we have carried out, for example;

• What influence would a larger, more diverse group of people have on the results?
• What influence would the views of service users have on the results?

So, What’s the PONT?

1. For the group we ran this test with, they seem to recognise that service user involvement in decision-making (co-production) will lead to decisions that benefit the service users (which is surely the point of public services?).
2. There are however some decisions that are probably better off taken without a prolonged discussion with the service user. To quote @ComplexWales  (Matt Wyatt) “I don’t want to be asked how I want to be defibrillated … just do it!”
3. And getting right back to where this series of posts started, it’s all about the context. Understanding the context and using risk management and taking decisions that are appropriate in that context. There is no one size fits all or universal method that applies everywhere, all of the time.

Finally. As mentioned earlier, this is an experiment for us and an example of us ‘working out loud, doing things in the open’. There is still a lot more we would like to do with this data. We are certain that we haven’t got things right and would appreciate any comments and feedback on what we have tried here. If anyone would like to have a look at the dataset and help expand our understanding, please get in touch, we would very much like to talk.

This post is linked to others that look at:

• Post 1. Context is everything. This is a brief description of what we did in the session and some observations on how people think they would respond to failure in the context of different risk management approaches.
• Post 2. Is common sense more useful than the rule book? This reviews the data we collected around how people use different approaches when they are making decisions about risk.