This isn’t an anti-checklists post.
It’s an illustration of why picking up an example of good practice in one location, and dropping down in another doesn’t always work. No matter how brilliantly conceived, beautifully constructed or obviously ‘good’ the original good practice might be.
So, next time a politician or clever speaker at a conference tells you “it’s just a simple matter of transferring good practice”, please ask them; “what about Atul Gawande and the Hospital Checklists?” (and then get sacked or thrown out of the conference).
Checklists are a good thing. The thinking behind checklists is difficult to argue against. Basically a checklist breaks down a complicated procedure into a series of logical, easily understood steps. This helps a person/operator successfully complete the procedure.
The origins of the modern checklist (which is a fascinating story) track back to 1935 and the adoption of the B17 Flying Fortress Bomber by the U.S. Airforce. The B17 checklist helped to reduce accidents caused by human error, and versions of it have been ubiquitous in the aircraft industry for the last 80 years.
Checklists are so effective that they aren’t just restricted to aviation. They have a role in everything from; ‘packing for your round the world trip’ through to auditing and ‘shutting down your nuclear reactor in an emergency’. Useful in any complicated situation where the human brain could become confused or muddled, particularly in a stressful situation. So, it’s no surprise that the medical world (eventually) picked them up.
Atul Gawande helped us love checklists. You’ve probably heard of Atul Gawande, author of the 2009 bestseller, The Checklist Manifesto? (Yes, I do own a copy, a birthday present from my Father in Law).
The book talks about how eight hospitals took part in a pilot of the World Health Organisation (WHO), 19 Point, Surgical Safety Checklist during 2007/8. The results were impressive:
- Post operative complications reduced by almost a third.
- Death rates reduced by almost a half.
The WHO went on to recommend that the Surgical Safety Checklist (or something similar) was adopted by all hospitals. The UK National Health Service (NHS) required that all its facilities use the checklist, and by 2012 over 2000 facilities worldwide had tried checklists. An impressive example of ‘evidence based good practice’ being rolled out across organisations.
If you want to get a feel for just how enthusiastically this example of good practice has been ‘rolled out’, have a look at You Tube and search for ‘WHO Surgical Safety Checklist’. You might be surprised by just how many diverse organisations and groups have uploaded videos, I was. You can lose a few hours browsing the 3000+ search results. ‘How NOT to do the WHO Surgical Safety Checklist’ is a particularly uncomfortable yet compelling video to watch.
Checklists haven’t worked like they should. This feels like a bit of a spoiler, but do have a look at this article from Nature; Hospital checklists are meant to save lives – so why do they often fail?
The Nature article uses an example of 200,000 procedures carried out at 101 hospitals in Canada, where there was no evidence of any reduction in complications and deaths following use of checklists. It also details the failure to match results of the Michigan Checklist, which was used to prevent problems associated with the introduction of catheters into veins (eurrgh!).
For me, the article is making a very clear point that; checklist aren’t the problem, it’s how the people who receive the ‘new’ checklist choose to use it. There are a number of ‘issues’ used to illustrate this: (have a look at the helpful graphic)
- Staff resisted, or failed to complete the checklist. Sounds like a bit of the ‘not invented here syndrome’ to me.
- The checklist was illogical or inappropriate. Fair point, one size rarely fits all.
- The checklist was seen as a waste of time. A bit like the first point, people don’t see it as useful.
- ‘Parachuted in’ solutions. It was just another ‘initiative’ dropped on front line staff by Managers and Administrators.
- It felt ‘Imposed’. People do have a tendency to resist things that are imposed upon them, regardless of how much it is the ‘right thing to do’ (I know I do).
- It didn’t fit the local context. The requirements of a checklist developed in a well-resourced American hospital might not apply in an under-resourced hospital in a conflict zone. You may not have the same standard of equipment or even the same number of people available to perform the task.
It’s all about the implementation. The Nature article goes to talk about the need for careful thinking about how you transfer good practice and the growth of Implementation Science.
As an example of proven good practice (which saves lives) the WHO Surgical Safety Checklist stands out. It’s been enthusiastically adopted in many places, just have a look at You Tube. However there are places where it has failed to have the desired impact. Based on this experience I think there is a case for thinking more widely about the process of implementation in the transfer of good practice and the role of Implementation Science. Good practice transfer is not always a simple case of ‘just do it’.
So, what’s the PONT?
- Checklists are a very effective way of reducing human error in complicated (and straightforward) procedures. An example of ‘transferable good practice’.
- Issues around ‘implementation’ the WHO Surgical Safety Checklist have prevented them from being fully effective in some settings.
- Universal acceptance and use of the Surgical Checklists might take time (Aviation checklists are after all 80 years old).
Related Post: Why is Good Practice such a bad traveller? http://whatsthepont.com/2012/08/19/why-is-good-practice-such-a-bad-traveller/
What's the PONT 
Had no idea the humble checklist originated in a B17! Live and learn. 🙂
You could replace the word ‘checklists’ with ‘computer systems’ and this would all be equally valid! There’s often a strong urge to implement a system because it’s working well elsewhere.
And just in the same way that people HATE having checklists thrust upon them, they really despise having to bend their working day to new software they don’t understand or think they need.
Last week I was pondering how context is EVERYTHING – particularly when trying to implement change. It’s one of the reasons that I think Working Out Loud is so vital – it’s an opportunity to narrate the journey of problem solving and co-create the solution together (thus defusing some resistance along the way).
Thanks Neil,
Context does absolutely count for so much.
There are very few situations that are exactly alike, allowing the exact replication of a process from one place to the next.
A ‘forced fit’ can sometimes be worse than nothing at all.
The point you make about how something is introduced (or imposed) is really important. I think many of us have an ‘in built’ resistance to meekly accepting imposed things, no matter how ‘good’ they might be for us.
To quote the Daleks, “resistance is futile Doctor…..”
….. but I’ll give it a good go anyway….
Cheers
Chris
The big mistake people make is to use checklists to enforce compliance and attempt to control the actual work. Crap! Checklists are best used before a long boring sequence of tasks to mentally prepare, or just after a short boring task to make sure you haven’t forgotten anything.
The idiots try to apply them by saying they make up for weaknesses or deficiencies in perfect teleologicall processes, so called human factors. That’s wholely inaccurate and only said by people who don’t really understand the cognitive science. People reject being patronised by an idiot, they rarely reject a handy list of boring stuff!
There’s also growing evidence that what it says on the list is mostly irrelevant, it’s the reframing experience that creates safety, by ritualising the transition from one social context to another. It’s more important that the theatre team perform the ‘checklist’ together to heighten collective situational awareness as they move from social norms, to formal roles around the operating table, than what it actually says on the checklist. #mindbombed
Thank you Matt,
I think the Nature article needs your quote “people reject being patronised by an idiot, they rarely reject a handy list for dealing with boring (but necessary?) stuff”
I think they might have been trying to get at that when they talked about the problems of managers/administrators dropping the checklists onto operational staff…… Code for “patronised by an idiot?”
I get point about re-framing as you move from one social context to the other, and it being as/more important than the detail of the checklist itself.
I do wonder if this is the same for all checklists?
I have a checklist for packing before a cycling holiday (spare inner tubes, x pairs of socks etc). Packing my kit is a solitary exercise. I wonder how that would fit?
The social context is the same (me packing at home)…… Unless….. It’s a mental shift to the social context of cycling?
I’m rambling now.
Would be useful to have a gander through the work that’s been done on this.
Thanks
Chris
You do not need to read some arsehole professor’s view on this malarkey to understand the difference between a two dimensional abstraction (the list) and a three dimensional methode plonge (doing it yourself). Tomorrow morning, brush your teeth using your other hand! The toothbrush pattern is embedded in your usual hand. The unusual hand requires a checklist!
Healthcare and aviation are two very different industries with two very different cultures. Checklists (and Crew Resource Management generally) were transplanted from one to the other despite this poor tissue match and, not surprisingly, were rejected!
The underlying genetics, however, are sound but require a Genetic Engineering approach to transfer them. Small changes at the coalface with the support of the staff will enable any benefits to spread through a unit or hospital by natural selection (staff generally want to do their best and will support initiatives which help them to do so) whilst failures will simply be bred out of the system.
Dr Lucian Leape told us way back in 1999 that “To Err is Human” but Error Management Training failed to be implemented with the result that our results now are little better than back 16 years ago.
Aviation needed the loss of almost 600 lives in the Tenerife disaster in 1977 to make it focus. What will healthcare need to force it to do likewise…??
Thanks Niall,
Sorry for the delay in responding.
I agree about the challenges of transplanting practices – the world of organisational antibodies is frequently ignored by those trying to impose change or force improvement – its a complex process.
In my slackness some one has come to the rescue (Think Purpose) and helpfully provided a comment on the difference between aviation and medicine and referenced the latest book by Matthew Syed, Blackbox Thinking.
The point about the difference in cultures being affected by the ‘visibility of failure’ is quite interesting.
Aviation – failure is very obvious so you have to do something about it
Medicine – the reasons for failure are less obvious?
Your point about Tenerife is well made.
A did get involved on a conversation about Standardised Mortality Rates a while back.
I wonder if the better use of data will help to move on?
Thanks for responding
Chris
I guess that why human beings are fall safe for broken checklists – maybe invest more in them and use the checklists as aide memoirs for effective good practice transfer.See David Denyer’s presentation at @Academi’s summer school 2014. http://www.daviddenyer.com/wp-content/uploads/2014/07/map.jpg. Black Hawk (1994) https://en.wikipedia.org/wiki/1994_Black_Hawk_shootdown_incident. The investigating officer couldn’t believe how this hadn’t happened sooner.
Thanks Clover
I sat through that presentation.
None of this is perfect.
All just tools to help our thinking, not replace it.
Chris
Hi, there’s a book called Blackbox Thinking by Michael Syed that I’ve downloaded a sample chapter of, and in the very first chapter he goes in depth into the difference between the aviation culture and their thinking about learning and failure, and the very different medical culture of cover up and deference to hierarchy. It’s very interesting but the thing I’ve realised is that they’ve got skin in the game, literally, compared with doctors and nurses. And its impossible to hide an aviation disaster compared with a sick patient dying. That’s what sick people do, right?
Thank you.
That point about ‘skin in the game’ and the consequences of failure (however) caused being obvious in aviation, compared to health is, a great insight.
The more I think about it the more it resonates.
– single person dies – reasons are a bit obscure – nothing much happens to learn from any mistakes or poor practice
– 100’s die in an air crash – every last fragment and piece of data is analysed to understand why – and prevent it happening agin – which permeates the working practice of a whole industry
I some respects this might link to why the pharmaceutical industry (which has the potential to affect 1000’s people) spend so much time on clinical trials before they release drugs?
I’ll have a look at that Matthew Syed book – thank you.
I saw him speak a while ago about ‘success’ – it was a bit like a British Malcolm Gladwell – talking about the 10,000 hours thing to turn you into an expert.
Thanks for the comment.
Got me thinking about a follow up to the checklists post.
Chris