Fi Mercer caught up with Dr Marie Bismark to go through the key tips from her insightful and informative keynote presentation at the VHA Governance Conference, April 2018.
Responsibility for quality and safety starts with the board
In discussing the background to her presentation Marie noted that: “In recent years, some high-profile events have highlighted the critical role of boards in shaping our health system. These include the tragically avoidable perinatal deaths at Djerriwarrh Health Services and significant failures of care at Oakden Older Persons Mental Health Service. Reviews of how to prevent such events in future repeatedly emphasise that responsibility for clinical quality and safety starts at the top – with the board.”
Marie went on to say that this raised the important question of “How best can boards go about leading this change?” Many efforts to improve quality and safety are based on anecdote, intuition, past practices, and the wisdom of experts. Yet, the history of medicine tells us that such approaches may not offer the best solutions and may even cause harm. A more evidence-based approach to governance requires an aptitude for asking the hard questions, a commitment to gathering the right kinds of information, the skills to interpret and weigh that information, and the courage to act on those findings.
Her presentation drew on her own experience as a practising clinician, a health lawyer, and a company director and shone a light on the pathway towards more evidence-based leadership for change.
Nothing new under the sun
Marie’s presentation started with a brief history of evidence-based medicine and pointed out that the concept is not new. For example, the Book of Daniel in the Bible records that King Nebuchadnezzar instructed some soldiers to have meat and wine while others were fed legumes and water to determine which would result in the best physical condition (the vegetarians were better nourished at the end of ten days).
During the Crimean War, Florence Nightingale gathered detailed data on the number of deaths from battle wounds compared to preventable diseases that could be reduced by better nursing care. At the same time, Florence was conducting nightly rounds of the wards – a wonderful early example of “leadership walkarounds”, in which someone could see the big picture through data while remaining connected with the realities of care at the bedside.
Marie described how throughout history many medical practices, based on “expert judgement”, have been at odds with the data. For example, it took fifty years from when James Lind first showed, in a randomised trial, that citrus juice cures scurvy before the British Navy routinely provided lemon juice aboard its ships. Over time evidence-based decision-making has become increasingly well established in medicine – at least in theory, if not always in practice.
Importantly, evidence-based decision-making requires judicious decision making: it’s not simply a process of “cookbook” decision-making. The evidence forms part of the decision, but boards still need to assess the needs of the local community and consider how best to implement findings in their own organisational context.
Evidence-based decision-making is “…a continuous interactive process involving the explicit, conscientious and judicious consideration of the best available evidence. itis essential to optimise outcomes for individual clients, promote healthy communities and populations, improve clinical practice, achieve cost-effective care and ensure accountability and transparency in decision-making within the health-care system.”
What’s the Board’s role in evidence-based decision making?
Marie said the board’s role, first and foremost, is to ask the right questions. In her own governance work, she uses a four-stage model, known as the four A’s of evidence-based decision-making.
1. Ask the right clinical question
“It is crucial for boards to have an enquiring mind”
In her presentation, Marie gave a good example of an aged services board receiving a report of a gastroenteritis outbreak in a nursing home. The board might ask: “Why haven’t our nurses been washing their hands?’ But this question assumes that the board knows where the problem lies, rather than asking a deep enquiring question to identify the real problem. A better question might be: “Where is the infection coming from?” as a subsequent audit may reveal that the real problem lies with the kitchen staff rather than the nurses.
2. Acquire the best evidence
The next step is to acquire the best available evidence. Marie uses the following evidence pyramid to describe the different types of evidence available and the importance of striving to base your decisions on the strongest available evidence.
A) Expert Opinion
Expert opinion is commonly offered around the board table as a basis for decision-making, but from an evidence-based perspective, it is less persuasive than hard data. Marie noted that in the boardroom opinions can come from:
- Anecdotal feedback from the community
- Interpretation of a single event
- Personal views of executives or board members
Marie noted the risk of boards deferring to the opinion of the “HIPPO”: the highest-paid person in the room. In health boards, senior clinicians and technical experts may also offer influential – but not always evidence-based – opinions.
When boards are asked to make a decision on someone’s opinion alone, they shouldn’t be afraid to ask: “What additional evidence would help us to make this decision?”
B) Observational Data
Observational data is the kind of data that boards are most used to seeing, and can range from a summary of adverse events that occurred over a month through to a large survey of patient experience. Having observational data can substantially improve the evidence-base on which you make a decision. However, observational data is not well suited to identifying the cause of a problem or assessing the impact of an intervention.
C) Controlled Trial
Controlled trials offer a stronger form of evidence for decision-making. They are commonly used in medicine, for example, to test the effects of a new medicine. Controlled trials are less commonly used by boards and policymakers, although there is a strong push for better evaluation of governance decisions as they can impact just as many lives as a new drug.
Marie emphasised that the benefit of having a control group is that it allows you to answer the question: “how will we know if this works”. Marie gave an excellent but simple example of an aged care provider with more than one campus. She noted that if the board wanted to understand the benefits of a new rostering system, they could trial it in a few sites first, and compare changes in staff satisfaction compared with sites that were still using the old system. For organisations with only one site, it is possible to do comparisons between different wards, before or after comparisons, or to work with neighbouring hospitals to assess the impact of an intervention.
D) Synthesis of evidence
The top level of evidence brings together findings from multiple sources of data to ensure that the results are consistent and replicable. Marie notes that: “One study may not provide the right or enough evidence.” She recommends the Cochrane collaboration as a great source of synthesised evidence on key issues for health service boards. See for example: Interventions for preventing falls in older people living in the community
Another form of synthesised evidence comes from pulling together information from patient surveys, staff experience, clinical audits, and sentinel events to look at one problem – such as medication errors – from several different perspectives.
3. Appraise the Evidence
The third step involves “sorting the wheat from the chaff’. Whenever you are presented with evidence, you need to assess whether it is reliable and relevant. Boards might like to ask: “Who generated the evidence, and what might they stand to gain?” For example, a study of the benefits of a computerised health record is less likely to be biased if it was conducted by independent researchers, rather than by the company selling the solution.
Marie also noted the important difference between accuracy and precision. Accuracy describes the difference between the results presented in a report and the actual value – for example, an adverse event report may be inaccurate if staff don’t feel safe to report critical incidents. As part of assessing the accuracy of data, boards might like to ask: “Who aren’t we hearing from?” to ensure that important voices can be heard even when they whisper rather than shout.
Precision describes the likelihood that you will get the same result if you measure something several times. For example, rates of falls in a hospital are likely to vary from month to month simply due to chance and it is important not to overreact to changes in data if they are not statistically meaningful. A helpful question for boards to ask is ‘Is this a significant difference, or could it be due to chance?’
4. Applying Evidence
The fourth and final stage involves having the courage to act on the evidence. Marie noted that good clinical governance is a cyclical process of understanding a problem, identifying a solution, implementing a solution, and then evaluating the impact. At all stages of the process, Boards should ask themselves “What outcome are we hoping to achieve? How will we know if this works?”
Conclusion
In her concluding comments, Marie assured board members that they don’t all need to be statisticians. However, they do need to have the confidence to ask questions about the evidence that lies behind important decisions. Just as all board members are expected to develop their financial literacy, even if they don’t have an accounting background, all board members should be comfortable with the basics of evidence-based decision-making.
Her keynote presentation was not only incredibly engaging but it also took a lot of the myths out of evidence-based decision-making for boards. If boards are willing to ask the hard questions, gather the best evidence, carefully weigh that evidence, and act on those findings, they will be well placed to lead improvements in quality and safety.