I was standing outside a restaurant the other week when I noticed a sticker proudly affixed to the
window, declaring that some organisation or another had awarded it three stars for food hygiene. So far, so good. Unfortunately, my eye was then drawn to the fact that there were a possible five stars that could have been awarded. Not so good. What was I to infer from this, I pondered? Was the kitchen moderately hygienic or the food moderately fresh?
Or were the standards very good on some days, but not so good on others? If so, which days of the week picked up the stars and which days didn’t? These were unanswered questions that would prevent me from crossing the threshold to discover the truth for myself.
It wasn’t too long ago that ridicule was raining down upon the hospital league tables and star rating systems. Unless information has both meaning and context, it serves only to confuse and mislead us. If the existence of the information is designed to facilitate choice, then it must add value to that choice and enhance the process. If it is intended to prioritise activity, it needs to ensure that time and effort is not diverted away from what really matters, towards some distracting cosmetic surgery on the performance measures and the league tables themselves.
The lessons of league tables
appear not to have been taken on board, because this somewhat populist approach has emerged from the recycling bin and will soon be available to confuse patients in their choice of GP, as well as their choice of hospital. How much better can it get? Laurence Buckman, chairman of the BMA’s GP committee, compared the government’s plans to Strictly Come Dancing, tempting GPs to encourage patients to ‘vote’ for them rather than getting on with the day job.
Criteria
It is possible to juggle the criteria of any league table in order to load the dice in favour of a particular outcome. Manchester United would never win the Premiership if all goals scored within Greater Manchester were discounted. Formula One motor racing fundamentally changed its rules this year, and instantly there were new winners and new losers among the teams and
individual drivers. And as we watched the same drivers racing around the same circuits as last year, we were drawn logically to the conclusion that it was the change of rules that made the difference and changed the outcome. Is Speedy Gonzales a worse driver because he is driving an inferior car? Almost certainly not. They are all good drivers, but some are made to appear better than others – for a while at least – because of the modified rules of engagement.
It is much the same with the league table approach to health, education, local government,
investment accounts, and a lot more besides. The exercise only helps me, as a consumer with a choice to make, if I happen to share the same values and priorities as the person who is drafting the rules and the assessment criteria. And what are the chances of that?
NHS Choices is the mother of all websites, a huge £80m extravaganza which (so far, at least) has bucked a long trend of spectacular NHS IT disasters. Critics have pointed out that some of its functions (like the ability for patients to post feedback on their experiences) replicate the independent online resources patientopinion.org.uk and iwantgreatcare.org. Some argue that the fact that the latter are independent of (and hence, not moderated by) the NHS itself, makes them particularly useful.
But, while of some value as part of the mix, this is still hopelessly subjective and why should you base your healthcare choices upon the perception of a total stranger, who may have their own agenda and may even – heaven forbid – be barking mad?
NICE (or more accurately these days, NIHCE), has just launched yet another initiative, NHS Evidence, which was foreshadowed by Lord Darzi in his final NSR report. Aimed primarily at healthcare professionals, NHS Evidence will ‘sort, sift and prioritise a range of information and award an accreditation mark to the most reliable and trustworthy sources of guidance’. Spare us, your Lordship – not another accreditation mark, please. Is it not slightly perverse that highly trained healthcare professionals need to be helped in this way in order to ensure that they are not misled by data of doubtful quality and relevance, while information is thrown at the great unwashed (sorry, ‘the public’) with no such caveats, on the assumption that if you throw enough information at them, they will make better choices. And where is the evidence for that?
Hidden truths
The potential problems are highlighted by the events at the Mid Staffordshire NHS Foundation Trust, site of the latest tragic outbreak of ‘death by data’. The Trust was not short of information, but awash with it. It had 5-6 years of wall-to-wall data, none of it terribly reassuring and some of it deeply worrying. Unfortunately, the source information was also poorly coded, and important facts that the Board needed to know, were obscured. The Healthcare Commission’s report questioned the wisdom of high-level indicators like ‘quality of care’ and ‘communication’, which concealed low-level and acutely important deficiencies that were threatening lives. For years people were shouting but nobody was listening.
It is estimated that between 400 and 1,000 more people died in the hospital over a three-year period, than would normally have been expected.
In the wake of this, NHS Choices is about to publish standardised mortality rates for individual hospitals. Fortunately, this information comes with a government health warning from Sir Bruce Keogh, NHS medical director, who stresses that this league table is a very blunt tool that is not enough in isolation for an objective assessment to be made.
In flagrante delicto
As recently reported in Pulse magazine, the next league table to grace these shores may well reveal GP prescribing of contraceptives. The driver for this is said to be concern over rising teenage pregnancy rates (increasing 2% year-on-year) which ministers believe ‘suggest that young people are not accessing effective contraception’. And what could be more useful in flagrante delicto than another league table? It would certainly give them something else to think about, which can only be helpful.
I particularly enjoyed the report of the much-vaunted pilot scheme recently undertaken in Southwark, Lewisham and Lambeth PCTs, in which contraception was made available to teenagers over-the-counter. Ministers welcomed the scheme as a flagship of innovation under its £10m awareness initiative. Might I suggest that some of this money be used to purchase large quantities of industrial strength duct tape, which can then be used to attach the lads to the counter?
Ideally, by a certain part of their anatomy. Then watch those teenage pregnancies plummet.
