3.07.2011

Framing Errors

There has been some good reporting recently on a particular form of cognitive bias called framing errors. (This should be distinguished from the way that computer programmers use the term. Here, we won’t be talking about the computer science problem of where a sequence of data is read starting at an incorrect point.) In our world, framing errors refer to differing decisions being reached based upon how data is presented to the user. Framing errors are a huge problem in medical informed consent and are heavily leveraged by litigators and juries. In fact, most legal proceedings involve the purposeful creation of framing errors on behalf of a particular client or viewpoint. Here's the scary part: almost everything you read about medicine in the popular media is rife with framing errors.

Let’s take a simple example: If an expert in epidemiology reported that exposure to substance ‘x’ over a ten year period would increase your risk of developing nose cancer by 33%, would you be in favor of draconian measures to prevent exposure to ‘x’? The answer should be, ‘depends upon the baseline risk of nose cancer within the population’, but that surprisingly is not how most people respond. If the baseline risk of nose cancer is 0.1% at ten years then the adjusted risk is actually around 0.133%. That seems a little less dire than a 33% increase. Said another way, after 10 years of exposure to ‘x’ there would be 4 cases of nose cancer per 3000 persons instead of 3 cases. The question of what one might be willing to tolerate for that reduction is very different from what one might consider to prevent a 33% increase without context. The moral of course is that all three ways of presenting this data are correct. But one gives a very biased perspective. Adjusted risks and incidence rates within a population are much better measures.

Let’s look at another example. Late night cable is awash with ambulance chasing liability attorney ads. A recent one is a great example. Some ‘non-attorney spokes person’ was bemoaning that a particularly advanced form of knee replacement prosthesis was associated with an 8% rate of failure. My golly - an 8% risk of failure? Sounds kind of high, and it may be; I’m no orthopod. But keep in mind that this is a knee replacement. You are replacing a worn out version of one of the most complex joints in the body. One subjected to huge stresses and of course being installed in less than stellar bone by definition. I’m actually surprised the failure rate isn’t higher to be honest. The litigator’s implication is that the failure rate is an example of poor design or malpractice during insertion. The assumption is that if some miraculous procedure unavailable to the last generation is less that 100% effective, it’s grounds for a lawsuit. Of course some of these same people would turn around in a lawsuit against big tobacco and claim that you can’t prove causation between cigarette smoke and a particular cancer.

As a doc my first response is that an 8% failure rate for a procedure that in essence is ‘polishing a ----’, ain’t that bad if a lot of people are walking as a result. The question should be how does this rate compare with other methods and what are the adjusted risks for the procedure. But that’s some of that pesky science that people seem to hate so much.

Framing entered into the pseudoscience surrounding vaccines as well. Enemies of vaccination claimed (a claim that has been subsequently completely debunked) that autism resulted from vaccinations at a rate of around 1 - 2 per 100,000 vaccinations. No one denies the tragedy of autism however, even considering the fact that the allegations were completely false, it would not have been clear cut even if they had been true. How can you say that Pliny!? Simple, the incidence of death and complications from the diseases that are covered by these vaccines is far higher than 1-2/100,000 in unprotected populations. We've forgotten that because vaccinations have been so effective and we haven't seen that kind of wild fire in a few generations. We are even now in the throws of an early outbreak of measles here in the NW - the US capital of Woo. It’ll be interesting to see how it spreads since out herd immunity is reduced around here by a lot of granola.

Another way people get exposed to framing confusion is through the use of confusing terms like cure, disease free interval, remission, survival, and quality of life. These terms show up a lot in oncology discussions and obviously mean very different things. Survival means exactly that. Is the person likely to be alive. Says nothing as to the status of the individual or whether the cancer is being kept at bay temporarily. Cure and disease free survival are more critical in making decisions. And these two are very different. For some specific kinds of malignancy, cure isn’t even a valid term. For others a certain disease free survival equates with cure.

Sometimes a patient is convinced to embark on a particular regiment for therapy by the promise of a extension of their survival. Almost never is the discussion had as to what that survival entails. Many think it’s 6 to 12 months of normal existence. It usually is not and a thorough understanding of the reality of those last weeks might change some minds.

Docs fall for this too. People talk about defensive medicine to avoid litigation but the risk is actually not nearly as high as most presume. Most of the time the risk can be minimized by good communication and documentation that addresses framing issues. I have this kind of discussion with parents of teen trauma victims all the time - the risk of occult injury with good examination vs the lifetime risk of radiation exposure from CT scans that very frequently add nothing to our management strategy even when a small positive finding is discovered. Too often my profession adopts some half backed policy to address a rare outlier rather than trying to educate the public or communicate with the patients.

With the application of evidence-based medical methods some of this should get resolved. These methods tend to be more directed at what serves a population most effectively. That means, however, that less emphasis will naturally be applied to the desires of one specific individual as it is today. But allowing unrestricted pursuit of any and all options no matter how marginally effective they are (backed by courts that encourage this notion) is a big reason why we are spending 2 trillion dollars each year and still only ranking in the 30’s or 40’s amongst nations with regards to our health and wellness.

So, next time you see some medical data, be careful to think about what it really means based upon the adjusted risk.

11 comments:

Michael Lockridge said...

Wisdom is rare. Bullshit is not.

In the Wild West there were a lot fewer attorneys, and a lot more large caliber hand-guns. The question is, have we actually moved forward?

How would you change the course? Apparently the status quo is beneficial enough to those with power that a change will probably be challenged. How, then, to bring about change?

Mining wisdom from bullshit is time consuming, and quite messy and smelly.

Pliny-the-in-Between said...

Mike we are moving the goal post a bit every day. The projects I have been involved with use computer systems designed to bowl over cognitive biases like framing. As we are using large numbers of non-MD's in these primary care roles we can better control these types of problems. The computers have no agenda except factual assessment. They also collect vast stores of experiential data to support their points. This works well in the Blue states at least ;)

mac said...

As I read this, I am reminded of a Mark Twain quote (that he, in turn, borrowed from Disraeli), "There are three kinds of lies: lies, damned lies and statistics."

It comes as no surprise, given our litigious society, that unscrupulus people take an acceptable failure rate and use it against medical practioners.
I wonder if suing lawyers for legal malpractice due to frivolous lawsuits would work ?

pboyfloyd said...

Good stuff as usual Pliny.

Just wondering how many people know 'in their gut' that this is true, but will go for the gold and try to punish those they wish were responsible for their loved-ones suffering? Even if that suffering was for all intents and purposes, self inflicted.

Pliny-the-in-Between said...

pboy - good question and no doubt that does happen. But what's most interesting is that with good communication and empathy with patients malpractice is less of a problem even in cases where actual mistakes were made. I guess it comes down to the universal level - if you show people that you give a hoot and take time to talk with them and get to know them, it becomes harder for them to think of you as the enemy.

Harvey said...

Couldn't agree more! For nigh onto forty years, I have been trying to teach my residents and colleagues that their best defense against malpractise suits is not insurance, but assurance (to the patient and his/her family) that you care about their well-being and that you are giving your very best effort to achieve it for them. People tend not to sue people that they have come to like and respect.

GearHedEd said...

Damn! Is nose cancer on the increase? Does the level of nose-picking have any effect? Is the size of the nose important?

AAARGH!

So many questions!

Michael Lockridge said...

I don't know how often it happens, but I have seen a lot of court cases that constituted law suits essentially to determine which insurance company will pay, and how much.

This is the use of the public court system to essentially settle a business issue. Seems wrong to me, and just encourages gold digging litigation.

Jared said...

Might I recommend the "DHMO" website as a source of fun statistics, intentional 'framing errors,' and biting sarcasm?

http://www.dhmo.org/

Pliny-the-in-Between said...

Jared, I like that site as well. I wonder how many people really miss the joke.

GearHedEd said...

I've heard this one before...

DHMO = dihydrogen monoxide
= water.

Haven't been to the site yet...