Keeping Score

“If you count something interesting, you will learn something interesting”

– Atul Gawande

Last year I read two excellent books by Atul Gawande:

I recommend them both.

One idea that has stuck with me from these is the power of simple tests or checklists.

An example he gives in “Better” is the Apgar Score, created by Dr Virginia Apgar in the early 1950s as a quick and easy way to assess the health or otherwise of a new born baby.

It is almost certainly the first test you passed in your life!

Prior to this many babies which were the wrong size (too small) the wrong colour (blue) or not breathing, perhaps as a result of a difficult birth, would die because it was assumed that nothing could be done to save them.  At the time, remember, over 3% of babies died during child birth.

The concept of the test was more or less dismissed when she first suggested it because it was thought to be too simplistic to be useful, but today is used all around the world, and has had a massive impact.  

Here is a quote from the book:

“The score turned an intangible and impressionistic clinical concept – the condition of new babies – into numbers that people could collect and compare. Using it required more careful observation and documentation of the true condition of every baby. Moreover, even if only because doctors are competitive, it drove them to want to produce better scores – and therefore better outcomes – for the newborns they delivered.”

He was recently interviewed by Charlie Rose:

http://www.charlierose.com/view/interview/10191

Here is an extract from that interview, where he talks about the same idea, this time in the context of general surgery: specifically a checklist used by surgeons at key points during a procedure, and the massive difference it has had to the outcomes where it has been used:

“We introduced a 90-second checklist that happens right before [the patient goes] to sleep, and then right before an incision, and right before [the patient] leaves the room, kind of like before takeoff, after takeoff and on landing.”

“It checks 19 things that if we missed them, we will harm you.  But it turns out we miss one of those steps about two-thirds of the time, and using the checklist in every hospital we put it in – we put it in eight hospitals that range from rural Tanzania to Seattle, Washington — and each hospital it went into reduced complications by double digits, on average by about a third.”

Something that was easy and made such a big impact would be popular, right?  Actually no…

“Q:  And how is the medical profession responding to this?”

“A: Well, there are two kinds of reactions.  We surveyed the teams afterwards, asked them what did you think.  And about 80 percent said, you know, in the end, it wasn’t as bad as I thought.  It was swift.  I thought it improved care.  But there was a solid 20 percent who said, this is a waste of my time.  I don’t think it improves anything.  But then we asked them one more thing.  We said, if you were having surgery, would you want the checklist?  And 94 percent said they would want it.”

It makes me wonder what other things we dismiss as being too simple to be useful.  Or, what things we oppose for ourselves but which we would think others should do.  “Do as I say not as I do” perhaps, or is it “I’m an expert, so I don’t need it”?

I’m sure this same idea could be applied to a wide range of different things.

For example, imagine a simple test that you could do at the end of each day which measures the impact of your actions on your own health:

  1. Did you eat well?
  2. Did you exercise?
  3. Did you get enough sleep?
  4. Did you spend time with family and friends?

How would you score?  

Would you want to know?

Maybe you would even share your results with others, in order to be able to compare?

And then, having created a false sense of competition, wouldn’t you be incented to do better?

Clearly keeping score can be pretty powerful, as long as you choose the right score to keep.

6 thoughts on “Keeping Score”

  1. This goes back to Peter Drucker too: “If you don’t measure it, you can’t manage it”.

    Although the addition of sharing your score and then competing with your peers is going to add a whole new level of motivation!

    Some things I’ve been measuring in order to manage them:

    1) my CO2 output (see my blog)

    2) my weight, every day – inspired by the Hacker’s Diet (http://www.fourmilab.ch/hackdiet/www/hackdiet.html)

    But while I do share my CO2 output, I don’t think I’ll be sharing my weight any time soon…

    ~ Carolyn

  2. Obviously, where safety of life is a major concern – Aviation, checklists have exisited for as long as there have been aircraft to fly. Properly used, they make flying a whole lot safer. They are there for 50-hour pilots and for 20,000-hour pilots. And they work.

  3. Rowan,
    In case you don’t remember me, I’m one of Phil Cockfield’s friends in SoCal – we met at ETech last year. I read “Better” last year and absolutely loved it. Great synopsis and I agree about checklists. My daughter has cystic fibrosis and I found his chapter investigating the amazing differences between average and excellent doctors and clinics. It turns out that most of us, by definition, are in the middle of the bell curve (“average” to use the word in it’s non-mathematical sense). The best doctors can be 10 or 20 times more effective than the average ones and it can be because of the disciplined application of things as seemingly mundane as checklists. This can apply to many disciplines in life.

    Thanks for the great post!

  4. @Carolyn

    Tracking weight is a tough one.

    In the past when I’ve tracked this for myself I’ve seen it fluctuate a lot and not necessarily correlated to things that I really care about.

    For example:

    I weighed myself at the gym a few times (at the same time of day, etc) and found the result moved as much as +/- 2kgs depending on how many times I filled my drink bottle while I was there
    Also, both times I’ve been training for a big event (see: 16km of hose, 5km of truth) I’ve seen my weight increase despite being in the best shape of my life at the time

    Another book I’ve recently read suggested using waist measurement as a better indicator.
    See: Think Before You Swallow by Noel O’Hare

    My brother recommends using the Look in the mirror test, which is also probably a more accurate (albeit subjective) measure. Although there are flaws with that approach too.

    And I know what you mean about keeping this sort of thing to yourself. Although, as long as you can get over the initial shock, being a bit public with this sort of data is probably a very effective way to ensure you stay disciplined! Perhaps something you’d just share with a few people? Still not sure I’d be brave enough to do that though. :-)

  5. @Bill: you’re right – and as a 5-hour student pilot, learning the checklists for startup and takeoff, I should have thought of that myself!

    @Rowan: I agree, daily weight fluctuations are the biggest problem in using weight as a measure.

    The classic chick-on-a-diet book is Bridget Jones’ Diary; Bridget weighs herself every day, and a one-pound gain sends her into a panic. I must admit, I’ve been there myself.

    The Hacker’s Diet (mentioned in my first post) directly addresses this problem by employing some mathematics: a simple weighted average, to smooth the daily/hourly fluctuations into a smooth curve – trending up or trending down.

    I have an Excel graph of the trend and daily data points, and I’ve used it for seven years now (!). Every time I think I’ve got it nailed and stop using the spreadsheet, my weight trends up again :-).

    For what it’s worth, I also track waist measurement daily – and for the whole seven years, during which I have been completely unfit, got fit and run a marathon, and got injured and relapsed into unfitness, waist measurement has always been pretty well correlated with my weight. So for me at least, they’re both good metrics for what I’m really interested in.

    One more thing has just occurred to me: what are all those group-weight-loss organisations, but a way of sharing your metrics and competing to improve them?

    ~ Carolyn

    1. Using a weighted average (no pun intended, I’m sure) makes so much
      sense.

      I’ve used this in a number of other contexts where measurements
      fluctuate – e.g. when tracking visits to a website over a week or
      month – but it just hadn’t occurred to me to use it in this case.

Comments are closed.