#### Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Options

# Goodhart's law

• Options
1.
edited November 2010

Having read the entry, I know that in London paramedics on mountain bikes are usually the first to arrive. That is generally good, as they provide immediate help and can radio an assessment to the approaching ambulance. (Somewhat to my surprise ambulances in Singapore - and in Japan - do not push through traffic and wait patiently at red lights).

It's interesting that this is called a "law". Are you seeing it in terms of say, carbon trading, a "good thing", being circumvented, such that the numbers look good, but the real outcome has not been achieved?

Comment Source:Having read the entry, I know that in London paramedics on mountain bikes are usually the first to arrive. That is generally good, as they provide immediate help and can radio an assessment to the approaching ambulance. (Somewhat to my surprise ambulances in Singapore - and in Japan - do not push through traffic and wait patiently at red lights). It's interesting that this is called a "law". Are you seeing it in terms of say, carbon trading, a "good thing", being circumvented, such that the numbers look good, but the real outcome has not been achieved?
• Options
2.
edited November 2010

Yep, that's the basic idea. Even "circumvented" isn't quite right: the idea is that if you set people a target they'll focus on acheiving the target by any means necessary, not on "achieving the target via the means you want used". It's a caution not to depend to setting targets and then assuming things are working when they're achieved.

On ambulances, my view is probably influenced by just having read the "book of the blog" of a London ambulance man. Can't find the exact posts I read, but here's some:

It sounds like an example of Goodhart's law, where "in the wild" a good response time correlates with being a good ambulance service, but once you reward response time people change things so that they get a good response time, event though the fast responder very often doesn't have the resources to actually do anything useful.

If there's a better example, or way of explaining it, that's easier to see I'd be happy for it to change.

Comment Source:Yep, that's the basic idea. Even "circumvented" isn't quite right: the idea is that if you set people a target they'll focus on acheiving the target by any means necessary, not on "achieving the target via the means you want used". It's a caution not to depend to setting targets and then assuming things are working when they're achieved. On ambulances, my view is probably influenced by just having read the "book of the blog" of a London ambulance man. Can't find the exact posts I read, but here's some: * [post](http://randomreality.blogware.com/blog/_archives/2004/3/15/21076.html) * [post](http://randomreality.blogware.com/blog/_archives/2005/2/4/302996.html) * [post](http://randomreality.blogware.com/blog/_archives/2008/6/4/3729332.html) It sounds like an example of Goodhart's law, where "in the wild" a good response time correlates with being a good ambulance service, but once you reward response time people change things so that they get a good response time, event though the fast responder very often doesn't have the resources to actually do anything useful. If there's a better example, or way of explaining it, that's easier to see I'd be happy for it to change.
• Options
3.

One famous story, which could be an example of Goodhart's law, is how slave ships used to be paid by how many slaves they took on board rather than how many were still alive at the end of the journey. This had effects you can guess: slaves were packed in, and many died. But in this case, some smart people eventually noticed this effect and switched the system. Lo and behold, a miraculous improvement in the care of slaves occurred!

Comment Source:One famous story, which could be an example of Goodhart's law, is how slave ships used to be paid by how many slaves they took on board rather than how many were still alive at the end of the journey. This had effects you can guess: slaves were packed in, and many died. But in this case, some smart people eventually noticed this effect and switched the system. Lo and behold, a miraculous improvement in the care of slaves occurred!
• Options
4.
edited November 2010

I think I haven't been terribly good at explaining things. Another way of putting the extreme case is "if you just give someone a target -- even one that's currently well correlated with the one you want -- to acheive, don't be surprised if they come up with new ways of acheiving the target that you really wish they hadn't". There are some more examples

(If I was feeling facile, I'd say this is a similar to the uncertainty principle in the sense that "observing something changes it", but I think that obscures more than it clarifies.)

Comment Source:I think I haven't been terribly good at explaining things. Another way of putting the extreme case is "if you just give someone a target -- even one that's currently well correlated with the one you want -- to acheive, don't be surprised if they come up with new ways of acheiving the target that you really wish they hadn't". There are some more examples * [here](http://boingboing.net/2010/04/29/goodharts-law-once-y.html) (If I was feeling facile, I'd say this is a similar to the uncertainty principle in the sense that "observing something changes it", but I think that obscures more than it clarifies.)
• Options
5.

There are several examples that come to mind:

The British offered some coin for every cobra head in India, to get rid of these animals, Indians of course started to breed them.

Frensh resp. British offered some coin for every British resp. Frensh scalp to the natives, until they found out that the natives sold them French resp. Britih scalps, too...

When is software testing finished? At IBM at a certain time, the management estimated that there are on average x bugs per Y lines of code, and they decreed that developers should continue testing and bugfixing until they had found and fixed 80% of the accordingly estimated numbers of bugs. Developers, of course, implemented "bugs" on purpose in order to "find and fix" them during testing, in order to finish testing earlier. I don't know, but am pretty sure, that the management paid huge bonuses to the developers that followed this stragey - until they found out the truth...

Comment Source:There are several examples that come to mind: The British offered some coin for every cobra head in India, to get rid of these animals, Indians of course started to breed them. Frensh resp. British offered some coin for every British resp. Frensh scalp to the natives, until they found out that the natives sold them French resp. Britih scalps, too... When is software testing finished? At IBM at a certain time, the management estimated that there are on average x bugs per Y lines of code, and they decreed that developers should continue testing and bugfixing until they had found and fixed 80% of the accordingly estimated numbers of bugs. Developers, of course, implemented "bugs" on purpose in order to "find and fix" them during testing, in order to finish testing earlier. I don't know, but am pretty sure, that the management paid huge bonuses to the developers that followed this stragey - until they found out the truth...