I’m just going to come right out and say it: I don’t believe that choosing a North Star Metric is the best option for 99% of social sector organizations.

From campaigns seeking “zero people unhoused” to education initiatives dedicated to getting students ready for high school, we’ve been overwhelmed by celebrated changemakers reducing complex issues to one simple metric and goal.

So, what’s a community who cares about making progress on the toughest issues to do?

The unfortunate reality is that most communities replicate what sounds good.

They start a big initiative, get lots of people involved, and then work tirelessly to get everyone to agree on one metric (often called a “North Star”) to serve as the supreme measurement tool and goal. 

Now everyone’s laser-focused. Just improve the numbers. It’s all so simple.

But here’s what usually happens next (with homelessness and high school readiness examples):

• The metric is translated into an action that can be more easily measured.
  • The number of people experiencing homelessness, which is a notoriously difficult number to track, is converted into the number of people placed into supportive housing.
  • High school readiness, a broad concept, is shifted to percent of students achieving a certain score on the statewide 8th grade math test.
• The responsibility for the new action metric is delegated.
  • Although many are involved in providing services to the homeless, nonprofits who provide supportive housing are targeted to increase the number of clients placed into supportive housing.
  • Schools target 8th grade math teachers and encourage them to “teach to the test” to increase scores.
• The metric is attained but the problem doesn’t get any better.
  • Supportive housing placements reach an all time high but the number of people experiencing homelessness continues to grow.
  • 8th grade math scores increase, but subsequent 9th grade GPAs trend downward.

When failure happens, most communities redouble their efforts (“more supportive housing!”; “after-school math tutoring!”), try to find a new all-encompassing metric, or just give up. 

It’s really disappointing.

To avoid making the same mistakes, I’m going to explain why metric-based failure happens, and what you should do instead.

How to avoid being bitten by a cobra

A short anecdote illustrates the danger.

During British rule of India, officials became concerned about the number of venomous cobras in Delhi. 

To address the issue, the government set a goal of reducing the cobra population. It offered a bounty for every dead cobra. The policy was initially successful and the snake population declined precipitously. 

However, enterprising Indians started breeding cobras to collect the bounty. Realizing their mistake, government officials halted payments. Because the snakes were now worthless, cobra breeders released the snakes into the wild and the population exploded.

The idea behind the cobra effect was later applied to social science by Donald Campbell, whose  research found that:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Some refer to this general idea as Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.”

Don’t confuse the metric with the goal

Metrics are super useful as a management tool. 

They help quantify effectiveness, allow comparison of results across a range of different interventions, and measure change over time.

The problem comes when people forget the larger strategy and start to think of one or more metrics as goals in and of themselves. This common confusion between goals and metrics is called surrogation.

For example, rather than seeing “number placed in supportive housing” as a proxy for ending homelessness, people start to focus exclusively on the metric as the goal. They forget that homelessness is a complex issue that can’t be reduced to one metric.

Or, schools start to focus on increasing 8th grade math scores as the end goal rather than seeing it as just one representation of high school readiness. They forget that student success comes from a variety of academic, psychological and social factors.

Once you understand surrogation, you can take a better approach.

The future is a holistic approach that incorporates big picture strategy and a range of metrics

A truism of modern organizations is “what gets measured gets managed.” 

But what’s quantitatively measured – like a North Star metric – doesn’t exactly capture the comprehensive goal. It just represents a part of what we care about.

We should manage these key indicators rigorously, but we have to simultaneously keep other other aspects of the problem and the big picture in mind.

It’s not quite as simple as complete dedication to a North Star metric, but it’s much more effective.

Here are four ways to start doing it:

1. Involve people who will implement the strategy in its formulation. Front line workers know that housing placements are just one of many important homelessness metrics. Math teachers know that everything can’t be reduced to one test score. Getting their input early and often can help avoid top-down tunnel vision. 
2. Use multiple metrics. Problems are multifaceted, so you have to have goals that are similarly holistic and broad. No one measure can capture all that’s important.
3. Play devil’s advocate to broaden beyond one metric. Assign one or more people involved to make the case against any North Star metric. This can help ensure that critical thinking is a part of the measurement discussion. It can also help avoid groupthink.
4. Try to hypothetically game it. Take time to explore how any one measure could lead to unintended consequences, or even be gamed (I’m looking at you would-be cobra farmers). It’s a fun exercise to imagine how stakeholders could use metrics to their own advantage. Note that every additional metric you use makes gaming the system more difficult.

See you again next week.

==

Whenever you’re ready, there are two ways I can help you:

→ I’m a strategic advisor for the toughest societal problems like poverty, crime and homelessness. People come to me when they want to stop spinning their wheels and get transformative, systems-level change.

→ I’m a coach for emerging and executive leaders in the social and public sectors who want to make progress on their biggest goals and challenges.

Let’s find out how I can help you become transformational.