THE EFFECTIVE PROBLEMSOLVER #089
When I first started working in public policy related to unemployment and job training, I made a mistake.
In my enthusiasm to push for change, I latched onto data from a small-scale training program that showed promising results.
I selectively used it to promote a big policy shift, convinced that if it worked for 20 people in one setting, it would work for 20,000 across a state.
It didn’t.
The results were underwhelming, and I realized later that I’d overlooked the very factors that made the program succeed on a small scale but fail at a larger one.
I had been disingenuous—not out of malice but out of a desire for something simple to be true.
I’ve since learned we can and must do better.
The evidence from Housing First
This brings me to Housing First, a popular approach to homelessness that often gets described as “evidence-based.”
At its core, Housing First is simple: provide people with stable housing first, and then address other issues like mental health, employment, or substance use.
On an individual level, Housing First has helped many people find housing stability.
But here’s the catch: most of the evidence supporting Housing First focuses on housing stability, not reducing homelessness at the community level.
If you put someone in a subsidized house, they are no longer unsheltered—they have a roof over their head.
But does that reduce homelessness across a community?
Does it address the causal factors, like mental health crises, drug abuse, or housing affordability?
The evidence here is murkier.
In cities like San Francisco and Los Angeles, which have heavily invested in Housing First, homelessness continues to rise.
Why?
Because, among a variety of reasons, what works for individuals in controlled, small-scale studies doesn’t automatically scale up.
When interventions become large-scale policies, new feedback loops emerge, and incentives can change in unexpected ways.
Programs designed to be nimble and tailored to individual needs can morph into rigid systems that don’t account for the complexities of entire communities. The transformation from programmatic intervention to policy is not seamless—it’s a scale shift that introduces new challenges and consequences.
Moreover, claiming something is “evidence-based” because it helped a small number of people, somewhere, sometime, once, can be dangerously misleading.
Using evidence is a continuous process – not a one-time data point
Evidence-based policy requires clear-eyed scrutiny: What evidence? From where? For whom? At what cost? And what are the unintended consequences when applied at scale?
This isn’t an argument against Housing First or other innovative approaches.
It’s a plea to stay open—to let the data speak honestly, to keep asking hard questions, and to acknowledge when evidence doesn’t translate.
We need to look critically at both individual and community outcomes and be willing to adjust our strategies when the results aren’t what we hoped for.
My experience taught me that clinging to a single solution—even one backed by well-intentioned data—is a mistake.
Tackling homelessness requires humility, curiosity, and a willingness to adapt.
Let’s be rigorous about what “evidence-based” really means and commit to learning from what works—and what doesn’t—at every level.
We owe it to the people we’re trying to help to do better.
See you in two weeks.
