Management, the startup way
Or: when what gets measured does not get managed
There is a widely known proverb saying “What gets measured gets managed” , generally attributed to Lord Kelvin, and then co-opted by legions of management consultants.
It makes a lot of intuitive sense. When you can turn something — anything — into a number, you can track its progression and you can assess your impact on it. In a store, the sales figure is a pretty good indicator of the health of the enterprise. Do some advertising, reshuffle the shelves, hire or fire employees: whether and how sales vary is a precious result.
We’ve grown accustomed to seeing metrics evaluating just about anything: a country’s economic output, inequality or even happiness, the likelihood of humanity screwing itself over in a definitive way, or how well baseball players do whatever they are supposed to be doing.
These metrics are supposed to help us manage what they measure, and they often succeed in doing that. That is, until they don’t.
When the measurement starts misbehaving
British economist Charles Goodhart was the first to point out a flaw in this approach, initially stating in 1975 that “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” It was later phrased as:
“Economic metrics tend to misbehave when targeted”
That may sound complicated, but it is in fact very simple to grasp. For instance, unemployment is an adequate measurement of a country’s economic health. But what if you explicitly target it, for instance to hold your campaign promise to reduce it to 5%? That can be done easily: just create millions of public sector jobs. Will it bring unemployment to the target? Yes, it will. Will it improve the economic health of the country? Most probably not.
It’s the same for many other metrics. Want to raise GDP? Start big, debt-fuelled public projects. Raise inflation? Print money Do quantitative easing. These metrics provide a good measurement of the economy, but as soon as they are managed directly their value collapses.
Why it matters outside of economy
While Goodhart was focusing on macroeconomics, you can see his law anywhere you look. Take software development: the number of lines of code written per day is probably a decent indicator a developer’s historic performance. Give her that as a target? You will have a bunch of garbage code. Evaluate a consultant by number of slides created? I assume the fonts will get bigger.
Examples abound across disciplines. The Economist pointed out in a recent article, that:
Incentivise executives with an earnings per share target, for example, and it is relatively easy to run the business towards meeting that target rather than focus on things that create long-term value such as capex.
On a different note, we could also mention Jack Black’s character in Mars Attacks, who turns out to be useless in combat despite his prowess at assembling guns.
Goodhart’s law can — and should — be generalized:
“Any simple metric used to assess the performance of a complex, multivariate system, will misbehave if targeted.”
The two key words are multivariate and targeting.
If your system depends on a single variable, targeting it will work. At a checkout, you could perhaps use the number of items scanned by hour as a decent performance metric — and even then, the cashier will end up disincentivized to perform other valuable tasks such as helping a customer.
And “targeting” is the other key. These metrics may very well work as performance indicators, so long as you do not target them explicitly.
So what can you do?
You could try to identify all of the dimensions on which performance should be measured and come up with a compound indicator leaving no room for manipulation. Please send me a note if you have achieved that, I’d like to hear about it.
Or you could do what start-up founders do, which is to measure very little, if at all.
How startups do it
The coder striving to meet a target number of lines or the manager working towards an earnings per share figure are essentially gaming the system: they are taking shortcuts in order to reach an arbitrary goal, which may or may not be correlated with their organization’s interest.
Why doesn’t this happen in a start-up? It doesn’t because there is nothing to game. Imagine a scrappy 4-people start-up team in a garage. Even if this team decided to track the number of lines of code written by each one of the members, no one would be able to get away with shit code by saying “yeah but I did deliver my 1000 lines”, because at the end of the day their customers and investors won’t care, and they will end up out of a job.
Start-up founders have nothing to game because they have ownership of what they build. They will succeed if their product works, not if any arbitrary metric increases.
Give ownership
The start-up team is working to accomplish something clear in everyone’s mind but which escapes capture by a single number. No one is there to tap them on the back for advancing an arbitrary number. By having ownership of this goal, they can be expected to give it their best.
Problems arise when you introduce layers of middle management who expect numeric results to put on reports. This will disempower the workers, who will then game the metrics. The whole organization will become bloated and inefficient, all the while middle-managers will be patting themselves on the back looking at their dashboards, seeing a number go up while their company goes down.
So you have to make a decision: do you want a number to make you happy, or do you want to get stuff done? For the former, go ahead and track something that seems to correlate with success. I suggest: number of sheets printed, number of songs played on Spotify… number of faxes sent maybe?
For the latter: constitute an interdisciplinary team of aces. Give them an objective in terms of company or product success. Treat them like adults. And get back to work.
