The narrowness of the data you seek out, the numbers you’re able to gather, the numbers you omit, the third parties with whom you choose to share your information – any of this can skew the so-called science of quantification.
“If you’ve got bad data, you’ve got bad analysis,” says Andrew McIntosh, a software developer at Uptime Software. “People who track their dreams can only track the dreams they remember and write down in their tracker. Or people try to skew the game, and our data becomes only what we want ourselves to be. You wear your Fitbit on the days you’re going to exercise, but you don’t wear it on the days when you’re sitting at the office.”
The numbers should always be subordinate to a more introspective understanding of behaviour, Mr. McIntosh says. “The data doesn’t prove anything or give you any deep insights quantitatively. It just helps you look at yourself better – it gives you the direction you’re going, and then it’s up to you if you want to change.”
Ms. Chua agrees with this qualified approach to quantifying: Her apps simply help her make connections that she otherwise might have missed. Sometimes her starting-point for self-improvement is an odd pattern in her graphs, but just as often it’s vague intuition.
“Something funny’s going on. So then you start digging round in the data, and it brings different things together – you take a look at how much sleep you’re getting versus the sunset/sunrise pattern. You’re not necessarily holding yourself to strict scientific standards, but you can still say, I think it’s got something to do with this.”
Numbers for her are not an end in themselves but a starting-point for reflection. “It’s not so much about productivity, about squeezing as much as I can out of every second, and more about asking, what do I really value?”
A good analogy here is the contentious approach to player assessment in baseball, a sport that has undergone a data-driven revolution. And so I phone Keith Law, an MBA and former analyst for the Toronto Blue Jays who now covers baseball for ESPN.
Mr. Law is a statistical savant with a special talent for ranking young prospects. But with the proliferation of available numbers, a significant part of his expertise depends on sorting out the data that matter from the data that don’t.
Smart baseball analysts, for example, used to rank pitchers according to the batting-average they allowed for balls hit into play. “But that probably isn’t all that much in their control. Once the ball is hit into play, the pitcher’s role is over,” Mr. Law says.
“You have to understand what the data means,” he continues. He could be talking about baseball or cholesterol levels because in this over-quantified world, the wise approach has to be the same. “More data is better but you have to have the temperament and self-discipline to take all that data, put it in the appropriate context and understand how not to react to it.”
Danger in numbers
We’ve always counted the bits and pieces of our lives – how many Hail Marys will expiate our sins – but finding use for the immense data we now can gather makes this ability to discern much more important. Do food apps make us healthier, or eating-obsessed? Do heart-rate monitors motivate healthy exercise, or push us over the edge? Should we trust the selective data that carries the illusion of science simply becaue it’s numerical?
“Detailed monitoring and quantification can be harmful,” says Nav Persaud, a doctor and lecturer at the University of Toronto. “Even when we’re quantifying a purely subjective state like pain or mood, a number gives the impression of scientific rigour and thoroughness. It’s pretty hard to explain what a mood is – people could disagree about it. And yet once you’ve applied a number, it’s difficult to ignore.”
Research has shown that frequent pain readings will increase the amount of pain people feel. A routine stress test can actually cause more harm than good. A question about anxiety will make a person more anxious.