Christian Rudder: ‘People have come to grips with the fact that Google is going to show you an ad based on something you searched for.’CHESTER HIGGINS JR/The New York Times
For years, part of Christian Rudder's job, as the co-founder of OkCupid, was to mine data from the online dating site's 12 million users.
His reports would inform the OkTrends blog he wrote, detailing patterns in the messaging, flirting and profiles on the site – mapping the data, if you will, behind human desire.
With his new book, Dataclysm: Who We Are (When We Think No One's Looking), he's branched out beyond the world of matchmaking, arguing that big data can reflect our own behaviour back at us, and help us see ourselves more clearly.
He talked to The Globe about the power of big data and why people need to exercise caution when wielding such an extraordinary tool.
Is the collection of massive data samples going to replace other ways of understanding human behaviour?
I'm not trying to replace anthropological or sociological behaviour science. I see big data as a supplement to those things. But this is definitely a new level: Previously, research involved a group of college kids, or whoever wanted to do a survey on the phone.
Now data sampling is much broader; we have an unprecedented vision of real life. Half the United States checks Facebook first thing every morning, for example. So that's an extremely meaningful sample.
But there are ways that the data we cull online are skewed, right?
Sure. For example, you're much less likely to hit the "Like" button on Facebook if you think you'll look foolish to all your friends.
And that "transparency" could bias things toward a sort of groupthink?
Right, but on a site like OkCupid, where your messages and your "likes" aren't public, there's really no reason to try and game the system. It's bad for you to lie on a dating site. So there's inherent honesty with that data.
You note in your book that big data can reveal truths we don't fess up to. For example, we say we're race-neutral when it comes to dating, but your research shows that black users, as well as Asian men, are consistently ranked lower on OkCupid's desirability scale.
Across the board – on all dating sites – the conversation around race and the actual actions of users are two different things. It's like when the black mayor of L.A., Tom Bradley, ran for Senate and people said they'd vote for him, but then changed their minds at the ballot box.
Do you think that algorithms, acting on big data, can end up doing more than just reflecting our behaviour? Might they be changing our behaviour, too?
It's indisputable. For instance, Amazon's recommendations simulate serendipity without achieving it. (Personally, I prefer to shop in bookstores.) I'd say that these filters, these algorithms, are absolutely homogenizing by design. These algorithms are essentially advice, remember. And we're getting all this advice from the same source.
So does that lead to an increasingly manipulated citizenry, where a few Silicon Valley titans – or their black-box algorithms – dictate behaviour?
I think people develop personal algorithms to counter the Silicon Valley algorithms, actually. They say: "You're going to reduce me to a number? Okay, I'm going to exploit your system right back." It's almost like an arms race. I guess I'm not pessimistic about it; I think there will be an equilibrium.
You got in a bit of hot water recently when you announced on your blog that OkCupid experiments on its users by changing the algorithm that matches them up. Why do you think people got so upset?
Yeah, that definitely rubbed people the wrong way. … It's a simple truth about how websites work but people haven't internalized it yet. People have come to grips with the fact that Google is going to show you an ad based on something you searched for, but not this … So listen, anytime you're being recommended something – a book, or a potential mate – it's the opinions of some guys who wrote the software that are putting that book or whatever in front of you. There's no solved way to recommend what people will like, though, so we're going to keep experimenting.
Maybe the experiments, and all those algorithmic decisions, upset people because they think of the Internet as a democratic, liberating space, where things are more neutral.
Exactly – I can definitely see why people find it upsetting; it doesn't fit the narrative they've been told. The way Silicon Valley talks about itself – there's this overconfidence almost to the point of condescension: this idea that "we've figured out what you like."
Do you feel the public is right to be so skeptical of the data collection done by corporations like yours?
Sites like ours, actually, are just trying to figure out what people are into. Our questions are asked in aggregate. Nobody at OkCupid or Facebook or Twitter cares about any individual person. Nobody is looking at their 12 million users and wondering what a single person is doing. Whereas governments are. Governments aren't asking: "What are guys into?" They're trying to decide if this one guy is dangerous. But I think when it comes to big data and corporations, people's fears aren't founded.
But the government uses Facebook data.
I guarantee you people at Facebook aren't happy about that.
This interview has been condensed and edited.