All Tech Considered
1:57 pm
Mon May 14, 2012

Algorithms: The Ever-Growing, All-Knowing Way Of The Future

Originally published on Fri August 17, 2012 3:24 pm

My favorite movie, Days of Heaven, is at the top of my recommendations list on Netflix. But I've never actually watched it on Netflix, so how did they know I like it?

"Every time a Netflix member streams a title from us, we learn a little bit more about what's interesting to them," says John Ciancutti, vice president of engineering at Netflix. Netflix has algorithms that mix customer information with other algorithms that group shows together. So if I enjoy Mad Men, it guesses that I might like other dramas with complicated male leads, like Breaking Bad — and it's right.

If the Industrial Revolution was about extending the power of human muscle with inventions like the car, then the computer revolution is about extending the power of the human mind — and algorithms are the key to its success. These formulas find search results, pick the top story on the news feed of your Facebook page, and determine things like your credit score and trades on Wall Street.

Every day Netflix has dozens of engineers improving its algorithms. A huge whiteboard in the hallway of Netflix headquarters displays numbers in a grid, part of a contest to see who can come up with the best algorithms.

"Just in the last couple of months, we've run tests where we've improved overall streaming hours for members with a new algorithm that's just a little bit better at making recommendations, which means it's so powerful as far as delighting members that they are more likely to stay with the service versus not," Ciancutti says.

Political Potential

For Netflix, it's about keeping customers. But algorithms tailored to figure out individual tastes and interests are also being applied to the political arena. When Mitt Romney is on local TV in Ohio, it's no surprise that he'd talk about local interests, like manufacturing jobs. But it would be even better if he could target his message directly to people who lost those jobs.

In this coming election, that's exactly what Democrats and Republicans will be able to do.

"There's been this explosion of data available over the last decade, frankly, at the individual level from voter files, from consumer sources, from other sources," says Tom Bonier, co-founder of Clarity Campaign Labs, a company that uses algorithms to help Democrats target voters. "You can send one piece of mail about choice to one household, and to their neighbors you can send a piece of mail about the environment."

In other words, there are more computers hoarding more data about us than ever before.

Algorithm As Crystal Ball

Sean Gourley is co-founder and CTO of Quid, a company that gets hired by governments and business to create algorithms. He says, "From an algorithms perspective, this is a great time to be alive. Algorithms are just frolicking in the mountains of data that they can play with."

Gourley uses algorithms to predict insurgencies in Iraq and Afghanistan and help banks map developing markets for new technologies. As an experiment for NPR, he maps the development of the Occupy Wall Street movement. He uses algorithms to sort through about 40,000 blogs and articles written since the movement began and group similar ideas together.

"You can't read all this as a human or even necessarily get what's going on, so you start to apply algorithms to help kind of cluster, sort, put topics around them and ultimately visualize [it]," he says. What Gourley's algorithm helps visualize is how ideas from the initial Occupy Wall Street rallies in New York spread to other groups and other parts of the country.

Gourley points to a computer screen. It's filled with dots that are grouped into color clusters and connected by thin lines. One cluster represents politicians talking about taxing the wealthy. It's far from the cluster that represents protesters.

"When the politicians talk about Occupy Wall Street, they're not really talking in Occupy Wall Street," Gourley says. "They're not talking within the cluster — they're talking separately. And so when they talk about taxes and they talk about inequality it's not really resonating [in the Occupy Wall Street cluster], or at least the language is quite different."

Gourley clicks on another smallish cluster that represents "Bank Transfer Day," an activist initiative that calls for people to move their accounts out of larger banks and into credit unions — you can see that people aren't saying much about it anymore. A more recent cluster represents the issue of the courts using Twitter feeds as evidence in cases against protesters, and yet another tracks the conversation around Occupy Oakland, which is dominated by talk of police violence.

Proceed With Caution

Gourley imagines that information like this might be useful to politicians, police and political activists.

"We can start to think about where it's come from," he says. "We can start to think about how it's evolved and we can start to think about where we want it to go and if we want to change the direction."

But Gourley has concerns about other possible uses of algorithms. Netflix is just trying to keep its customers watching movies, which isn't so bad, but he wonders what would happen if Google Maps knew that you were looking for a new car. Maybe when you looked up directions to a party, it would suggest a route that passes right past a dealership.

"[Has] it got your interest at heart or has it got making money from ads at heart?" Gourley says.

What worries him most is that we humans haven't yet evolved to be as wary of algorithms as we are of used car salesmen.

Copyright 2013 NPR. To see more, visit http://www.npr.org/.

Transcript

ROBERT SIEGEL, HOST:

It's ALL THINGS CONSIDERED from NPR News. I'm Robert Siegel.

MELISSA BLOCK, HOST:

And I'm Melissa Block.

The industrial revolution extended the power of human muscle with the invention of machines, such as the steam engine. The computer revolution is extending the power of the human mind. And one key to that revolution is algorithms. They find search results, pick the top story on your Facebook newsfeed, determine your credit score.

And as NPR's Laura Sydell reports, algorithms are used to predict how ideas travel through society.

(SOUNDBITE OF MUSIC)

LAURA SYDELL, BYLINE: My favorite movie is "Days of Heaven" and it's at the top of my recommendations on Netflix. But I've never watched it on Netflix, so how do they know that?

JOHN CIANCUTTI: A Netflix member streams the title from us and we learn a little bit more about what's interesting to them.

SYDELL: John Ciancutti is the vice president of engineering at Netflix. Their algorithms mix customer information with algorithms the group chose together. So, if I enjoy "Mad Men," I might like other dramas with complicated male leads.

(SOUNDBITE OF TV SERIES, "BREAKING BAD")

BRYAN CRANSTON: (as Walter White) My name is Walter White. There are going to be some things that you'll come to learn about me.

SYDELL: Netflix says I like "Breaking Bad" and they got that right. Every day, Netflix has dozens of engineers improving its algorithms. A huge white board in the hallway of Netflix headquarters has numbers on a grid. It's a contest over who can come up with the best algorithms.

CIANCUTTI: Just in the last couple of months, we've run tests, where we've improved overall streaming hours for members with a new algorithm that's just a little bit better at making recommendations. Which means it's so powerful as far as delighting members that they're more likely to stay with the service versus not.

SYDELL: For Netflix, it's about keeping their customers. But algorithms tailored to figure out individual tastes and interests are now being applied to the political arena.

MITT ROMNEY: Well, I happen to believe that going forward over the coming decade, you're going to see a lot of manufacturing come back to America.

SYDELL: When Mitt Romney is on local TV in Ohio, it's no surprise he's talking about local interests like manufacturing jobs. But it would be even better if he could target the message directly to people who lost their jobs. In this coming election, both the Democrats and the Republicans will be able to do that.

TOM BONIER: And there's been this explosion of data available over the last decade, frankly, at the individual level, from voter files, from consumer sources, from other sources.

SYDELL: Tom Bonier works with Clarity Labs, a company that uses algorithms to help Democrats take that data and target voters.

BONIER: You can send one piece of mail about choice to one household. And to their neighbors, you can send a piece of mail about the environment.

SYDELL: More computers and hoarding more data about us than ever before.

SEAN GOURLEY: So from an algorithm's perspective, this is a great time to be alive.

(SOUNDBITE OF LAUGHTER)

GOURLEY: You know, algorithms are just frolicking in the mountains of data that they can play with.

SYDELL: This is Sean Gourley, the co-founder and CTO of Quid. His company is hired by governments and businesses to create algorithms. Gourley has developed algorithms to map and predict insurgencies in Iraq and Afghanistan. His company helps banks map developing markets for new technologies.

As an experiment for NPR about Occupy Wall Street, Gourley sorted through 40,000 blogs and articles written since it began and grouped similar ideas together.

GOURLEY: You can't read all this as a human or even necessarily get what's going on, so you start to apply algorithms to help kind of cluster, sort, put topics around them and ultimately visualize.

SYDELL: What Gourley's algorithms help visualize is how ideas that germinated at the initial Occupy Wall Street rallies in New York spread to other groups and parts of the country. Gourley opens up a computer screen filled with dots grouped into color clusters that look like different solar systems with thin lines that connect them. One cluster represents politicians talking about taxing the wealthy. It's far away from the protesters.

GOURLEY: When the participants talk about Occupy Wall Street, they're not really talking in Occupy Wall Street. They're not talking within the cluster. They're talking separately. And so, when they talk about taxes and they talk about inequality, it's not really resonating up here or at least the language is quite different.

SYDELL: Gourley clicks on another smallish cluster. It's bank transfer day. You can see that people aren't talking about that so much anymore. A recent cluster has the issue of the courts using Twitter feeds as evidence in cases against protesters. He clicks another cluster of ideas. This one has to do with Occupy Oakland, which is dominated by talk of police violence.

GOURLEY: We can see a conversation with our eyes.

SYDELL: Gourley imagines that information like this might be useful to politicians, police and political activists.

GOURLEY: And we can start to think about where it's come from. We can start to think about how it's evolved, and we can start to think about where we want it to go and if we want to change the direction.

SYDELL: But Gourley has concerns about other possible uses of algorithms. Netflix is just trying to keep its customers watching movies. That's not so bad, but he wonders what would happen if Google Maps knew that you were looking for a new car. And when you were driving to a party, it suggested a route that went right past a dealership.

GOURLEY: Has it got your interests at heart or has it got making money from ads at heart?

SYDELL: What worries Gourley the most is that we humans haven't yet evolved to be as wary of algorithms as we are of used car salesmen.

Laura Sydell, NPR News, San Francisco. Transcript provided by NPR, Copyright NPR.