{"id":648761,"date":"2013-03-25T21:10:35","date_gmt":"2013-03-26T01:10:35","guid":{"rendered":"http:\/\/gigaom.com\/?p=622699"},"modified":"2013-03-25T21:10:35","modified_gmt":"2013-03-26T01:10:35","slug":"liking-curly-fries-might-not-mean-youre-smart-when-mere-data-isnt-enough","status":"publish","type":"post","link":"https:\/\/mereja.media\/index\/648761","title":{"rendered":"Liking curly fries might not mean you\u2019re smart: When mere data isn\u2019t enough"},"content":{"rendered":"

You might have heard recently about a study finding that liking \u201ccurly fries\u201d on Facebook correlates strongly with high intelligence<\/a>. Publications such as Wired have written about it<\/a>. Quid Founder and CEO Sean Gourley cited it during a presentation at Structure: Data<\/a> last week. A faction of the European Union parliament even pointed to the study<\/a> as yet another reason to prohibit data mining by web companies.<\/p>\n

However, if you\u2019re like me, hearing anybody repeat that curly fries data point as fact likely sends shiver down your spine. It\u2019s not that it\u2019s not true \u2014 it very well might be \u2014 but that it\u2019s nearly useless information without more background.<\/p>\n

That\u2019s right, the old correlation versus causation argument is front and center once again. In all the big data world, it\u2019s probably the biggest fallacy there is, no matter how you look at it. No, getting value from big data always doesn\u2019t require giving greater credence to correlation than causation. And, no, relying on correlation isn\u2019t inherently some sort of an ethically or scientifically questionable practice.<\/p>\n

Really, the choice between relying on correlation or striving to find causation probably depends on what you\u2019re trying to do.<\/p>\n

When there\u2019s nothing at stake, correlate away<\/h2>\n

Let\u2019s be honest: If all I\u2019m concerned with doing is boosting clickthroughs, selling more products or predicting the movies you want to see<\/a>, correlations probably will work just fine. I don\u2019t really care why, for example, Mac users book more-expensive rooms on Orbitz<\/a> \u2014 I just care that they do.<\/p>\n

You visit my site, my system sees you\u2019re using a Mac (or that you like curly fries, or any other attribute it can associate with you) and it shows you content that it thinks you\u2019ll want to see. It\u2019s not a perfect approach, but it\u2019s probably a far cry better than the old method of just showing everybody the exact same content.<\/p>\n

And when you\u2019re collecting potentially petabytes of user data and trying to serve ads in near real time, strong correlations might be about the best things you can hope to find. It\u2019s a volume-and-velocity business, and heavy examinations of why any two (or more) things are related to one another might not always provide a high return on investment.<\/p>\n

A more extreme example of when correlations might suffice would be something like machine-to-machine systems that need to make decisions in real-time in order to prevent disasters. The people charged with running these systems might not know why a certain series of events often precedes a particular outcome, but it\u2019s better safe than sorry.<\/p>\n

You can\u2019t make a difference \u2014 or real decisions \u2014 with correlations<\/h2>\n

But if you\u2019re trying to use big data to make a meaningful difference in the word or to make decisions that can have significant real-world consequences, mere correlations probably won\u2019t cut it. This is what Evgeny Morozov warns about in relation to crime in a recent New York Times <\/em>column<\/a>. It\u2019s what Gourley had in mind when talking about data science <\/em>versus data intelligence<\/em>. It\u2019s why the current discussion around machine learning almost always includes a human aspect<\/a>, as well.<\/p>\n

Many of the reasons for not acting on correlations alone are based on privacy and a whole collection of civil, constitutional and human rights. You simply can\u2019t profile and then arrest, for example, people based on what their Likes suggest they might be. You probably shouldn\u2019t make decisions about people\u2019s financial<\/a>, health or general well being based on mere correlations, either.<\/p>\n

Heck, I wouldn\u2019t even serve ads that delve into personal information such as health, sexual orientation or intelligence without a very strong reason to believe I was accurate (and express consent to serve those ads). And the Facebook-curly-fries study is full of correlations that could be potential landmines<\/a>, a small portion of which are visible in the chart below.<\/p>\n

More correlations from the \"curly fries\" study. Source: Proceedings of the National Academy of Sciences<\/a><\/p>\n

More correlations from the \u201ccurly fries\u201d study. Source: Proceedings of the National Academy of Sciences<\/p>\n<\/div>\n

But these are all situations where the fear of incorrectly profiling someone occasionally \u2014 and being sued as a result \u2014 might overpower the desire to do good most of the time. The data Darwinism that my colleague Om Malik wrote about recently<\/a> extends beyond just peer reviews and social-media ratings, and one shouldn\u2019t take the role of playing God (or catalyst for evolutionary change, to continue the Darwin metaphor) lightly.<\/p>\n

Sometimes, though, correlations aren\u2019t enough because you really want to solve a problem or perhaps build a great product. As Gourley explained at Structure: Data, even using correlative data to predict insurgent attacks in a place like Iraq is relatively easy, but predicting the likelihood of events doesn\u2019t stop them. Stopping them requires really understanding and addressing the root causes of the attacks.<\/p>\n

The same goes for stopping disease outbreaks, figuring out why programmers make more mistakes during certain seasons<\/a>, stopping gun violence<\/a>, or just capitalizing on that knowledge about curly fries or hotel-room bookers in order to build products that touch upon the deeper rationales for liking those things. You can fight the symptoms, so to speak, or you can cure the disease.<\/p>\n

So feel free to try selling the next guy you see eating curly fries on a documentary about Dostoevsky, but don\u2019t expect him to care. It might be that there\u2019s some strong connection between curly fries and intelligence; of course, it might also be that intelligent people \u2014 entirely coincidentally \u2014 tend to live within walking distances of an Arby\u2019s. But no one has asked about that.<\/p>\n

Feature image courtesy of Shutterstock user Tobias Arhelger<\/a>.<\/em><\/p>\n

\"\"<\/p>\n

<\/a><\/p>\n

Related research and analysis from GigaOM Pro:<\/strong>
Subscriber content. Sign up for a free trial<\/a>.<\/p>\n