What Google search tells us about ourselves

May 21, 2017

Discovering what people really think has often proved elusive. Just ask those pollsters who assured us that Donald J. Trump had no chance of becoming the 45th president of the USA. I am fascinated by the gap between what people say and what they really think or do. Sometimes we lie to ourselves – we really do have good intentions, but we do lack self-awareness. Other times we hide what are socially unacceptable views. What social scientists need are ways to get around these biases. I have already blogged about big data and its potential. One key area is Google search. The omniscient Google is our friend, confidante and confessor. We have all googled something that we wouldn’t dream of asking somebody in the flesh. Such search queries are anonymous, or at least that’s how we feel. Every time we type in a search we reveal something about ourselves. It is like a societal x-ray of our collective hopes fears and desires. In particular, Google’s anonymous, aggregate data can also tell us about the dark sides of our thoughts and behaviours. This tool is the subject of a new book by Seth Stephens-Davidowitz, Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

Seth Stephens-Davidowitz created a map of racism based on searches with racial slurs and racist jokes. He then studied how this affected voting across the United States. He used it to analyse how it affected voting in the 2012 presidential election. What he discovered was that in those areas of the country with the highest number of racist searches Obama’s results were markedly worse than those of John Kerry, the unsuccessful white Democratic candidate in 2004. This variable was far more relevant than education levels, age, church attendance, or gun ownership was. Although Obama won, the effect was important. Obama lost roughly 4 percentage points nationwide just from explicit racism. He was, however, able to get back 1% or 2% from higher African-American turnout. In 2012 the conditions were favourable the the Democrats. But this data is also germane to what happened in 2016 and the rise of Trump. According to Nate Cohn, the biggest predictor of Trump support in the Republican primaries was the racist searches. We need to be careful – correlation is not causation. Nevertheless it does provide a partial explanation of the Trump phenomenon.

What can Google tell us about Sex lies and videos? One revealing fact is that in 2015 2.5 billion hours of porn were seen on Pornhub, the largest pornography site on the Internet. To put this number in perspective, it is more than the entire history of our species on Planet Earth. And in surveys 2.5% of men say they are gay. But Google tells another story; 5% of male porn searches are for gay porn. There are more gay searches in tolerant areas, such as California, than in places like Mississippi. But the difference is not that high 5.2% compared to 4.8%

We parents do tend to want to project things onto our kids. In fact, of all Google searches starting “Is my 2-year-old…” the most common next word is “gifted.” We like to think that as parents we have equivalent expectations and dreams for our sons and daughters. But the abovementioned question is not asked equally about young boys and young girls. Parents are two and a half times more likely to ask “Is my son gifted?” than “Is my daughter gifted?” They show similar biases when using other phrases related to intelligence. Stephens-Davidowitz asks if the parents are simply picking up on legitimate differences between young girls and boys. In fact, at this age girls tend to have larger vocabularies and use more complex sentences. In American schools, girls are 11% more likely than boys to be in gifted programs. Nevertheless, parents seem to find that their male progeny are the gifted ones. With their daughters their concerns are more about appearance. “Is my daughter overweight?” is googled approximately twice as much as is “Is my son overweight?” this is despite the fact that whereas 30% of girls are overweight the corresponding figure for boys is a little higher- 33%.

A team of researchers from Columbia University and Microsoft analysed data from tens of thousands of anonymous users of Bing, Microsoft’s search engine. They coded a user as having recently been given a diagnosis of pancreatic cancer based on unmistakable searches, such as “just diagnosed with pancreatic cancer” or “I was told I have pancreatic cancer, what to expect.” The researchers wanted to discover what symptoms were strong predictors of a diagnosis. They examined the searches that had been made before the actual diagnosis, comparing the few who were finally diagnosed with the cancer to those who weren’t. Here’s how Stephens-Davidowitz explains what were remarkable results:
“Searching for back pain and then yellowing skin turned out to be a sign of pancreatic cancer; searching for just back pain alone made it unlikely someone had pancreatic cancer. Similarly, searching for indigestion and then abdominal pain was evidence of pancreatic cancer, while searching for just indigestion without abdominal pain meant a person was unlikely to have it. The researchers could identify 5 to 15 percent of cases with almost no false positives. Now, this may not sound like a great rate; but if you have pancreatic cancer, even a 10 percent chance of possibly doubling your chances of survival would feel like a windfall.”

In Everybody Lies Stephens-Davidowitz talks about the digital truth serum. The truth about what we think is so hard to find that we need every tool at our disposal. As a sceptic of opinion polls and surveys I like the idea of using proxies. However, as I pointed out in my post about big data there is a danger of finding spurious correlations, with cherry picking on an industrial scale. Nevertheless, I do feel that this book has hit on something.


Filter Bubbles, post-truth politics and the rise of populism

January 15, 2017

I know that there are other ways of seeing the world, and I’m happy that people have them, but I just don’t want to be in their world.”  Contribution to BBC Seriously podcast – Bursting the Social Network Bubble


What a fascinating time it must be to be studying politics. Although for university academics it must be somewhat disconcerting too. Indeed maybe they well have to bin the traditional textbooks. There is a convergence of factors – filter bubbles, our post-truth world and the rise of populism – that are convulsing modern politics.

I remember hearing about a book called The Filter Bubble when it came out in 2011. Author Eli Pariser’s central thesis was that social media algorithms are selectively shaping what a user sees on their feeds based on information about them. Consequently, we are in a bubble in which the news we receive serves to confirm what we already believe:

The filter bubble tends to dramatically amplify confirmation bias – in a way, it’s designed to. Consuming information that conforms to our ideas of the world is easy and pleasurable; consuming information that challenges us to think in new ways or question our assumptions is frustrating and difficult. This is why partisans of one political stripe tend not to consume the media of another. As a result, an information environment built on click signals will favour content that supports our existing notions about the world over content that challenges them.”

Pariser’s premise did seem to ring true, but I didn’t pay that much attention to it. But the events of the last twelve months have validated Pariser’s work. We have seen the effects of these echo chambers. There are two negative aspects to this phenomenon. Firstly, we are failing to exercise our critical faculties. We do tend to want to only hear views and facts that confirm our worldview. But these filter bubbles go beyond this; opposing ideas are not just wrong, they are totally alien. We just cannot imagine where they could even come from. This is why I criticised the use of safe spaces and no- platforming at universities. This is not just a problem of dumb people being taken in by dumb ideas. Sometimes the worst offenders can be the highly educated. Being articulate, they are better able to justify their prejudices.

In this polarised world we are also have different people accessing different facts. This is what is known as the post-truth world, the realm of fake news. In the pilot programme of The Colbert Report on October 17, 2005, American television satirist Stephen Colbert coined the term truthiness. According to Wikipedia, it refers to a truth that a person making an argument or assertion claims to know intuitively from the gut or because it feels right without regard to evidence, logic, intellectual examination, or facts. It may be a fake word invented by a fake person, but it does capture something. Both the Brexit campaign and Trump campaigns were characterised by extreme mendaciousness, but it goes beyond right-wing populism. Liberals and the left can live in a fantasy world too. I did like that quote from Kellyanne Conway, Donald J Trump’s campaign manager:

The most fake piece of news I heard all along up until Election Day and still hear from some people is that Donald Trump couldn’t win. How’s that for fake news?”

Trump is the ultimate manifestation of the rise of populism, especially right-wing populism. It’s not just the Donald and Brexit; Marine Le Pen may yet become the National Front President of France. Similar populist revolts can be seen in Austria, in Germany and the Netherlands. On the left we have the rise of left wing populism in Greece and Spain, and of course the rise of Jeremy Corbyn and Bernie Sanders. It does seem that the right have been more successful at the populist game.

I view populism, whether it be of the left or right, as a nightmare. What I hate about it is the way they peddle simple solutions, a vision of the world which denies the existence of divisions of interests and opinions within the people. It sees a Manichean world of pure people and the corrupt elite. Political opponents lack legitimacy and there are always scapegoats to blame for the country’s ills. Populism tends to get ugly when it gets into power. I have chronicled the horrible failures of Chavismo and Peronism in Venezuela and Argentina respectively. Apart from all the all obvious things to dislike about Trump, I am horrified by his views on trade, many of which he shares with Bernie Sanders. I think the populists will be found out once in power, but they might have done a lot of damage by then

I am a bit pessimistic. There is no doubt that liberal democracy is in crisis. There are problems with globalisation and inequality can have toxic effects on society. You can’t understand the success of Berlusconi and Putin without looking at the chaos and failure that preceded them. Of course there is nothing unprecedented about any of this. Yes, we have new technology but we have still seen many of these phenomena. People have always tended to follow the news that reflects their ideology. Fake news is nothing new. And populist movements have arisen before. The comparison between Trump and Hitler is ridiculous. Historian Niall Ferguson found a better analogy with the now forgotten figure of Denis Kearney, leader of the Workingmen’s Party of California. Kearney belonged to a movement of nativist parties and Anti-Coolie” clubs whose goal was to end Chinese immigration into the United States. Indeed, he was behind the slogan “The Chinese Must Go!” Curiously he was an Irish immigrant himself. But then Trump is he son of a Scottish immigrant and grandson of a German.

I often quote that Chinese curse: May you live in interesting times – I fear that we are about to experience this at first hand.

A couple of videos

January 15, 2017

Here are a couple of videos related to this week’s topic:

Beware online “filter bubbles – Eli Pariser


Johan Norberg – Identity Politics on the Left and Right


Noise, damned noise and leaf-blowers

December 11, 2016


A few centuries ago, someone invented a brilliant device: a long pole with bristles attached to one end. They called it a broom. It has been refined over the years, but it serves the same purpose for which it was designed. It sweeps up fallen leaves into nice neat piles so they can be collected and dumped elsewhere. Leaf-blowers cannot do that.

The clue is in the name. They blow leaves. They redistribute them. They shift them from one place to another. In fact, they are extremely good at that. If you are a little leaf just lying there quietly you stand no chance. You and all your companions will be swept up into the air with great force and deposited somewhere else.

What they cannot do is blow the leaves into nice neat piles so that the what they cannot do is blow the leaves into nice neat piles so that they can be taken away. For that you need a broom or, if we’re talking grass and borders, that other ingenious invention: a rake. John Humphries writing in The Daily Mail

Let’s find the guy who invented the leaf blower. Let’s follow that guy to a peaceful spot he loves and then let’s blow some fucking leaves. Bill Weir, ABC newsreader in a tweet.


Last week’s No Such Thing as a Fish podcast featured this fascinating fact about leaf-blowers:

Running a leaf-blower for 30 minutes creates more emissions than driving a Ford F-150 pickup truck 3800 miles. According to the Fish panellists, enough to get from Covent Garden, where the show is recorded, to Jerusalem. I didn’t know whether to call this a fact or factoid. It does seem to be barely credible. I ought to send it off to the BBC’s More or Less podcast to be fact-checked, as is the fashion these days.

The Fish fact came from James Fallows, national correspondent for the prestigious magazine, The Atlantic Monthly.  Fallows, has led an ongoing campaign against what he calls “the Leaf-blower Menace.” Leaf-blowers certainly provoke strong reactions. BBC journalist John Humphries is not alone in his opprobrium of these devices. Brian May, the Queen guitarist protested about “leaf madness” after being woken and disturbed by blowers in his Kensington and Chelsea borough. Rock stars never used to complain about blowers in the sixties, but I suppose May is 69 now.

The anti-blower community is extremely active on social media. Facebook has groups such as Ban All Leaf-blowers, Death to Leaf-blowers and Million People Against Leaf-blowers. What strikes me about all this is how impassioned the language is. One critic invoked Satre: “Hell is other people, with leaf blowers”, whereas another went for the Freudian angle: “Give a guy a leaf blower and he wields it like an oversized penis.”

Being a bit lazy myself, I can see the attraction of leaf-blowers; I think convenience is a good thing. However, I am sceptical about their value, but I can’t say that they make my blood boil. I live in a flat in Madrid, so my neighbours don’t tend to use leaf-blowers. What we do have are the municipal workers who do visit our street of a morning. How can we explain the rise of this machine?

Wikipedia defines a leaf blower as “a gardening tool that propels air out of a nozzle to move debris such as leaves and grass cuttings.” It is basically a reverse vacuum cleaner, blowing instead of sucking. They have traditionally been powered by two-stroke gasoline engines. It must be said that leaf blowers aren’t the only garden tools that use a two-stroke engines; many lawn mowers also rely on this technology.  Although you now have what is known as a blower vac, which can suck in leaves and small twigs via a vacuum, and shred them into a bag.

The first recorded example of people using air pressure to remove leaves is said to come from Japan in the 19th century. Gardeners employed bellows to clean up the mossy ground of their landscape gardens. But, for the modern machine there is some uncertainty about who invented it. According to Wikipedia, the leaf blower was invented by Dom Quinto in the late 1950s. It had originally been introduced to the United States as part of an agricultural chemical sprayer. But the manufacturers soon discovered that many consumers were removing the chemical dispensing parts from the device, leaving only the blower. The manufacturers realised it had a lot of potential as a common lawn and garden maintenance tool. The Wikipedia entry does say “citation needed”, so we need to be cautious. Indeed, The New Yorker claims leaf-blowers originated in Japan, in the 1960s as a tool for dispersing pesticides onto fields and fruit trees. I side with the online encyclopaedia in this dispute, but I thought I would put it out there. Curiously, when they first started becoming popular in the 1970s, leaf-blowers were seen as an environmental godsend in California, as drought conditions there meant that the use of water for many garden clean-up tasks was banned.

It is true that they are faster than using a rake. But, there are far more downsides. First is the noise. They are incredibly loud and the noise has spikes, which makes it more irritating. Then you have the pollution. The use of fossil fuels is, as we saw above, profligate. Fallows claims that around one-third of the petrol that goes into this sort of engine is spewed out, unburned, in an aerosol that has been mixed with oil in the exhaust.  This leaves a horrible stench of petrol in the air. They are needlessly blowing dust, allergens, toxins, pollutants and pathogens into the air we all breathe, especially harmful for small children or those with allergies.

I’m no technophobe, but I can’t really defend leaf-blowers. Maybe in the future they will be quieter and more eco-friendly. For professional gardeners and landscapers there is the Mean Green Blast Battery Powered Backpack Leaf Blower. With the blower and the battery the backpack weighs over eleven kilos. They lithium battery, which takes three hours to charge, gives it an autonomy of 65 minutes. They claim it is super quiet, which is 56 db. It creates zero emissions and requires no gas and little maintenance. What’s not to like?


The price – it comes in at $1,695.95. Maybe we should indeed go bake to the rake and broom for the moment.

Ads, damned ads and online advertising

October 22, 2016

This week I heard a fascinating interview with Columbia Law School professor Tim Wu on NPR’S Fresh Air. An open Internet advocate, Wu is increasingly worried about the direction in which the World Wide Web is heading – advertising just keeps getting heavier and heavier and heavier. Wu has just written a history of advertising: The Attention Merchants: The Epic Scramble To Get Inside Our Heads.

Many of the founders of Internet were utopians. According to Stewart Brand, we owe it all to the hippies:

Forget antiwar protests, Woodstock, even long hair. The real legacy of the sixties generation is the computer revolution

But as time has gone on it has gone what has taken place has diverged from this vision. The internet has lost its innocence.  Yes, a lot of content is free, but this comes at a price. We are constantly being bombarded with advertising. Not only is it ubiquitous, it is becoming increasingly difficult to close. We like the idea that that we can get stuff for free. This is a notion that is particularly prevalent on the internet. Newspapers are caught in this dilemma. Going behind a paywall doesn’t seem to be a particularly viable option. You lose a lot of influence this way; newspapers have traditionally wanted to be at the heart of the debate. But giving everything away doesn’t work either. Advertising just doesn’t bring in enough revenue for newspapers. There are some companies making money from online advertising – the Internet powerhouses Google, Facebook and Twitter.

Wu describes Google as the most profitable attention merchant in the history of the world. They began as a very idealistic company, but what they didn’t have was a business model. Their route to wealth would be with advertising. What is particularly ironic about this is Google’s founders, Larry Page and Sergey Brin actually hated advertising, which they thought would corrupt the goal of the search engine, which is to try to give you the most important information, not what someone paid to be there. They wrote all this in a paper, The Anatomy of a Large-Scale Hypertextual Web Search Engine, which they wrote while studying at Stanford in 1998.

We are in what is known as the attention economy. Human attention is indeed a scarce commodity. As we have such digital overload, companies struggle to get into our brains. To attract interest one of the online marketer’s most potent weapons is clickbait. Merriam Webster defines this as:

online material (such as headlines) designed to make readers want to click on hyperlinks especially when the links lead to content of dubious value or interest.

The earliest citation on the Wordspy website goes back to 1999. In reality what they are doing is using many of the techniques perfected by tabloid journalists. All we have is a 21st century version of The Sun’s notorious Freddy Starr ate my pet hamster.

Wu’s central idea is that if we really care about content, we should be willing to pay for it. He gives food for thought. An ad-free version, he claims, would cost $12 a year. I don’t take such an alarmist view. True, it can be irritating when you are on the mobile phone. But I don’t find Facebook ads particularly irritating. I see it as a reasonable trade-off. I’m still in awe at everything that you can get on the web. There has been a loss of idealism, but I have never been a cyber-utopian. I know that they are looking to find my weaknesses, but I don’t think that we are such passive victims.

20 Diversion Tactics Highly Manipulative Narcissists, Sociopaths And Psychopaths Use To Silence You

October 22, 2016

No, you’re not going to learn these diversion tactics. This is just clickbait, which I got from the ContentForest clickbait generator I found online. Here are some others:

People Are Tweeting Their Most Awkward Moments And It Is Cringingly Hilarious

The pope’s lunch plans tomorrow are exactly why we love the dude so much.

Auto Mechanics Hilariously Recreate Renaissance Paintings

A Woman Is Posting Feminist Messages Written On Period Pads All Over Her City

16 Things We Forget To Thank Our Moms For

24 Pictures That Will Make You Feel Better About The World

A Scottish boy couldn’t stand a preacher’s homophobic rant, so he whipped out his bagpipes.

26 Poses Every Single Person Will Immediately Recognize

Corgi Puppies Running In Slow Motion Will Overwhelm You With Cuteness

Young Couple Aged 70 Years With Make-Up To See If They’d Still Love Each Other

A Dog, 8 Birds And A Hamster Are The Most Unusual Best Friends Ever

This Is For Everyone Suffering From Freezing Office Syndrome

Remember that pizzeria that was feeding the homeless? See what happened when you shared their story.


How was it for you?

April 10, 2016

How we became obsessed with rating, ranking, reviewing and measuring.

As a Harvard undergraduate, Mark Zuckerberg could have been expelled when an early version of Facebook, which ranked girls on how hot they were, brought down the university’s server because of the high level of traffic. He was accused of breaching security, violating copyrights and violating individual privacy by creating the Facemash website. After a disciplinary board hearing the future billionaire was put on probation and required to see a counsellor.

We love ranking things. In his first novel, High Fidelity, Nick Hornby tapped into this obsession. The protagonist, Rob Fleming, was obsessed with making top-five lists: most memorable split-ups, in chronological order, Best Films of All Time and even Bands Or Musicians Who Should Be Shot If It Came to the Musical Revolution. But now thanks to the IT revolution we have all become like Rob Fleming. We rate, we rank, we review, we measure and we also track. We do it to ourselves, our friends, the companies we purchase products from and also the places where we eat, drink and sleep.

The growth of e-commerce in particular, has led to an explosion of rating. The online auction and shopping website, eBay was one of the pioneers of this way of ranking people. It was obviously important for the fledgling website to establish trust; you were buying from complete strangers, so you wanted to know if you were you going to ever see your money again or get your package delivered. And in the sharing economy of Airbnb and Uber trust is a vital component too. Indeed, the Uber rating system isn’t one-way. Customers also get rated. You’re more likely to be a courteous passenger if you know getting a bad score might result in you being cut off from the convenience of Uber in the future. These systems don’t always work well. The Uber driver who went on a random shooting spree that killed six people in Michigan last month had an Uber rating of 4.73. However, this phenomenon is not limited to the online world. Dr. Harold Shipman, who may well have been murdered more than 200 of his patients, was described as a wonderful GP by many of the families of his patients.

Research carried out by Consumer Business Research at Deloitte UK has shown that 81% of consumers read the customer reviews, on e-commerce and websites. This is not just a passive activity. 40% of consumers post reviews. I don’t tend to write reviews. Recently I bought an e-reader from the Spanish company Energy Sistem (sic). As well as coming without a built-in dictionary, it had serious battery issues. I just took it back to the shop. I didn’t write one of those scathing online reviews recommending that you never buy any of their awful products.

Companies have become vey needy in recent years. You go to IKEA and as you leave the store, they have this smiley-button based system:

ikea smiley

Buy online and you are asked to rate the experience.  A similar thing happens if you phone you telephone company. It’s all too much. As Anne Karpf pointed out in the Guardian: “If I have a transcendental experience with a bin liner, my supplier will be the first to know. But otherwise, forget it.”

One thing is writing reviews about products, but the rating is invading the personal sphere. Peeple is a mobile app allows you to rate people. The slogan for this app is “Character is destiny” and according to its website, the objective is to turn character into a new form of currency.  There are three criteria for evaluation: professional, personal and dating. After a storm of criticism about potential cyberbullying and harassment when the company announced the plans for the app in September 2015, the service became opt-in; people could only be rated if they had registered with the service. The five-star-rating system also went to be replaced by Positive, Neutral or Negative. You can write a negative review, but unless the person on the other end approves it, it won’t appear on their profile. The app was finally launched just over a month ago, and is available for download in the US and Canada.

China has a more sinister use of rating people. Sesame Credit trawls data from social networks and online purchase histories, to give people a score for how good a citizen they are. In reality this seems to mean how obediently citizens follow the party line. As the video above explains, “If you post pictures of Tiananmen Square or share a link about the recent stock market collapse, your Sesame Credit goes down. On the other hand, Share a link from the state-sponsored news agency about how well the economy is doing and your score goes up.” Users whose friends have low obedience scores could also lose points.

We need to put all this into some kind of context- less than sixty years ago China was in the grip of the Great Leap Forward, which was followed soon after by the Cultural Revolution. Nevertheless, there is something insidious about this attempt at promoting social obedience.


If you are interested in this subject, check out this ABC Radio National podcast, Future Tense, Rate, rank, review and measure.