User data means power

The internet has made possible  what power structures have tried and, usually, failed to do throughout the history: knowing everything there is to know about their citizens. Now there is no need for anyone to ask us for information – we gladly give it away and we are doing this in larger and larger numbers.

Think about the first time you’ve connected to the internet: the sheer amount of information available was dazzling. You could travel the world in a few seconds without leaving your home. Everything other human being seemed closer than ever and things looked pretty safe as you were just one of the other millions of people connected to the internet. Anonymity instilled a sense of freedom that was not possible in the real life.

Years have passed, the internet improved, the number of internet connected users increased exponentially and some companies started wondering – who are this users and how can we find out more about them? Apparently – pretty much.

We had a look at how companies use large data to improve their marketing efforts. Facebook and Google are some of the biggest data-handlers in the world and by offering free web services such as social networking or search these companies gather hundred of millions of users on a daily basis. These users have certain interests, profiles, friend connections and are willing to give away all this information without much thought.

If, say, a Coca-Cola representative would approach us on the street and started asking us questions regarding personal data, interests in different areas,information about our friends we would be rather skeptical, wouldn’t we?

Actually that’s what basically happens every time we search something on Google, update our Facebook profile, read an article on the web or simply send an email. Even though some information is anonymized, even though there are laws that may interfere with privacy breaches, the truth is there are some companies that hold real time information on large masses, information that can be used either at a macro or micro scale. From individuals to countries such companies know a lot of things and became increasingly good at harnessing the power that lies in these bits of data.

Micro and macro implications

Micro implications

On a micro level individuals basically offer some of their most intimate information to companies that use it for marketing purposes. There is a saying stating that “if you are not sold anything, you are the product being sold”. That holds true to both companies mentioned above. There is no secret Facebook and Google make most of their revenues through advertising. To help increase advertising efficiency both companies need to know as many things as possible about the person viewing the ad.

Both companies thrive on information users offer, knowingly or not. Whether is the page you are viewing, information on your Facebook profile – you tell advertisers how to better sell their products.

Another implication of sharing so much data is that you become predictable. Even though we look at ourselves as unique, special individuals, the fact is we are not. We are creatures of habit and habits turn into patterns. When some important events in our lives happen our behavior is even more likely to become predictable. Target used customer data to find out when their buyers start dealing with pregnancy. Based on a series of products future mothers are more likely to purchase they managed to target those exact customers, sometimes even before their friends or family found out.

You might think that companies and other organizations can track you only if you choose to use your real identity. Actually no. There are several techniques developed to help deanonymize internet users. One of these techniques is based on stylometry, the analysis of writing style. Although information on this subject dates back a few centuries, the internet made possible the analysis of large chunks of data.

Be it your blog, your Facebook profile or movie reviews you posted online  the fact is you leave traces on the internet through your writing style. Even if you publish a text anonymously and make sure you are not traceable by classic means, stylometry can point towards you. Arvind Narayan, a computer scientist focused on “breaking data anonymization, and more broadly […] digital privacy, law and policy” explains here how this can happen, what are the necessary steps, what are technological requirements etc.

Macro implications

china flag
China is allegedly one of the leading cyber-warfare powers at the moment

Although micro implications are interesting, they are just the tip of the iceberg. With enough data anything is possible. And I mean everything. Think stock markets, military, health and epidemic research, economy, global intelligence. Basically all the power structures our civilization depends upon can find large data extremely interesting.

In 2010 China routed traffic intended to some very discrete US organizations through its servers for roughly 18 minutes. “Not too long”, one might think, but enough to cause a 300 page report for the US congress. If 18  minutes worth of internet traffic routing caused such a stir, imagine how much of an impact information passed through Facebook and Google have on the global political scene.

With enough data stock market crushes and bubbles could be predicted, social movements could be news before they even happen, just like military strikes or economic crises. One thing is for sure: there is great power in the data provided by users online.

Mobile, Social search, Photos and iOS integration makes Facebook Stocks Rise

After dropping more than 50% since the IPO, Facebook’s market cap restarted growth following Mark Zuckerberg’s on-stage interview at TechCrunch Disrupt. The company closed today with a 7.62% increase in its share price. Several factors could have lead to this fortunate turn of events but the keywords are: mobile, social search, photos and iOS integration.

facebook market capitalization growth
A surprising growth in Facebook market capitalization

Facebook’s focus on mobile

On stage, Mark Zuckerberg made it very clear that Facebook is focused on mobile growth. A very bold statement, probably targeted at investors that so far have had their fair share of drama, was “On mobile we are going to make a lot more money than on desktop”.

Facebook has more than 488 million mobile users and the numbers are growing fast, due to increase in smartphone and mobile internet adoption. Recently the company introduced new ways for advertisers to target mobile users through sponsored stories. The advertisers were not so fast to switch to mobile ads, however: although mobile revenues are estimated at about $72 million this year, the figures are below Twitter’s estimates.

This situation is sure to change as Facebook’s focus seems to follow the mobile trend.

Facebook search

Probably the most important thing Zuckerberg mentioned was the fact that Facebook now serves 1 billion search results per day, “without even trying”. To put that in perspective – that is 10 times the number of Bing searches and approximately 30% of Google’s searches. Imagine that – 30% of the world’s largest search engine.

With social input Facebook search can potentially deliver better results than Google. After all, Google is not really good at answering questions but rather locating information. Facebook users usually ask their friends for help on different issues and this type of behavior creates a huge pool of data Facebook can use to answer questions in a very efficient way. Even “without trying”, Facebook has recently started monetizing its searches through contextual ads.

Facebook and Instagram Photos

instagram facebook
Image source

Zuckerberg mentioned that there is no hidden agenda in Instagram’s acquisition. They want to help the app grow and so far they increased exposure by 1100%.

Photos seem to be a very important area in future Facebook development. Although it’s obvious that photos have a positive psychological effect on users and increase revenue through photo-page delivered ads there is probably something that we don’t know yet.

iOS integration

Facebook is deeply embedded in iOS 6

iOS is the most popular mobile operating system on the Internet, with an astonishing 65.27% of all mobile internet users. Today Apple announced several news, including the long awaited iPhone 5, the new iPod touch and some social features based on Facebook and Twitter social relationships.

Having been integrated in the world’s most popular OS means big exposure for Facebook. Even more – it means an increase in revenues.

As I mentioned a few days ago Apple and Facebook were planning and started rolling out a deeper iTunes integration. Although Facebook is just starting monetizing its mobile users, Apple is one of the best at this game. Using Facebook’s social features Apple can sell even more, bringing a new stream of revenues for both companies.

It seems as though George Soros knew what he was doing when he purchased Facebook stocks

The sad truth about print media

Print is dying. You don’t have to be fortune teller to understand this. You just need to be rational enough to accept the facts.

After all print was never about the technological print process. It was about written content, ideas, news, thoughts. Along the way some people failed to grasp the changing media landscape and technology made the printed media obsolete.

decrease-print-revenues
Print advertising decreases. Online increases. But does it increase fast enough? Source: State of the Media – 2012

The internet let individuals and smaller media companies reach potential readers. With low overheads such small editorial operations let the founders survive on advertising alone. “Free” content meant a larger access to potential readers. The increase in internet adoption, increase in online publishing websites popularity and increase in advertising revenue meant that in a very short time these websites became serious threats to print industry’s revenues.

How did it get to this?

Print publishers failed to adapt on several key moments:

1. The internet made online publishing possible: This was the most important moment. This was the moment a certain key element previously reserved to large media companies became publicly available. Anyone with access to a computer, internet, a few bucks could potentially start a online publishing company. Of course, funding was scarce, especially after the dot-com crash, advertising revenue was almost impossible to reach so most publishers decided they weren’t going to join the club. At the moment the decision may have sounded rational as there was no point in spending money on something that many considered a fad. There were no economic incentives so the publishers decided they were not going to risk established business over some geeky internet stuff. It was all downhill from here.

2. Online independent publishers started monetizing their audience: Through advertising online publishers started making at least some kind of money. With low operational costs some of them reached break even. However large media companies were still dismissive of this new trend as revenues were still way below their radar. Were this companies to seriously consider the online alternative they may have just detracted the public from the “free” advertising sustained model independent publishers adopted. Maybe, just maybe, a paywall solution could have been accepted were it proposed and generally introduced at the time. In my opinion Rupert Murdoch’s idea of pay per written content is not feasible on a large scale. It may work on a select few buyers that need accurate financial information, for example, but it’s pretty hard to impose such a model on, say, a daily news website.

3. Print revenues started declining: Even when revenues started declining large media companies were not able to seriously consider the online alternative. With a short term vision they treated their, by now existing, online division as the “town freak”. Rather tolerated then encouraged and developed. Focus was (and with most print-based companies still is) the print. That’s where the revenue (still) came from – that’s where they thought they should focus on. Wrong. Readers  switched sides. Advertisers switched sides. The whole world switched sides. After all, Amazon is already selling more ebooks then hardcover and paperback books. Why wouldn’t newspapers and magazines get the same treatment?

The print companies faced denial, denial, denial and now they face survival. They now face the sad truth that print is going to die and there’s nothing they can do about it. Maybe just survive the crush.

(II) Ogilvy to Zuckerberg: How did Amazon, Apple, Facebook and Google change consumer research and targeting ?

In my last post I talked about the shift in consumer targeting that happened once the Internet went mainstream. Several highlights were the short history in consumer targeting, information regarding Amazon’s personalized recommendations and Apple’s usage of consumer data to increase music and app sales.

Now we’ll have a look at how two of the largest and fastest growing technology companies use consumer data and behavior to deliver ads. As Facebook and Google’s business model heavily relies on advertising they have to make sure ads are delivered efficiently to increase revenue.

However, trying to increase ads relevance and user experience can sometimes lead to unexpected (?) outcomes. Both companies had had their fair share of legal troubles regarding users privacy. For example last year Facebook user tracking practices lead to a request by US congressmen for the Federal Trade Commission to investigate the company. Apparently Facebook would track users web traffic even after they logged out. By linking browsing history, location and time of visit  to account information (list of friends, preferences, browser) the company could potentially extend its user profiling to some very intimate data. Apparently the issue was corrected and now Facebook stopped linking browsing data to user profiles. Even so, the anonymized data can provide the company with some very good insights.

What are Google and Facebook’s revenues?

As stated above both companies rely heavily on advertising revenue. 96% of Google’s 2011 $37.9 billion revenue came from advertising. Industries that pumped most money in Google’s Adwords program were Finance and Insurance ($4 billion), Retail ($2.8 billion), Travel and Tourism ($2.4 billion) – source.

Meanwhile Facebook reported “only” $3.1 billion in advertising revenues last year. Even though the numbers are visibly lower than Google’s, Facebook advertising revenue increased 69% and topped Yahoo in 2011.

Just to give you a perspective on how big this figures are Publicis, the largest advertising group, a 86 year old company, operating in 104 countries reported a $7.7 billion revenue in 2011.

Having established that online targeting leads to generous revenues, let’s have a look at how Facebook and Google manage to efficiently target consumers using technology:

How does Facebook target users?

facebook logoFacebook increase in popularity coined the term “social media”. This term describes web and mobile platforms where organizations or individuals communicate through different types of media (text, image, video etc.). As more and more users started using Facebook the available content increased, social links improved as users added more and more friends.

Facebook recognized the opportunity in consumer targeting using social preferences (Ex. “Your friend likes X Brand. You should too.”). Interestingly Facebook managed to give user profiles a real – life feeling by encouraging people to bring their friends along. Of course few people could recognize nicknames such as “MickeyMouse1982” so users started adding their real names, than their birthday, location etc.

Soon enough Facebook had a few hundred million demographic profiles at hand. These profiles were interconnected so influence groups could easily be determined. In a genius move Facebook introduced the “Like” button and later “Share”.

By using the “Like” button users would essentially hand over to Facebook their personal preferences.

As publishers saw that articles posted on Facebook were more likely to become viral and increase traffic they adopted the Like/Share widgets and later the Facebook Connect signup system. As these widgets could track user behavior by transferring traffic data back to Facebook the social network now knew what users were interested outside the platform.

Combining this data Facebook launched and improved in time their Facebook Ads platform. With more than 20% of all web traffic plus data on web traffic outside its social network, the company could potentially target ad delivery better than most other media companies. Let’s review what kind of data Facebook has at its disposal to target users:

  • consumer demographics: users enter their demographic information during signup or later as they use the social network
  • social networks: Facebook knows who is a friend of who, who is more likely to have his or her posts liked, shared or commented on. Basically it knows who is most likely to influence their peers actions with a granularity almost impossibly to achieve by others
  • consumer preferences: every time a user clicks a like or share button, comments, posts a status, photo or video it basically signals Facebook on some of his or her preferences regarding a wide array of things (music, products, news) that could later be used to show relevant ads.
  • web traffic: by tracking user behavior through like, share or social widgets Facebook registers data that even anonymized can show insights on a scale that no other company can

These are the most important factors in Facebook efficient ad targeting. Weather advertisers choose to use classic ads, sponsored stories or promote several posts the company takes into account this data to maximize exposure and engagement.

How do Google ads become “contextual”?

google logoProbably the most disruptive technology company in the past two decades, Google relies on user data, behavior and semantics to deliver the contextually targeted ads.

To deliver ads, Google needs data. Where does it get it from?

Where does Google get data from?

  • indexed and ranked web pages: even though the number is not really known as Google is secretive about its data centers, it’s estimated that indexed data is stored in more than 30 data-centers. These data centers hold 35 to 50 billion pages at any given time. They are ranked according to an algorithm initially designed by Larry Page and Sergey Brin and improved in time.
  • web page analytics: Google Analytics is used by more than 10 million web sites. As Google hosts data regarding traffic and user behavior on these sites it can predict user behavior and ads most relevant to potential consumers.
  • email information: even though information is anonymized Google makes good use of mails hosted on it Gmail platform. With more than 350 million users in Jan 2012 the data flow through Google’s emailing platform is astonishing.
  • searches: Google responds to almost 3 billion searches every day. By analyzing searches and user paths Google can determine what are the most popular search results and how can this information be used to optimize ad targeting and delivery.
  • Google+ is the company’s response to Facebook’s rise in popularity. It already has more than 170 million registered users (mostly active). Having answered the need for information in social networking targeting Google further improved its advertising targeting capabilities.
  • Android is Google’s mobile operating system. Though buggy at start, Android is now on its way to world domination in terms of mobile operating system.

Basically Google knows a lot about a lot of potential consumers and uses these data to increase efficiency in ad targeting.

Having a look at how the likes of Amazon, Apple, Facebook and Google use research and targeting , we can surely say that conventional (old ?) knowledge on the matter is becoming increasingly obsolete. As technology replaces human input research and targeting becomes real-time.

Unfortunately some privacy issues arise when people become “users” or “consumers”. On this matter – soon.

From Ogilvy to Zuckerberg: How did Amazon, Apple, Facebook and Google change consumer research and targeting ?

Conventional (TV, print, radio) advertising often relies on research and targeting methods such as focus groups or demographic targeting to increase brand awareness and sales. These methods seem to be more and more outdated as targeting technology is already delivering better results.

A (very) short history of advertising research and targeting

In the past, as media was unidirectional (broadcaster to consumer), there were few ways retailers could efficiently target potential consumers. Advertisers would use consumer profiles and split purchasing options through demographic indicators (age group, location, education, sex etc.). By using statistic results they could outline marketing opportunities for certain demographic groups (Ex. “Women between 25 to 35 years, urban, having higher education are more likely to buy Product X”).

Having (theoretically) discovered a potential consumer profile they would then buy media in newspapers, radios or TV stations that would best appeal to that certain demographic group.

David Ogilvy
David Ogilvy

Of course this is just a skeletal description of the whole targeting process but it explains the process pretty well. Many companies have benefited greatly from this targeting and advertising system. Most of the brands we now know and buy were built this way. Even now, decades after the likes of David Ogilvy were setting up the rules on research-based advertising, the system is virtually unchanged.

“I notice increasing reluctance on the part of marketing executives to use judgment; they are coming to rely too much on research, and they use it as a drunkard uses a lamp post for support, rather than for illumination.” – David Ogilvy

How did the Internet change research and targeting?

Few could have predicted the impact Internet was to have on commerce and economy. Even less would have guessed how this initially “exotic” media would impact research and targeting.

20 years ago there was no marketing concept that could explain AdWords targeting and not be considered science-fiction.

Internet targeting and advertising renders most of conventional knowledge on research obsolete as technology has achieved what was once impossible. 30% of all human population is now in reach of all advertisers and they can now target more than just demographics.

Behavioral marketing is a concept that could not be possibly be achieved with conventional media. Using consumer behavior rather than demographics advertisers can target real time preferences and individuals rather than demographic groups. Say a user is known to have previously visited a car dealership website. He then browses websites in search of reviews on different car models. The car dealership could potentially target this exact user and serve him the most informative ads. Advertising ROI is sure to increase this way.

Some companies have become increasingly good at Internet research and targeting. One of them is now the most valuable company in the world in terms of market capitalization. Let’s have a look at how Apple, Amazon, Facebook and Google use large data to target and monetize consumer traffic.

How did Amazon, Apple, Facebook and Google changed consumer targeting ?

Amazon personalized recommendations

amazon logoAmazon is well known for its personalized products recommendations. How can it do this? Short answer: large data on consumer purchases and mathematics. Longer answer: Amazon holds a patent on its product recommendations which you can have a look at here (issued in sept. 2006). Although rather technical it focuses on certain key elements:

  • user profiling: Amazon holds valuable data on user demographics and previous purchases. Using this data it can map users in specific consumer groups. User profiling combines shopping cart contents, item ratings and recent purchases as purchase intents seem to change in time.
  • similar products information: say you bought three SF books. Similar products would be other books in that category. Some of these books would be more popular in terms of item ratings, reviews, views and purchases.
  • item affinity is the probability of some products to be purchased together. Say you are buying a Kindle on Amazon. You are very likely to buy a cover or case to protect your device. That means these products have a high affinity index
  • driver items are those products that are most likely to drive traffic to store. Again – the Kindle, Amazon’s best seller is not only a driver item but also a platform that insures further product purchases.
  • user path: the consumer will follow a certain path until it ads a product to the shopping cart or confirms a purchase. These paths are very important as they can be used to “guide” consumers to products they are most likely to purchase.

Using these information (and probably more) Amazon can first map users in consumer groups (1), extract popular, affinity and driver products (2), compile most profitable user paths based on previous history and other users actions (3) and than recommend the items most likely to increase basket size.

Recently Amazon announced the launch of its Kindle Fire product. This product is built on a Android platform and uses a proprietary web browser called Silk. The browser optimizes web traffic by routing it through Amazon’s servers. As Amazon already holds information on user profiles (users will have to login to synchronize their book collection) and now data on web traffic it can further improve its recommendations.

Apple Genius recommendations

apple logoAlthough Apple does not explicitly state it monitors iOS user actions it doesn’t deny it either. If it does, however, it might access a huge pool on users data such as web traffic, mobile purchases, locations, call history, social networking information (through access to contacts information, call history, SMS and iMessage history etc.). Basically everything there is to know on its customers profile.

For now the most visible way Apple uses data to increase sales is iTunes Genius, the music and video recommendation system. iTunes Genius uses purchase history and iPod activity to recommend potentially interesting songs, albums or videos.

Although iTunes Genius probably uses a system similar to Amazon’s it is not yet known to be as accurate. The performance issues are probably connected to the number in customers and items on sale. Amazon has a wider products inventory and a larger pool of potential customers. This leads to a larger database and increased accuracy.

Technology based companies have changed the way we think of consumer targeting and advertising. Innovation lead to profits and behavioral targeting will probably develop in the future. Tomorrow we’ll have a look at how two of the largest advertising – revenue based companies, Facebook and Google, use large data to improve consumer targeting. Stay tuned.

The UN hijacks the Internet

Some countries, including Russia, China and Iran, well known for their democratic efforts and respect for civil rights ( 🙂 ), seek more control over the Internet, through the UN International Telecommunications Union (ITU).WCIT logo

As this year’s World Conference on International Telecommunications (WCIT-12) is preparing for its December the 3th opening several documents have leaked on http://wcitleaks.org/  . Although these documents were not actually classified they were never made public. After they leaked the public opinion could see why: countries like China, Russia, Iran, Tadjikistan and Pakistan are asking for a complete change of the Internet as we know it.

What is ITU and what does it do?

The ITU was founded in 1865 to help regulate telegraph communications. Over the years it has evolved into an organization managing international telecommunications regulations (ITRs). As its last signed protocol dates back to 1988, previously to the invention of the Internet, some of the government representatives in the UN thought of how might the Internet be controlled. As a result some preposterous proposals surfaced on wcitleaks.org. Here are some of them:

UN proposals on Internet management

  • China proposed that governments should be in charge of internet traffic nation-wide. Online companies operating inside a given territory should use the Internet in “a rational way” or as WSJ puts it – handle over control and internet traffic information.
  • UN could handle online content however it finds appropriate
  • Russia and some of the Arab countries proposed access to Internet communications (like checking users emails, Facebook accounts etc.)
  • Iran and Russia proposed a charging method similar to international calls (users could be charged more for international traffic). Some European countries backed a similar proposal.

Why the Internet should stay just the way it is ?

putin - internet
Vladimir Putin, a well known civil rights supporter

I find such proposals not only misguided but also economically dangerous and short sighted. For the last 20 years the world benefited greatly from the Internet being a somehow free media. Yes, the Internet is now manged by some US based organizations but this structure brought us a level of information we have never seen. Almost one third of the world is now connected and free to share ideas. Education, economy and general well being – they all improved with the free Internet.

I discussed  the dangers of over-regulating the Internet in a previous article. At the time I had no knowledge of such disturbing news. Even more disturbing is the fact that public media is not yet taking as much interest as it should in this matter. Forget ACTA, SOPA and all the other things the Internet stood against. This conference is a turning point in human history, just like the invention of the Internet was.

Should the proposals discussed above be accepted and enforced we will see a huge set back in economic development, education and civil rights. After all – the Internet is changing the status quo in a civilization that has been so far lead by fear, ignorance and hate. In a world that faces poverty, over population, countries armed with nuclear weapons, can we really accept to bury this diamond in the dirt?

Further reading: