Social data is the canary in the coal mine for Big Data

I spoke at the Boston eMetrics Symposium yesterday on the subject of Big Data; specifically, how we will build organizations that can adapt to it, extract insights and act on them in a meaningful way.

People generally discuss Big Data in technical and even aspirational terms (see this month’s Harvard Business Review dedicated to the topic), but there’s less attention to the organizational impact. Who will have access to it? What will we do with it? What challenges will we face? Who, if anyone, will “own” it? And how big a change is this, actually?

Let’s be clear: Big Data–or at least elements of it–is here already. We’re seeing it in a focused way in some of the case studies that HBR mentions. One example, in “Big Data: The Management Revolution,” addressed the topic of plane arrival times, which pilots generally (and often incorrectly) estimate. The issue is that errors either direction produce productivity impacts that translate to significant economic impact. So one major airline improved its ETAs by engaging PASSUR Aerospace to crunch tons of data on plane locations, weather patterns, flight schedules, along with proprietary data, greatly improving arrival time estimates.

But we’re also seeing the effects of Big Data beyond the type of highly sophisticated decision support application above. Social data, after all, is a type of Big Data. It meets the “Three Vs” criteria: Velocity (speed), Variety (as IBM says, “text, sensor data, audio, video, click streams, log files and more”) and, most certainly, Volume.

Social data is the “canary in the coal mine” for Big Data: we can learn a lot about how Big Data will thrive (or not) by looking at what has happened with social data thus far. I covered many of these issues in my research reports, “A Framework for Social Analytics” and “The Social Media ROI Cookbook.” Consider this (all data courtesy Altimeter Group):

  • The average enterprise-class company has 178 social media accounts
  • The average enterprise-class company has 13 departments actively engaged in social media
  • Fifty-six percent of companies we surveyed stated that the biggest challenge to measuring revenue impact of social media was “inability to tie social media to business outcomes.”

Some of the other impacts we see, both quantitatively and via anecdotal evidence, have to do with the strain on organizations as they seek to integrate social data into decision-making:

  • Lack of analytics expertise
  • Poor or disparate tools
  • Inconsistent analytical approaches
  • Unreliable data

From a maturity model standpoint, we’re seeing this (rough) progression for social data organizations:

One drawback of this progression, at this point, anyway, is that it encompasses only social data, which threatens to become a silo of its own if it is not integrated with enterprise data such as CRM, Business Intelligence and Market Research data, not to mention the many varieties of Big Data coming from sensors, mobile devices, the Web, and so forth. We’re seeing early signs of this as companies such as SAP, eBay, Oracle, Adobe and Salesforce partner with or acquire technologies that integrate social and enterprise (and ultimately even external economic) data.

Some people have proposed the idea of a “Chief Data Officer” to oversee this process, and that is likely a fine idea, but not in isolation. While it’s a tiny sample size, most of the heat during and following my talk focused on the cultural impacts of Big Data: Who owns it? How do we share it? And how do we make the transition from a culture where decisions are made by “HiPPO” (Highest Paid Person in the Organization) to one that is data driven?

It’s a bit of a chicken-and-egg scenario (to continue the bird metaphors) as organizations need leadership to make the leap of faith needed to invest in data-driven organizations, and leadership needs the data to convince them that this is a leap worth taking.

Plenty more to mine here; I’d love your thoughts as always.

About susanetlinger

Industry Analyst at Altimeter Group
This entry was posted in Big Data, Predictive Analytics, Research, Social Analytics, Social media, Social media measurement. Bookmark the permalink.

2 Responses to Social data is the canary in the coal mine for Big Data

  1. richardstacy says:

    There is a Big Problem with Big Data – at least with Big Social Data – that no-one seems to have spotted. This problem is actionability. This is not actionability in a technical or organisational sense as outlined in your post, it is actionability in a social sense. To use data acquired in a social environment, you need to play by social, not technical, rules. And this dictates that you have to get an implied permission to both acquire and use the data, otherwise you end up as a digital stalker – and no-one likes stalkers. The only reason we can use social data at the moment is because individuals have not understood the full consequences of the fact that the social platform owners (Google or Facebook and thus advertisers) can both see and use all of your data. When the implications of this become fully recognised, individuals will revoke their permission – which they will do by abandoning social platforms that don’t allow them to retain ownership and control over the data they share and also by rejecting approaches from data users to which they have first not granted a form of explicit or implicit permission.

    Social data is a social challenge – not a technical or managerial challenge.

    More on this


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s