Social data is the canary in the coal mine for Big Data

I spoke at the Boston eMetrics Symposium yesterday on the subject of Big Data; specifically, how we will build organizations that can adapt to it, extract insights and act on them in a meaningful way.

People generally discuss Big Data in technical and even aspirational terms (see this month’s Harvard Business Review dedicated to the topic), but there’s less attention to the organizational impact. Who will have access to it? What will we do with it? What challenges will we face? Who, if anyone, will “own” it? And how big a change is this, actually?

Let’s be clear: Big Data–or at least elements of it–is here already. We’re seeing it in a focused way in some of the case studies that HBR mentions. One example, in “Big Data: The Management Revolution,” addressed the topic of plane arrival times, which pilots generally (and often incorrectly) estimate. The issue is that errors either direction produce productivity impacts that translate to significant economic impact. So one major airline improved its ETAs by engaging PASSUR Aerospace to crunch tons of data on plane locations, weather patterns, flight schedules, along with proprietary data, greatly improving arrival time estimates.

But we’re also seeing the effects of Big Data beyond the type of highly sophisticated decision support application above. Social data, after all, is a type of Big Data. It meets the “Three Vs” criteria: Velocity (speed), Variety (as IBM says, “text, sensor data, audio, video, click streams, log files and more”) and, most certainly, Volume.

Social data is the “canary in the coal mine” for Big Data: we can learn a lot about how Big Data will thrive (or not) by looking at what has happened with social data thus far. I covered many of these issues in my research reports, “A Framework for Social Analytics” and “The Social Media ROI Cookbook.” Consider this (all data courtesy Altimeter Group):

• The average enterprise-class company has 178 social media accounts
• The average enterprise-class company has 13 departments actively engaged in social media
• Fifty-six percent of companies we surveyed stated that the biggest challenge to measuring revenue impact of social media was “inability to tie social media to business outcomes.”

Some of the other impacts we see, both quantitatively and via anecdotal evidence, have to do with the strain on organizations as they seek to integrate social data into decision-making:

• Lack of analytics expertise
• Poor or disparate tools
• Inconsistent analytical approaches
• Unreliable data

From a maturity model standpoint, we’re seeing this (rough) progression for social data organizations:

One drawback of this progression, at this point, anyway, is that it encompasses only social data, which threatens to become a silo of its own if it is not integrated with enterprise data such as CRM, Business Intelligence and Market Research data, not to mention the many varieties of Big Data coming from sensors, mobile devices, the Web, and so forth. We’re seeing early signs of this as companies such as SAP, eBay, Oracle, Adobe and Salesforce partner with or acquire technologies that integrate social and enterprise (and ultimately even external economic) data.

Some people have proposed the idea of a “Chief Data Officer” to oversee this process, and that is likely a fine idea, but not in isolation. While it’s a tiny sample size, most of the heat during and following my talk focused on the cultural impacts of Big Data: Who owns it? How do we share it? And how do we make the transition from a culture where decisions are made by “HiPPO” (Highest Paid Person in the Organization) to one that is data driven?

It’s a bit of a chicken-and-egg scenario (to continue the bird metaphors) as organizations need leadership to make the leap of faith needed to invest in data-driven organizations, and leadership needs the data to convince them that this is a leap worth taking.

Plenty more to mine here; I’d love your thoughts as always.

Building a Measurement Organization for the Big Data World from Susan Etlinger

Before You Freak Out About Pinterest and the Zappos Study…

Image courtesy Stock.Xchng

A lot of people have been buzzing today over this Bloomberg story both announcing Zappos’ launch of a recommendation engine based on Pinterest pins, and stating that–although Pinterest drives the highest number of shares compared to Facebook and Twitter–it also contributes the lowest revenue of the three.

Several publications picked up the thread this morning (Laura Hazard Owen of GigaOm wrote a story including some pointed criticism of the Zappos recommendation engine, while Business Insider posted this widely-shared piece), and the takeaway–that Pinterest drives low revenue for brands–has been widely circulated on the interwebs.

This is odd on several levels; why would Zappos tout a new recommendation engine on the one hand, while admitting it’s not effective at driving revenue on the other? But before we jump to conclusions, let’s take a look at the Bloomberg story and ask a few pointed questions.

1. What was the methodology of the study?

This is a tough one, because we don’t have the data or know the methodology of the Amazon/Zappos analysis. Some questions to consider:

• How many sales/users does this study represent?
• Do these numbers represent average order value per social platform, or average order value per share?
• For that matter, how was a share defined and calculated? Is a pin being equated to a tweet or a facebook wall post? What are the precise metrics being compared?
• What was the time frame of the study? A week? A month? A quarter? More?

2. What methodology was used for conversion attribution?

The way the conversions were attributed will have a dramatic impact on the results. Questions to consider:

• How was the data tracked? Was it tagged? If so, and if the analytics tool is using last-click attribution, the conversion rates could be (not saying they are, but could be) under counted because of the fact that people rarely see content and convert (i.e., buy) within a single browser session. Sometimes you see that sweet hat (shirt, painting, necklace, muscle car, mansion, whatever), then you go away, come back and buy weeks or even months later–on another device, even. (Note: this is why time frame of the study is so important). On the other hand, if you did an A/B test (Within a six-month period, Pinterest users spent an average of $X, non-Pinterest users spent an average of $Y), you could get a different view of the revenue generation value of Pinterest vs. non-Pinterest users.
• What is the content of the posts from Facebook and Twitter?  Is it just straight shares, or does it include coupon-related affiliate links? You need to know whether this was an apples-to-apples comparison to determine the relative revenue generation performance of the social platforms. No fair to compare a share to a coupon, whatever social network it comes from.

3. How accurate is the recommendation engine?

It’s too new and we have too little experience to have a real sense of this, but it stands to reason that the quality of the recommendation engine will affect conversion results.  Here’s an example from the GigaOm story: the fact that I pinned a black-and-white dress does not actually make me more likely to buy a men’s sky-blue tie. But maybe that’s just me.

4. How do people usually behave on these social networks?

Here’s where we need to think about socialgraphics; that is, how people actually use social networks. Some points to consider:

• Age of the Network. Facebook and Twitter have been around for several years. Pinterest is still relatively new, with a lot of new users who are still figuring it out.
• Behavior on the network. Twitter is a fast-moving 140 characters, the Facebook Timeline passes quickly but allows for more editorializing, yet Pinterest allows for a more permanent record of one’s preferences.  As a result, something I pin in September might influence your purchase in March. You cannot track that with last-click attribution, so Pinterest would not get credit for that sale. You have to ask: does that seem logical, given how high the sharing behavior is on Pinterest?  Again, we don’t have the data, but a starting question might be: do one or more social networks have the advantage when it comes to last-click attribution? That could explain why Twitter scored so high.  Is there another way to prove revenue impact across Pinterest, Facebook and Twitter? Does it tell us something different, or does it confirm what we found the first time? How does it compare to other studies, such as the research released earlier this year by Shopify?
• Relationships on the network. How close are the ties between Facebook friends, Twitter followers, Pinterest pinners? We are still very early on in our understanding of how people in a social graph are influenced by each other, relative to how people in an interest graph behave.

Finally, there are just too many unanswered questions to draw a conclusion about Pinterest based on the Amazon/Zappos data. But this story raises some really important issues about how we measure revenue impact of social media, because it highlights the analytical traps that we’re vulnerable to if we assume that all networks behave in the same way.

My advice, whether you’re concerned about this particular study or the revenue contribution from your own social media programs, is to use this example as a thought-starter to make sure that you are asking the right questions, drawing useful results and–of course–making informed decisions.

For more information about how brands are determining the revenue contribution of social media, please see The Social Media ROI Cookbook.

I’d love your thoughts. Any questions you have? Points I missed? Please tweet, comment or share however you prefer.

It’s [Always] Time to Re-examine Social Media Risk

Courtesy sxc.hu

“Imagine the 1:35 a.m. call informing you that your new product scheduled for release next week has been accidentally leaked because of some Flickr images. Or the 5 p.m. call that the regulators are going to be in your office first thing tomorrow because of a response that one of your customer service reps tweeted to a customer.”

In his research report, “Guarding the Social Gates: The Imperative for Social Media Risk Management,” Altimeter Group analyst Alan Webber lays out those two chilling scenarios and more.  While the ability for customers, partners, employees and others to communicate with and talk about your brand is a tremendous opportunity, it carries significant business risk.

Here are the four top risks, according to Alan’s research:

1. Damage to brand reputation
2. Releasing confidential information
3. Legal, regulatory or compliance violations
4. Identity theft or hijacking

But what do we really mean when we talk about social media risk? Alan defines it this way:

The likelihood that a negative social media event will happen
(multiplied by)
The impact that negative event will have if it does happen 

The report lays out a framework–along with pragmatic recommendations–to help you identify, assess, mitigate and evaluate these risks.

Keep in mind, however, that as technologies, your social media strategies, the regulatory environment and even the political climate changes, your risk profile changes as well. This is not a one-and-done effort; it’s an ongoing function that requires a holistic approach to risk in the context of social strategy.

If you doubt this, take a look at today’s Washington Post story about how children’s advocacy groups are urging the FTC to investigate a number of large brands for allegedly violating violate the Children’s Online Privacy Protection Act (COPPA).

Here’s Alan’s report, available for download. Please use it to protect your customers, your company, colleagues and partners and your brand reputation.

Guarding the Social Gates: The Imperative for Social Media Risk Management from Altimeter Group Network on SlideShare

Shut the Front Door! Here Comes Media Convergence

One of the knottiest topics we’ve been discussing at Altimeter Group lately is the way media are converging. Back before the social web and the ability of consumers to share and publish their own content, there were pretty clear lines of demarcation: advertisements (paid), marketing collateral (owned) and press clips (earned). And the twain didn’t really meet.

Today, that’s all changed. Here’s a TV commercial for Oreos that aired not too long ago.

So, what is it? Paid? Owned? Earned?

Yes.

• It’s certainly paid, in that it’s a commercial that aired on network television;
• It’s owned, in that as soon as you click you are redirected to YouTube;
• And it’s earned, in that I just shared it with you, and was responsible for your click.

Jeremiah Owyang and Rebecca Lieb recently published a research report, “The Converged Media Imperative: How Brands Must Combine Paid, Owned, and Earned Media,” that takes on the questions surrounding how advertising and media are converging, how this convergence is disrupting marketing, and what marketers and their agency partners must do to unlock the value of converged media within their organizations.

This is an area I discussed in some detail in “The Social Media ROI Cookbook“; the fact that measuring media now depends on the state in which you find it. I’ll be working with Rebecca and Jeremiah to flesh out the measurement implications over the next several months; in the meantime, here are several resources to spur your thinking.

Rebecca and Jeremiah will be hosting a webinar to discuss the results of their research on September 13 at 10 am PDT.  To register, click here, which will make that webinar and report both owned (via Altimeter) and earned (via this blog post).

Here’s the report, to read, download and share:

Does the Social Web Perpetuate or Expose Disinformation?

This is the fake NYT Op-Ed
http://www.opinion-nytimes.com/2012/07/29/opinion/keller-a-post-postscript.html

I woke up Sunday morning to a tempest on Twitter. Apparently, Bill Keller of the New York Times had published an op-ed defending Wikileaks. It was retweeted by none other than Jeff Jarvis, Dan Gillmor and Nick Bilton–some of the top media minds in the country–until, not too many hours later, it was discovered to have been entirely fabricated.

Keller repudiated the post (in ALL CAPS), Jarvis, Bilton and others rapidly apologized for having been taken in, and Harvard Berkman fellow Zeynep Tufekci, NSA whistleblower Thomas Drake, Reuters columnist Anthony DeRosa and dozens of others immediately began sleuthing to determine what had happened. Tufekci discovered a fake New York Times domain name registered on March 30,  tweeting “This was in the works for a while.” Later, she added that “Whoever did this also had http://blocknytimes.org  registered.”

We’ve seen similar dynamics many times before.  Nearly a year ago, rumors of Muammar Gaddafi’s death began trending on Twitter before being retracted. Numerous celebrities–including Brad Pitt, Eddie Murphy and Robert Pattinson–are regularly the targets of death hoaxes.

Here’s a timeline of the Gaddafi death rumor of August 21, 2011, courtesy of Topsy.  (Gaddafi actually died in October of that year):

Source: Topsy Labs
http://topsy.com/s/from%3Amalikalabdeh+gaddafi?offset=31&om=baa&order=date&page=4&window=a

Of course, it’s not only politicians, journalists and entertainers who are vulnerable to confirmation bias and the accelerating effects of social media. Businesses are as well. Snopes.com is rich with examples of urban legends that have sparked public imagination, dread and disgust. Did you know, for example, that bat excrement is a secret ingredient in mascara? That Lucky Strike cigarettes were so named because some packs contained marijuana cigarettes? Right, because they’re not true.  We can all think of examples–from last week, from childhood. Thanks to the tireless folks at Snopes, and now others on the web, they eventually get corrected.

So, does the Internet make disinformation worse or better?  Yes. As we see from yesterday’s New York Times kerfuffle, not only can you spread disinformation quickly via social media, but you can detect it too.

Says Rishab Ghosh, Chief Scientist, Topsy, “In social media you can track where a message came from, see who the first person who tweeted it out was and whether they are reliable. You don’t have a guarantee of whether it’s true or not, but you can see how it started.”

So, in that spirit, here are three simple things Ghosh recommends you do to identify disinformation on the web:

1. Check the URL. The mistake Jarvis, Bilton and others made before retweeting the erroneous NYT op-ed was to forget to check the URL. Had they done so, they would have recognized that is wasn’t legitimate. The fake column began “http://www.opinion-nytimes.com…” while real New York Times Op-Eds look like this: “http://www.nytimes.com/pages/opinion/index.html”.
2. Check the story independently against the real source.  The story did not appear on the New York Times Opinion page.
3. Check the Twitter account. The story was tweeted by a fake account attributed to Bill Keller.  At first glance, it looked like Keller’s actual account (@NYTKeller) but turned out to be, according to Nick Bilton, spelled with an uppercase “I” and a lowercase “l”. Tricky, tricky.

Source: Flickr, Nick Bilton
http://www.flickr.com/photos/nickbilton/7668795876/

To be fair, this was a pretty elaborate fake, and it took in, albeit briefly, some of the most sophisticated media folks in the country. So the lesson here is this: any of us, no matter how smart, is susceptible.

Disinformation is a real business risk, and, like other sorts of black-hat tactics, is becoming more sophisticated all the time.  We are seeing promising advances in influence detection algorithm technologies, and soon technologies will emerge that will use machines to help flag the characteristics of reliable versus unreliable content. Until then, a healthy amount of skepticism and a few spot checks are in order.

I’d love your feedback, tips and tricks for identifying false content. Please leave your ideas in the comments, or if you post elsewhere, I’ll link back to you.

The Social Media ROI Cookbook: Measuring the Revenue Impact of Social Media

Everyone talks about the challenges of measuring the revenue impact of social media, but how are top brands actually doing it? And are they successfully measuring ROI?

My colleague Andrew Jones and I spoke with 16 brands, 38 vendors, 3 agencies and 4 ecosystem contributors, and surveyed 71 social media and analytics practitioners to answer this question and provide pragmatic recommendations.

The result is our  “Social Media ROI Cookbook,” which aims to identify and describe the most effective “recipes” for measuring the revenue impact of social media that we have seen adopted to date. Here’s what we learned:

Only 30% of brands we surveyed consider themselves to be “effective” or “extremely effective” at connecting social media to revenue. The top three challenges: Inability to tie social media to business outcomes (56%);  Lack of analytics expertise and/or resources (39%); Poor tools (38%).

To be fair, we are in a period of transition: Jeremiah Owyang’s “A Strategy for Social Media Proliferation,” published earlier this year, showed that “companies average 178 corporate-owned social media accounts.” At the same time, the research for this report found that 63% of measurement organizations surveyed have 1-2 people focused on social data. That’s not a healthy ratio.

While revenue is important, it isn’t everything. 84% of survey respondents reported that the primary business impact of social media was not revenue generation, but “insight that helped us meet customer experience goals.”

Altimeter Group identified six primary top-down and bottom-up approaches and developed three case studies that illustrate how organizations measure the impact of social media on revenue. The best practices blend top-down and bottom-up measurement methods to provide a holistic view, informed by granular evidence.

While these six ingredients are consistent, the emphasis each company places on them depends on the nature of their business. There is no “one-size-fits-all” approach. The four most important criteria for determining the measurement mix are:

• Business: the nature and structure of the business
• Product: the nature and type of products or services offered
• Media: type of media being used
• Customer: the nature and type of customer(s)

Where’s it all going?  There are four main themes: mobility (to connect the online and offline worlds); digital identity (to connect the consumer wherever s/he may be); More, bigger, faster data to demonstrate as well as predict influence; and, finally, integration, integration, integration: with other social data, with enterprise data and with external data.

The truth is, we’ve only just scratched the surface of what we will be able to know and understand about the impact of social in the years to come, and the process of writing this report emphasized the gaps over and over. But we have to start somewhere. As Ken Burbary, Chief Digital Officer at Campbell Ewald admits in the report, “I feel like we’re all data chemists at this point, trying to put a bunch of stuff into our beakers to see if it works.”

The Social Media ROI Cookbook from Altimeter Group Network on SlideShare

As always, we’d love your ideas, questions and thoughtful disagreement; after all, this is an ongoing project. Please leave a note in the comments, or if you choose to post about the report, we’ll link back to you.

Many thanks to all the ecosystem contributors and colleagues who made this report possible.

Related Posts

• Campalyst, “The Social Media ROI Cookbook,” a study by Altimeter Group
• Adam Singer, Google, writing for TheFutureBuzz: Worth a Read: Altimeter Social Media ROI Cookbook
• Jay Baer, Convince and Convert: Not Tracking Social Media ROI is Your Fault
• Ekaterina Walter, Intel, writing for The Next Web: Measure your brand’s effectiveness with The Social Media ROI Cookbook’s recipes
• Jay Baer, Convince and Convert, and Eric Boggs, Argyle Social: Social Pros Podcast 26: Real People Doing Real Things in Social Media
• Stacy Lukasavitz, Whirlpool: Altimeter finally ‘splains the social media ROI situation and gives you a recipe!
• Barry Levine, CMSWire: Social Media ROI Cookbook Offers ‘Recipes’ for Measurement
• Kyle Lacy, ExactTarget: How Brands are Measuring the Impact of Social Media ROI
• Eric Boggs, Argyle Social: Thoughts on Altimeter’s Social Media ROI Cookbook
• Lisa Joy Rosner, NetBase: The Elephant in the Room: Social Media and ROI
• Jeremiah Owyang of Altimeter: Altimeter Report: The Social Media ROI Cookbook
• Keith Paul of EMC: Eating from the Social Media ROI Cookbook

Working with Industry Analysts: Eight Tips for Entrepreneurs

Before I joined Altimeter Group, I worked in Marketing and PR/Analyst relations. During that time, I counseled hundreds of entrepreneurs on how to work most effectively with industry analysts. Now that I’m an analyst myself, I find that most of what I advised then was correct, with a few tweaks along the way as the industry has grown and changed.

This is the substance of a talk I prepared for The Alchemist Series, a series of events, including panel-driven discussions and lectures, that explore significant milestones in the trajectory of the development of a business.  The deck is based on my experience on both sides of the industry analyst fence–as well as the input of my colleagues at Altimeter Group. To that end, I’d love to hear from other analysts and, of course, entrepreneurs themselves, about experiences and questions they’ve encountered along the way.

Thanks, feel free to share, and I hope you find it useful.