Let’s End The Magical Thinking About Social Media ROI

Last year at SXSW, I participated in a panel on social media measurement with Margaret Francis of Exact Target, Megan Costello of Crimson Hexagon, Blake Robinson of CNET and Chris Lightner of Edelman, called “Advanced Integrations of Social Media Analytics.” In retrospect, not the most user-friendly of titles, but it was essentially a workshop, with framework, examples and tips on social media measurement.

We prepped intensely, meeting weekly to run through the most recent and useful examples, tips and tools we could find. Our slot was the highly-coveted Tuesday at 3:30 session, by which point, we figured, we’d have to yell over the sound of the vacuums to the one hungover guy who’d missed his plane and was snoring in the back. The room was packed; people need and want to know this stuff.

In the interim, I published my research report, “A Framework For Social Analytics,” in which I interviewed about 40 brands, vendors and agencies on how they measure the business value of social media. It’s an emergent, imperfect discipline, still as much art as science, but there’s no question: companies are measuring social media, many with rigor, significant investment, and most importantly, useful results.

All of which is to say: Social media is measurable. It can be done. To say social media can’t be measured is a cop-out, and dangerous for anyone who works in an organization that, say, plans to make or raise money at some point in the future.

This year, I thought it would be interesting to go to SXSW as a tourist and see as many sessions and meet as many people as I could. I met with a bunch of companies including some interesting stealth startups whose ideas I’m sure we’ll be talking about next year. They take the measurement of social extremely seriously, and they also understand, as is a foundational philosophy at Altimeter Group, that it is innately driven by relationships. These ideas co-exist happily in their minds and business models.

So I was disappointed to miss Olivier Blanchard at today’s Social Media ROI panel, even more so to hear people whose opinions I respect recycle the same magical thinking about social media measurement that I thought (hoped?) we’d dispensed with already. Olivier quoted a lot of it on his blog post today. Here’s a representative example:

“Asking if there is ROI for Social Media is like asking if there is an ROI of the telephone or a pencil.”

Actually, no. It’s not. Social media is different. It’s not “just another channel,” because telephones and pencils simply don’t empower consumers the way that social media does.

Asking about the ROI of social media is a legitimate question, and the people who paid money and took time to attend the panel today (as do the people who funded their trips and programs and are expecting some insight when they return) deserve thoughtful and specific answers. I can think of a couple dozen people working in this space as analysts, authors, consultants, technologists and practitioners who would have sent the audience away with plenty of valuable examples. Olivier has written a book on the subject, I’ve written a research report (and am starting work on another), and there is no lack of serious work in this area. Really.

So, for the last time, we need to dispense with this magical thinking about social media; that somehow it exists on a plane beyond accountability or understanding. Jeremiah Owyang recently released a report on social media management systems in which he found that the average company he surveyed has 178 social media accounts. Are we really naive enough to believe there was no investment in these properties worth evaluating? Will they magically receive funding next year? Do we even know what they are?

We measure lots of things, and they seem to withstand the process perfectly well. We learn from what we measure, and we get better as a result. It’s our responsibility, whether we work for a magazine, an oil company, a global charity or ourselves.

So, whoever does this type of panel next year, I challenge you to treat these questions–much more importantly, your audience–with the respect they deserve. We’ll all be the better for it.

</end rant>

About susanetlinger

Industry Analyst at Altimeter Group
This entry was posted in Predictive Analytics, Social Analytics, Social media, Social media measurement, Uncategorized. Bookmark the permalink.

19 Responses to Let’s End The Magical Thinking About Social Media ROI

  1. Tac Anderson says:

    It is sad we still have this conversation. Two years ago I wrote about this http://www.newcommbiz.com/social-media-is-not-a-telephone-or-pants/

    My main argument:
    “Having a phone at work is a cost of business. At one time having phones at everyone’s desk was something that had to be justified. Now having a phone is a cost and managed accordingly. Do you really want social media to be managed by cost and not by return? Guess which gets more budgets?”


    • Harold Schambach says:

      Very well put. I attended both panels you mention above (one in 2011, one yesterday) and could not believe what I heard in yesterday’s panel. Just because something is difficult to measure, doesn’t mean we shouldn’t try. Keep fighting the good fight


  2. Keith Paul says:

    You can rant any time 😉


  3. Thank you Keith and Harold. I was hoping we’d moved past this issue, but apparently not. So yes, I’ll keep fighting the good fight. Really appreciate your attention and feedback!


  4. Eric Swayne says:

    Hi Susan,

    Greatly respect your work with Altimeter, and since I moderated this panel, I wanted to offer one point of clarification. Like you, I have a deep physical nausea when we talk about social ROI in terms of the ROI of a pencil / telephone / email / carrier pidgeon / other obvious objects. Liz (Strauss) holds these opinions, and while she’s a fantastic personal friend, I couldn’t disagree professionally any iota more. Unfortunately, many of the individuals there picked up on her soundbites, shared them, and many on the outside felt this was the recommendation of the entire panel. I can 100% assure you it was not. Liz’s role was to provide diversity of opinion – something the SXSW staff consider heavily when selecting panels, and a point of view to be offset by the rest of the panelists. I am indeed disappointed we weren’t able to share that counterpoint with the audience in such a way that it carried beyond the walls of the meeting room.

    Thanks –



  5. To all respondents. If someone wanted one (one) recommendation for a social media measurement, publishing and marketing platform, what would your answer be?


    • Hi Chris. It depends on your business objectives, where you sit in the organization, resources, marketing strategies and tactics, other tools in the mix; it’s not an easy answer. The other thing to consider is that this is a fast-consolidating set of four tool markets: monitoring, measurement, publishing and content management; that means that most companies have to make serious choices and tradeoffs based on a number of criteria. If you’d like to discuss further, please reach out to me at my altimeter group address. Thx!


  6. Thank you. That is all. 🙂


  7. You can get the full context of the panel here Susan. http://storify.com/mattridings/a-case-study-in-crossing-a-line . It was a long ways from perfect, but a little more balanced than you’ve represented here I think 🙂


  8. Eric and Matt – I’m really glad you posted here, because this is such an important discussion to have. I respect that you’re continuing the conversation rather than retrenching.

    I want to clarify that I did attend the panel (I was on the floor at the front) so I heard all that was said. What I couldn’t tell, because of where I was sitting, was who was saying what. And I also want to emphasize that I respect you all individually and have for some time, but that doesn’t change my opinion that the panel as a whole didn’t deliver on what it promised. The promise was to “explain the metrics you should be focusing on,” but the conversation veered off early on into a debate about what social media ROI really means, which unfortunately sent things into the weeds.

    I completely empathize with the time constraints and panel size, the fact that you unexpectedly lost Olivier and that Craig was sick; I’ve been there and I know how tough it is. But at a minimum the panelists should have anticipated that the ROI question needed to be addressed quickly and crisply so you could move on to what people were really there for: case examples and recommendations of how companies like theirs actually measure social media results. And while there were some insightful and important points (like Craig’s about opportunity cost), and a couple of useful examples of how Make-a-Wish is measuring the business value of social, they got lost under the weight of social media platitudes, which didn’t provide the kind of value and insight that the audience came for.

    Anyway, my two cents. These things happen; sometimes these events take on a life of their own. In any case, I really appreciate you reaching out and am happy to discuss further if you’d like.

    Thanks again,


  9. See Susan, that’s an opinion I can totally respect. I’ll be blunt and own up to the fact that I can’t speak to what the session promised, I knew the title of the session (What’s so bleeping hard about ROI?) and came prepared to react and let the discussion lead me vs. leading the discussion. It’s a topic I speak on frequently so perhaps I leaned too much on my belief that my experience in single presenter and workshop environments would translate just fine to a panel such as this. I have no doubt that I could have spent more time trying to coordinate with Eric, my schedule was busy but I had 2 days advance notice so it’s not like I couldn’t have found a way.

    The challenge with large panel venues like this from where I sit is how to balance the higher level viewpoint that someone with more maturity on the topic has such as yourself, with that of the throngs that came up afterwards who only heard those things that validated what they already believed (yes, many platitudes included). When I say something like ‘benchmarking, activity isolation, and correlations still work just fine’ and find a third of the room have no idea what I mean, I’m not sure where to go with that. Do I try and educate (and dominate the other panelists time in the process?) on that or move on and add value to the others who did understand?

    We could pretty easily eliminate much of the time ‘wasted’ by finding panelists who all agree with one another and are at the same levels, then insuring that a panel description filters the audience down to those at the appropriate stage so that the message adds value to all. But at that point isn’t a single presenter in a workshop setting more appropriate? (which is what I’m typically in) In essence, we’d need to remove any debate from the panel. Given the panelists in this case I’d have to say that it was put together for the express purpose of encouraging that debate. Whether that was the right move or not, it dictates the outcome in regards to the amount of depth that can be attempted.

    All of that is mainly a theoretical question about these panels in general more than any kind of excuse for ours in particular. For ours specifically let me apologize to you that you didn’t get what you expected out of it. That’s obviously never what I want to hear.

    Cheers, and thanks for the constructively positioned feedback.



    • Hey Matt,

      Thanks so much for this. I think you’ve hit on a really important issue here, and one that has more to do with SX itself: the need to curate these panels and clarify which are educational versus debate-oriented. My belief is that the time to debate whether we should be measuring the value of social is long gone (I’ve written ad nauseum about this) in much the same way as we should be past the point where we are questioning whether social media is of value at all. Yes it has tremendous value for business. Yes we can prove that. So we should.

      I have a huge sense of urgency about this because I hear it all the time from people who are under intense pressure to justify all the time and expense that their companies have invested, both to maintain social media budgets and to increase them,

      So maybe that is the lesson here; that we need to clarify expectations first. And sometimes it all goes to hell anyway, but at least we tried. Thanks for hanging in there to bash this out.


      • What you call ‘hanging in there’ I call ‘trying to always improve’, but I appreciate it 🙂

        And thank *you* for knowing how to disagree without being disagreeable. That sometimes gets lost in these exchanges.

        Hope the rest of SxSW was productive for you.



  10. Totally was, thanks. And I think this was productive too; I learned something and got to know you a bit better. That’s a win in my book.


  11. Pingback: Google Addresses Revenue Impact of Social Media with New Social Reports | Thought Experiments

  12. Pingback: Google Addresses Revenue Impact of Social Media with New Social Reports

  13. I thought Eric did all he could to keep this panel on point, and that getting a group to stay focused can be difficult and sometimes not possible. I also think they stayed true to the title of the session, which implied a discussion, not a demonstration of how to make it work. The mistakes I see being made is trying to measure Social engagement with the same tools we measure every other digital touch point. In my view email, search, even banner ads, have spoiled marketers into thinking everything can be and must be measured with the same metrics used to gauge success in other mediums. So instead of looking to what is most important to measure, and how, too many are trying to figure out how to measure everything we are accustomed to measuring, but in a new medium. Initially as you are building your Social Media audiences, and testing, I have three stages with which I measure… #1 is Audience growth, #2 is Reactivity… getting them to take an action (can be as simple as a RT, reply, or clicking on a link), and #3 Stickiness… keeping them coming back, engaged and interacting.These are not difficult to measure and do not require rocket science.

    I am a relationships guy, in that camp, and a firm believer that what differentiates the value of social is the relationship building… interactions, engagement, emotional connection. But we can NEVER forget Metrics are incredibly important… what are the numbers we’re aiming for? What will tell us if it (the implementation and use of the tool / process) was a success? How will we get that information and make sense of it in a way that can inform our business strategy? These same questions – plus a few new ones — need to be asked as we begin taking social media integration seriously in our businesses and marketing strategies.

    Defining metrics around social media advertising and marketing campaigns has been challenging enough that for a while many people said it simply could not be done… or adhere to the “why measure, we know it works” philosphy. Now, however, we are learning that social media measurement (re: use and impact) IS possible – just not always using traditional metrics and methodologies. But remember it is not just the practitioners who are muddying the pond, but the brands who are desperate to keep measurement in the realm of familiarity.

    Let’s measure what we can measure with tools we are accustomed, and then be sure to look outside the typical measurement box and find ROI where we may not be used to looking. And most important keep looking for a viable solution/solutions, share what we do and discover, and remember that measurement will be different for all depending upon goals, what other marketing mediums we are employing, and how we integrate.


  14. Pingback: Let’s End The Magical Thinking About Social Media ROI | Social Intranet and Mobile | Scoop.it

  15. Pingback: Altimeter Group - Research and advisory for companies challenged by business disruptions

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s