Measuring the Effects of Social Media Marketing

Colin DelanySeptember 21, 200812 comments
Share
Tweet
Share
Email

Cross-posted on K Street Cafe

A tough question came up in a conversation with a visiting group of Danish communications professionals last week — how do you actually measure the effectiveness of social media outreach? At that moment, the questioner seemed to be looking for some grand sweeping mechanism, but I think the reality is much more complicated: how you measure social media depends on what you’re trying to make it do.

Trying to Grab Hold of a Cloud

Here’s the problem: as with so much communications work, the effects of social media outreach can be quite diffuse. Say your advocacy campaign has a video on your issue out on YouTube — how do you measure the influence it has on the public mind? Some thing with that network of activists you’ve laboriously built up through Facebook — how do you find out how much good they’re actually doing you?

In traditional media outreach, we usually rely on proxy measurements to get hints of the answers to questions like that. How many press hits did we get, and in what publications? How many reporters participated in our media briefing calls? On the Hill, the answers are sometimes more concrete — your bill either passed or it didn’t — but most legislative change is (excruciatingly) incremental. Goals are often more long-term than getting a measure passed (or defeated) in a single session, and so we’re back to proxies.

Let’s apply this idea to social media and see what we get. A couple of quick considerations — first, don’t fall into the habit of focusing on what you can measure rather than on what matters. Proxies should be proxies, but they can become ends in themselves when we forget that they’re really just indicators of something more significant. Obsessing about process rather than results is a frequent trap in the nonprofit/advocacy space.

Second, online technologies often suffer BECAUSE they’re measurable — for instance, when a 1% click-through rate for online ads is considered to be good, people can start to question why they’re bothering to advertise on the ‘net at all. What’s not so obvious is that we usually don’t have equivalent numbers for other media (in most cases, you don’t have a mechanism in place to measure the conversion rate for a single print or TV ad), and the effective “click-through rate” for other forms of outreach may be WORSE!

Goals Provide Yardsticks

In the advocacy world, most outreach efforts involve one or more of three basic goals: trying to affect government legislation and/or regulation, trying to build a base of supporters for long-term fundraising and/or grassroots action, and trying to influence the broader public policy discussion. Right away, it’s obvious that the first two are going to have the most concrete measurables: for instance, you get what you want on the Hill (more likely, part of what you want) or you don’t. Likewise, support-building usually equals list-building, and you either build a list or you don’t.

For Hill/regulatory work, the measurables are likely to include counts of grassroots advocacy emails sent, constituent visits to district or DC offices, constituent calls to legislative offices, supporter quotes in mainstream media news stories, and letters to the editor in local newspapers. They may also include analogs to traditional media work, with blog hits and mentions in influential online discussion groups and listserves substituting for mainstream media hits. The most useful hints will probably come from your lobby team, since they can measure staff and member opinions directly — they’re likely to hear about it if you’re flooding congressional offices with calls. The ultimate measure of effectiveness is whether or not a congressmember comes to support your side, though it may be hard to isolate a single cause for the conversion.

Grassroots support-building is easier to gauge, since it almost always comes down to list-building. Regardless of whether you’re building your database through social media outreach, at in-person events, through phone banks, etc., you’re still building a list! Besides the raw numbers of supporters that come in through different channels, you can also look at the proclivity of members to take action — do the people you bring in through Facebook or through that clever online game contribute as much as people recruited through other means, for instance. A list is almost always a means to an end rather than an end in itself, but modern CRM (customer/constituent relations management) tools will let you slice and dice your database to help you get the most out of your outreach regardless of why (or how) you’re doing it.

The most difficult social media goal to measure is almost always going to be your influence on the public discussion. Here we’re close to traditional media work again, and the yardsticks shouldn’t sound too alien to anyone who’s spent time in a press shop. How many times are you mentioned in the top-level blogs in your topic area? How many bloggers were on your conference call, and how many posts resulted? How many times has your online video been viewed, what websites have embedded it on their pages, and was it picked up on Digg and other social news sites? How many people have put your promo graphic or widget on their MySpace profiles? You might also measure your effectiveness by your enemies — how many comments and video responses did your YouTube masterpiece spark, and how many hostile bloggers reacted violently to what you said?

If you’re really making a splash, you may see the results of your work in public opinion polling, but again it’ll be hard to separate out the influence of any one communications channel. A more obvious indicator is the spread of your message beyond the channels you’re using directly, for instance when your online video gets picked up on cable news and is seen by millions.

Let’s Get to Work

Going back to the original question, social media outreach in the political space is more measurable than it might seem at first glance, since you should have concrete goals to hold it up to (and if you don’t have concrete goals, you probably ought to start asking why you’re in this business at all!). It may not always be possible to pin down the exact results of a given outreach effort, but proxy measurements should let you compare the relative usefulness of different strategies, at the very least. Just remember, don’t focus on the measurable and forget about what you’re actually trying to get done.

– cpd

Share
Tweet
Share
Email

(Robot-Selected) Related posts:

Written by
Colin Delany
View all articles
12 comments

  • Thanks for this posting and the useful materials on your site! I agree that it’s always difficult to get useful measures of the impact social media activity in the advocacy / political world. Unfortunately, I think too many groups think of these measures in terms of “quantity” (i.e., how many people are reading the blog, how many people are in our group on Facebook, etc.) instead of quality. The value of social media is that it can help groups reach that powerful one to five percent of people that care enough about an issue to actually do something on it — and that’s when true change can happen.

  • http://www.e-benchmarksstudy.com/

    It’s not social media, per se, but I always find the M+R E-Benchmarks study valuable when discussing this topic, because 1) you see how low an average response rate can go; and 2) it contains arguments about how to evaluate user engagement that you might not have thought about before when trying to persuade the skeptics.

  • Great point on keeping focused on goals in developing measurements.

    After spending 15 years as a consultant working with organizations to shift online policy debates, I reached the conclusion that the lack of real measurements has led organizations to both invest in measurable tactics that have no pubic policy impact (i.e. – sending email to Congressional offices) and continue to focus their resources on less efficient offline tactics.

    In regard to measuring the impact on the public discussion, their is an emerging field (NOTE – I now work for one of the companies in this new filed – http://www.morningside-analytics.com/ ) growing out of academic work on social networks to quantify the movement of issues throughout the blogosphere.

    Measuring the direct impact of issues that are moving through personal email and social networking sites remains an obstacle. However, the growth of publicly accessible blogs provides a barometer of conversations (online and off) that are happening and therefore provides a way to measure the impact of social media campaigns.

    While measuring posts on top level blogs or total links can provide some insight, looking closer at how the issue is then moving and shaping the debate can provide real comparative data on the success of the effort.

  • ROI, metrics and measurement are just a few of the buzzwords on the lips of social media professionals these days. To be sure, we have all experimented a bit with blogs, wikis, YouTube, Facebook and others. The result is more than often visually and aesthetically compelling, but is the impact? Now is a good time to ask.

    In any case, most research agencies (disclosure, I work for one) will tell you that what matters are the objectives you’ve set for yourself, whatever medium you are working with (offline or online). Indeed, influencing public opinion within the context of public policy issues may have different meanings to different advocacy groups. The Tobacco industry may be satisfied with the fact that some research report arguing tobacco may not be primarily responsible for lung cancers made its way to a limited number of highly visible medical blogs or other social websites. However, health adovcacy groups might not rest until their message “tobacco kills” has been hammered in all the online social spaces favoured by teenagers.

    It all boils down to setting concrete and measurable objectives before your campaign starts. Then, all you often have to measure are proxy metrics as the one that counts most, i.e. the evolution of each concerned / targeted individual’s mindset, is often hard to grasp, even with traditional techniques such as opinion surveys. Anyhow, before you wonder whether your action made some impact on your target group, you have to make sure the issue was brought to their attention, through proxies…

    Anthony Hamelle

  • Same here – quality over quantity. While there is value in, say for example, retweets, the value is more about exposure. But having constituents right thoughtful communications that tell personal stories is likely to be more effective. Now the trick will be how to make quality happen at scale.

  • As someone who works in the social media monitoring and analysis space (disclosure – http://www.ensomo.com) I agree that one needs to look at the quality over the quantity of social media mentions. With today’s tools, you can go far beyond the numbers and look at the actual words, phrases and concepts people are using around your candidate or issue. You can find out if people are using the words from last night’s speech. Are they saying them in a positive or negative light? Did an opponent’s ad suddenly change the conversation away from your issues? If there is a spike in the number of conversations you can find out what is driving that and perhaps adjust your key messages to keep that volume at this new higher level. There are dozens if not hundreds of ways to analyze social media data. While the methods of measuring social media success in the offline world mentioned in this article are important, social media itself, if properly monitored and analyzed can offer reams of data on the success or failure of any action taken during a campaign. Unfortunately, the free or low cost tools are not of much help here. There is an investment to be made in the technology and the people with enough knowledge about the campaign and the tool to use it properly, but the amount of actionable data returned more than justifies the expense.