Monday 12th October 2009 by Stuart Glendinning Hall

A key factor in the success of your community is demonstrating that it can deliver results. This is linked to the fact that the role of the community manager has risen in importance in the private sector as how they can help generate value is better appreciated. Because part of that value is intimately connected to setting up and monitoring the value created through the community and feeding that back to the host organisation, and the community itself. That’s a neat fit on paper but in faced with the busy schedule of community activity what do you look for to measure and monitor and why?

To start to answer that question, when to be frank there’s already plenty of debate about how exactly to best to achieve that, let’s go back to basics. Instead of trying to figure out the return on the investment (ROI), a standard reference point for both businesses and charities, I wish to suggest a more attractive proposition. Wouldn’t it be ideal if the process by which you measure success is also the same one which in turn delivers those results? So that you are measuring for a real purpose and not just to hit targets. And that’s why measurement in the context of key performance indicators work so well, rather than just standalone community metrics from visits to conversions, work so well as they provide the framework for both measurement and performance.

A balanced approach to performance

Working with our clients we ensure that our traditional balanced scorecard approach to setting KPIs matches up with their current framework of objectives and metrics as neatly as possible. The scorecard approach first collates existing objectives under the four ‘perspectives’ of finance, customers, business processes, and internal learning and growth. If most objectives are grouped just under one or two headings, this highlights gaps in a balanced business approach. Once that hurdle is overcome the benefits of the approach become clear. By simply following the structure of setting objectives under the four perspectives, setting suitable metrics (quantitative and qualitative), targets based on a quarterly review, capturing results and taking action based on these results, you are creating both a means to measure and a means to improve performance. It helps at the beginning of the process if each department considers how its existing objectives can contribute to community KPIs. But it may be this integration takes place later on in the community development cycle too. At the other end of the spectrum it helps if a named individual is made responsible for addressing performance based on the results review.

Nothing personal, it's just business

One aspect worth highlighting is the value of collecting qualitative as well as quantitative measures of success against metrics. Certainly from the viewpoint of a recent marketing poll, for business-led communities, the metrics of success are spread across both subjective and objective measures. Indeed the results of the poll discovered that for marketers when asked for the most important metric to track in social media, the value of ‘virality’ and ‘sentiment’ predominated, not ROI:

• Virality (the reach of your brand and how much your message is spread), 35%
• Sentiment (positive, negative or indifferent consumer reaction), 32%
• Financials (the effect social media has on your bottom line), 20%
• Volume (number of comments, blog posts, tweets, links, etc. about your brand), 11%
• Other, 2%.

And indeed what business-led communities and communities for membership organisations have in common, is the desire to move away from communication with customers/members based on transactions to one based on relationships. How better to start working towards that strategic aim than by measuring the likes of sentiment as part of your KPIs? And let’s face it ten tears after the social media bible Cluetrain’s famous adage that ‘markets are conversations’. Again business is leading the way, take this example from June 2009 in which Irfan Kamal describes the approaches Ogilvy PR are taking to make sentiment analysis work for clients, driven by the desire to uncover the patterns of consumer action and take full advantage of it.

Looking for golden nuggets

What can member organisations learn from this? Firstly, to reiterate that setting up sound KPIs not only helps measure progress but also helps drive performance which results are linked to action. Secondly, that as understanding of social media’s power and scope develops that qualitative measures such as ‘sentiment’ not only need as much prominence as traditional metrics, they are starting to lead the way to extracting value from communities. A practical approach to this is for community managers and moderators to keep a track manually of positive conversations, for example counting the number of frozen discussions where a clear positive ('nuggets') or negative conclusion has been reached. That's something we cover in the SiftGroups Community Healthcheck programme for example. A less directly provable measure could be a reduction in the number of issues dealt with by your customer service desk, but it's that's also an example of how it can be hard to isolate what was the causal factor behind an improvement. When budgets allow there’s also the opportunity to test out sentiment using surveys, whether systematically scientific market research approaches, or through more informal ones at your own events and conferences.

Responding to the need for conversions

Picture credit: http://blog.wiserearth.org/a-simple-model-for-community-engagement/

Finally in terms of sentiment from a community manager’s viewpoint it’s also worth thinking about conversion rates, a particular theme for membership organisations. For example it’s a common desire to show how communities can help not only retain existing members but also convert low fee paying student or guest members into full paying members.

In theory this should be not such a tough task as community management success depends on encouraging visitors to become members, and members to become contributors as the graphic above shows. Indeed SiftGroup’s own core community metrics includes a key measure of the numbers of readers to posters, so community managers can better monitor and respond to that conversion need. Of course it's an important caveat to realise the needs of a community change over time as it matures, and this tool is designed to be simple and flexible enough to support that development. After all as Dennis Howlett underlined in a recent blog "it takes much more time and commitment than people realize in order to get somewhere you can say there is enough data to return sensible measures".

 
Plotted above based on 90-10 is a linear relationship between readers and posters starting st 90 readers to 10 posts, and in the top right ending with 900 readers to 100 posters' contributions. At intervals the stats for readers and posters are plotted against this line, showing for example in 'Discussions' that there is a disproportionate amount of posters.

A similar strategy to target students to become engaged, and to benefit from the community, therefore also needs to go hand in hand with measuring conversion rates, and again taking action on the basis of regular reviews of targets. 

It’s not rocket science, but once you’ve done the work to set it up and get it working as part of your community KPIs the simple act of measuring and monitoring will in turn drive the performance you require to show that all important return on investment.