Ed Mitchell over at Platform Neutral picks up on the issue of Return on Investment (ROI) for on-line communities, quite rightly identifying this as growing topic for debate and argument over the coming months. In Ed's words:
"ROI for ‘communities’ is going to appear on our horizons for proper this year. We will have the debate from the sponsors asking if it’s worth it, and another one about which way around it should be - ie: ROI for whom? The sponsors, or the participants. Likewise there is much talk about the ROI of ’social media’, and all of the projects I am working on have measurement built into them at the strategy level"
Personally, I haven't quite cracked this nut for the virtual communities using the IDeA platform (240 of them at the last count). Since many of the communities are closed, self-organising networks, I only really see the platform-wide metrics. So, for example, I can see:
- Total number of communities
- Total registered users
- Total contributors
- Members per community
- Total topics
- Threads with responses
- Responses per thread
- Participating users
- Total number of blogs
- Total posts
- Total comments
- Total number of Wikis
- Total edits
- Articles per community
- Total number of documents
- Total number of document comments
- Number of Documents per community
- Total number of messages
- Number of users sending messages
- Number of users receiving messages
- Total events posted
- Events per community
I can get the same raw data from each individual community (but it's a tedious process doing this for 240 communities!) - though this does at least give a more accurate indication of the health of the individual community (where my definition of 'health' means 'activity'), but none of this information can provide me with a 'value' or ROI - no matter how I slice and dice the data.
Hence the need for a more qualitative approach, in the form of membership and user surveys etc. Picking up on Ed's comment that all of his community projects have measurement built into them at a strategy level - which sounds like the right approach - it should also be noted that even this can be difficult to quantify in terms of an ROI. The IDeA community strategy was developed to "improve local government services". I can infer from the platform statistics that the communities appear to be active, but I cannot yet connect a specific output (service improvement) to the work of any individual community.
So, allowing for the subjective nature of user surveys, I still think they are more likely to give an indication of ROI if not an absolute measure. If I'm right, the next step is in asking the right questions!
The Online Community Research Network are doing a survey on this, so please have a read, fill it in, and pass it on. The more responses the better.
ROI for social networks isn't nearly that complicated. When Ragan started "MyRagan" and Melcrum started their network, I asked both of them how they were going to measure their success and they both said "membership." I wondered how "membership" translates into revenues, but then both organizations do conferences and the rule for conferences is, the bigger the mailing list the more successful the conference. So the true ROI for those networks, is ultimately tied to the attendance at their conferences. It's as simple as that. What's the desired outcome to the organization? and then you take the cost of the network and subtract it from the revenue benefits.
Posted by: KD Paine | 11 February 2008 at 12:01 AM
Dear Katie,
this sounds great if your business is to do with getting bums on seats, but I'm not really referring to the ROI of the organisation, rather the ROI of the community. In this instance, the organisation is the Improvement & Development Agency, and one of the measures at the organisation level is the number of registered users on the platform (though even this doesn't easily resolve into an ROI). The main difficulty I was trying to communicate is measuring the ROI of a community of practice. There are over 240 of these communities currently active. Taking one at random - say the "Preventing Violent Extremism"- one obvious ROI measure might be a reduction in violent assaults. However, there is enormous difficulty in tracing back an outcome (e.g. reduction in violent assaults) to collaborative decisions or policies generated by the CoP. Quite simply, the data and mechanisms for collecting the data do not exist. Consequently, we have to base any ROI on a purely subjective assessment of how well the CoP is working.
Not sure if this helps or not.
Steve
Posted by: Steve Dale | 13 February 2008 at 04:28 PM