Measuring the value of communities of practice

Ed Mitchell over at Platform Neutral picks up on the issue of Return on Investment (ROI) for on-line communities, quite rightly identifying this as growing topic for debate and argument over the coming months. In Ed's words:

"ROI for ‘communities’ is going to appear on our horizons for proper this year. We will have the debate from the sponsors asking if it’s worth it, and another one about which way around it should be - ie: ROI for whom? The sponsors, or the participants. Likewise there is much talk about the ROI of ’social media’, and all of the projects I am working on have measurement built into them at the strategy level"

Personally, I haven't quite cracked this nut for the virtual communities using the IDeA platform (240 of them at the last count). Since many of the communities are closed, self-organising networks, I only really see the platform-wide metrics. So, for example, I can see:

• Total number of communities
• Total registered users
• Total contributors 
• Members per community
• Total topics 
• Threads with responses
• Responses per thread
• Participating users
• Total number of blogs
• Total posts
• Total comments
• Total number of Wikis
• Total edits
• Articles per community
• Total number of documents
• Total number of document comments
• Number of Documents per community
• Total number of messages
• Number of users sending messages
• Number of users receiving messages 
• Total events posted
• Events per community

I can get the same raw data from each individual community (but it's a tedious process doing this for 240 communities!) - though this does at least give a more accurate indication of the health of the individual community (where my definition of 'health' means 'activity'), but none of this information can provide me with a 'value' or ROI - no matter how I slice and dice the data.

Hence the need for a more qualitative approach, in the form of membership and user surveys etc. Picking up on Ed's comment that all of his community projects have measurement built into them at a strategy level - which sounds like the right approach - it should also be noted that even this can be difficult to quantify in terms of an ROI. The IDeA community strategy was developed to "improve local government services". I can infer from the platform statistics that the communities appear to be active, but I cannot yet connect a specific output (service improvement) to the work of any individual community.

So, allowing for the subjective nature of user surveys, I still think they are more likely to give an indication of ROI if not an absolute measure.  If I'm right, the next step is in asking the right questions!

The Online Community Research Network are doing a survey on this, so please have a read, fill it in, and pass it on. The more responses the better.

Ed Mitchell over at Platform Neutral picks up on the issue of Return on Investment (ROI) for on-line communities, quite rightly identifying this as growing topic for debate and argument over the coming months. In Ed's words:

"ROI for ‘communities’ is going to appear on our horizons for proper this year. We will have the debate from the sponsors asking if it’s worth it, and another one about which way around it should be - ie: ROI for whom? The sponsors, or the participants. Likewise there is much talk about the ROI of ’social media’, and all of the projects I am working on have measurement built into them at the strategy level"

Personally, I haven't quite cracked this nut for the virtual communities using the IDeA platform (240 of them at the last count). Since many of the communities are closed, self-organising networks, I only really see the platform-wide metrics. So, for example, I can see:

• Total number of communities
• Total registered users
• Total contributors 
• Members per community
• Total topics 
• Threads with responses
• Responses per thread
• Participating users
• Total number of blogs
• Total posts
• Total comments
• Total number of Wikis
• Total edits
• Articles per community
• Total number of documents
• Total number of document comments
• Number of Documents per community
• Total number of messages
• Number of users sending messages
• Number of users receiving messages 
• Total events posted
• Events per community

I can get the same raw data from each individual community (but it's a tedious process doing this for 240 communities!) - though this does at least give a more accurate indication of the health of the individual community (where my definition of 'health' means 'activity'), but none of this information can provide me with a 'value' or ROI - no matter how I slice and dice the data.

Hence the need for a more qualitative approach, in the form of membership and user surveys etc. Picking up on Ed's comment that all of his community projects have measurement built into them at a strategy level - which sounds like the right approach - it should also be noted that even this can be difficult to quantify in terms of an ROI. The IDeA community strategy was developed to "improve local government services". I can infer from the platform statistics that the communities appear to be active, but I cannot yet connect a specific output (service improvement) to the work of any individual community.

So, allowing for the subjective nature of user surveys, I still think they are more likely to give an indication of ROI if not an absolute measure.  If I'm right, the next step is in asking the right questions!

The Online Community Research Network are doing a survey on this, so please have a read, fill it in, and pass it on. The more responses the better.