February 18, 2007

Measuring Community

These days, many folks are trying to understand how to measure 'community'.

I thought it might be fun to explore the development of a "community index", using the statistics from my website.

You can download the community index spreadsheet here. Only change cells that are shaded yellow.

For those of you who want the details behind the spreadsheet, read the appendix of this article.

For the rest of you, here's the Executive Summary:

  • Community can be measured, trends can be understood.
  • In the case of my blog, I had a better "community" in October and November, when my audience was small, but comparatively vocal.
  • In December, my visits went way up, courtesy of Google. This hurt my "community index", because these visitors only came to the site to find specific information. Increased visitors did not translate to increased comments, or a significant number of visitors coming via an RSS-reader referring URL.
  • In January, my "community index" tanked. I had a barrage of visitors courtesy of Reddit. Those visitors largely viewed just one article. As a result, I didn't grow my "community".
  • My February results are extrapolated to represent a full month. Notice that I grew my base of folks who visit from a referring URL of an RSS reader. I gained a significant number of people who type in my URL. My search visitors have plateaued. But the number of comments have stagnated. Because comments have stagnated while visits increased, my "community index" is nowhere close to where it used to be.
It becomes clear that metrics can be created to illustrate how community can be measured.

In the case of my blog, the "community index" I developed is consistently declining. As my audience grows, the larger audience is less engaged with my blog.

These concepts translate well to e-commerce. Brands can pick a series of metrics appropriate for their business, and measure how engaged their customers actually are. Might any of my web analytics readers be willing to take a shot at an e-commerce version of this?

What do you think, folks? Is there any validity to this style of measurement?


Appendix:

Row 4: Enter the number of posts you wrote in the past month.

Row 5: Enter the number of comments you received in the past month.

Row 6: Comments per 1,000 visitors are calculated. This is a good way of understanding how many comments you are getting, given the varying amount of traffic you have.

Row 7: Enter the number of visitors that came from a referring URL of an RSS reader. I chose this route because not everybody uses Feedburner. In addition, it doesn't really matter how many folks subscribe to your feed. What matters is how many people choose to visit your site because they liked what they saw in their RSS reader.

Row 8: Enter the number of people who visited your site by typing in your URL. These visitors should be more loyal than others, given that they are willing to type your URL.

Row 9: This is a calculation --- it basically represents all other site visitors, and ultimately measures the number of folks who came to your site via a referring URL. These visitors should be more valuable than those who visit your site via search.

Row 10: Enter the number of people who visited your site due to a search (Google, Yahoo!, MSN, etc.). These should be the least engaged of any of your visitors, and will consequently count less when measuring community.

Row 11: Enter the number of pages viewed. The algorithm gives extra credit to visitors who view more than one page. If page views are truly becoming meaningless, then this algorithm will help, because I am only counting page views as a small percentage of the total score.

Row 12: Enter the total number of actual site visitors.

Column H: This is an adjustment column. If you don't like my weighting scheme, you can adjust it here. If you want to discount page views by half, change the adjustment value of "1.00" to "0.50". If you want RSS visitors to count twice as much, change the adjustment value of "1.00" to "2.00".

Column I: You don't adjust this column, you manipulate it by changing values in Column H. However, this column is ultimately what drives the measurement of community.

  • 50.00 points are awarded for each comment per 1,000 visitors.
  • 2.50 points are awarded for each visitor who comes from an RSS reader.
  • 1.00 point is awarded for each visitor who types in my URL.
  • 0.65 points are awarded for each visitor who comes via a referring URL.
  • 0.25 points are awarded for each visitor who comes via a search engine.
  • 0.05 points are awarded for each additional page (beyond the first page) that a visitor views on your site.
The spreadsheet multiplies values you enter by your adjustments, and my point scheme, to arrive at "community points". Community points, divided by visitors, yields the "Community Index".

What do you think?

2 comments:

  1. I don't really have a comment to make, Kevin -- I just thought I'd bump up your community index score by leaving a comment.

    Just kidding -- this is a great approach, and while I'm sure some might quibble about the details of the calculations, no metric is perfect.

    But here's what I couldn't help but think was missing (in your perspective, not in your calculation):

    1) Market share. I completely understand your desire to have engagement, but the question is how big is the pool of potential engagees, and what % of them are you engaging? Your index score may be low, but then again, if there are only 5 of us out there who are potential community members, and we're highly engaged, then you're doing well.

    2) Time lag. I don't know if you can track the stats to measure this or not, but just because somebody comes to your site this week and doesn't comment, doesn't mean that they won't comment sometime down the road. It's the marketing funnel: You want to continue to build awareness, even if it doesn't immediately convert (ie, they comment).

    ReplyDelete
  2. These two comments add 0.02 to the community index!

    I don't have a good way to measure time lag, though that makes sense to me.

    Market share --- agree, there is a finite audience that will leave comments.

    Ok, folks, who wants to add an additional 0.01 to my community index by leaving a comment??!!

    ReplyDelete