Debunking the Analysts

I am a fan of the open source analysts RedMonk and 451 Group. I think Alex Fletcher from Entiva has good open source insights and Jeremy Owyang from Forrester share great information on his blog about social media. However, when it comes to IT buying decisions I have never been much of a fan of the big analyst firms.

I remember having breakfast with Sendmail’s Eric Allman after he was the CTO at Sendmail Inc. He told me that when he had problems to solve everyday it was easier to figure out what direction to take his project/software in. When he became a CTO he didn’t solve his own problems and that made it difficult to set product directions. I suspect the same goes for analysts who rely on interviews, briefings, and surveys for a large part of their information. They are largely getting their information second hand and even then what they are receiving is probably to some degree suspect.

Aaron Fulkerson from MindTouch has a thoughtful blog post speaking to the recent Gartner report, The State of Open Source, 2008. Aaron makes a lot of the good points but here’s the one that I think is interesting. I suspect many if not most CIOs that respond to Gartner surveys are ignorant of how much open source is in use in their enterprise. Aaron shares this anecdote:

Specifically, I don’t think they fully grasp how pervasive open source is, how quickly it’s growing, nor do they fully grasp the benefits. They are absolutely correct that the use of open source is “elusive.” I spoke with Chris Maresca of Olliance Group recently who shared with me a story of an audit he had performed for a particular organization to establish their usage of open source. The organization is very large and was very keen on auditing their own systems; meaning, they had far more of a grasp of their systems than your typical enterprise-sized company. Said org had a well educated prediction of their usage of open source, but by the end of the engagement Chris had shown them their usage of open source was, at least, twice or three times more than they had realized. Open source literally permeated their entire IT infrastructure and they didn’t even know it.

Check out Aaron’s point of view he’s put an interesting spin on Gartners findings.

For more Mark Hinkle, visit his Socialized Software blog.