Can crowdsourcing disrupt Gartner? Tech community reacts to the Grid

May 9, 2013

Yesterday, we at G2 Crowd announced the Grid as our crowdsourced alternative to Gartner’s Magic Quadrant for rating enterprise software and technology.

Tech community reactions to the Grid

TechCrunch and Techworld wrote insightful stories about our Grid, and we’ve already received tons of great feedback from the enterprise tech community around the world that I want to share.

What users love

One common reaction to the Grid announcement was excitement; many of our peers in the tech community share our discontent with the status quo.

Here are the attributes of the Grid that users love:

  • Inclusive: Many innovative vendors have to wait years and spend hundreds of thousands of dollars just to get included in a Magic Quadrant. Marc Benioff once called analysts, “Whores to the established software vendors.” Innovators are included on the G2 Grid as soon as 10 users share authentic reviews at no charge.
  • Real time: Traditional quadrants are only published around every two years and are based on research that is already six months old at time of publication. The Grid updates in real time, based on new reviews, ratings, and social signals.
  • Democratic: Traditional analyst ratings rely on “magic” – typically the opinion of one analyst. The Grid ratings incorporate the opinions of thousands of real users and a host of online social signals. Everyone has a voice and a vote.
  • Social: “Social” to traditional analysts means attending user conferences and having vendors buy them steak dinners. The Grid incorporates social signals and sentiment by mining big data in millions of posts on Twitter, Facebook, and LinkedIn.
  • Free: Gartner often puts the pressure on clients from the start by having a sales rep in the room the first time they meet their analyst. The time and money invested in kowtowing to analysts might be better spent on innovation. G2 Crowd does not charge for access and influence.

User concerns

On the flip side, many industry watchers shared valid concerns and skepticism. Can a crowdsourced Grid really disrupt Gartner’s Magic Quadrant?

Here are some of the questions they raised, and our responses:

  • How can I trust online reviews? With reports stating that 30 percent of online reviews are fake and reviewers using aliases, such as “Han Solo” or “Chewbacca” on review sites, trust is certainly an issue. G2 Crowd designed the Grid to put trust first. Users must sign up with their LinkedIn profiles, so we can validate their identities and prevent users from rating their own or their competitors’ products.
  • How can user reviews possibly differentiate vendors in terms of complex qualities, such as innovation and leadership? G2 Crowd asks users many questions, including whether a vendor is taking a product in the right direction, that provide meaningful clues and color. We’ll explore this topic more in a future post.
  • Do you have enough data to be statistically significant? In some categories, such as CRM, the answer is becoming yes. For vendors such as Salesforce and Microsoft, we have gotten hundreds of ratings and reviews. Once we get more than 20 complete reviews from real users, the data starts to become very meaningful. Traditional analysts ask for only five customer references – and they only get the best ones.
  • Large enterprises and SMBs have different needs. Do you mix all the data? We can segment all the ratings and reviews by variables, such as company size, industry, geography, role, and title. We will roll out premium features with the ability to slice and dice the data to address divergent needs.
  • Don’t analysts play a vital role in advising companies on strategy? Many companies do need advice that goes beyond crowdsourced data. We believe that such advice can also be found from the crowd; we plan to facilitate outreach to peers and experts as identified by the crowd on our site.

We are still in early days and look forward to getting your input as we tune our model to help change the game in enterprise technology.

What do you think about the Grid from G2 Crowd? Do you love it or fear it?

Explore the CRM Grid and more