By Kaitlin Pike (Marketing & Community Manager, Web 2.0 Expo)
Since it was released just two weeks ago, the Startup Genome report has inspired hundreds of blog posts, articles and discussions about the “science” behind successful startups and the causes of business failure here in Silicon Valley.

While the team behind the Startup Genome still has miles of analysis to wade through (including information from their new updated survey), they’ve already put together a number of summaries about the data including their post on VentureBeat — The 7 signs of failure for internet startups and 14 key findings on their own blog. Here is a quick flavor of what the report authors are saying:

  • “Founders that learn more are more successful.”
  • “Startups that pivot once or twice raise 2.5x more money.”
  • “People who work half time are able to raise money, but ~24x less than founders who go full time.”

As startup business folks, we’ve learned to rely on data, data, and more data to test, plot, and move forward with our ideas, so it’s only natural a report of this nature has sparked such fervent interest. It’s also generated a significant amount of (hopefully helpful) criticism. Jason Cohen wrote a thoughtful response in which he listed two alleged statistical fallacies in the report, including Survivor Basis:

  • “They interviewed 650 companies, which means extant companies, which means we don’t know which of these trends contributed to success and which are just trends that everyone is doing, even the companies who failed.”
  • “The Startup Genome Project looks only at standing log cabins and thus isn’t telling us what separates the successes from the failures. If the failures follow the same patterns as the successes, the “patterns” are descriptive but not helpful in guiding us to building better companies.”

Read the end of Jason’s post for a reply from Startup Genome and this interesting article on the methodology of the report. I share Jason’s hesitations, but also join him in his celebration of the idea behind the project and would love to see improved iterations.

Gender, Anyone?

Along with the already mentioned doubts, I spotted a big red flag in the report (which inspired the title for this post) — Where was the data on gender? Or race? Class? Education level?

It’s an unfortunate truth that gender, skin color, your parents’ connections, or whether you joined a particular club makes the difference between landing that VC meeting and being told “Sorry.” Success and failure as defined by the Startup Genome is limited to hitting benchmarks and following best practices.

And while such information helps guide us, it certainly isn’t the full –- or even half -– the story behind success in the Silicon Valley.

Moving Forward

I spoke with Max Marmer, one of the lead authors of Startup Genome, about this and other points. He said the authors have included a question in the new survey asking whether the participant is male or female, and have included a question about education (and will be looking into personality types down the line). Beyond this, they are not looking further into social factors behind success.

I appreciate what Max and Startup Genome have begun, but until factors such as gender and race can be followed and tested (a likely impossible task), we cannot claim to have a repeatable process (and the “science” they hope to accomplish) behind startup success – just interesting data.

Further reading:

About the guest blogger: Kaitlin Pike is the Marketing & Community Manager for Web 2.0 Expo. A startup veteran, she may be able to help you out (on a consultant-basis/for pie) with your copywriting, marketing, or social media woes. She’s also the charming organizer of SF Nightowls, a late night coworking group. You can bug her about startup failures, late night coworking, and whatever else is on your mind on Twitter @kcpike.