Survey Monkey and Amazon Mechanical Turk Validated My Startup Idea
By Lindsay Harper (Founder, Swayable)
So you have your genius idea that you just know everyone is going to love, why on earth would you need to do user testing? After all, since the idea is genius everyone is just going to “get it” from the first visit right?
To validate your genius idea (or invalidate it and save you wasted time, sweat and tears) as well as get actual users to test your site once you think it’s ready for mass market, survey your potential and current users.
Testing your hypothesis won’t cost you a lot in money, or time for that matter, and will provide a great deal of insight to improve your product/service dramatically.
Here’s what I learned about viability testing:
- Create a Survey Monkey first, then use Amazon Mechanical Turk to direct Turkers to click on the survey link and to complete the survey.
- Test your survey with a small group first and keep tweaking it until you get the right level of response.
- Sort and prioritize the results. Don’t expect yourself to fix everything that every users tells you, so prioritize the feedback.
- You can use MTurk for consumer-facing as well as B2B products.
- Good tool for testing price points in Mechanical Turk to see where the sweet spot is for your survey/feedback.
Let’s start from the beginning.
Step 1: Validate Your Idea:
Assuming you’ve already written down at least a concept of your idea to make sure there aren’t any low hanging flaws to automatically rule it out. Now it’s time to do a quick viability survey. As a bootstrapped entrepreneur and sole founder of Swayable, I had to come up with a creative way to to do this since I couldn’t afford the cost of a research firm.
So I got creative and used the service from Amazon called Amazon Mechanical Turk. I spent a whopping $27.50 to do this survey and what I received back was invaluable.
I learned from this one survey where my MVP (minimum viable product) was, and ideas of how people would use my service. The reason why you don’t want to skip this step is because why would you want to pursue an idea if you can’t get any validation that people would use it or see value in it? (Your friends and family should not be your only validation, sorry!)
To create a viability survey on Mturk (Amazon’s mechanical turk), I recommend using a combination of Survey Monkey for the survey portion and then MTURK to distribute the survey request out to Turkers (Turkers are the people that complete the task for you via Mechanical Turk). Mturk works great for products that are pretty broad like Swayable is.
If your product is more niche, you can still create a survey and just use other distribution methods like running an ad on Google Adwords or Facebook ads, using a contest for completing the survey, etc. Get creative to drive your target audience to your survey if Mturk is too broad of an audience for you.
Here are the questions I put together to test Swayable on Mturk, along with providing a brief description of what Swayable would offer, asking my survey recipients:
- What’s your Gender
- What’s your Age
- Would you use the service as described above.
- Give me 3 examples of how they would use the service or would see others using this service.
- General feedback or ideas on the service, why they would or would not use it.
The Mturk Survey Results:
I found out that 73% of the people surveyed would use my service (that validated the idea for me).
Then found that of the 27% that wouldn’t use the service, WHY they wouldn’t use the service, then I added those common “no” reasons into my MVP product for launch, in hopes that I could convert even more.
I also found out to my surprise that more men than women said they’d use the service. I fully expected it to be the other way around.
You can read more about details about my viability survey for Swayable.
Step 2: Your product is built, now test it with users!
Fast forward a few months and your product is working well enough. You’ve tested it to death, had friends test it and feel like most of the bugs are worked out. At this point you feel ready to push out the beta.
DO MORE USER TESTING! Not necessarily needing to test with you product demographic, you can test easily with a broad audience. Once again, I turned to Amazon’s mechanical Turk to user test Swayable cheaply but effectively.
At this point I literally just need users to test registration and work flows to make sure they don’t encounter any bugs. You do have to make sure you follow the Amazons TOS (Terms of service) and either tell your testers to make up a name and fake email, or give them credentials to use, because you can’t require them to use “real” information.
I continue using Mturk as well as usertesting.com to get user feedback any time I make feature updates. By the way, my feature updates are features requested by users! Keep in mind that no method is full proof, but I am of the mind that trying something is better than doing nothing.
About the guest blogger: Lindsey Harper has over 13 years of experience in project management, event management and marketing consulting. Since 1998, she’s been building, marketing and selling small websites as a hobby. By 2008, she sold all her “hobby” sites on Sitepoint.com and focused on her marketing consulting business. Currently, she is working on Swayable and blogs about her startup experience at www.harperlindsey.com. Follow her on Twitter at @harperlindsey.