Leon Noel has one of those classic, well-rehearsed business start-up stories. In 2009, Noel, then a junior at Yale majoring in biological anthropology, was trying to get subjects to answer questions for a research project. He tried to collar passers-by only to have most of them wave him off. It was a pain. It was inefficient. There had to be a better way.
So he and another Yale undergrad, Harley Trung, a computer science major, started working on a solution, which became SocialSci, a Web site that connects researchers and participants. Soon thereafter Noel and Trung did the Zuckerberg/Gates thing and dropped out of the Ivy League to work on the company full-time. So far they’ve raised more than a million dollars in funds. Roughly 300 researchers have used the service and they’ve attracted a sizable pool of regular participants, about 15,000 from 35 countries.
But Noel and Trung realized quickly there were challenges. Among them was rooting out the liars, the cheats, the participants who were in it solely for the compensation and didn’t care about providing accurate answers.
Not all of their strategies worked. At first, they asked prospective participants to photograph themselves holding up a piece of paper with their names and locations. Soon they started getting multiple photos of the same person with different locations, donning various lame disguises.**
So they looked for other screening methods. Now they throw curveballs like asking participants to type in a random word
rather than answering the multiple-choice question on the screen (if they’re not carefully reading the question, they’ll blow it). They track how quickly they answer: If you’re too fast, you’ll get flagged, but if you try to game the system by slowing down, they’re watching for that, too.
They follow whether a participant is being consistent survey to survey. Are you a 34-year-old woman in one survey and a 19-year-old guy in another? Or what if in one study you write that your mother and father are black, but in another you list yourself as white? The software will notice.
Noel doesn’t argue the system is foolproof, and he’s sure some liars slip through the net -- but that’s true with every survey method. Because participants are often taking multiple surveys, the company can rate them on how careful and honest they’ve been in the past. Those who don’t build up credibility aren’t offered new surveys. The more credible you are, the more likely you are to earn the $5 Amazon gift card or whatever reward is being offered in return for your information.
SocialSci has some better-known, better-financed competitors, like Amazon’s Mechanical Turk. But Noel hopes that tailoring the site to academic researchers will help set it apart. It’s a simple, bright idea, and the company has already made some headway. As for whether they’ll return to Yale to finish their degrees, Noel isn’t sure. “We’re not going back yet,” he says.
(** Noel e-mailed to say that they actually tested the photograph method on Mechanical Turk to see if people tried to game the system (they did). It was never actually run on SocialSci.)