Blog Coordinator

Knobe's X-Phi Page

X-Phi Grad Programs

« Formal Semantics and Experimental Philosophy | Main | The process of publishing a paper: some data from one person's experience »

05/08/2015

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Shen-yi Liao

Here is another recent popular press article on MTurk: http://www.washingtonpost.com/blogs/monkey-cage/wp/2015/05/04/researchers-are-rushing-to-amazons-mechanical-turk-should-they/

Eddy Nahmias

For the first time in my x-phi experience, I have just used mTurk. In my past studies, I've almost* always used students (first at FSU and now at GSU, where I've probably had over 3000 students in critical thinking courses take surveys, mostly about free will, responsibility, choice, etc.).

A few armchair reflections:
-mTurk is amazingly fast (got 65 subjects in a matter of hours) and easy.
-given the time they took and the lack of failing comprehension and attention checks (4 questions in all, with only a handful of misses), they seem to be doing the task well.
-they differed in small but (probably statistically significant) ways from the GSU students on a couple scenarios that both groups took.
-they are less demographically diverse than the GSU sample in some noticeable ways--fewer females (44%), many more white people and fewer of other races (GSU is non-representative with about 40% African-American), more liberal (60% on liberal side of scale, 20% neutral, only 20% on conservative side), and way less religious (than GSU and certainly than US population: 60% 'not religious at all' and 12% more in the slot between that and 'somewhat religious').

Anyway, I'd be interested to hear if x-phi researchers using mTurk have found the participants to be reliable and to work well for their purposes.

*For the Free Will Inventory Thomas created (with 4 others including me), we used Qualtrics to get a representative sample of US participants.

The comments to this entry are closed.

3QD Prize 2012: Wesley Buckwalter