Blog Coordinator

Knobe's X-Phi Page

X-Phi Grad Programs

« Mapping Human Values | Main | Philosophy and Data Science »

01/30/2014

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

jonathan weinberg

Do you think your data in the climate survey could speak at all to the question as to whether members of different groups might respond differently to learning that they have a "wrong" intuition? On lots of philosophical cases it looks like there's a fair amount of variation in the folk responses, and so one hypothesis here might be that some folks are more (and unfortunately) easily discouraged than others, when told that their intuition diverges from the philosophical norm.

(I'm pretty sure this idea is not mine; I think I must be either stealing it from the Buckwalter & Stich, or maybe from the Q&A at the SPP session in which they first presented these ideas a few years ago.)

Morgan Thompson

Hi Jonathan,

The main claim of Buckwalter and Stich’s paper is that the mismatch between female undergraduate student’s intuitions and those intuitions treated as “correct” by their instructors is a partial explanation of the underrepresentation of women in philosophy. We are primarily interested in causes of the underrepresentation of women and other minorities in philosophy, so we focused on their arguments and evidence for this main claim.

However, you’re right that Buckwalter and Stich also suggest that any students (regardless of gender) with intuitions that conflict with the intuitions their instructors treat as “correct” may be discouraged from continuing to take philosophy courses. I think this suggestion comes up right before the conclusion of their paper.

On p. 22 of our paper, we report some initial findings from the 2012 version of our climate survey. While we did not ask students about their intuitions about any thought experiments, we did ask them whether they felt their opinions differed from those of their peers. We found that students’ perceptions of having different opinions from their peers mediated the relationship between gender and willingness to continue taking philosophy courses. That is, women were less likely to agree that their opinions differed from their peers when compared to men and students who reported having different opinions also reported more willingness to continue. This seems to provide some evidence against Buckwalter and Stich’s idea that students who perceive their intuitions as differing significantly from the norm will be less likely to continue taking philosophy courses.

We have not yet analyzed the data from our 2013 climate survey to see if we’ve replicated this finding. We also asked students whether they felt their opinions differed from their instructor’s opinions and whether they had strong opinions about the philosophical issues discussed in their class, but again, we haven’t analyzed this data yet.

Christian Mott

Just thought I'd add a shameless plug for the Experimental Philosophy Replications page here:

The data from this (excellent) paper alongside data from another attempt to replicate the Buckwalter & Stich result (and replications and non-replications of 22 other experimental philosophy papers) are up at :

http://pantheon.yale.edu/~jk762/xphipage/Experimental%20Philosophy-Replications.html

And if anyone is willing to share any additional data on this question, published or unpublished, please send it along!

Wesley Buckwalter

I wanted to write in to thank you for your valuable work contributing these data points to the ongoing discussion. I think the most important thing, especially with respect to such an important question, is to follow the data. The fact that you were unable to find the same effects that we reported may very well indicate that the differences are not as robust or widespread as we had originally thought. This is an important development, even if it suggests that our original hypotheses were wrong.

There are many interesting issues raised in your paper, but for now, I wanted to make three quick observations. First it looks like you did find differences, such as in brain in a vat. It looks like you might have found a difference in Gettier had you used the preferred, chi-square test, in that circumstance. It also looks like there were trending effects, such as twin earth. This is to say, lack of statistical power may be a very real issue in detecting some of the small effects we reported. Second, I think even small effects can be of great importance. For instance, upon repeated exposure to similar examples of inquiry, differences might disappear or they might increase. Or perhaps they would continue at the same small but steady rate, resulting in gradual attrition over time. They might also interact with a number of other, larger contributing factors. Those are complex questions that go beyond our investigation, but suggest that even small effects can potentially contribute to our understanding. Third, when we started this project back in 2009, it seemed that many researchers were hesitant, unable, or unwilling to report gender results in their work. This led to a file drawer problem that was part of what inspired the original paper. Hopefully another positive outcome leading from this dialectic is that more researchers will be inspired to share this information in the future.

Wesley

Geoff Holtzman

I want to mention that I have presently have a manuscript under review in which I revisit the gender data from Holtzman that Wes and Steve discussed (2014), and compare it to the Philosophy PhD data from that study that I reported (2013). It is noteworthy that Eddy's team (2014) failed to replicate some of my results, despite recruiting enough participants to yield sufficient statistical power. Hamid also has a manuscript available at the link Christian posted above, in which he presents interesting arguments about issues in study design (including order effects that I failed to control for in my own study), though his statistical power was generally less than that used by me or Eddy or the standard .80 level used in social psychology. I think both Hamid's manuscript, and mine (which I can share directly) are useful supplements to this discussion, and anyone interested in the topic should definitely check out some of the published pieces on the topic, especially the ones by Carrie Figdor et al. (2013) and Louise Antony (2013).

The comments to this entry are closed.

3QD Prize 2012: Wesley Buckwalter