Blog Coordinator

Knobe's X-Phi Page

X-Phi Grad Programs

« Wright in the News | Main | Registration Open: Second Workshop of Experimental Philosophy Group UK »

07/14/2011

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Joe Paxton

Hi Marcus,

Interesting results! I wonder, though, if the two sets of results might undermine (or at least complicate) your normative argument in the following way:

Your results suggest that utilitarianism in the footbridge case correlates positively with a particular set of bad personality traits, while liberalism correlates negatively with a (somewhat?) distinct set of bad personality traits. However, in pretty much every study I've ever run using the footbridge case, liberalism is positively correlated with utilitarianism in the footbridge case.

You want to conclude from your results that utilitarians are dispositionally bad, and that liberals are dispositionally good (at least relative to conservatives). But if utilitarians are liberals, then what are we to conclude?

From your papers, it appears that the same group of subjects were used in the two studies. So you should be able to see whether utilitarianism is positively correlated with liberalism in your data. I'd be interested to know whether this is the case.

-Joe

Marcus Arvan

Hi Joe,

Thanks for the helpful comments - I'm glad you found the results interesting! The 14 day trial-edition of SPSS I used to run my analyses has expired, so I can't run full statistical tests right now (I'm attempting to install another trial version as we speak). I just ran a quick correlation test with Excel, however, and did find a positive correlation between self-identified liberals and utilitarian judgments in the footbridge case. If this holds up (and again, I'll try to run the tests with SPSS later), you're right that this complicates my normative picture. Since you ask, "If utilitarians are liberals, what are we to conclude?", let me offer a tentative guess.

Notice that the "Bad News" study found one positive correlation between psychopathy and a liberal judgment (the view that it is bad or wrong to restrict civil liberties to protect against terrorism), and three postive correlations between psychopathy and conservative judgments (on free markets, capital punishment, and waging war in violation of UN resolutions). This suggests that different liberal and conservative judgments may *both* reflect some element of psychopathy. If that is so, then perhaps what is going on is this:

(1) Liberals are dispositionally good on
*average*(relative to conservatives);,
but,

(2) The footbridge case is one very
*specific* case in which
liberal/utilitarian judgments are
dispositionally worse (relative to
conservative judgments).

Obviously, this explanation is rather speculative at present, but it seems to me a plausible way to make sense of the data. Your thoughts?

Eric Schwitzgebel

Could accurate self-report of dark triad traits interact with political view? E.g., could conservatives and liberals be equally dark by real-world behavioral measures but conservatives self-report more accurately?

APinillos

Hi Marcus,

I have a couple of comments. It might be useful (at least to me) to put in the paper (or here in the comments) the 27 question measure used to assess the dark traits. Also, in the paper you mention that this test "only measures these traits at a sub-clinical level." I wasn't sure what that meant in this context. Maybe you can help me here.

Also, you seemed to have run stats for 80+ correlations. The chances are that you will reach significance for some of them on the conventional .05 threshold even if there was no genuine relationship between the variables. Correcting for this should leave you with a lot fewer significant results, I think.

Marcus Arvan

Hi Eric and APinillos - thanks for taking the time to comment. Here are some of my initial thoughts.

Eric: it's certainly possible that conservatives self-report more accurately. I wonder, though, how one might test the hypothesis. Any ideas?

APinillos: the Short D3 inventory is available online. Just google paulhus personality labs. As to your worry about significance and the number of tests I ran, this is something I have discussed at some length with Adam Feltz (who I thank for initially bringing it to my attention). Here are three quick thoughts:

1.) The worry certainly underlines the need for replication of the results. I'd like to pursue further studies myself soon. Seeing as this is the first experiment I've ever run, I'd be more than happy if anyone is interested in collaborating to run some more experiments. I could certainly use some experienced assistance! If there are any takers please feel free to let me know here or through email at marvan@ut.edu.

2.) If I'm not mistaken, the traditional .05 threshold means there is a 95% chance the result is significant. That means that for any indicated significant result, 95 times out of 100 the result is genuine. But in that case if I ran 80 tests at most one of my purported findings is likely to be spurious, no?

3.) Finally, it is my understandings that worries like yours are alleviated a good deal if one has stated clear hypotheses before running the experiment (which I did). After all, if one has a clear hypothesis, gives reasons to think that it may be true, and then one's results verify the hypothesis, one thereby has some real reason to think the results are not a type 1 error (false positive) -- or so I've heard.

Anyway, I do think your worry underlines the need for replication, so again, if anyone is interested in collaborating, please do let me know. Thanks again to both of you for the stimulating comments!

APinillos

HI Marcus,

If you set the significance threshold at .05 this basically means that if you redo an experiment 100 times, 5 of them will likely show up as showing a significant effect in your stats program IF nothing is going on in reality. Since you ran 80, you will likely get approximately 4 significant correlations for free in the case where nothing is going on in reality. Of course, this is complicated somewhat since you are not doing the same experiment over and over. In my view, the threshold for significance should be set lower in this case. (But i don't think we are going to get to them bottom of this in this blog. It might be useful to look up discussions on Bonferroni corrections).

A related issue is that there might be "double counting" since your independent variables are likely correlated with each other. For example, a view about the death penalty might be correlated with narcissism maybe only because views about the death penalty are correlated with views about guns (these are examples I am just making up on the spot, not having your paper in front of me). One way of addressing this in SPSS is by using the partial correlations options. This will allow you to get closer to the genuine relation between the variables. I think that if you are interested in showing that *more* conservative views are *genuinely* related with some bad personality trait, it would be helpful to go this route.

Im not sure these are criticisms as much as friendly suggestions for getting better (in my opinion) information from the valuable data you have collected.


-Angel

Marcus Arvan

Hi Angel,

Thanks for the helpful suggestions. I've heard different things in different places regarding your main worry, but I'll look more closely at Bonferroni corrections, etc. I'll also take a closer look at partial correlations.

Best,
Marcus

The comments to this entry are closed.

3QD Prize 2012: Wesley Buckwalter