Blog Coordinator

Knobe's X-Phi Page

X-Phi Grad Programs

« What should we be teaching in philosophy of mind courses? | Main | Why Gettier Cases are misleading »

03/13/2016

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Josh May

Interesting issue, John. I'm sympathetic, but I wonder: Is the problem with welcoming or encouraging WORK aiming at unfinding, or is the problem simply encouraging such AIMS? In other words, I worry about discouraging this work just because of the aims, since it can be quite important work when the right conclusion is drawn. In the spirit of keeping science self-corrective, shouldn't we worry about discouraging work that's critical of previous results?

John Turri

Thanks, Josh. Good question! As I see it, the problem pertains to people aiming for that outcome while being prone to hastily concluding that they have achieved it. I don't have any problem with work that actually demonstrates the outcome. I don't even have a problem with the aim itself. (My thick fallibilist skin might make me peculiar on this point — others I've spoken to disagree.) But my sense is that when coupled with the tendency toward motivated inference, the aim causes problems. These problems include eroding beneficial sentiment and distorting the record. As a counteractive incentive, I suggest operating with a defeasible presumption against work that aims at unfinding. There might be other solutions.

I agree that critical work is very important and I definitely don't want to discourage anything merely because it is critical.

Jason Shepard

Hi John, I think I have worries similar to Josh's. I was hoping you could say more. If you are willing to, I am hoping we can engage with a real example, such as the following:

Rose, Buckwalter, and Nichols (RBN; 2016) "Neuroscientific prediction and the intrusion of intuitive metaphysics" (http://tinyurl.com/RBN-neuroprediction), if I am understanding you correctly, should count as an attempt of unfinding. In the paper, RBN provide evidence that results reported in Nahmias, Shepard, and Reuter (NSR; 2014) "It's OK if my brain made me do it: People's intuitions about neuroscientific prediction and free will" (http://tinyurl.com/NSR-Neuroprediction1) were uninformative--not just better explained by some other explanation but genuinely uninformative--because we did not take into account an error people were making when they processed our scenarios.

While I think their unfinding claim was a bit hasty, the community would be worse off if their paper was not published for, at least, two reasons: (1) They drew the community's attention to a potential methodological issue that, if right, may be a problem for many x-phi and philosophy-inspired-psychology experiments. (2) Their work has forced my colleagues and me--and I would guess others--to think more carefully about what people believe about free will and how we should be measuring these beliefs. As we all work through the issues RBN raised, I am sure we will make progress on coming to a better understanding of people's beliefs about free will.

Why would the community not want such insightful and challenging work published?

Jason Shepard

PS I should note that your answer to Josh appeared while I was composing my post. Your response to Josh helped answer my worry. But let me ask a related question: As a reviewer or editor, what do you look for when you are trying to determine if the attempt at unfinding should be published? (Or, if you prefer, what do you think we should be looking for as a reviewer or editor?)

John Turri

Hi, Jason. Thanks for pressing on this! I definitely think that the community should want insightful and challenging work published. At the same time, I think that there are other considerations in play here. I proposed a defeasible presumption as a hedge against some non-ideal factors. Do you think there is a better solution? Or maybe we're just stuck with them because anything we try will just make things worse overall?

Jason Shepard

Hi John. Thanks for pressing me! After given your proposal more thought, I've come to appreciate your position more and more. There is a certain way of doing research that seems to undermine a sense of community and cooperation, namely the sorts of research projects that go beyond being critical and actively seek to undermine previous research. I, too, believe a strong sense of community and cooperation is something we should strive for. But I think there is a way to frame "undoing" research that can come across less adversarial . Unfortunately, we as a community are not always very good at doing this. I also believe that there are competing interests that, perhaps, trump the desire to maintain a strong sense of community and cooperation, namely the interest to publish research that advances the field. Perhaps the approach I would feel most comfortable would be not to hold a presumption against recommending this sort of work for publication but to encourage the author(s) to make adjustments to the tone of their work in the revise and resubmit phase. But ultimately I think recommendation to publish comes (primarily) down to two factors: (1) Does the work advance the field? (2) Are the methods sound?/Are the arguments compelling? If the answer is "yes" to both of these questions, then the research should be published, regardless of tone. If the answer is "no" to either of these questions, then the research should not be published, regardless of tone.

(Apologies for framing my earlier question in such a loaded way. That was unnecessarily adversarial of me!)

John Turri

Hi, Jason. I actually wasn't thinking of this primarily as something to be litigated as part of the formal review process, but rather as a value that we as people bring to our work and seek to encourage in others. Still, as you point out, the review process provides opportunity for us to encourage a more human touch.

Factors 1 and 2 do seem to make a good case for publishability. Of course, it can be notoriously difficult to assess "how important" the advancement is. Other important qualities I'd add are that the project is appropriately contextualized and that the results are responsibly interpreted. (Perhaps you were thinking of these as falling under 1 and 2.) The interpretation stage is where the hastiness I described occurs, especially in virtue of overlooking potential limitations or alternative explanations.

Of course, I also strongly encourage people to try their best to not take research developments personally. That is very important! And, at the end of the day, I think we should prefer advancing the field to sparing people's feelings. But, human nature being what it is, I think that the field will advance more in the long run if we don't routinely force a choice between those two things.

PS: And sorry for missing your earlier PS. I initially overlooked it in the approval queue.

Jason Shepard

Thanks, John! As is often the case, what may first appear as a disagreement, however slight, is often discovered not to be after a bit of dialog. I think your proposal is right. We all can be better members of our community without sacrificing the publication of good research, if we try to be a little more self aware of tone and (possible) hasty conclusions, especially when these hasty conclusions are of the kind you note.

Wesley Buckwalter

Hey Jason,

I just wanted to chime in to say that I definitely didn’t think your paper was uninformative. I thought the findings presented by NSR were definitely suggestive of some very powerful factors at work in people’s judgments about futuristic imaginative scenarios. I’m also glad to hear that you found the methodological issues raised by RBN instrumental moving forward. I never thought of RBN an “unfinding” claim before, though maybe on some definitions it basically amounts to that. In my view of the exchange, I am just not yet confident the original conclusion has been sufficiently demonstrated by data, independently of the particular theoretical dispute in question.

Wesley

John Turri

Well said, Jason, and thanks for helping me get clearer in my own thinking about all of this!

The comments to this entry are closed.

3QD Prize 2012: Wesley Buckwalter