Blog Coordinator

Knobe's X-Phi Page

X-Phi Grad Programs

« Thanks for Your Patience... | Main | Knowledge on Saturday »

07/21/2008

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Ron Mallon

A nice effect!

Ron Mallon

Question: You describe your own explanation as related to, but broader than, the speakers/semantic reference ambiguity.

Why isn't your explanation, rather, a competitor to it? Couldn't you ask, e.g. of the clarified narrator's probe, whether it was probing speakers' or semantic reference?

If that's right, it seems your result is more significant, counting against not only the MMNS explanation, but the most common alternate explanation offered by philosophers of language. Is that right?

Justin Sytsma

Good question. This is an issue that Jonathan and I have spent quite a while talking about; but, neither of us are philosophers of language and we’d love to hear other people’s thoughts on the issue.

My feeling is that there is the possibility for a speaker’s reference / semantic reference confusion because of the knowledge asymmetry in the story. That is, because the subject comes to have information about Gödel that John lacks, it sets up a situation in which some (but not all!) statements that John might utter using the name “Gödel” would be best interpreted (from the subject’s perspective) as referring to Schmidt. The story, of course, doesn’t give any such statement.

What we’ve done is to offer a more clear mechanism by which the knowledge asymmetry in the story could explain the variation found in the original study. But, in clarifying the test question to emphasize either John’s perspective or the narrator’s perspective (or, put another way, to emphasize whose knowledge to consider in answering the question), it seems that we made steps toward clearing up the larger ambiguity that opens up the possibility of a speaker’s reference / semantic reference confusion.

If the clarified narrator’s perspective question is roughly just as open to a speaker’s reference / semantic reference ambiguity as the original question, then I think we are entitled to the more significant claim that you suggest. I’m just not sure if that is the case.

Lisa Lederer

Great study! Your findings reveal one factor besides, causal-historical/descriptivist intuitions about reference, that may have accounted for much of the between-groups variation.

Interestingly, I recently had an e-mail exchange about the original study with a colleague from Beijing that suggested another possible source of this variation. After reading the Godel probe, she wrote:
"I would say that I will choose A if I were doing the question. Because
I would think the person actually discover the theorem and the person
who found the manuscript both contribute to the work, in that sense,
the name Godel could refer to the entity that contains both people,
which primarily include the person who discover it . . .
I do think meaning referent transfer is not uncommon in our language.
There are cases to refer to a group of poets by the name of the
location they frequently met, then the location name transfer to the
leader of the group, and then may refer to someone whose works has
similar styles."

This piece of anecdotal evidence raises a possibility. Maybe the difference between EAs' and Ws' responses to the probe is partly due to a cultural difference in the kinds of entities that proper names can refer to; my friend's response suggests that in East Asia, they can refer simultaneously to several people, or even to a place. Thus maybe for EA participants, "Godel" is neither the person who best fits the given description nor the person historically associated with it, but rather the whole collection of people (and even places) that have satisfied the description throughout history.

In fact, research in cultural psych suggests that EAs see the person as continuous with her (social and physical) environment. For example, Masuda et al. (2008) found that EAs', but not Ws', ratings of a cartoon person's emotions varied depending on the facial expressions of surrounding people in the cartoon. So it makes sense that an East Asian proper name could refer to a number of people.

I note that Machery et al. (2004) themselves note that "although our results are consistent with this hypothesis [that the difference we found is due to Ws' tendency to make causation-based judgments], they fail to support it directly." The suggestion I make above merely mentions an additional factor that may have contributed to their findings.

A future study of intuitions about reference might allow participants to choose a third option: that when John says "Godel," he is talking about BOTH the man who discovered the theorem and the man who stole the manuscript.

Justin Sytsma

Thanks Lisa -- very interesting!

I wonder about how best to test the possibility you raise. My concern is that there are a few ways to take an answer of "both" on the original Gödel question: Subjects might answer both because they think that some "Gödel" statements made by John are best understood as referring to Gödel, some to Schmidt; or, they might answer both because they think that the name "Gödel" refers to a conglomeration of these two men. I'm not sure how best to disentangle these in the test question. Any ideas?

Edouard Machery

Justin and Jonathan

This is really a great paper and it certainly challenges our original work. A few comments:

1. Your interpretation of our original results supposes that when some subjects answer that John uses “Gödel” to talk about the person who really discovered the incompleteness of arithmetic, they mean to say that John uses “Gödel” to talk about the person who he thinks discovered the incompleteness of arithmetic.

In light of your results, I find this plausible, but I should note the following caveat. I feel that the presence of the adverb “really” in the question undermines your hypothesis. If I were to give an answer along the lines of (A), I would say something like “When John uses the name ‘Gödel’ he is talking about the person he thinks discovered the incompleteness of arithmetic” but I’d be reluctant to endorse the following answer “When John uses the name ‘Gödel’ he is talking about the person who REALLY discovered the incompleteness of arithmetic.”

The use of “really” suggests a contrast with what the speaker thinks; it is thus strange to find it in the content of a propositional attitude that is attributed to the speaker.

2. In Studies 2 and 3, around 25% of your subjects have descriptivist intuitions. You argue that this is not substantial variation. Your arguments are reasonable, but not decisive, as I am sure you realize.

Consider the following analogy. Suppose that 25% of English speakers find question 1 perfectly grammatical, while other speakers find question 2 grammatical:

1. That the dog barks is a German Shepard?

2 Is the dog that barks a German Shepard?

This variation could not be rejected as mere noise.

3. I agree that your results raise some doubts on our finding of a cross-cultural difference, although I would not put these doubts as affirmatively as you do. So, I am now engaged in testing your probe and other probes in Mongolia, Russia, India, and Kazakhstan. We’ll have clearer ideas soon.

Cheers

Edouard

Justin Sytsma

Thanks Edouard! Great comments! I’ll just note a few thoughts in response.

1. This is a good point, and something that we should note in the paper. The inclusion of the adverb “really” in the (A) description does seem to emphasize the narrator’s perspective, not John’s, as it indicates a contrast between the real discoverer and an implied imposter. My view is that although “really” serves to emphasize the narrator’s perspective, it is insufficient emphasis. In fact, in our pilot testing of the three probes discussed in Section 3.1, we removed the word (the descriptions for each read: “(A) the person who discovered the incompleteness of arithmetic? Or (B) the person who got hold of the manuscript and claimed credit for the work?”). We found a strong preference for (A) across each version of the question using these descriptions.

Original (“When John uses the name ‘Gödel,’ is he talking about”): 9.1% (B) answers (N=11);

John’s Perspective (“When John uses the name ‘Gödel,’ does John think he is talking about”): 0% (B) answers (N=12);

Narrator’s Perspective (“When John uses the name ‘Gödel,’ is he actually talking about”): 15.4% (B) answers (N=13);

Although the numbers here are low, I take this to indicate that you are right to indicate the importance of the “really” in emphasizing the narrator’s perspective; however, it also suggests that Westerners have a strong preference for answering the question from John’s perspective and that this is only partially overcome by the inclusion of the adverb.

2. I agree here. I suspect that there is some residual noise in this figure, but also feel fairly certain that a minority of Westerners have descriptivist intuitions about the Gödel case. If our coding of subjects’ explanations is to be trusted, that would place the number in the 10-20% range. It strikes me that this weakens your argument, in that the variation is much less than found on the original probe, but that as you note this isn’t decisive against it. In the end, I’m just not sure how much variation is needed for an argument of this type to be compelling: I found 40% rather compelling, but wouldn’t find 5% very compelling at all. I’d be interested in people’s thoughts on this.

3. I’ll check over our language here. The point we wanted to make was that in light of our results, the evidence in favor of cross-cultural variation in semantic intuitions is not compelling; but, I don’t want to claim that we’ve made a positive case against there being such variation. I can’t speak for Jonathan, here, but I take this to be a very live possibility and feel fairly neutral about it (i.e., I don’t have a strong expectation one way or the other). I can’t wait to see the results for subjects in Mongolia, Russia, India, and Kazakhstan!

Billie Pritchett

I don't understand the nature of the debate between Machery, Mallon, Nichols, and Stich vis-a-vis Kripke. I'm no Kripke-ite, but suppose it is the case that Kripke's causal account of names, or something like it, is correct. Would it make much of a difference if intuitions about the matter varied, if indeed the initial account was correct?

Justin Sytsma

I think it would be interesting if a significant percentage of people had intuitions at odds with the correct theory. But, the primary issue concerns the use of intuitions as evidence in developing theories of reference. Machery, Mallon, Nichols, and Stich are *responding to* work that makes evidential use of intuitions; against that, they argue that if there is significant variation in these intuitions between and within cultures, then it is unclear whose intuitions we should rely on; they then do an experiment showing cross-cultural and intra-cultural variation in responses to a prominent example from the literature.

Jonathan Livengood

So, I originally sent these comments directly to Edouard and Justin, but I will post them here in the interest of getting a wider discussion audience. These questions should be regarded as open to everyone. I hope no one minds:

I'm curious about how much consensus you think would be necessary for intuitions to count as uniform. I want to push on this in a few ways. Let me know what you think.

(1) Suppose you are in a room with ten other people discussing some issue. Suppose that you do not know enough about the material to make an informed judgment of the arguments or the evidence for yourself. If eight discussants take up one view and two take up the other view, would you be more inclined to say that there is a consensus that is being flouted by the two or would you be more inclined to say that there is simply no consensus? Do you think that there is a fact of the matter here?

(2) Does the amount of agreement needed for consensus depend on the question being asked? That is, does our standard of evidence (is that the right phrase?) depend on the domain in which we are looking for agreement? Now, it seems to me that with this kind of question, we need quite a bit of disagreement to really challenge the uniformity conjecture, because I think this kind of question is especially noisy. But maybe you disagree. Maybe it is not the question but the subjects that matter? I mean, I would expect very little disagreement on a mathematical question among mathematicians, but I would expect fair variation among non-mathematicians. In that case, what do you make of the philosopher data? Would you have made a different argument with that data than the argument we have in fact made? Don't you think that finding no difference between "experts" who are supposed to exhibit remarkable theoretical agreement and folk itself weakens any challenges to the uniformity conjecture from this amount of agreement on this precise question?

(3) What do you think it means to have compelling evidence in the first place? I take it that evidence is compelling when one cannot reasonably dissent. Of course, this sneaks in another weasle word: reasonably. Even so, it seems that 40-60% of one answer is plenty of variation to compel people to give up on uniformity; whereas, 0-10% is plenty of uniformity to compel people to accept uniformity (in whatever the relevant domain). Now, what about 10-40%? (Or, since these bounds are already fuzzy, maybe 15-35%.) What we said was that 25% was not enough for a compelling case against uniformity. I agree that it is also not enough for a compelling case in favor of uniformity. But do you really think that a compelling argument against uniformity can be based on 25% disagreement? There is a residual question here, in that if no compelling argument can be made for either side, the debate is left to individual choice or taste. At least from the perspective of traditional philosophers, it seems to me that the right thing to do is favor uniformity. But whatever one does, it is (I think) in some sense evidence-free at present.

(4) Would an empirical study measuring how much agreement is needed in order to have "consensus" make any difference to you or move the argument along at all? I'm imagining asking people whether or not there is consensus with various distributions of answers (sort of like what I asked you in (1). Would it matter if such empirical work were on folk or philosophers?

none

Interesting... This issue about variation and uniformity is a really important issue for all of the experimental philosophy interested in diversity in intuitions, and worth some thinking about.

It seems to me that the issue is really about when one can reasonably infer that one answer represents "noise" to the signal. If we had perfect uniformity, we're inclined to think that we've got a question that nearly perfectly elicits the signal we're looking for. That could be wrong, but it's our best guess about what's going on.

When we've got less uniformity, it can be that some large percentage of the answers represent noise or it can be that there is genuine variation in the population (or both).

There has always been variation in intuitive judgments (even among philosophers), but one moral of what Machery et al. suggest is that it is a mistake to assume that the variation is all the result of noise or error. Nichols and Ulatowski in a related vein suggested that there is individual variation in response to Knobe-effect cases - variation that does not represent error but rather diversity in the cognitive mechanisms used to produce the answers. Against this background, the big question is not whether the distribution of answers is 90%-10%, but whether it is plausible to think that the 10% represents genuine, systematic, non-errant diversity or whether it is simply noise. (So I wouldn't really be so interested in an experimental study of "consensus.")

To the extent that the answers grow more uniform, it seems to me it shifts the burden of proof back on the theorist that suggests the diversity is genuine. But supposing the theorist could show that there are populations A and B that systematically answer in distinct ways, it seems to me that it doesn't really matter whether population A accounts for 5% or 12% or 80% of the total (that's just a fact about historic reproduction rates). What matters is that the variation is systematically produced.

Justin Sytsma

I think that our results are fairly interpreted as showing some genuine diversity in semantic intuitions (maybe along the lines of 15% amongst native English speakers). But, I'm not sure why it would be the case that showing *any* genuine diversity in semantic intuitions, here, would be sufficient. Part of the issue is exactly what the role of the empirical results is in the argument: Certainly nobody would have thought, even prior to testing, that there was *no* genuine diversity in intuitions about the Gödel example; but, then, if the actual percentage doesn't matter, why isn't the argument just as good from the armchair?

Ron Mallon

I see now that typepad signed me out before I posted (the 1:25 pm post was mine)...

Justin writes, "Certainly nobody would have thought, even prior to testing, that there was *no* genuine diversity in intuitions about the Gödel example..."

In the epistemology case, that's not the way it played out. Faced with the mere conceptual possibility of genuine diversity, some did simply deny that it was real.

I think people are tempted by that as well in the reference case. One simply suggests that some portion of the answers (and it really doesn't matter if it's a majority or a minority) result from some sort of error. Indeed, I suspect this is on display among the critics - none of whom (so far as I remember though I haven't re-checked) - seem much inclined to concede diversity in semantic intuitions.

If that's the right reading of the dialectic, then the role of the experiment is to focus the mind on the possibility that diversity is real, and to shift the burden of proof against the presumption that it is not real. That reading seems to fit very nicely with what Machery et al. actually says, viz. "Thus, we conclude that it is wrong for philosophers to assume a priori the universality of their own semantic intuitions." If you are right that no one really makes this assumption, then this conclusion of the paper doesn't really establish anything interesting.

Max Deutsch

In response to Ron's earlier question about ambiguities in the probe questions, my 2 cents is that, yes, even the clarified narrator's probe is ambiguous between speaker's and semantic reference. Subjects might think that "John is actually talking about" the person he intends to be talking about, and this person might not be the semantic ref of 'Godel'. I don't think this means, as Ron suggests, that Justin and Jonathan are offering a *competitor* to the proposal that some of MMNS's subjects were expressing intuitions about merely speaker's reference. The original probe question might be ambiguous in a variety of ways. One ambiguity might explain some amount of the initial variation, while another might explain (some of) the rest. More than 25% of nonphilosophers answered (A), even to the clarified narrator's probe. Perhaps these subjects not unreasonably judged that there is a clear sense in which John is "actually talking about" whoever it was who discovered incompleteness (i.e. Schmidt) when he uses 'Godel'. On the other hand, I'm not sure why Justin and Jonathan take their perspectival ambiguity to be "similar" to the speaker's/semantic ref ambiguity, beyond the fact that they're both ambiguities in the original probe question.

One very startling result reported in S&L's paper is that 22% of their subjects gave (B) as their answer to the *John's perspective* probe! Are these subjects simply mistaken?

(Long parenthetical: Actually, I can imagine circumstances according to which it would not be wrong to think that John thinks *de re* of the proof stealer that he (the proof stealer) is the referent of some of his (John's) uses of 'Godel'. So maybe those who answer (B) to the John's perspective probe are thinking of these circumstances and interpreting the probe question in a de re fashion? Sheesh it's hard to do precise xphi!)

If so, what's to prevent the anti-xphi philosopher from saying: "People just get it wrong sometimes. If there are groups of people who think that 'Godel' refers to Schmidt in the Godel-case, or that agents in Gettier cases really know the relevant propositions, they are simply mistaken. Gettier and Kripke both *argue* for their judgments about their cases, after all. It is not as though they simply point out that their judgment is the intuitive one and leave it there.

Justin Sytsma

Ron –

I probably didn't word my reply carefully enough. What I meant to point out is that prior to the 2004 study, I hope it wasn’t the case that philosophers thought that absolutely everyone would share Kripke's intuitions about the Gödel example. For example, you (MMNS) write – "it has turned out that almost all philosophers share the intuitions elicited by Kripke's fictional cases, including most of his opponents" (2004, B3). But, if some (small minority) of philosophers don't share Kripke's intuitions about this case, then there isn’t complete universality (and surely philosophers aren’t unique in this, so we should expect that some non-philosophers don't share Kripke's intuitions). The claim of complete universality is surely too strong. If we accept as a starting point, however, that there is almost certainly some (perhaps very small) percentage of people who have divergent intuitions about the Gödel example, though, then the force of the empirical work must largely lie in showing that that percentage is larger than might have been thought.


Max –

This strikes me as correct: There is room for multiple ambiguities in the original probe… and there is probably still room for a speaker’s reference / semantic reference ambiguity in the narrator’s perspective and clarified narrator’s perspective probes. Let me add, however, that I think that there is less room for a speaker’s reference / semantic reference ambiguity in those probes than in the original.

The way in which we see these ambiguities as being similar is in that they both derive from the knowledge asymmetry that is at the heart of the Gödel example. Because there is this asymmetry, it is possible that (1) the subject could believe that John might intend to talk about Schmidt by using the name “Gödel,” and (2) the (A) and (B) descriptions could be decoded differently from either John or the narrator’s perspective. This gives us two basic ways in which a subject with roughly Kripkean intuitions might nonetheless answer (A) on the original probe: She might think that John intended to be talking about Schmidt and decode (A) from the narrator’s perspective, or she might just decode (A) from John’s perspective.

I don’t think that our probes do a good job of teasing these possibilities apart (I think that the narrator’s perspective and clarified narrator’s perspective probes make some steps toward clarifying both ambiguities). Nonetheless, it seems that this could be done. For example, we might separate out clarifications made to the body of the question from clarifications made to the descriptions. Perhaps:

Perspectival Ambiguity:
When John uses the name “Gödel,” is he talking about: (A) the person who (unbeknownst to John) really discovered the incompleteness of arithmetic? Or (B) the person who is widely believed to have discovered the incompleteness of arithmetic, but actually got hold of the manuscript and claimed credit for the work?

Speaker’s / Semantic Ambiguity:
When John uses the name “Gödel,” regardless of who he might intend to be talking about, is he actually talking about: (A) the person who really discovered the incompleteness of arithmetic? Or (B) the person who got hold of the manuscript and claimed credit for the work?

Do you (directing this toward any interested parties) think that it would be interesting to try to tease these apart more and get a better experimental answer as to how much of the variation on the original probe might be accounted for in these different ways?

Jonathan Livengood

Peirce, my favorite philosopher, once remarked on the power of testimony in shaping our beliefs, saying that on the testimony of others, a man can be convinced that he himself is mad. Incidentally, this appears to have actually happened to Georg Cantor.

I think this is relevant to the question of whether people who do not share dominant intuitions are mistaken. Being in a minority does not make one mistaken; however, being in an extreme minority is a good indication that one is mistaken and it is just as good from the perspective of belief/credibility. I don't know very many philosophers who are convinced by Berkeley's Master Argument (although they should be!); this doesn't mean that hold-outs are wrong. However, if I were the only one convinced by the argument, I should have some doubts, at least. Moreover, untutored people would be wise to severely test my claims insofar as they differ from a large majority.

A question can now be put to Ron: at what point are hold-out intuitions not worth mentioning or worrying about? If one in five billion humans dissents, is that enough for a philosophically interesting argument that human intuitions are not uniform? One in fifty billion? (Assuming we kept measuring for the next couple hundred years.)

Supposing this amount of dissent is agreed to be no support for the philosophical argument being run by MMNS, we are now haggling over price: just *how much* variation is interesting?

jonathan weinberg

The more pressing questions aren't really "how much?" but "what kind?" and "how can we tell?"

As Ron puts it, the presence of a smallish minority is still a problem so long as you can't tell "whether it is plausible to think that the [minority] represents genuine, systematic, non-errant diversity or whether it is simply noise." For some sufficiently small total number of dissidents, I suspect that "noise" becomes the default explanation. Moreover, for small enough numbers, one has to worry about the amount of noise in one's experiments as well (which is why 80% or so is often considered basically the ceiling for a lot of survey instruments).

Nonetheless, in percentage terms, even 10% of the whole world would still be a _lot_ of people, after all -- hundreds of millions, and something in the neighborhood of the entire population of Europe. So I suspect that the percentage values for which "well, that's just noise" is a reasonable response -- the values that separate substantive-dissent-minority from rounding-error-minority -- are going to be pretty darn small. And we are thus stuck with the "how many" question just not generally being able to do the work we might have hoped it could do. It's really going to matter what the fault lines for the diversity turn out to be, and what kind of epistemic story one might be able to tell in order to suggest why the minority view might, in some cases, be easily overridden (e.g., colorblindness), or in some cases, be valorized (e.g., the Stanovich & West stuff).

And you're just not going to be able to develop such a story without doing experimental work. (This is the "how can we tell?" question I mentioned above.) This just isn't something that is even remotely discernible from the armchair. And the fact that our armchair methods have been so ill-equipped to look for or notice what diversity there is, is a reason for concern about those methods; or, at least, about those methods when unsupplemented by experimental methods as well.

Lisa Lederer

Sorry for not responding to your question sooner, Justin. Phrasing the probe in the following way might test the possibility I raised:

"One day, John mentioned the name ‘Gödel.’ Was he talking about . .”

This seems to specify that we are talking about a single instance, so it would be less likely that participants would interpret “both” in the response option to mean “different people on different instances.” To make sure, we could also simply add to choice (c), “both at the same time.” Does this seem sensible?

Also, if my friend (the author of the e-mail message above) is right, we could probably find historical evidence that in China, a single proper name has been used to refer to (what we would consider) multiple entities. While it is possible that the use of proper names in literature does not match with people’s intuitions about what they mean in everyday conversation, this seems unlikely.

Justin Sytsma

Let me try a different approach:

Say that we all accept that roughly 15% of Westerners have descriptivist semantic intuitions about the Gödel case (so we have a substantive-dissent-minority). What is the positive argument for this minority opinion undermining the use of the majority intuition as evidence in this case?

Note that unless I’m missing something, MMNS (2004) don’t give any argument to this effect. Instead, they assume the much greater variation that they show and focus on the cross-cultural case. For example, they write:

“We find it wildly implausible that the semantic intuitions of the narrow cross-section of humanity who are Western academic philosophers are a more reliable indicator of the correct theory of reference … than the differing semantic intuitions of other cultural or linguistic groups. Indeed, given the intense training and selection that undergraduate and graduate students in philosophy have to go through, there is good reason to suspect that the alleged reflective intuitions may be reinforced intuitions. In the absence of a principles argument about why philosophers’ intuitions are superior, this project smacks of narcissism in the extreme.” (2004, B9).

As given, this argument cannot be repeated for intra-cultural variation within Westerners if we accept that the intuitions of academic philosophers for this case squarely reflect the majority opinion.

So, what is the argument here? An argument could, of course, be run that what the minority shows is that the answer is not patently obvious, or that intuitions about the Gödel case are not in the same boat as intuitions about 2+2, for example. It seems to me, however, that that point is pretty clear and that the empirical work was supposed to do something more than this. In thinking about this, the arguments that I come up with are essentially theoretical arguments against using intuitions about cases that are not patently obvious as evidence; but, such arguments don’t really rely on showing the size of the minority and it wouldn’t much matter if the minority constituted 5% of Westerners or 1% or just a few philosophers in Pittsburgh.


Lisa –

If this is correct of Chinese speakers, do you think that it would support the claim that they have descriptivist intuitions, or point out another confound? Let me give a common American example: While Lassie is a proper name, I feel pretty comfortable in using it to actually refer to any member of a group currently consisting of (I believe) nine collies in the blood line of the original Pal. In a case like this it seems like a proper name has become more like a label of office (like Caesar or Pope). If Chinese speakers are simply much more willing than English speakers to let a proper name morph in this way, that would seem to be distinct from having descriptivist intuitions.

Ron Mallon

Max-
I agree with Max that both explanations may have a role to play in explaining different groups of subjects’ replies. By calling them “competitors” I meant only to draw attention to the idea that where one explanation is successful, the other is displaced from explaining particular subjects’ responses. Thus both the speaker’s reference/semantic reference hypothesis and the Sytsma/Livengood hypothesis compete with the semantic diversity hypothesis put forward by Machery et al, even though it’s possible that each of them explains the responses of some subjects/groups of subjects. This is compatible with the analysis Max offers, so I suspect we agree on this.

With regard to Max’s closing query: When you say,
“Gettier and Kripke both *argue* for their judgments about their cases, after all. It is not as though they simply point out that their judgment is the intuitive one and leave it there,” I am assuming (at least limiting ourselves to Kripke’s case) that you mean that there are other arguments in the text. That is surely right. Still, the question is whether intuitions are a source of evidence for selecting the theory of reference. If they are, then evidence about them is relevant to the arguments or appeals in which they figure.

Justin’s first post-
I think we’ve got some kind of disconnect here: the question everyone is raising about ambiguities in the probe is meant to draw attention to the fact that answers to the original probe may not reflect bona fide semantic intuitions. (This is what I meant by “genuine diversity.”) When I said that I thought many philosophers do assume universality, I meant that they assume universality of semantic intuitions. The quote you give from our paper is ambiguous (though I think in a perfectly innocuous way) between whether what almost all philosophers shared was semantic intuitions or simply intuitive judgments (which can have many causes).
So, and in keeping with the previous posts, on my view, what’s relevant is whether we think there is diversity in semantic intuitions. The reason that the cross-cultural work is important is that it suggests diversity produced by cultural difference, which suggests that it isn’t simply noise (though – as critics point out and the original paper conceded) it doesn’t prove that we’ve revealed semantic diversity. Regarding the conclusion of semantic diversity, I don’t think the percentages matter except in the sense that where the percentages become small, one starts to suspect that they are simply noise because as a methodological fact, experiments never perfectly realize ideal conditions.

Jonathan L.-
I think it might be useful to distinguish three questions.
1. Whether there is semantic diversity is a metaphysical question.
2. When the number of a diverse semantic minority becomes so small that it is justifiable to assume it doesn’t exist (on the grounds that the signal from the minority looks like noise) is an epistemological question.
3. When the number of a diverse semantic minority is so small that it is not important is a question of philosophical significance.

On the metaphysical question: MMNS ‘04 have suggested that there is evidence consistent with semantic diversity, and tried to raise questions about what that would mean for the philosophy of language and arguments from reference.

On the epistemic question: I have no idea where the border is, and I don’t think you do either. Still, even where the minority is very small, other evidence could override the supposition that what we’re seeing is noise. For example, data showing that some individuals systematically answer diverse probes in distinct ways in possession of full information etc. would override our treating even small minorities as noise. Data showing that diversity correlates with other factors like culture can also show that it’s not just noise (though it wouldn’t show that it’s not just error).

On the general philosophical question: This question is too general to address here, but I certain am not arguing that the pervasiveness of semantic diversity does not matter at all. For example, I think that some of the arguments in Mallon et al. (forthcoming) regarding arguments from reference does depend on semantic diversity being significant. But exactly when it’s okay to ignore is going to depend on exactly what assumptions and arguments philosophers are making – what the assumption of greater universality is doing.
In any case (and in agreement with Jonathan W.), I persist in insisting that the main issue is not how much semantic diversity but whether there is semantic diversity and how we tell.

Justin’s second post:
You write, “What is the positive argument for this minority opinion undermining the use of the majority intuition as evidence in this case?”

It seems like there's an epistemic claim and a metaphysical claim here. The epistemic claim is this: given the data patterns simple surveys reply, as a group of respondents grows smaller, we are more entitled to treat the replies as not exhibiting genuine semantic diversity. I’ve conceded that percentages may have some role to play in whether we think the data reveals semantic diversity, a role that is especially strong as the minority goes to zero (because we start to think it’s noise), but a role that might be overridden with further study.

But it seems like you are arguing for a metaphysical claim: viz. that if we have a statistically small group that reveals semantic diversity (viz. all their judgments about reference are systematically in line with an alternate theory when all disambiguating factors have been checked for), that nonetheless, they are wrong about reference.
If this is what you are arguing for, then doesn’t this seem like a peculiar form of linguistic-centrism (similar to saying that some other linguistic group speaks the wrong language, on the grounds that the group is small)? Isn’t the much more tempting thing to say that: “this group of people has one concept of reference and this other group has another?” Or is this different than what you mean in some crucial way? (I’m wondering if Jonathan W. and I aren’t just in the grips of a very different idea about what’s "obvious" to say in this sort of case).

Justin Sytsma

Thanks again all for the comments! This is very helpful… and it points out that Jonathan and I should try to make section 4.1 of the paper clearer.

In particular, it seems that there is some confusion over what we think our results show. We agree with Ron that “the question everyone is raising about ambiguities in the probe is meant to draw attention to the fact that answers to the original probe may not reflect bona fide semantic intuitions. (This is what I meant by ‘genuine diversity.’)” With regard to this we feel that we clearly show that many responses to the original probe do not reflect bona fide semantic intuitions. For the original probe, aggregating across our studies, 41.2% of Westerners surveyed (N=140) answered (B). Clarifying the probe, 73.9% of Westerners (non-philosophers) surveyed (N=119) answered (B).

Despite the ambiguity in the probe, however, our work nonetheless supports the conclusion that there is *some* genuine diversity in semantic intuitions about the Gödel case among Westerners. We suspect that the actual number is somewhat lower than the 73.9%, above, suggests (because we doubt we’ve fully clarified the probe, etc., perhaps because of residual speaker’s/semantic ambiguity as Max argues). I think our coding of subjects’ explanations suggests that a reasonable estimate of the actual diversity is 80%-90% (or, splitting the difference, that roughly 15% of Westerners have descriptivist semantic intuitions about this one case).

(If I understand correctly, this puts us (or at least me) on the same side as Ron on the first two of the three questions he distinguishes in responding to Jonathan. My primary concern is with his third question.)

My view is that 85% genuine agreement would be enough to make talk about “our intuitions” for this case reasonable (i.e., there is a clear consensus opinion). What I’m not sure about is whether the corresponding level of genuine diversity (15%) undermines the use of the consensus opinion about the Gödel case as evidence in philosophical theorizing. I’m not arguing that it doesn’t, but asking why we should think that it does.

That is, I don’t mean to be putting forward a positive argument of the type that Ron suggests: “But it seems like you are arguing for a metaphysical claim: viz. that if we have a statistically small group that reveals semantic diversity (viz. all their judgments about reference are systematically in line with an alternate theory when all disambiguating factors have been checked for), that nonetheless, they are wrong about reference.” Rather, we are asking what the finding that roughly 1 in 7 English speakers have descriptivist semantic intuitions about this one case (the Gödel example) implies about the use of intuitions about this example in theorizing. My suspicion is that the arguments that might be given to say that this diversity undermines the use of the consensus opinion in theorizing are arguments that could be run just as well if the genuine diversity was 5%, or 1%, or just a few philosophers. (And, I would have thought that nobody actually thought that there would be total universal agreement about an example like the Gödel case… so these would be arguments that could have been run just as well without any empirical data.)

In other words, one can certainly argue against the use of one’s intuitions about cases in philosophical theorizing. (Heck, I’m all in favor of doing so!) What is not so clear to me is why showing that a small percentage of people within the same culture hold the opposed intuition does much to strengthen the arguments against that brand of philosophical theorizing.

Lisa Lederer

Justin,

I do think the possibility I raised is another potential confound, rather than further evidence that the Chinese have descriptivist intuitions.

The comments to this entry are closed.

3QD Prize 2012: Wesley Buckwalter