Expert Intuitions About Knowledge

This is kind of a follow-up to Joshua Knobe’s very helpful post on the state of play in the experimental philosophy of expert intuitions. Even though there is already plenty of psychological research on expert intuitions in general (check here for a recent summary), we are a bit skeptical about mere arguments from analogy that try to generalize from these wider results to expert intuitions about philosophical thought experiments (see e.g. here or here). One of the paradigms of psychological research on expert intuitions is the famous fire fighter who e.g. intuits that a burning house is about to collapse. But intuitions about thought experiment cases are very different from that. They are not causal intuitions, and they are also not predictions under uncertainty. In a good thought experiment, all the relevant facts are already stipulated, or at least can be stipulated if needs be. We think that these disanalogies considerably weaken the generalization from psychology to philosophical expert intuitions. Be that as it may, one can hardly disagree with Josh’s claim that “the most direct way of going after an issue like this one is to look at experimental studies that actually examine the philosophical intuitions of professional philosophers” (which one of us also recommended in an earlier paper).

Strikingly, work on expert intuitions about knowledge is still mostly absent from Josh’s list of available papers on expert intuitions, even though this is one of the classic targets of experimental philosophy (the paper by Vaesen and Peterson may seem like an exception here, but most of their one-sentence vignettes are clearly not thought experiments in the standard sense). Alex Wiegmann and I conducted two studies that may help to begin filling this gap. Before reporting our most interesting results, we want to highlight some of the distinctive features of our methodological approach.

First, unlike many of the existing studies, we set the standard for philosophical expertise fairly high. In our first study, the final sample of 84 experts only includes people with a PhD in philosophy who have epistemology as one of their areas of competence. In contrast, other studies often merely require that people took a few college courses in philosophy, which clearly strikes us as an insufficient operationalization of philosophical expertise.

Second, in order to actually engage the alleged expertise of philosophers and to prevent that they merely draw e.g. on their memorized verdicts about famous cases, such as the Gettier cases, we tried to come up with new versions of existing thought experiments from the epistemological literature. In addition to that, some of the cases we chose should be unfamiliar even to many epistemologists. For example, we included a version of one of Gilbert Harman’s famous cases on the theme of evidence that one does not possess – yet not his well-known assassination case, but the the unopened letter case, which is pretty much ignored today (cf. Harman 1973, Thought, ch. 9). We also chose cases where there is a reasonably clear consensus about their intuitive evaluation in the epistemological literature. Harman’s cases, for example, are today widely considered as inconclusive. For illustration, here is our Harman-inspired vignette (called ‘Painting’):

A CEO has owned a famous painting for a long time. Just this morning, she showed it to a colleague in her office where she can watch the painting every day. In the evening, the CEO returns home from work and is about to check her email. What she will find there is an urgent note by her secretary that says that her painting was stolen shortly after she had left her office. In fact, the CEO’s colleague merely wanted to play a trick on her, and so it was actually him who used the secretary’s email account to send her this note. The painting itself is still on the CEO’s office wall, just as it always was.

How much do you disagree or agree with the following claim:

At the time when the CEO returns home, but before she actually checks her email, she knows that the painting is still on her office wall. 

strongly disagree, disagree, mildly disagree, mildly agree, agree, strongly agree 

The expertise-friendly hypothesis behind our first study was that the philosophical experts – in contrast to the lay people – should come significantly closer to the verdict about the cases that one would expect based on the epistemological literature. Since we used a six-point Likert scale to measure the subject’s responses, one would expect, for example, that the mean expert verdict about our Harman-inspired case should be significantly closer to the middle of the scale than the mean lay verdict.

Now, the “good news” from the philosopher’s point of view is that this really turned out to be the case in all but one of the six tested cases (two of which where control cases – one a clear case of knowledge, the other a clear case of non-knowledge) – see chart below.

But here comes the “bad news”: except in the control case of non-knowledge, all of the mean expert responses are above the mean of the Likert scale, that is, the relevant cases are on average evaluated as cases of knowledge. Since we also included a case that is inspired by Ginet’s fake barn scenario (called ‘Sculpture’), this came as a bit of a surprise to us.

 

For this reason, we conducted a second study with vignettes that the epistemological literature clearly regards as non-knowledge. Overall, we included two variations on the fake barn theme (‘Sculpture’ and ‘Dollar’), and one case that is modeled after lottery-style cases (‘Exam’). Interestingly, we basically got the same result (see chart below). Even though the mean expert verdicts were closer to the negative side of the Likert scale than the mean lay verdicts, they where all clearly on the positive side. That is, even our experts mildly agree, on average, that all of the non-control cases are cases of knowledge, despite the fact that the epistemological literature treats cases of this kind as clear cases of non-knowledge. By the way, we were also able to replicate the respective results from our first study almost exactly, which suggests that there is a considerable robustness to our findings.

 

We are curious to learn what readers of this blog might think about our findings. And please contact us if you want to see some of the details of our data.

 

Addendum (08/21/2013) – the other case descriptions

Sculpture

The director of a sculpture museum is so impressed with recent improvements of hologram images that she decides to perform a secret test on the visitors of her museum. To this end, she orders hologram images that even art experts cannot visually distinguish from the real sculptures in her museum, and she replaces all but one of the sculptures by their hologram image. As the director had expected, no one realizes any difference between the hologram images and the real sculptures. One day, the world’s greatest Rodin expert is visiting her museum. The expert is standing in front of a famous marble sculpture by Rodin, which is the only real sculpture that is presently on display in the museum, and she thinks to herself: “I’m facing one of Rodin’s famous marble sculptures now.”

Monitor

A security guard monitors a number of video screens that show what is going on in an office building across the street. At midnight, the guard checks her video screens and sees, on five different screens that show five different offices, that people are still working in their offices. She thinks to herself: “There is still someone in the building.” Four of the five screens are in fact working properly. But due to some unusual malfunction, one of the five screens actually shows a videotape from the night before, and today this particular office is already empty.

Exam

A professor of medicine learns that one in a billion students die while taking their final exam. In about five minutes, the professor is set to substitute for a sick colleague in conducting a final exam of a student that she had never met before. The professor thinks to herself: “The student will survive her final exam.” And in fact, the student does survive her final exam.

Dollar

A waiter was recently hired by a restaurant in a remote part of the country where, unbeknownst to everyone, most of the circulating dollar bills are fake. The restaurant manager owes the waiter ten dollars for an extra hour that he worked. She takes a ten dollar bill from the cash register and hands it over to the waiter, and this leads the waiter to believe that the restaurant manager has just given him ten dollars. In fact, this particular ten dollar bill happens to be one of the few genuine dollar bills in the area.

Control Non-Knowledge

A little girl likes to play a game with flipping a coin. She sometimes gets a “special feeling” that the next flip will come out heads. When she gets this “special feeling”, she is right about half the time, and wrong about half the time. Just before the next flip, the little girl gets that “special feeling”, and the feeling leads her to believe that the coin will land heads. She flips the coin, and it does land heads.

Control Knowledge

Before leaving the house, a woman wants to check whether the light in her bathroom is off. She opens the bathroom door, and it is completely dark inside. She thinks to herself: “Alright, the light in my bathroom is off.”

Comments

[Dustin Locke]

Very interesting stuff. Can you tell us more about your method for finding expert subjects? Also, could you share the vignettes for 'Sculpture' and 'Dollar'? Thanks!

[Eddy Nahmias]

Hi Joachim, fascinating results. I feel like I took this survey and probably responded above the midpoint for most of the relevant cases. My intuition on the painting case is that she knows the painting is there. Can you post the other cases in comments so people can see them? My gut reaction to your results is that they indicate that the 'super-experts' who use these cases to try to raise the standards for knowledge (or reliability) are trying to raise them too high, certainly higher than the ordinary standards for knowledge. Contextualists should be happy enough with that response and these results, right? Also, did you have any tests for participants' confidence in their judgments?

[Joachim Horvath]

Hi Eddy,

by 'super-experts' you probably mean those philosophers who publish papers on knowledge in top-journals. You might be right that these super-experts set the standard to high (maybe because they are hyper-scrupulous, or maybe because they just like to make life hard for themselves because the love philosophical puzzles?). But on the basis of our data, this is of course pure speculation, for we haven't tested the super-experts as a separate group. I do remember, however, that Alex said that it doesn't make much of a difference if one looks only at people with a philosophy PhD who have epistemology as one of their areas of specialization (which should come pretty close to super-expertise).

I don't yet see that our results fit the standard contextualist story so nicely. As a contextulist, one would predict that the experts' tendency to ascribe knowledge is always lower compared to the laypeople. But in case of Monitor, which is an unusual type of case that is clearly evaluated as knowledge in the literature, the experts are actually more likely to ascribe knowledge than the laypeople. So, fitting this into the contextualist framework is probably not as straightforward as you suggest.

By the way, you'll soon find the other case descriptions in an addendum to our post above.