For everyone who has commented on the paper and offered feedback, thank you very much. Two attempts at replication have been made: one by John Turri (n=196 after comp checks), the other by Joshua Knobe (n= 136 after comp checks). Here is an overview of their findings. The replication data will be made available upon email request.
First, John’s study replicates the primary finding of our paper: participants attribute knowledge in the fake-barn vignette. Josh’s study does this as well.
Second, John’s study does not replicate the correlation between age and knowledge attribution. John’s population is demographically different from the population tested in our study, but with 19 individuals 49 years of age or older, there is some representation of older individuals. Josh’s study does not replicate the correlation either. Josh’s population had 20 individuals who were over the age of 40.
Third, John’s findings suggest a trend of age and gender interaction on knowledge attribution. For men, there is a trend that higher age predicts lower knowledge scores, while there is no trend for women. Neither trend is significant. However, Josh’s findings do not suggest a trend of this sort.
The original study and the replication studies were distributed using different means, and also different population demographics. One or both of these differences may explain why the age finding failed to replicate. The age range of the original study may be different from the online studies, which may affect the knowledge score correlation. More plausibly, the older population surveyed in the original study (in person on paper) may be unlike the population surveyed in a replication (online via the m-turk service). If they are different, more research must be done to assess which population, if either, is more representative of older people more generally.
I take these results to suggest that, if there is a relationship between age and knowledge attribution in fake-barn cases, it may be more complex than the original study was able to measure; knowledge attribution may be affected by a variety of interacting demographic factors, age included. This is in line with the cautious conclusion of the original paper. I would like to see more studies on the topic of intuitions and age, which perhaps will further elucidate their relation, if any.
Once again, I would like to thank John and Josh for running the replications, and everyone who has commented on the blog posts.