Pereboom @ Science, Religion, & Culture


Blog Coordinator

« X-Phi in Scientific American | Main | Asymmetry of Intentionality Judgments in Causal-Deviance Scenarios »

10/24/2011

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Ray Lopez

Interesting but the analysis may be flawed. The authors should have employed a 2x2 ANOVA to analyze the results, rather than t-tests. Also, on the graphs, show us your error bars.

Jonathan Weinberg

@Ray - Is your concern about the assumption of equality of variance? Is that really such a problem, if the size of the samples are about the same?

Error bars would be nice, though.

Shen-yi Liao

Really cool stuff!!!

I wasn't sure what Figure 2 is supposed to show. Is the difference between Kinkade one-exposure and Kinkade multi-exposure statistically significant?

Mark Phelan

I’ll respond to the questions about the analyses since, in connection with experiment month, I ran them.

Ray, thanks for the question. The reason I opted for running T-tests rather than an ANOVA had to do with the fact that the numbers of Millais paintings vs Kinkade paintings were so different. Remember, the study looked at sixty paintings, 12 of which were late Millais painting and 48 of which were Kinkade paintings. The relevant analyses—the means of which are captured in the second bar graph—compare via T-tests 1) mean liking ratings for the 6 single-exposed Millais paintings to mean liking ratings for the 6 multiply-exposed Millais paintings, and 2) mean liking ratings for the 24 single-exposed Millais paintings to mean liking ratings for the 24 multiply-exposed Millais paintings. In other words, the dependent variable for each comparison is an average of the liking-ratings participants gave for all the paintings in the relevant condition. But since the number of paintings in each of the Kinkade conditions was much higher than the number of paintings in each of the Millais conditions, there’s a sense in which the dependent variable for the Millais paintings is not the same dependent variable for the Kinkade paintings. True, both are average liking-ratings, however they are averages over much different numbers of paintings. Doing T-tests on each painter instead of an overall ANOVA standardized the number of paintings from which the dependent variable is constructed—for one T-test, we are comparing the average liking-score for 6 singly exposed Millais paintings to the average liking-score for 6 multiply exposed Millais paintings; for the other T-test, we are comparing the average liking-score for 24 singly exposed Kinkade paintings to the average liking-score for 24 singly exposed Kinkade paintings. (Perhaps Margaret can address the reason why the numbers were so disparate for the good and bad artist. I believe it had to do with the desire to avoid canonical good landscapes and the difficulty of finding a large number of those by the same artist.)

I’m willing to consider, though, that perhaps the fact that the average is based on such dissimilar numbers of paintings is irrelevant (after all, it is the same dependent variable in the sense that it is an average liking-rating). In that case, the ANOVA is the appropriate analysis. Thus I have now gone back and run a 2 factor (Artist, Exposure) repeated measures ANOVA. Here, we find a main effect for Artist (Millais preferred to Kinkade, p=.011, Greenhouse-Geisser test), and a main effect for exposure, in the opposite direction of Cutting (single-exposed preferred to multi-exposed, p=.022, Greenhouse-Geisser), however, the interaction is not significant (p=.095, Greenhouse-Geisser). So, these results tell against Cutting's hypothesis—here mere exposure resulted in overall less preference. But these results don't, in and of themselves, reveal a difference for good and bad art. However, the overall means and also the previously conducted T-test results do suggest the difference for good and bad art. And, anyway, the real key points in favor of the hypothesis are the between participant results for unexposed controls verses the exposed test group.

I believe Margaret et al have the standard errors for the means, and can report those.

The comments to this entry are closed.

3QD Prize


Knobe's X-Phi Page