If a result fails to replicate, then it is not a finding but rather an unlucky aberration. This post isn’t about replication failure (or replication-failure failure). It’s about something I call “unfinding.”
The following combination is not an uncommon occurrence:
(1) a result (“R”) replicates and is a finding, but
(2) different stimuli or procedures produce a different result (“R*”).
In many cases, the combination of (1) and (2) provides evidence for a more detailed or precise interpretation of R. Of course, in any particular case, people might reasonably disagree about whether the added detail constitutes a significant advance in our understanding of the underlying issues. Still, I think that there should be a presumption in favor of welcoming and encouraging work aiming to add detail.
Unfinding goes beyond merely reinterpreting R. Instead, unfinding occurs when (1) and (2) support a more radical conclusion, namely:
(3) R is uninformative (relative to the primary research question).
Uninformative findings can be set aside. They do not constrain future research on the topic, except insofar as it should not repeat the mistake.
Here is a hypothetical example illustrating an unfinding. Someone hypothesizes that the concept of physical beauty is partly constituted by visual symmetry. In a series of studies, manipulating visual symmetry strongly affects beauty judgments (R). This is taken to support the hypothesis. Subsequent research reveals that the symmetry manipulation was systematically confounded with changes in hue or saturation, and once those factors are controlled for, symmetry does not affect beauty judgments (R*). In this case, I think it’s fair to say that the original result is not genuinely informative relative to the primary research question. As an indication of this, if subsequent research on beauty judgments failed to find an effect of visual symmetry, the authors would not be obliged to explain the apparent inconsistency with the original result.
Unfinding is a possibility for just about any result, and we should be open to it in principle. Nevertheless, I do not think that there should be a presumption in favor of welcoming or encouraging work that aims at unfinding.