Advances in Experimental Epistemology

James Beebe's is the first anthology devoted exclusively to experimental philosophy of knowledge. It is essential reading for scholars interested in the role that stakes and the possibility of error play in the attribution of knowledge, in particular, the role that experimental studies of these contextual details can play in the choice between semantic contextualism (Cohen 1999, DeRose 1992, 1995) and subject-sensitive/interest-relative invariantism (Stanley 2005, Hawthorne 2004). Beyond that, Derek Powell et al.'s "Semantic Integration as a Method for Investigating Concepts" should interest experimental philosophers looking for novel materials and tasks, and Jonathan Weinberg's "The Promise of Experimental Philosophy and the Inference to Signal" should enrich the ongoing, meta-philosophical discussion of what experimental philosophy can add to traditional, "armchair" philosophy.

Throughout the volume, data are brought to bear on whether knowledge attributions are sensitive to local detail. Most of the studies ask participants to respond to a series of vignettes in which one to three local factors, such as stakes and/or chance of error, are varied. After participants read each vignette, they are asked to rate on a Likert scale how strongly they agree or disagree with a statement that attributes knowledge to the vignette's protagonist. By comparing participants' responses to the different vignettes, one can determine whether there is a correlation between changes in stakes, for example, and participants' willingness to attribute knowledge to the vignette's protagonist. Note that this is not a poll in which participants' responses are reported as votes for a certain philosophical position. This is a controlled experiment that seeks to uncover a regular, empirical relationship, ceteris paribus, between two observables, in this case, researchers' descriptions of stakes and participants' ratings of knowledge attribution. One might predict that participants will be more likely to attribute knowledge to the protagonist in scenarios where the stakes are low and errors have little practical cost. Knowledge is easier to obtain in these scenarios, the intuition goes, because less evidence is needed. Such a prediction would threaten epistemic purists, who do not think considerations of practical interest should appear among the truth conditions for "knowledge" sentences; the strength of evidence alone should determine whether a subject knows.

In "Experimental Evidence Supporting Anti-intellectualism About Knowledge," Ángel Pinillos and Shawn Simpson use evidence-seeking probes to investigate the level of evidence required for knowledge ascription in the face of changing stakes. Their second experiment includes two high stakes conditions, in which a plane could be hijacked if the protagonist were to err, and two low stakes conditions, in which a nice guy could remain in coach instead of moving to first class, if the protagonist were to err. In results that conflict with "negative first-wave" studies (Buckwalter 2010), they find that practical interests play a role in folk knowledge ascriptions: increases in stakes correlate with increases in evidentiary requirements, as participants require the protagonist to check his work at least twice more in high stakes conditions, but only 1.6 times more in two low stakes conditions (21).

Critics worry that Pinillos and Simpson's participants are thinking about whether or not the protagonist counts as responsible, given the possible consequences, and not directly about the word "knows" and how it is used.^[1] As Wesley Buckwalter and Jonathan Schaffer (2013) write: "Everyone can agree that the practical consequences of error matter when it comes to considering what one needs to do, normatively speaking". Pinillos and Simpson suggest in their third study that their earlier experiments do get at the concept of knowledge, because even if participants thought they were answering a question about what the protagonist Jessie should do, that question requires participants to consider the number of times Jessie would have to check the list, i.e., at what point Jessie could be said to have knowledge. In fact, participants say that Jessie should examine the manifest 1.75 more times to be in the right, and they respond that Jessie needs to examine the manifest 1.87 more times before he counts as knowing to an ordinary person (27). Because the means for the normative prompt and the knowledge prompt correlate across all four conditions, Pinillos and Simpson hold we have reason to think that subjects accept the Reason-Knowledge Principle (RKP): it is appropriate to treat proposition p as a reason for acting iff you know that p. Jessie needs to check about two more times, in order to act well, because he needs to check about two more times, in order to know. The data in this third study certainly do not rule out the authors' defense here, but arguing that the data support the "because" in the previous sentence could be a stretch.

In "The Mystery of Stakes and Error in Ascriber Intuitions," Buckwalter also worries that Pinillos and Simpson's studies tell us little about how the concept of knowledge is used, because participants do not distinguish between words like "knows," "believes," "guesses," and "hopes" (160-62). Stakes matter, but they matter equally to the ascription of several different mental states.^[2] Buckwalter further questions stakes effects on the ordinary evaluation of third-person knowledge sentences by varying the protagonist's speech act and making the possibility of error salient. Across all levels of stakes, when the possibility of error is made salient, participants tend to report that the protagonist's assertion "I know" is false and the protagonist's denial "I don't know" is true (156). More than stakes or error, the form of the speech act to which participants are asked to respond has the most pronounced effect on knowledge ascription, which Buckwalter attributes to accommodation (152, 156): "knowledge sentences involving an assertion, no matter the error or stakes of the case, were more likely true than knowledge sentences involving denial" (154).^[3]

Pinillos and Simpson respond to Buckwalter in typical X-phi fashion: with a further study. They ask participants to answer "hopes" and "knows" questions concurrently. When participants are asked to consider these questions one after the other, in either order, a statistically significant difference emerges. The "knows" means are in the high threes and the "hopes" means are in the mid-to-high ones (34). Less evidence is required for the protagonist to hope he has no typos than to know he has no typos in both the low and high stakes conditions, yet the stakes effect persists in the "knows" conditions. This suggests that when participants see the terms "hopes" and "knows" together, they do discriminate. Provided with a contrast class, they assume that researchers are trying to ask two different questions and please them accordingly.^[4]

Picking up on this issue, Nat Hansen analyzes a series of elegant experiments by Mark Phelan (2014) that find practical interest effects appear in within-subject designs, that is, when each participant is exposed to every condition, but disappear in between-subject designs, that is, when each participant is exposed to only one condition. Philosophers reading epistemology papers experience contexts jointly, as in the within-subject design. So Phelan's data raises the possibility that philosophers "are mistakenly offering theories that aim to explain what turns out to be merely an artifact of their particular experimental design, rather than a fact about judgments made in ordinary circumstances" (78-9).

Critics could object that the stakes effects are genuine, and not artificial. It's just that when contexts are evaluated separately, the stakes are not a prominent feature of the context; they're not salient. But Phelan does not accept this response, because an equally salient factor to stakes was detected when contexts were considered separately: he was able to discern a reliability effect (80). Hansen's objection is that stakes and reliability are not equally salient: it is more difficult to evaluate what is at stake in a single context than it is to evaluate the reliability of an information source in a single context, because in a between-subject design, in which participants read only one vignette, there is no scale available to assess stakes. In contrast, "the reliability of an information source has a clear upper and lower bound: a source can be 100 percent reliable, or completely unreliable" (83-84). Hansen succeeds in motivating an equipollent position here. But to make his case, he would have to address the fact that in most high stakes vignettes the protagonist faces either the death of himself or a family member. Even after reading only one vignette, participants understand that the stakes are very high, which generates, at least, the top end of a scale.

Joshua Alexander, Chad Gonnerman, and John Waterman test Jennifer Nagel's suggestion (2010) that when the possibility of error is made salient to participants, they egocentrically require the vignette's protagonist to respond to that possibility: "we treat him in the detailed story as sharing our concerns and failing to act accordingly" (100). Participant psychology is sensitive to contextual detail (e.g., chance of error), but the concept of knowledge is not. In their second study, the authors vary who is aware of a defeater -- i.e., the protagonist and participant, or just the participant -- and the protagonist's epistemic energy -- i.e., whether the protagonist actively "entertains" the possibility of being wrong or whether this is left "neutral." Across all four conditions, participants are on the fence as to whether the protagonist "knows," but they generate the same on-the-fence numbers in all four conditions (104-05). Participants hold the protagonist to the same epistemic standard regardless of whether he is aware or unaware of the possibility of error, which seems consistent with Nagel's suggestion.

To convincingly show that participants reading the narrator cases are epistemic egoists withholding knowledge ascriptions from the protagonist on the basis of their awareness of the error possibility, the authors would have to demonstrate that participants were able to distinguish the narrator and subject conditions in this between-subject design. In short, the narrator vignettes may need to be more explicit: "You know, as an outside observer, that a white table under red lighting would look exactly the same. But, from John's perspective, the possibility of error does not exist. The possibility that the table could appear red but really be white is not on the table for John." The question asked to participants, then, would be: "To what extent do you agree or disagree that, from John's perspective, John knows the table is red?" This might bump up the knowledge attribution numbers for the protagonist in the narrator cases. Invariantists would complain that these changes stack the deck against invariantism, but to refuse to make such changes is to stack the deck against the alternative.

In "Winners and Losers in the Folk Epistemology of Lotteries," John Turri and Ori Friedman run five experiments to investigate why some forms of justification are more compelling than others. According to epistemologist's intuitions, testimony seems to trump statistical reasoning: in the "skeptical lottery judgment," even if the odds are very high that a ticket is a loser, the ticket holder believes the ticket is a loser, and the ticket is in fact a loser, people will say the ticket holder does not know the ticket is a loser. However, in the "non-skeptical lottery judgment," if a newscaster reports that a ticket is a loser, the ticket holder believes the ticket is a loser, and the ticket is in fact a loser, people will say the ticket holder does know the ticket is a loser. Testimony seems to trigger knowledge ascription in the non-skeptical lottery judgment, but even overwhelming odds will not trigger knowledge ascription in the skeptical lottery judgment.

The authors find in their first two experiments that non-philosophers do in fact share both the skeptical and non-skeptical lottery judgments. Moreover, though epistemologists argue chance of error should inhibit knowledge ascription, in the "Odd News" case, which includes both statistical and testimonial forms of justification, sixty-six percent of participants report that the protagonist "knows" the ticket is a loser, despite the fact that ninety percent report a chance of error (52). The mention of statistical reasoning triggers chance of error, but the mention of testimony swamps that to trigger knowledge ascription. The human connection that comes with testimony may be a more forceful driver of knowledge ascription than the presence of absence of chance of error (which accompanies the presence or absence of statistical reasoning).

In the fourth experiment, the authors are able to boost knowledge ascription in statistical conditions by moving away from vignettes about lotteries and writing vignettes about the chances of having Obama's/Brad Pitt's phone number on the serial number of a dollar bill. The authors hypothesize that the "formulaic expression" of lottery cases accounts for the difference (59, 66). But a closer look at those vignettes reveals that, in many of the cases of increase, human testifiers, in the form of the evening news and Stan, have been added to the statistical story. It's only in State Odds versus Mafia that stats trump testimony, and this could be due to participants' general suspicion of the Mafia.

David Sackris and Beebe offer three polls to support the claim that there are, in fact, many individuals who will ascribe knowledge in the absence of (authentic) justification, after Crispin Sartwell. On the basis of the second study, the authors have the ability to think about the ways in which two variables affect knowledge attribution: (1) how the knowledge was acquired (i.e., a priori or a posteriori) and (2) the point at which the protagonist knows (i.e., while delusional or afterwards). But they conclude only that across their five demon/dream-justified, true belief cases, 54.4 percent of participants were willing to ascribe knowledge to the protagonists. The highest mean was 4.85 -- in between 4, "neither agree nor disagree," and 5, "slightly agree," on a Likert scale (187).

Given Turri and Friedman's work, it's hard to determine whether the demon testimony reads to participants as lack of authentic justification (as the authors hope), because it's a demon, or as authentic justification (which would be troubling), because it's testimony^.[5] But more importantly, the studies do not show that knowledge attribution is not sensitive to the presence and absence of (authentic) justification in contexts in which protagonists have true beliefs, because there is no control condition. To get at this issue, the authors would have to design an experiment in which, at the very least, the protagonist has justified, true belief in one vignette and only true belief in another vignette. They could then add vignettes with demons, dreams, and countervailing evidence. The study on offer is a provocative first step, however. If knowledge ascription is not in fact sensitive to the presence or absence of justification, future work in this area could have quite a bit of explanatory power in the social-political realm.

Derek Powell, Zachary Horne, and Pinillos review worries about using survey instruments and propose a new way to identify the concepts relating to the concept of knowledge. During language processing, underlying meanings, not original forms, are coded into long-term memory, which means people recall semantic information better than verbatim utterances. When presented with a list of words like "glass," "pane" and "shade," for example, participants later recall the word "window," even though it never appeared on the list; "the words in the list semantically activate the word 'window' . . . they cause people to form or retrieve stored mental representations associated with the word" (126).

Using a model of language processing developed by Dedre Gentner, Powell et al. find that after participants read the critical verb "thought" alongside words picking out the concepts of truth, justification, and belief, participants later falsely recall words picking out the target concept of knowledge (i.e., words like "knew") more frequently than they did in a control condition in which they read the critical verb "thought" alongside words picking out belief only (132). In a second study, participants were more likely to falsely recall "knew" in a justified, true, belief condition and in a Gettiered condition than in a false belief condition (133). The authors believe their studies support the claim that the folk have a traditional JTB concept of knowledge. But their results seem better equipped to support Sackris and Beebe's claim that truth matters more than justification -- authentic or otherwise -- in knowledge ascription.

In an early paper distinguishing experimental philosophy from empirical philosophy, Jesse Prinz noted that there are several methods experimental philosophers might employ, beyond surveys, to investigate correlations between folk concepts and/or terms (Prinz 2008, 203). Since then, experimental philosophers have conducted reaction time studies with perceptual-motor tasks (Arico et al. 2011, Guglielmo and Malle 2010) and studies in which verbal protocols were analyzed in response to visual targets (Chin-Parker and Bradner 2010), among other non-survey-based studies. Powell et al.'s false recall study with its memory task is welcome addition to this newly plural methodological landscape.

Experimental epistemology makes two implicit claims: (a) that the empirical relationship discovered between the two observables indicates something about how the philosophical concepts (of knowledge and stakes, for instance) are ordinarily understood and employed in social transaction -- something about ordinary socio-linguistic reality -- and (b) that the ways in which concepts are ordinarily understood and employed in social transaction should constrain our philosophical theories (of knowledge, for instance). Most experimental philosophers want to study, represent, and use ordinary language practices as evidence to bear upon philosophical theory choice. In the closing essay Weinberg responds to the "pressing question of how x-phi can make more direct contributions to traditional, mainstream sorts of projects in philosophy, addressing first-order questions in such areas as ethics, metaphysics, or epistemology" (194).^[6] The issue here is not whether x-phi counts as philosophy. As Weinberg notes: "That experimental philosophy is philosophy just is a settled question at this point" (193). Rather, the issue is how we might use the science to clarify "the nature of knowledge itself" (194).

Weinberg explains that x-phi studies can: (a) gather probabilistic evidence for regular correlations and (b) rule out noise -- distorting factors that interfere with our otherwise truth-tracking cognitive functions. When there is no prima facie reason to suspect that a distorting factor is at work, he recommends we follow Reid in trusting the verdicts our cognitive capacities deliver, until we have some positive reason to be suspicious (195). For example, participants might regularly respond with strong agreement that the protagonist of a vignette only "knows" when the stakes are low. But further experimentation could indicate that the sensitivity of knowledge ascription to stakes was simply the artifact of a distorting ordering effect or the font in which the vignette was presented. If stakes affect knowledge attribution in some circumstances but not others, the observed sensitivity to stakes could be "just a bit of local noise" and not "contextually robust" (197).

It is not clear how Weinberg will recognize influential factors as "distorting" influential factors without deciding in advance which correlations are the right ones to find and which ones are not. He acknowledges that there can be circumstances in which a sensitivity is lost, even when we are dealing with a contextually robust correlation. He then reassures that we need not worry about these cases if other relevant factors are lost in those circumstances as well (198). If knowledge ascriptions become insensitive to the truth or falsity of the agent's beliefs, for example, we know these are wacky circumstances. We should not be concerned, then, when knowledge ascriptions become insensitive to stakes as well. This response requires Weinberg to presuppose that knowledge is factive, which prejudges the science.^[7]

Experimental epistemologists continue to speak of their science as getting at the nature of knowledge itself, as if there were such a thing. In Weinberg's version, x-phi can help us separate "truth-tracking drivers of philosophical judgment" from "non-truth-tracking ones," such as font style (194). But these ontological attitudes are unnatural ones for scientists to hold (Fine 1996).

REFERENCES

Arico, Adam, Brian Fiala, Robert Goldberg, and Shaun Nichols (2011), "The Folk Psychology of Consciousness," Mind and Language 26: 327-352.

Beebe, James, and Wesley Buckwalter (2010), "The Epistemic Side-Effect Effect," Mind and Language 25(4): 474-498.

Buckwalter, Wesley (2010), "Knowledge Isn't Closed on Saturday: A Study in Ordinary Language," Review of Philosophy and Psychology 1 (3): 395-406.

Buckwalter, Wesley and Jonathan Schaffer (2013), "Knowledge, Stakes, and Mistakes," Noûs. 

Chin-Parker, Seth and Alexandra Bradner (2010). "Background Shifts Affect Explanatory Style: How a Pragmatic Theory of Explanation Accounts for Background Effects in the Generation of Explanations," Cognitive Processing 11:3: 227-249.

Cohen, Stewart (1999), "Contextualism, Skepticism, and The Structure of Reasons," Philosophical Perspectives 13: Epistemology: 57-89.

DeRose, Keith (1992), "Contextualism and Knowledge Attributions," Philosophy and Phenomenological Research, 52(4): 913-929.

--- (1995), "Solving the Skeptical Problem," The Philosophical Review, 104(1), 1-52.

--- (2011), "Contextualism, Contrastivism, and X-phi surveys," Philosophical Studies 156(1): 81-110.

Fine, Arthur (1996), The Shaky Game, University of Chicago Press.

Gentner, Dedre (1981), "Integrating Verb Meanings into Context," Discourse Processes 4(4): 345-371.

Guglielmo Steve and Bertram Malle (2010), "The Timing of Blame and Intentionality: Testing the Moral Bias Hypothesis," Personality and Social Psychology Bulletin 36: 1635-47.

Hazlett, Allan (2010), "The Myth of Factive Verbs," Philosophy and Phenomenological Research 80: 497-522.

--- (2012), "Factive Presupposition and the Truth Condition on Knowledge," Acta Analytica 27: 461-78.

Hawthorne, John (2004), Knowledge and Lotteries, Oxford University Press.

Knobe, Joshua (2003), "Intentional Action and Side Effects in Ordinary Language," Analysis, 63: 190-193.

--- (2010), "Person as Scientist, Person as Moralist," in Joshua Knobe and Shaun Nichols, eds. (2014), Experimental Philosophy: Volume 2, Oxford University Press, 195-228.

Nagel, Jennifer (2010), "Knowledge Ascriptions and the Psychological Consequences of Thinking about Error," The Philosophical Quarterly 60: 286-306.

Phelan, Mark (2014), "Evidence that Stakes Don't Matter for Evidence," Philosophical Psychology 4: 1-25.

Pinillos, Ángel (2012), "Knowledge, Experiments and Practical Interests," in Jessica Brown and Mikkel Gerken, eds., New Essays on Knowledge Ascriptions, Oxford University Press.

Prinz, Jesse (2008), "Empirical Philosophy and Experimental Philosophy," in Joshua Knobe and Shaun Nichols, eds., Experimental Philosophy, Oxford University Press: 189-208.

Sartwell, Crispin (1992), "Why Knowledge is Merely True Belief," Journal of Philosophy 89: 167-180.

Stanley, Jason (2005), Knowledge and Practical Interests, Oxford University Press.