While epistemology and ethics are probably the two areas where "experimental philosophy" has made the biggest splash, its impact on the philosophy of language has not been insignificant, and this excellent collection captures much of the state of the art in what is increasingly referred to as "experimental philosophy of language".
In his introduction, Jussi Haukioja suggests that philosophers who might "otherwise be skeptical about experimental philosophy" may think that experimental results have a role in the philosophy of language at least in part because of "the difficulty between drawing a clear line between philosophy of language and linguistics" (1). However, it is noteworthy that the collection has comparatively little discussion of those areas of philosophy of language that interact most with linguistics. Indeed, seven out of the eight papers focus on a set of questions that linguists such as Noam Chomsky (1995, 2000) insist are squarely on philosophy's side of the line, namely the debates between defenders of 'causal' and 'descriptive' theories of reference that came to prominence with the publications of Saul Kripke's Naming and Necessity and Hilary Putnam's "The Meaning of 'Meaning'".
The starting point for most of the papers is thus not experimental work in linguistics, but rather Edouard Machery, Ron Mallon, Shaun Nichols and Stephen P. Stich's 2004 paper "Semantics, Cross-Cultural Style" (hereafter "MMNS"). That paper notoriously argued that the intuitions that Kripke supposedly appealed to in criticizing descriptive theories of names were characteristic only of Western subjects, and that they were far less likely to be shared by East Asians. While westerners typically (though far from universally) take the name "Gödel" to refer to the man who historically bore the name (even if the important identifying information associated with the name were not true of him but actually true of another man), East Asians were supposedly more likely to take the name to refer to whoever fit the associated descriptions. That paper has already generated a large literature (including earlier papers by the authors in this volume), and this collection is not unusual in treating such questions as being at the center of the field.
The first paper, Max Deutsch's "Kripke's Gödel Case", argues that this whole debate starts off on the wrong foot with MMNS's assumption that Kripke makes any "evidential appeals to intuition" in his arguments against descriptive theories of names (7). According to Deutsch, rather than being tested against our intuitions, "A theory of reference is tested against the referential facts, many of which we already know (e.g. that 'Einstein' does not refer to the inventor of the atomic bomb)" (12). It is such manifest referential facts that drive Kripke's arguments, not any appeals to intuition. Deutsch bolsters this reading of Naming and Necessity by arguing that Kripke's oft-quoted remark:
some philosophers think that something's having intuitive content is very inconclusive evidence in favor of it. I think that it is very heavy evidence in favor of anything, myself. I really don't know, in a way, what more conclusive evidence one can have about anything, ultimately speaking. (Kripke 1972, p. 42)
is best interpreted as not pertaining to his judgments about scenarios like the Gödel case, but rather as part of a defense of the intuitive notion that we can make judgments about necessity that are not relative to descriptions -- judgements such as those expressed when we point to Nixon and say that he could have lost the 1972 election (Kripke 1972, p. 41). In short, Kripke's claim is "not that intuiting p is heavy evidence for the truth of p", but rather that "a notion's presence in the thought and talk of ordinary people -- its possession of intuitive content -- is very heavy evidence for its meaningfulness" (25).
Still, even if we accept Deutsch's arguments about how we should interpret Kripke, it isn't obvious that this would show that, as Deutsch puts it, "experimental work on intuitions about the Gödel cases is irrelevant" (25). After all, even if Kripke is appealing to semantic facts (and not just our intuitions about them), such appeals play an effective role in his arguments because they are uncontroversial. The MMNS results, if they stand, would suggest that that these appeals are far more controversial than philosophers have traditionally assumed.
In "Testing Theories of Reference", Michael Devitt also denies that intuitions should play a significant evidential role in the philosophy of language and argues that our metasemantic theories should be based on actual linguistic usage instead. Devitt claims that MMNS are right to criticize philosophers for testing theories against their own intuitions but wrong in thinking that this situation would be improved by moving in "more armchairs for the folk" (34). Rather, the primary goal of experimental semantics "should not be testing theories against anyone's referential intuitions but rather testing them against the reality that these intuitions are about: theories should be tested against linguistic usage" (31). Still, testing theories of reference against linguistic usage turns out to be no simple matter because of what Devitt calls the "implicit-scare-quote" problem, namely, that "elicited production . . . provides the evidence we need only if speakers are not implicitly distancing themselves from their usage" (55), and since the experiments designed to control for this worry discussed in his paper produced what he considers "baffling" results (53), this methodological problem for Devitt's approach remains very much in place.
While he doesn't respond to the criticisms of MMNS that we see with Deutsch and Devitt above, Machery, in "A Rylean Argument against Reference", responds to another well-known line of criticism of that paper, namely that philosophers can ignore the sorts of diversity that MMNS find because they should be concerned with "expert" intuitions rather than the frequently confused intuitions of the "folk". Machery responds to this approach with "a broad Rylean argument about reference" (65), where Rylean arguments are understood as concluding that philosophers' claims about a given topic are nonsensical on the grounds that their use of the relevant concept "contravenes proper use, as manifested . . . by laypeople's use of the predicates" (65). Machery argues that reference doesn't have any essential theoretical role, since it turns out to be needed to explain neither the truth conditions of our utterances nor our ability to communicate. Consequently, unless we can assume that philosophers are talking about something that is pre-theoretically grasped, "it becomes unclear whether philosopher's concepts of reference and extension are about anything at all" (66). However, such a pre-theoretical grasp of the concept is just what philosophers critical of Experimental Philosophy often deny when they dismiss the relevance of folk intuitions.
The use of intuitions (in at least the philosophy of language) is then defended by Daniel Cohnitz, in "The Metaphilosophy of Language". He argues that, by contrast to those areas in philosophy whose objects may seem "independent of the intuitive interpretation and production of linguistic items by competent speakers" the realm of semantic facts is arguably "constituted" by such facts about competent speakers (87). Cohnitz makes a case for this by defending a more general "meta-internalism" about how reference is determined. Unlike the stereotypical semantic internalist, who takes what our terms refer to to be determined by 'internal' factors, the meta-internalist can admit that reference is determined by external factors such as the term's causal history, but insists that these external factors are relevant because of factors more internal to the speaker. These factors needn't be actual beliefs about the relevance of a term's causal history; rather, all that is required is that speakers "usage, including their dispositions to correct their usage, is sensitive to information that happens to be about the causal-historical chain in question" (94). Cohnitz argues that most meta-semantic theories are meta-internalist in this sense (92), and that anyone who is a meta-externalist (such as Devitt 2006) takes on an implausible commitment to the possibility of "semantic secrets" whereby terms systematically refer to items that are "irrelevant to the contents transmitted in communication" (93).
The next few papers focus on extending the sorts of concerns raised by the MMNS results from the analysis of proper names like "Gödel" to the analysis of natural kind terms like "water" or "gold". The first, Sören Häggqvist and Åsa Wikforss's "Experimental Semantics: The Case of Natural Kind Terms", argues that the "available empirical evidence poses a serious challenge to Kripke's account of natural kind terms and lends some support to cluster theories for these terms" (109). In particular, they discuss recent work on kind terms by Jussi Jylkkä, Henry Railo and Haukioja (2009) and argue that a cluster theory accounts for those results better than either a straight causal/historical theory or a 'hybrid' theory that predicts that such terms are ambiguous between a causal/historical and a more descriptivist reading (125).
Just such a hybrid theory is defended by Ángel Pinillos in "Ambiguity and Referential Machinery", where he argues for the claim that "names and natural kind terms are ambiguous between causal-historical and descriptive interpretations" (139). Pinillos defends his position with an appeal to a set of experiments about people's judgments about whether or not the medieval use of "Catoblepas" referred to Wildebeests or not, and the larger significance of precisely these experiments about speaker intuitions is challenged in Genoveva Martí's "General Terms, Hybrid Theories and Ambiguity: A Discussion of Some Experimental Results". Like Devitt, Martí argues against the relevance of such experiments on the grounds that they (along with the original MMNS experiments) only capture the subjects' reflection on their practice, not the facts about their actual use (158). Martí admits that our use might ultimately support some sort of ambiguity view (166), but she argues that this would not undermine the most important point we should take from Kripke's work, namely, we need to reject any conception of reference "according to which reference has to be mediated" (169).
The final paper, Mark Phelan's "Testing Transparent Ascriptions: A Plea for an Experimental Approach", represents a shift in focus from all the previous ones in that it is unconcerned with issues that come out of Kripke and Putnam's arguments for semantic externalism. Rather, Phelan's paper investigates the question of whether utterances such as "I believe that dinner is supposed to be at 8:00" always literally report just a psychological state, even if they sometimes implicate a hedged claim about the world (the "Gricean view"), or whether the truth conditions of such sentences can vary depending upon whether they are primarily meant to convey information about the world or about the speaker's mental state (the "direct expression view") (173-74). Phelan presents a number of experiments that seem to show that speakers' intuitions support the direct expression view (195). His paper is closer in style and substance to other work described as "experimental pragmatics", and its ending the collection raises the question of just how we should see the relation between experimental philosophy of language and experimental pragmatics. It is true that many who do experimental pragmatics seem to prefer to describe their work under that name, but it's unclear whether this reflects a substantial difference in topic, in methodology, in training, or just a preference for a label that is associated with less controversy.
All in all, while the collection is more focused than the title might suggest, it provides both an excellent introduction to the main issues driving what typically goes by the name of Experimental Philosophy of Language, and a taste of a broader conception of what the field could be.
Chomsky, N. 1995. "Language and Nature". Mind, 104, 1-61.
Chomsky, N. 2000. New Horizons in the Study of Language and Mind. Cambridge: Cambridge University Press.
Davidson, D. 1977. "Reality Without Reference", in Davidson, D. Inquiries into Truth and Interpretation, Oxford: Oxford University Press, 1984.
Devitt, M. 2006. Ignorance of Language. Oxford: Oxford University Press.
Jylkkä J., Railo, H. and Haukioja, J. (2009), "Psychological essentialism and semantic externalism: evidence for externalism in lay speakers' language use", Philosophical Psychology, 22, 37-60.
Kripke, S. 1972, 1980. Naming and Necessity, Cambridge: Harvard University Press.
Ludwig, K. (2007) "The Epistemology of Thought Experiments: First-person approach vs. Third-person approach" Midwest Studies in Philosophy 31, 128-159.
Machery, E., Mallon, R., Nichols, S. and Stich, S., "Semantics, Cross-Cultural Style." Cognition, 92, B1-B12.
Machery, E. and Stich, S. (2012) "The Role of Experiment" in Russell, G. and Graff, D. The Routledge Companion to the Philosophy of Language. New York: Routledge, 2012 (pp. 495-512).
Nichols, S., Pinillos A. and Mallon R. (2016) "Ambiguous Reference" Mind, (published online Feb 3, 2016).
Phelan, M. 2014. "Experimental Pragmatics: An Introduction for Philosophers". Philosophy Compass 9/1 (2014) 66-79.
Putnam, H. 1975. "The Meaning of 'Meaning.'" Reprinted in his Mind, Language and Reality, New York: Cambridge University Press. 1975. pp. 215-271.
Sperber, D. and Noveck, I. (2004), Experimental Pragmatics, New York: Palgrave MacMillan.
 Haukioja himself attributes "much of the current interest in experimental philosophy of language" to MMNS (2), and the paper on "The Role of Experiment" in the Routledge Companion to the Philosophy of Language (Machery and Stich 2012) focuses exclusively on these issues.
 Along with more 'epistemic' facts such as that beliefs like "Einstein invented the atomic bomb" are not immune from error even when they are the sole description that a speaker associates with a name (15).
 As with the rest of the papers in this volume, I won't have space here to discuss the actual experiments.
 Kirk Ludwig (2007) is cited as a paradigm of this sort of response (76).
 Machery appeals to Donald Davidson 1977 for the former claim, and Chomsky 1995, 2000 for the later.
 The medieval use of "Catoblepas" seems to have originated from encounters with Wilderbeests, but medieval Europeans were almost never exposed to them and they had a number of Catobeplas-beliefs -- such as that they had scales and that looking them in the eye would cause immediate death (146) -- that were manifestly untrue of Wilderbeests.
 As they appear in Nichols, Pinillos and Mallon (2016).
 A movement which also raised its profile considerably in 2004, though this time with the publication of Dan Sperber and Ira A. Noveck 2004 rather than MMNS.
 Phelan's own introduction to Experimental Pragmatics (Phelan 2014) leaves this unclear.