Adam Pautz’s excellent book is an introduction to the philosophy of perception within the analytic philosophical tradition. What is discerning about the book is that it focuses on the question of what constitutes the character of experience (henceforth, the character question), which is one of the most interesting and perplexing questions in the philosophy of perception. While the book targets primarily upper-level undergraduate and graduate students, the novelty of some of its empirical arguments will be of interest to postdoctoral students as well as professional philosophers working on the philosophy of perception.
The character question pertains to how things seem to a perceiver. When you successively view, say, two nearly identical tomatoes, the character of your experience remains the same (both tomatoes seem the same to you) even though the object of your experience changes as your gaze moves from one tomato to the other. In this case, you have the same type of experience. However, when you successively view, say, a tomato and a lemon, the character of your experience changes (the tomato seems different than the lemon to you) along with the object of your experience as your gaze moves from the tomato to the lemon. In this case, you have two different types of experiences. Pautz frames the character question in terms of what it is to have a certain type of experience, which takes the form of the following definition:
To have an experience with a certain character (that is, to have an experience of a certain type) just is to ______.
The book provides a comprehensive examination of four prominent accounts, each of which aims to fill in the blank, viz., sense datum theory, naïve realism, internal physical state theory, and representationalism. The book is organized as follows. The first chapter examines the sense datum theory and compares it to naïve realism. The second chapter examines the internal physical state theory. The third and fourth chapters explore various versions of representationalism. The fifth chapter compares representationalism to contemporary naïve realism.
The unifying theme of the book is the external-internal puzzle. This puzzle arises because what we seem to experience as being “out there” seems to be determined by what’s happening “in the head.” Pautz argues that answering the character questions requires that a theory respects both essential external directness and internal dependence.
Essential external directness is a pretheoretically plausible constraint, according to which “it is part of the essence of some types of [sensory] experience that in having those [sensory] experiences it seems that there are objects with certain [perceptible] features” (74). Your visual experience of a tomato, for instance, is essentially externally directed in the sense that it involves the seeming presence of a round object.
Internal dependence is the idea that internal processes play a role in our experiences of properties or objects. Pautz argues that empirical evidence indicating “good internal correlations” between experiences of color properties and neural responses provides support for the claim that color experiences are shaped by neural processes. Pautz cites numerous studies, including a study on macaques, the findings of which indicate that behavioral judgments of the similarity between colored stimuli closely match the similarities between the neural responses to these colors by a population (“glob”) of neurons in the posterior interior temporal cortex (the V4 complex). Pautz argues that many of the most prominent theories fail to respect both essential external directness and internal dependence.
Pautz argues that naïve realism provides an “act-object” explanation of the character of experience, according to which the character of experience is constituted by or grounded in the experiencing of perceptible properties (e.g., roundness or colors) of actual objects. On this account, to have an experience-type of a certain character just is to experience the actual character of material things, e.g., the redness and roundness of the tomato. Differences in the character of experience, in the normal cases, are constituted by or grounded in differences in the material objects. Naïve realism attempts to accommodate external directness in normal cases by maintaining that when you experience a red tomato, there is a red tomato in front of you. However, Pautz presents evidence indicating that the similarities and differences of normal color experiences do not correlate with similarities and differences of spectral reflectance types, which are thought to be the most likely candidates for the colors (MacAdam 1985). For example, even though blue objects (such as a blueberry) look more similar to purple objects (such as a grape) than green objects (such as a leaf), studies indicate that the spectral reflectance type of a blue-looking object is more similar to that of a green-looking object than it is to a purple-looking object. These findings present serious difficulties for naïve realists since they indicate that the character of experience is grounded in the experiencing of color properties.
Pautz further argues that naïve realism fails to accommodate internal dependence across the board. Naïve realism holds that what properties or objects you experience does not depend on your internal physical states. However, cases of illusion and hallucination suggest that, contrary to naïve realism, the character of experience is not grounded in the experiencing of perceptible properties. Suppose that you have an illusory experience of a red tomato. The tomato seems orange to you. But since the tomato is not orange, the character of your experience is not grounded in the experiencing of the redness of the tomato. Similarly, when you hallucinate a red tomato, your experience cannot be grounded in the experiencing of the tomato and its properties since there is no tomato.
Contemporary naïve realism attempts to solve the external-internal puzzle by positing that your experience of an actual red tomato differs from your experience of a hallucinatory red tomato. To account for the fact that these two experiences seem (introspectively) the same to you, they adopt an indiscriminability theory, according to which to have an illusory or hallucinatory experience with the tomato-like character is to be in a situation “reflectively indiscriminable from (not knowingly different from) experiencing” an actual tomato (209). Pautz, however, argues that indiscriminability theory is implausible since it treats illusory and hallucinatory experiences are consisting in nothing but an inability to know that you are not successfully experiencing the redness of the tomato.
Sense datum theory also provides an act-object explanation of the character of experience. However, on this view, to have an experience-type of a certain character just is to be aware of non-material objects, viz., sense data, generated by neural processes in the brain. For example, for you to have an experience of a ripe tomato just is for you to experience a tomato-like sense datum (a non-material object) generated by neural processes in the brain. On this view, differences in the character of experience are constituted by or grounded in differences in sense data. Sense datum theory is consistent with evidence indicating poor external correlations between experiences of sensible properties. The properties you experience are not, after all, properties of external objects but rather properties of sensa data generated by your internal states. Nevertheless, since sense data are external objects, albeit non-material, your experience is externally directed. Sense datum theory is thus also consistent with evidence indicating good internal correlations between experiences of such sensible properties and neural responses in normal experience. Since sense data are generated by the neural process in your brain, your experience is also internally dependent. Sense data theory, therefore, solves the external-internal puzzle, but only at the cost of positing internally generated non-material objects.
Internal physical state theory and representationalism reject the act-object explanation in favor of an existence-neutral explanation of the character of experience, which Pautz calls the “seems-gambit.” On this explanation, your brain has an “innate capacity” to generate such “seeming states.” For example, when you experience a tomato in front of you, regardless of whether you undergo a veridical experience, an illusion, or a hallucination, it seems to you that there is tomato in front of you that has the properties of being round and being red.
Internal physical state theory identifies the character of your experience with an internal physical property, which is an intrinsic property of your brain. On this view, to have a certain experience-type of a certain character just is to be in a certain neural activation state. Pautz argues that evidence indicating good internal correlations between experiences of such sensible properties and neural responses in normal experience support this account since they indicate that differences in the character of experience just are differences in the intrinsic neural states that generate them. By identifying the character of experience with an internal, intrinsic property of the brain, the internal physical state theory can, therefore, accommodate internal dependence. But in doing so, it fails to account for external directness. For, it cannot explain why an intrinsic property of your brain seems externally directed. Internal physical state theory, therefore, cannot solve the external-internal puzzle.
Representationalism says that to have an experience with a certain character just is to experientially represent a complex array of actual or possible perceptible properties. Representationalists can explain “experientially represents” using the Ramsey-Lewis method:
All sensory-perceptual experiences consist of a basic mental relationship R between subjects and ways things might be such that: R plays the cognitive-access role, R is existence neutral, and R plays the character role. (100)
Experientially represents is thus understood in terms of experientially seems. Having an experience of a red tomato, for example, essentially involves standing in some relation R to the redness of the tomato. This relation R plays the character-role: differences in the character of your experience just are differences in the way your experience represents the world. Since this relation R is existence neutral, representationalism can account for cases of illusion and hallucination. In illusory experience, a red tomato seems to you to have a color (say, orangeness) that it lacks. In hallucinatory experience, it seems to you that there is a tomato present even though no object is present. Representationalism says that in both cases, your experience of a red tomato involves standing in some existence-neutral relation R to a red tomato. Pautz offers a careful examination of three of the most prominent versions of representationalism: response-independent, response-dependent, and internalist-nonreductive representationalism.
Response-independent representationalists maintain that the experiential relation R just is the detection relation. Take colors, for example. On this account, colors are response-independent properties. Your experience represents color properties because your visual system has the function of detecting the colors (much like the function of a thermometer is to detect temperature). Your color experience is, therefore, externally directed. It follows that response-independent representationalism can account for external directness. Pautz, however, argues that it cannot account for internal directness. To this end, he makes the following structural-mismatch argument (see also Pautz 2020).
Imagine a possible world in which humans have the function of detecting the same properties under biologically normal conditions as humans in the actual world. However,
because of naturally evolved differences in [their] postreceptoral wiring, their V4 neural representation of [a blue-looking blueberry] resembles their V4 neural representation of the green-looking leaf more than their V4 neural representation of the purple-looking grape—the opposite of how things stand in the actual world. (159)
This presupposition is consistent with evidence indicating good correlations between color experiences and neural processes. Now, suppose that while in this possible world humans tend to sort a blue-looking blueberry with the green-looking grape, in the actual world humans tend to sort a blue-looking blueberry with the purple-looking grape. Given their evolutionary history of humans in this possible world, the best predictor of their color experiences is their V4 neural representations. We may thus suppose that if a human in this possible world were to look consecutively at a blueberry, a grape, and a leaf, she would have a greenish experience of the blueberry instead of the bluish experience her counterpart would have in the actual world. Pautz, however, argues that response-dependence representationalism “delivers the mistaken verdict” that the human in the possible world and the human in the actual world would have the same color experiences despite vast neural and behavioral responses” (159). It follows that, even in normal experience, response-independent representationalism cannot account for internal dependence.
Response-dependent representationalism attempts to solve the external-internal problem by defining colors in terms of dispositions of objects to normally cause certain neural responses (e.g., V4 neural representations) in perceivers. Accordingly, what color properties you are R-related to depends on your internal physical states. This account accommodates internal dependence since it maintains that your experientially representing color properties depends on your neural processing. But it also accommodates external directness since your color experiences represent dispositional properties of external objects. Pautz, however, argues that while response-dependent representationalism solves the external-internal puzzle, its success is undermined by the fact that it cannot provide a satisfactory answer to the representation question: What is it for you to experientially represent a perceptible property?
To explain the problem, Pautz asks us to suppose that you and your counterpart view the same red sphere which seems reddish to you but orangish to your counterpart. On this account, the sphere has two response-dependent properties: it has the disposition to cause red experiences in you and the disposition to cause orange experiences in your counterpart. But it also has a response-independent property: being round. This suggests that answering the representation question requires specifying a relation R that is such that both you and your counterpart bear R to different response-dependent properties of the sphere but also to the same response-independent property. According to Pautz, finding such a relation seems rather implausible.
The last account Pautz considers is internalist-nonreductive representationalism. According to this account, while experientially representing sensible properties depends on neural processes in the brain, sensible properties cannot be reduced to anything physical. When you are experientially representing a red tomato, for example, you stand in an irreducible experiential relation R to its perceptible properties (its redness and its roundness).
Pautz favors this account because “external directness and internal dependence lead directly to it” (173). In defense of this claim, Pautz asks us to imagine a possible world where there is nothing but a brain in the void (BIV), which “popped into existence out of nowhere, and purely by chance it undergoes the same neural activity as your own brain” (82). Suppose, for example, that the BIV has an experience of a red tomato. In having this experience, the BIV stands in the experiential relation to the properties of being red and being round. However, by hypothesis, the BIV has no interesting relation to these properties since it is cut off from the external world. The experiential relation is distinct from the detection relation since experientially representing the properties of being red and being round is not reducible to its undergoing neural states that detect these properties. Pautz argues that this conclusion can be generalized to normal experiences of humans in the actual world. When, for example, you view a red tomato, you experientially represent the properties of being red and being round. This experiential relation, however, is not reducible to your undergoing neural states which detect the color and shape of the tomato, “because the BIV example shows that these two things are separate” (173).
Pautz argues that although internalist-nonreductive representationalism solves the external-internal puzzle, it is somewhat mysterious. For, how can you stand in an irreducible experiential relation to perceptible properties that cannot be identified with neural states by virtue of undergoing such neural states? Pautz suggests that future research in neuroscience may provide an answer to this problem by showing that “what perceptible properties you experientially represent can be systematically ‘decoded’ by looking to internal processing alone” (239).
Pautz’s suggestion stems from the evidence cited throughout the book, which indicates a failure of any stable correlations between these experiences and sensible properties and good internal correlations between experiences of sensible properties and neural responses. The former establish that relation R cannot be a detection relation. The latter seem to suggest that our experiences of perceptible properties can be explained solely in neuroscientific terms.
Pautz’s suggestion is consistent with empirical evidence indicating that biological visual systems cannot rely on the detection of perceptible properties because these properties are conflated in the retinal image (Bohon et al. 2016; Lotto and Purves 2000). The visual system must instead learn to transform inherently ambiguous retinal stimuli into unequivocal experiences of perceptible properties which can guide behavior. However, the fact that the visual system does not have the function of detecting perceptible properties does not entail that our experiences of these properties can be explained purely in neuroscientific terms. Indeed, studies indicate that the visual system evolved, through trial and error, to rely on the recurring scale-invariant patterns to rank color experiences (Purves et al. 2015; Lotto and Purves 2000). This suggests that the character of color experience does depend on external stimuli (albeit not perceptible properties). The information about recurring scale-invariant patterns the visual system relies on to rank color experiences is based on the frequency of occurrence of image patterns that have promoted behavioral responses which were pertinent to the survival and reproduction of our species in the past. Evolution, as is well known, does not build neural machinery from scratch. It utilizes existing neural circuitry and modifies it as needed to help organisms survive and thrive. As the inherited neural circuitry of our visual system continued to evolve, the neural links that improved its reproductive fitness prevailed over those that did not. But none of these neural circuits would have been built in the first place if, over the course of evolution, the visual system didn’t have to rely on a probabilistic strategy to transform inherently ambiguous retinal stimuli into unequivocal color experiences. It follows that what perceptible properties humans experientially represent cannot be systematically explained by reference to internal processing alone.
Pautz acknowledges that there is still no perfect solution to the puzzle of perception despite the significant innovations in philosophy that resulted from advances in science. That much we can all agree on. The book nevertheless succeeds in moving the discussion forward. Through its careful examination of the character question, it provides a fertile ground for a fruitful debate about the role of neuroscience in solving the puzzle of perception.
REFERENCES
Bohon, K. S., Hermann K. L., Hansen T., and Conway B. R. (2016). Representation of Perceptual Color Space in Macaque Posterior Inferior Temporal Cortex (the V4 Complex). eNeuro 3. 10.1523/ENEURO.0039-16.2016.
Lotto, R. B. and Purves, D. (2000). An empirical explanation of color contrast. Proceedings of the National Academy of Sciences, 97(23), 12834–12839.
MacAdam, D. L. (1985). The Physical Basis of Color Specification. In MacAdam, Color Measurement: Themes and Variations. Berlin: SpringerVerlag, pp. 1–25.
Purves, D., Morgenstern, Y., and Wojtach, W. T. (2015). Will understanding vision require a wholly empirical paradigm? Frontiers in Psychology, 6:1072. 10.3389/fpsyg.2015.01072
Pautz, A. (2020). How does color experience represent the world? In Derek Brown and Fiona MacPherson (Eds.) Routledge Handbook of the Philosophy of Color. New York: Routledge.
Russell, B. (2009). The value of philosophy. In Steven M. Cahn (Ed.) Exploring philosophy: and introductory anthology. New York: Oxford University Press.