Between Perception and Action

When I tie my shoelaces I perform an action. Bence Nanay's primary aim is deintellectualizing what is involved in the production of human actions of this kind. According to him, the vast majority of our actions are like the actions of infants and non-human animals in being performed under the guidance of perceptual states. Nanay calls the perceptual states that precede actions 'pragmatic representations'. Pragmatic representations are the cognitive components of the immediate mental antecedents of action, and they represent action properties -- properties, such as the size of an object relative to my grip size -- that are relevant for the performance of an action (p. 39). Most of our actions are not performed under the guidance of sophisticated propositional attitudes such as beliefs. (p. 2). If we were to take the title of the book as a question, we would answer it by saying that between perception and action lies mostly just perception.

Chapter 1 introduces the main thesis of the book as just summarized. Chapter 2 spells out the notion of pragmatic representation. Central here is Nanay's argument that pragmatic representations are both perceptual and necessary for action. Chapter 3 is devoted to placing his view vis-à-vis others in the philosophy of perception. He surveys representationslism and direct realism, siding with representationlism, and discusses the issue of the richness of perceptual content. Chapter 4 argues for naturalizing action theory. Chapters 5 and 6 constitute an expansion. Chapter 5 is about a variant on pragmatic representations: pragmatic mental imagery. Pragmatic mental imagery is quasi-perceptual, and it differs from perception only in the degree of determinacy of its content. The final section of this chapter distinguishes pragmatic mental imagery from Tamar Gendler's notion of alief. Chapter 6, concerns vicarious perception, where we perceive properties that are relevant to the actions of others. Nanay argues that vicarious perception is a primitive way of engaging with others, and it is relevant to issues in social cognition.

Actions are intended by Nanay to be different from both adaptive behaviors, such as reflexes, and from movements. Nanay takes himself to follow philosophical tradition in supposing that the difference between an action and a movement is that "there is a mental state type, call it M, that makes actions actions . . . . If our bodily movement is triggered (or maybe accompanied) by M, it is an action. If it is not, it is not an action." (p. 82). So actions differ from movements because they are triggered, guided or accompanied by a mental state (p. 15). One of Nanay's main contentions is that these mental states are, in very many cases, not beliefs and desires as traditionally understood. Belief-desire psychology that conceives of beliefs and desires as syntactically tructured, rationally cohesive mental states is often inadequate.

In place of beliefs and desires, Nanay introduces a 'special' kind of mental state (p. 36). He calls this special kind of mental state 'pragmatic representation'. Pragmatic representations are perceptual states, arising in response to sensory stimulation, and they are the cognitive components of the immediate mental antecedents of action. Although necessary for the performance of an action, pragmatic representations are not sufficient. A conative component that moves to act is also necessary. Nanay does not discuss the conative component in detail, so it is a hard to tell how this component compares to desires.

Pragmatic representations are often not accessible to introspection, and they tell us about 'action properties' -- properties the representation of which is necessary for the performance of an action. This is reminiscent of J. J. Gibson's idea that we perceive affordances -- properties such as edibility that suggest an action to the perceiver. But Nanay takes himself to hold a position that is less revisionary than Gibson's (pp. 11-12). Action properties are fairly mundane properties, such as the size of an object as it relates to one's grip size, the weight of an object as it relates to one's capacity to lift it, and the location of an object as it relates to the agent who interacts with it. Action properties are relational properties ascribed to an outside entity (p. 39).

Pragmatic representations share a number of features with beliefs. Firstly, they are representational states -- states that represent the world as being some way. They also have content and accuracy conditions. Unlike beliefs, however, they lack linguistic and syntactic structure. If one conceives of propositions as syntactically structured, then pragmatic representations also lack propositional content. Nanay's proposal is that pragmatic representations should replace beliefs and desires as the basic building blocks of our mind (p. 2). When we explain how I tied my shoelaces we can just appeal to my perception of, say, the size, weight and spatial location of the shoelaces (p. 32).

There are many passages that are worth discussing in this provocative and well-written book, and I cannot do justice to all of its parts in this review. I leave chapters 5 and 6 untouched. Readers interested in imagination and in social cognition should read these chapters. Instead, I will focus on a general difficulty with understanding the scope of the main claims of the book.

Consider, first, Nanay's critique of belief-desire psychology. He seems to want to displace belief-desire psychology as the primary explanatory framework of human action. In chapter 4, for example, he argues -- a bit too briefly -- for an account of decision-making that is based on imagination (p. 87). In chapter 5, he argues for the quasi-perceptual status of imagination itself, and in a number of passages he says that we do not need beliefs and desires to explain most of our actions (p. 32).

It is a question, however, whether Nanay is better interpreted as defending a weaker claim -- one that he sometimes makes -- namely that there are some actions that are explained by contentful states, where such states are not beliefs. The actions in question are those that require immediate worldly feedback. His general criticism of belief-desire psychology is otherwise open to a common objection. The common problem with attempts to do without belief-desire psychology is that one ends up explaining something different than what belief-desire psychology explains.^[1] This point is typically made by using the distinction between actions and movements. Belief-desire psychology makes sense of actions, while other theoretical frameworks -- such as the one afforded by neuroscience -- explain specific movements.

Since Nanay takes himself to explain actions, this criticism may not apply to his view as it is. Yet it seems a similar point holds. Actions can be individuated at different levels of grain. When I tie my shoelaces, I need to represent the shape and location of the laces. These representations explain how I am able to interact with the laces as spatio-temporal particulars. But they hardly explain why I am tying the shoelaces considered as such -- that is, as things that tie shoes. In order to explain why I am tying my shoelaces considered as such, I need to appeal to further representations -- for example, to a representation that tying my shoelaces secures my shoes, and to a representation that this gets me ready to go out. These further representations look like beliefs. I didn't tie my shoelaces just because I perceived their size and spatial location. I tied them because I thought I needed to get ready. If I didn't represent the shoelaces as instrumental in getting ready, I wouldn't have tied them, even if I did perceive their size and spatial location.

Similarly, I may be happy walking around with untied shoes until I am about to enter a nice restaurant and I tie them. In this case, I perform an action that is puzzling without ascribing to the actor both inferential abilities and a belief that untied shoes are inappropriate in nice restaurants. Appealing to pragmatic representations of the size and location of the laces leaves much unexplained.

It seems that this problem applies to virtually all the examples of action we can come up with: turning on the heat by turning a nob (p. 45), raising a cup of coffee (p. 39), killing a mosquito (p. 22). These are hardly just 'the tip of the iceberg' (p. 32) or 'the icing on the cake' (p. 1) of human actions. In explaining these actions -- as described -- we need to appeal to representations of what Nanay calls 'thick action properties' such as the property of 'being a facilitator of turning on the heat'. He does not want to say that thick action properties are represented in perception (p. 43), so he has to concede that many actions are not explainable by appeal to perception alone. His criticism of belief-desire psychology is then too weak as it stands. Belief-desire psychology was arguably never interested in explaining fine-grained actions and 'ongoing bodily movements' (p. 22), and Nanay does not provide an alternative for explaining actions described commonsensically.

The idea that some actions are accounted for by appeal to perceptual/representational states, while not dislodging belief-desire psychology, does contrast with some recent enactivist accounts of mental activity.^[2] According to such accounts, many of our actions are 'feedback-heavy', and they require postulating no representations. Nanay is nicely pointing out that there is a middle ground where we introduce representations but such representations are not beliefs.

An issue with how to understand the scope of Nanay's book arises also when we consider the claim that pragmatic representations are the necessary antecedents of action. They are what make actions actions (p. 3). Since pragmatic representations are perceptual states, one may wonder whether this claim is true: couldn't some actions be preceded by non-perceptual states?

Nanay makes it clear that his claim only covers basic actions -- that is, actions that are not performed by performing another action (p. 13). This means that his claim excludes a lot of actions. It excludes: (1) complex (non-basic) actions, such as bidding on an auction by raising one's hand (p. 13); (2) mental actions, where no bodily movement occurs, such as counting in one's head (p. 17); (3) Ballistic actions that require no perceptual feedback, such as jumping as high as one can (p. 30); (4) actions initiated by imagination or mental imagery, such as imagining a glass and pretending to toast with it; and (5) actions that are preceded by a conative but non-cognitive state, such as blinking, swallowing, or tapping one's foot to the music (p. 85).

Nanay may have left out some actions from this list. If, for example, I remember a dance move and do it, I perform an action whose antecedent is a mnemonic, non-perceptual state. There doesn't seem to be any direct feedback loop between my move and what I see, hear or touch. Perhaps Nanay would explain these cases similarly to how he explains (4) -- actions initiated by mental imagery. Chapter 5 argues for the claim that mental imagery is quasi-perceptual, and I lack the space to discuss this chapter.

The worry, however, is that conceding these exceptions attenuates significantly the necessity claim. Since pragmatic representations are what make actions actions, we need a story of the action-status of actions that require no such representations. And given the somewhat vague difference between basic and non-basic actions, we may wonder whether most actions are non- basic. I tied my shoelaces, for example, by tying those specifically shaped things.

Nanay's introduction of pragmatic representations into our mental ontology is also perhaps better seen as a claim about content. The claim is that we need to admit action properties into the content of unconscious perception. Pragmatic representations are not sufficiently different from quotidian perceptual beliefs or perceptions. They do not look like a new or 'special' psychological notion in the way that, say, Gendler's aliefs are intended to be.^[3]

Pragmatic representations do not have syntactic structure. In this sense, they are not propositional, and some philosophers may not like this characterization of perception. But those who are okay with perception being unconscious and non-syntactic -- from Andy Clark, to Tyler Burge to Christopher Peacocke -- may want to call pragmatic representations simply perceptions.

As a matter of fact, it is not clear why pragmatic representations are not perceptual beliefs, something that further problematizes Nanay's criticism of belief-desire psychology. In chapter 2, he proposes an argument for the perceptual status of pragmatic representations. His argument is based on the consideration that there are cases where our behavior is guided by unconscious states that are in conflict with our conscious experiences and beliefs. Instances of perceptual learning and of visual illusion are examples (pp. 23-28).

Reasons of space prevent me from discussing Nanay's argument. Presently, what is interesting is that Nanay runs the argument without offering an account of the difference between perceptual and non-perceptual states (p. 23). Since he is willing to think of pragmatic representations as having propositional content -- provided that we have a non-syntactic understanding of propositions as sets of possible worlds (p. 37) -- we may wonder how such representations are different from unconscious beliefs acquired through the senses.

As for the question of what kind of properties these unconscious action-guiding states represent, Nanay has an interesting proposal. He introduces action properties, and his examples are fairly convincing. It seems plausible that to explain our interaction with objects we need to represent properties that allow us to grab, lift and dodge them.

Although Nanay's position is different from Gibson's, it is a question how the two relate. I wonder, for example, in what way a relational property such as weight for lifting is different from 'liftability' and what argument Nanay has for excluding affordances from the content of pragmatic representations. Affordances seem to qualify as action properties.

One final issue with the scope of Nanay's claims arises in chapter 4. In this chapter, he argues for naturalizing action theory. By 'naturalizing action theory' Nanay means that action theory should be: (a) sensitive to empirical results (p. 77); (b) continuous with the empirical sciences in not privileging the philosophical to the scientific (p. 78); and (c) not relying heavily on first-person methods, such as introspection (p. 80). Although these points seem reasonable, it is not clear how action theory would benefit from them. For example, how does empirical evidence bear on the relation between intention and action? Or on the difference between something happening to an agent and an agent doing something?

Nanay may appeal to pragmatic representations to clarify what actions are. But there are two problems with this appeal. One is that pragmatic representations are not empirical evidence. They are theoretical notions. The other is that it is unclear that pragmatic representations help. To distinguish actions from movements, Nanay introduces pragmatic representations that are described as the cognitive components of the immediate mental antecedents of action. As it is, this way of characterizing action is circular: to understand how action is different from movement, we appeal to mental states, and such states are themselves individuated by appeal to the concept of action.

Although I raised some critical worries, Between Perception and Action is an interesting attempt to deintellectualize the mind, and it is worth the attention of those interested in a middle ground between antirepresentationlist/enactivist accounts of mental activity and more traditional computational and linguistic positions.^[4]