Language, Consciousness, Culture: Essays on Mental Structure

 

Introduction

According to a familiar research framework, we can explain various aspects of cognition and behavior by functionally characterizing the mind/brain in terms of computations over structures. Leading questions then include: what are these structures and computations (what are the primitives, and how are they combined and processed), how are they acquired (are any innate or innately constrained), how are they neurally realized, and what is their evolutionary history? Ray Jackendoff has been among the most creative and wide-ranging contributors to this enterprise. In particular, he has advocated applying the methods and insights of linguistics to less well-trodden cognitive domains, such as musical cognition and the interface of language and perception — while demanding as well that linguists attend more to the constraints imposed on their own theorizing by results in neighboring fields.¹ Jackendoff’s most recent book continues these efforts, providing in particular his most sustained discussion of the mental structures implicated in social cognition.

Language, Consciousness, Culture is immensely stimulating. Jackendoff excels at weaving together lucid overviews, suggestive large-scale connections, and intriguing specific hypotheses and observations. Anyone working on its topics will benefit from studying the book.² In what follows, I summarize its contents and then raise some questions concerning Jackendoff’s views on consciousness and on theory of mind.

Summary

The first part of Language, Consciousness, Culture comprises Jackendoff’s augmented 2003 Jean Nicod Lectures and provides a wide-lens view of various cognitive domains. Chapter 1 lays out the computational framework and argues that the mind is modular in that it utilizes a variety of distinct, incompatible structural formats. Jackendoffian modules — individuated, but also prima facie isolated, by their proprietary structural formats — interact in virtue of interfaces that establish cross-modular correlations among structures. Some details are developed in Chapter 2 (a précis of Jackendoff 2002) with respect to phonology, syntax, and semantics, modules implicated in linguistic competence. Their structures differ not just in their constituents, but also in their combinatorial principles (e.g., syllabic structure, unlike syntactic and conceptual (= semantic) structure, is hierarchical but not recursive). Indeed, a central claim of Chapter 2 is that, contrary to mainstream generative linguistics’ assumption that syntax alone supplies generative principles, linguistic competence has a “parallel architecture.” The modules interface via triples of cross-indexed phonological, syntactic, and conceptual structures stored in long-term memory. These include both specific lexical entries and more abstract schemata (“rules”), which differ, on Jackendoff’s view, only in that the latter contain variables that the former have filled in. Jackendoff argues inter alia that the parallel architecture provides a better mesh with accounts of linguistic processing, a compelling account of idioms and “syntactic nuts,” and consistency with an incremental evolutionary path to language.

Chapter 3 asks after the functional correlates of consciousness, specifically with respect to consciousness of language. (Cf. Jackendoff 1987, 1996.) On Jackendoff’s view, the mental structures correlated with consciousness have two kinds of constituents: content features and valuation features. The content features of linguistic experience mostly mirror auditory images of phonological structure. In particular, we do not experience conceptual structure — i.e., the meaning of what we say. Candidate valuation features include ±external (percepts vs. imaginings) and ±self-initiated (in the case of language, experienced as your own speech vs. as another’s). Jackendoff’s “Intermediate-Level Hypothesis” maintains that these structures correspond to where top-down processing first comes into play and attention is attracted and anchored. Because attention affects resource-allocation, consciousness of language enhances thought: the attention drawn to a phonological structure yields a greater allocation of resources to its associated conceptual structure as well, leading to greater inferential processing.

Chapter 4 speculates on the mental structures involved in complex actions, with special attention to shaking hands and making coffee. Jackendoff pursues analogies with linguistic competence, arguing, for instance, that these structures exhibit hierarchical complexity — indeed, recursive structure, contrary to Hauser, Chomsky, and Fitch’s (2002) claim that language is unique among human cognitive capacities in this regard. He claims moreover that, as with language, the principles that constrain the construction of these structures operate over a vast lexicon of actions stored in long-term memory, as well as stored action-schemata whose variables can be fleshed out on-line as context requires (perhaps thus avoiding problems that bedeviled “frames” and “scripts” of old). But language is not just a species of action. It retains its uniqueness — and its modularity — in virtue of its proprietary structural formats and its particular functional role.

Chapter 5 advocates a mentalist approach to social cognition by drawing parallels with familiar considerations concerning language. The unlimited number of understandable social situations suggests a combinatorial rule system (only partly available to consciousness). That we acquire social competence from imperfect evidence and receive explicit instruction only about the consciously accessible rules suggests unlearned resources for social learning, innate and perhaps in part specific to social cognition. Jackendoff argues further that linguistics has a particular interest and role in the study of social cognition. First, a number of important linguistic issues involve social terms: the difference between a request and an order, for example, invokes social dominance. Second, linguistic evidence can provide insights into the conceptual structures involved in social cognition. After outlining how we conceptualize stable social relations (kinship, alliances, dominance hierarchies, and groups) as well as more evanescent relations, Jackendoff discusses rules and other normative principles, arguing inter alia that systems of norms in general offer a more promising object of study than moral principles in particular.

With Chapter 5 in place, the stage is set for the book’s second part, which mostly provides detailed examinations of specific conceptual structures implicated in social cognition. Chapter 6 argues that the syntactic distribution of perception verbs requires a basic constituent in conceptual structure — ‘X EXP Y’ (glossed as ‘X experiences Y’) — that divides these verbs into those that imply a theory of mind (‘sees’) and those that do not (‘looks at’). Theory of mind, on Jackendoff’s view, does not constitute a distinct module but rather amounts to the possession of particular resources in the conceptual module. Chapter 7 analyzes how we use psychological and evaluative predicates to frame objective and subjective claims (‘Problem P isn’t interesting’ vs. ’I’m not interested in problem P’). The objective case involves a conceptual constituent ‘YA’ for a generic person, introduced in the previous chapter’s discussion of experience verbs without overt experiencers (‘The stew tastes wonderful’). Chapter 8 takes up our conceptualization of intentional action and the linguistic expression of intention. It’s argued that ‘intend’ belongs to a class of actional attitude verbs restricted to self-initiated, non-past-directed actions. Chapter 9 distinguishes eight kinds of conceptualized value and discusses which ontological categories they apply to, their objective and subjective linguistic expression, and their varying conceptual roles. The value system is characterized as an abstract, multi-dimensional calculating system that guides action — albeit typically in a “quick and intuitive” fashion. Chapter 10 concerns how we conceptualize fairness, reciprocity, and exchange. Drawing upon Chapter 8’s discussion of joint intentions, it distinguishes freely chosen reciprocity (including reciprocal altruism) and agreed-upon exchange. Chapter 11 turns to the conceptualization of rights, obligations, and the authority to impose obligations. Rights are analyzed as a relation between an Actor and an Action; obligations require as well someone to whom the Actor is obligated.³ Along the way, Jackendoff argues that the language used to characterize these domains (terms of possession for rights, of constraining force for obligations) does not support the hypothesis that these concepts arise from metaphorical extension; rather, the rough association between independently existing conceptual clusters motivates the linguistic collocations. Finally, Chapter 12 briefly speculates about which aspects of social cognition may be innate, which of those may be shared with other primates, and which may be specific to social cognition as opposed to part of a more general cognitive capacity.

Comments

Language, Consciousness, Culture, beyond up-dating Jackendoff’s views on a variety of topics, adds a distinctive perspective to the study of social cognition. The comparison with linguistics delineates a framework for the field, and the examination of how social judgments are linguistically expressed supplies a new source of evidence. If at points Jackendoff raises more questions than he (sometimes speculatively) answers, he also sharpens those questions and indicates directions along which answers may be found. The book succeeds as well because it is so rich and thought-provoking: quite a few of its brief asides could themselves form the basis of an interesting research program. I’ll try to indicate this richness by raising a few questions about just two of the many topics touched on: consciousness and theory of mind.

Consciousness

I’ll remark on three aspects of Jackendoff’s views on consciousness: the Intermediate-Level Hypothesis, the impossibility of conscious thought, and the valuation feature ±self-initiated.

Jackendoff, recall, holds that the functional correlates of consciousness are intermediate-level structures, where top-down processing first comes into play and attention is attracted and anchored (phonological structures in the case of linguistic experience). But Prinz (2007), following Bisiach (1992), argues that subliminal perception and neglect show that the occurrence of an intermediate-level structure is not sufficient for consciousness. He suggests that the structures must be attended to as well. Jackendoff, however, while accepting that consciousness and attention have a “close relation” and that consciousness is necessary for attention, does not claim that attention is necessary for consciousness. If I understand him aright, at least some of his cases of “fringe awareness” would exhibit consciousness without attention. (Cf. Jackendoff 1997, p. 197.) Unless he would dispute the putative counter-examples, Jackendoff must in some other way complete his account.

An alternative to Prinz’s amendment would be to isolate those intermediate-level structures appropriately related to conceptual structures. This could take several forms. The functional correlates of consciousness might be those intermediate-level structures “poised” (adapting Tye 1997) to activate structures that conceptualize the intermediate-level structures’ content features — or the correlates might be those conceptual structures themselves. Alternatively, the correlates might be “higher-order” conceptual structures that incorporate the intermediate-level structures — or the intermediate-level structures together with conceptual structures indexed to them.⁴ However developed, consciousness would thus indeed require an intermediate-level structure (so the data that motivate Jackendoff’s account would be accommodated), but the occurrence of such a structure would not suffice for consciousness. With the possible exception of the “poised” proposal, Jackendoff rejects these and other “fashionable” views of consciousness (pp. 85-6). But to do so he must answer Prinz’s worry in some other way.

I turn to my second remark on consciousness. Jackendoff maintains that we are not conscious of thoughts — that is, conceptual structures. However, we can be conscious of our thoughts “indirectly” (p. 107):

We are aware of the content of our linguistically expressed thought only by virtue of experiencing phonological images associated with them, plus other images that are inferentially nonefficacious. (p. 83)

It is possible for us to be conscious of our thoughts in a way that is impossible for animals: not through awareness of the thoughts themselves, but through awareness of phonological structures associated with thoughts, which animals lack. (p. 84)

It is unclear, however, in what sense awareness of phonological structure enables consciousness of thought at all. To be sure, on Jackendoff’s view, it is the association of phonological structures with conceptual structures that enables us to understand them. Perhaps it is more contentious to claim that we understand what we think via the experience of an associated phonological structure that in turn activates the very same conceptual structure that gave rise to the phonological structure. But, in any event, what is wanted is some connection between this understanding and indirect consciousness of thought. After all, phonological structures are just as much associated with syntactic structures. Should we also say that we are indirectly conscious of syntactic structures in virtue of their correlation with phonological structures?

Jackendoff wants to find within his account of phenomenal consciousness — or, conscious experience — some place for a notion of conscious thought, even if only an ersatz. (This is needed not only to accommodate common sense, but also to cash out his own theorizing: Jackendoff does not hesitate to distinguish thought that is and isn’t accessible to consciousness — cf. my summary of Chapter 5.) But perhaps conscious thought is just a different phenomenon. Suppose I am asked what I am thinking and answer, “I was wondering whether that linguistic claim has a counter-example.” As I searched for a counter-example, my conscious experience may have included various phonological images. But among them need not have been a phonological image of, say, ‘I wonder whether that linguistic claim has a counter-example’ or of ‘Does that claim have a counter-example?’. If my report of what I was thinking is accurate, then it seems I can consciously think that P without experiencing a phonological image correlated with the relevant conceptual structure.

Finally, some remarks about one of Jackendoff’s candidate valuation features: ±self-initiated (in the case of language, experienced as your own speech vs. as another’s). Among Jackendoff’s marks for being a valuation feature are that it apply to multiple modalities and that it to some extent combine freely with other evaluation features. Accordingly, Jackendoff notes that ±self-initiated applies as well to proprioceptive experience, where it freely combines with ±external: consider voluntary bodily motions and involuntary motions, as well as voluntary and involuntary images of bodily motion. Jackendoff notes, however, that external, +self-initiated] visual experience seems impossible (though cf. p. 88, fn. 4). This would seem to hold for taste as well, but not for touch. Why is that? Here’s a possible explanation, albeit one that would threaten ±self-initiated’s status as an evaluation feature. We can speak and touch voluntarily. But while we can also choose to see in the sense of putting ourselves in a position to see (e.g., by opening our eyes — and also choose to see something in particular by directing our gaze or otherwise moving), when our eyes are open and otherwise ripe for seeing, arguably the seeing is not something we voluntarily do. The suggestion, then, would be that seeing (likewise tasting) is not accompanied by a relevant doing; and, with those experiences that are accompanied by a relevant doing (hearing the product of one’s speaking, visually imagining), it’s the doing that supports the “feel” coded by +self-initiated. That is to say, we do not, for instance, experience a bit of language as +self-initiated; rather, in such cases the auditory experience is accompanied by a +self-initiated proprioceptive experience: we experience, as the case may be, speaking as +self-initiated. As Jackendoff (pp. 87-8) is careful to note, “valuations are not characteristics of one’s experience as a whole [RJ’s italics]. Rather, they are attached to particular percepts and/or images.” But if +self-initiated is attached to proprioceptive experiences in these cases, then ±self-initiated does not seem to be multi-modal after all.

It might be replied that [-external, +self-initiated] visual experiences — i.e., voluntary visual imagery — and other voluntary imaginings suggest otherwise. After all, it’s not clear that our experience of mentally acting is proprioceptive. But it doesn’t matter whether it is; it would suffice that we experience mental doings as well as the image, whatever the sensory modality and however the senses are individuated.5 Another reply might be that self-initiated visual, auditory, and gustatory experiences suffice for multi-modality. But, in the absence of +self-initiated cases, what justifies claiming that the functional correlates encode a lack of self-initiation as opposed to denying ±self-initiated’s applicability to these modalities altogether? Finally, one might grant that the feature is not multi-modal but deny that this threatens its candidacy: multi-modal applicability may be a useful mark for a candidate valuation feature to have even if not all valuation features are in fact multi-modal. The discussion nonetheless illustrates a general methodological issue: it can sometimes be difficult, when two kinds of experience A and B as a rule co-occur, to distinguish introspectively which has some felt feature - all the more so if they are experienced as pair-wise connected (cf. p. 97).

Theory of Mind

I conclude with two remarks on Jackendoff’s views on theory of mind — the ability to ascribe beliefs, desires, and intentions (p. 164), as well as experiences (pp. 206-7), to others. The first concerns its relation to social cognition and our dualistic tendencies; the second concerns Jackendoff’s claims about ‘looks at’ and ‘sees’.

Jackendoff emphatically distinguishes theory of mind and the capacity for social cognition. First, we attribute mental states to tigers but do not consider them individuals with whom we can have social relations. Second, some social statuses — e.g., clan membership — can obtain regardless of what mental states we attribute. Third, monkeys seem to possess social cognition but not theory of mind.

These considerations are open to debate. First, there are no cognitive blocks to my respecting or taking myself to be respected by a tiger, or to my taking myself to be dominant over or dominated by one. The social possibilities are attenuated, but so are the mental states I can ascribe.⁶ Second, though clan status might not depend on the particular mental states ascribed, it might depend on taking the individual to be a (potential) bearer of such states. As for monkeys, recent research on this contentious topic arguably suggests otherwise (see, e.g., Lyons and Santos 2006).

But suppose we grant Jackendoff’s claim. I wish to raise a worry for his further claim that it is the nature of social cognition that accounts for dualist intuitions. Jackendoff (see also Jackendoff 1993, p. 77), like Boyer (2001), Bloom (2004) and others, holds that our dualistic tendencies — e.g., “culturally ubiquitous belief in supernatural entities … that survive death”, “no problem conceptualizing persons coming to inhabit different bodies” (p. 162) — can be explained by a " ‘natural joint’ in our intuitive modes of thought" (p. 163). The joint is not carved between modules: on Jackendoff’s view, physical cognition, social cognition, and theory of mind do not constitute modules, either in his “proprietary format” sense or in any other sense “remotely close to the way the term … is commonly used in the literature.” (p. 207)⁷ Their development — both phylogenetic and ontogenetic — rather involves the availability (and appropriate processing) of the relevant kinds of conceptual structure. The physical, the psychological, and the social constitute different domains — or, “planes” as he also puts it — within our conceptual resources.

My question concerns the particular joint upon which his explanation rests. For Jackendoff, the explanation of our dualistic tendencies is not that we think of human beings as occupying both the physical and the psychological domains; it’s that we think of them as occupying both the physical and the social domains. But it is unclear that this is so. These matters are difficult to sort out because, as Jackendoff notes, the domains as a matter of fact closely overlap even if they are not coextensive. But consider Jackendoff’s example of non-overlap: do our dualistic tendencies fall short of tigers? In fact, it seems just as easy to conceptualize animals surviving their bodily death, metamorphosizing, body-switching, etc. as it is humans. (Cf. Bering and Bjorklund 2004.) It is not clear that this is only so if one anthropomorphizes them or treats them as honorary persons (as Jackendoff suggests we sometimes treat pets).

Finally, I turn to ‘looks at’ and ‘sees.’ As noted, on Jackendoff’s view, possession of theory of mind involves possessing certain kinds of resources for building conceptual structures. He argues, in particular, via an analysis of the semantics of ‘looks at’ and ‘sees,’ that one crucial part of theory of mind is possession of a conceptual primitive ‘X EXP Y’ (glossed as ‘X experiences Y’), possession of which enables the attribution of experiences to others. On Jackendoff’s analysis, the conceptual structure associated with a sentence of the form ‘X looks at Y’ includes at least the following two tiers: [X SENSE_visual Y] and [X AFF], while that associated with ‘X sees Y’ includes at least: [X SENSE_visual Y] and [X EXP Y].⁸ The tier with ‘AFF’ (glossed as ‘affects’) marks what is encoded as Actor and Patient, standard tests for which are the acceptability of, respectively, sentences beginning ‘What X/Y did was …’ and ‘What happened to X/Y was … .’ (Actor and Patient need not align with thematic roles such as agent, theme, goal, etc.) Jackendoff argues that ‘EXP’ is needed as well to capture the fact that, while the subject of ‘X looks at Y’ is an Actor, the subject of ‘X sees Y’ is not — nor typically can ‘see’ occur in the progressive.

Jackendoff suggests that the combination of ‘SENSE’ and ‘EXP’ "encode[s] the conceptualization of the ‘feel’ of one’s relation to a percept, which has the feature combination [external, self-initiated]." (p. 206) But it also enables one to ascribe perception to others. On his analysis, then, the difference between the semantics of ‘looks at’ and ‘sees’ marks a major divide in the possession of theory of mind. Without ‘EXP,’ one may have experiences, but one cannot conceptualize them and thus cannot attribute them.

It’s not clear, however, that one can attribute looking at without attributing visual experience or at least the capacity for it. A blind person might be able to direct her eyes towards something, but she cannot look at it. Jackendoff notes that one can look at something without seeing it. But what matters is whether one can look at something without seeing anything, or without having any visual experience. If one can’t, it seems that the capacity to attribute looking at already involves the relevant part of theory of mind. Indeed, Jackendoff’s analysis seems itself to acknowledge this. For what the conceptual structures associated with ‘looks at’ and ‘sees’ have in common is ‘SENSE_visual’. Perhaps I am reading too much into ‘SENSE_visual’ (glossed as ‘senses in the visual modality’): Jackendoff writes that “SENSE+AFF yields the verb look, which denotes an observable action,” in contrast to ‘see’ which is “part of theory of mind.” (p. 206, my italics) ‘Looks at’ does indeed denote an observable action, but what it denotes differs from what ‘directs one’s eyes towards’ denotes. If ‘SENSE’ doesn’t capture this difference, then arguably Jackendoff needs to add something else that does.⁹

One might go further and suggest that ascriptions of looking at ascribe more aspects of theory of mind than do ascriptions of seeing. Mehler and Dupoux (1994, p. 139) claim that ‘look’ has an element of intentionality that ‘see’ lacks in that it involves an action, not a state. They presumably don’t mean just that the subject of a ‘looks’ sentence passes the test for AFF or has the thematic role of agent, which is also the case with ‘The wind made Bill sneeze,’ but rather that looking is typically done intentionally or is conceptualized as subject to voluntary control.¹⁰

References

Bering, J., and D. Bjorklund (2004) “The Natural Emergence of Reasoning about the Afterlife as a Developmental Regularity,” Developmental Psychology 40, pp. 217-33.

Bisiach, E. (1992) “Understanding Consciousness: Clues from Unilateral Neglect and Related Disorders,” in A.D. Milner and M.D. Rugg (eds.), The Neuropsychology of Consciousness, Academic Press, pp. 113-38.

Bloom, P. (2004) Descartes’ Baby: How the Science of Child Development Explains What Makes Us Human, Basic Books.

Boyer, P. (2001) Religion Explained, Basic Books.

Eilan, N. (2005) “Joint Attention, Communication, and Mind,” in N. Eilan, C. Hoerl, T. McCormack, and J. Roessler (eds.), Joint Attention: Communication and Other Minds, Oxford University Press, pp. 1-33.

Gross, S. (2005) “The Nature of Semantics: On Jackendoff’s Arguments,” The Linguistic Review 22, pp. 249-270.

Gross, S. (2007) “Reply to Jackendoff,” The Linguistic Review 24, pp. 423-9.

Hauser, M., N. Chomsky, W. T. Fitch (2002) “The Faculty of Language: What Is It, Who Has It, and How Did It Evolve?” Science 298, pp. 1569-79.

Jackendoff, R. (1987) Consciousness and the Computational Mind, MIT Press.

Jackendoff, R. (1992) Languages of the Mind, MIT Press.

Jackendoff, R. (1994) Patterns in the Mind, Basic Books.

Jackendoff, R. (1996) “How Language Helps Us Think,” Pragmatics and Cognition 4, pp. 1-34.

Jackendoff, R. (1997) The Architecture of the Language Faculty, MIT Press.

Jackendoff, R. (2002) Foundations of Language: Brain, Meaning, Grammar, Evolution, Oxford University Press.

Jackendoff, R. (2007) “Linguistics in Cognitive Science: The State of the Art,” The Linguistic Review 24, pp. 347-401.

Jackendoff, R., and F. Lerdahl (2006) “The Capacity for Music: What’s Special about it?” Cognition 100, pp. 33-72.

Landau, B., and L. Gleitman (1985) Language and Experience: Evidence from the Blind Child, Harvard University Press.

Landau, B., and R. Jackendoff (1993) " ‘What’ and ‘Where’ in Spatial Language and Spatial Cognition," Behavioral and Brain Sciences 16, pp. 217-238.

Lerdahl, F., and R. Jackendoff (1982) A Generative Theory of Tonal Music, MIT Press.

Lyons, D., and L. Santos (2006) “Ecology, Domain Specificity, and the Origins of Theory of Mind: Is Competition the Catalyst?” Philosophy Compass 1, pp. 481-92.

Mehler, J., and E. Dupoux (1994) What Infants Know: The New Science of Early Cognitive Development, Blackwell.

Prinz, J. (2007) “The Intermediate-Level Theory of Consciousness,” in S. Schneider and M. Velmans (eds.), Blackwell Companion to Consciousness, Blackwell, pp. 247-60.

Sperber, Dan (2005) “Modularity and Relevance: How Can a Massively Modular Mind Be Flexible and Context-Sensitive?” in P. Carruthers, S. Laurence, and S. Stich (eds.), The Innate Mind: Structure and Contents, Oxford University Press, pp. 53-68.

Sperber, D., and D. Wilson (1986) Relevance: Communication and Cognition, Blackwell.

Tye, M. (1997) Ten Problems of Consciousness, MIT Press.