Unifying the Mind: Cognitive Representations as Graphical Models

Placeholder book cover

David Danks, Unifying the Mind: Cognitive Representations as Graphical Models, MIT Press, 2014, 287pp., $40.00 (hbk), ISBN 9780262027991.

Reviewed by Steven Horst, Wesleyan University


The two main themes of David Danks's book are nicely summarized by its title. The first, developed in Chapters 3-6, is that many forms of cognition can usefully be viewed as employing a particular form of representation called a graphical model. Chapters 7-10 argue the second claim, that the graphical-model-based approach might "unify the mind", and go on to compare it with several other accounts of cognitive architecture that seek to do the same (connectionism, ACT-R, Soar, and Sigma). While model-based approaches to cognition are quite common in artificial intelligence and cognitive science, they seldom find their way into philosophical discussions of the mind. I am continually puzzled by this, and am quite pleased to find Danks bringing to a philosophical audience a particular notion of a model (that of a graphical model) and its concrete applications to cognitive science.

Danks introduces the notion of a graphical model in Chapter 3, which also contains more technical subsections marked with asterisks so that readers not interested in the mathematical formalism can decide whether to skip them. Danks writes,

At their core, graphical models can be understood as compact representations of relevance relations, where different types of graphical models represent different types of relevance (e.g., informational, causal, probabilistic, communicative). They thus address a key challenge for any cognitive agent: namely, determining what matters and, often more importantly, what can be ignored. . . . At their cores, these models all have a graph: a diagram of nodes and edges that encodes the qualitative relationships.

From a purely mathematical perspective, the nodes are simply objects, and the edges can be undirected (A -- B), directed (AÕB) or bidirectional (AÖB). (pp. 39-40)

The nodes and edges (the lines between nodes) can represent various types of objects, properties, states of affairs, and relations, and hence the graph itself is underdetermined without an interpretation specifying that, for example, a unidirectional arrow in a particular graph represents causal or statistical relevance. When graphical models include quantitative information (such as degree of statistical relevance), the parameterization is also supplied independently of the graph. Graphical models are thus a very flexible form of representation, though not suited to every situation, and not all formalisms that make use of node-and-edge representations are graphical models.

Chapters 4-6 attempt to show that the notion of a graphical model can be applied to research in a number of areas of cognitive psychology. Chapter 4 applies the account to causal cognition, including causal inference and causal reasoning, though Danks provides reason to think that graphical models cannot be made to fit in the same way with accounts of causal perception. Chapter 5 applies it to concepts, categories, and inference, making a case that several major theories of concepts can be interpreted in terms of graphical models, and then shows how graphical models allow straightforward interpretation of reasoning with concepts. Chapter 6 shows how causal knowledge represented in graphical models can be used efficiently in decision-making, with the important twist of looking at models encoding causal knowledge in terms of decision-making procedures that exploit the most likely nodes for intervention rather than risking combinatorial explosion by considering an entire search space. This, in turn, is the linchpin of one of the most interesting potential implications of Danks's account: that it might provide a way of explaining how the mind is able to assess relevance -- a problem that has plagued AI since its second generation, as has been a repeated theme for critics of AI at least since Dreyfus (1979) criticized Minsky's (1974) frame theory for an inability to assess relevance (and hence to choose appropriate frames or decide when to change frames) using standard algorithmic methods. If graphical models have information about relevance built into them via the adjacency of nodes connected by directional links, they may provide some traction on a major problem that some have supposed to be unsurmountable.

While the non-technical parts of Chapter 3 provide a point of entry to readers unfamiliar with graphical models, 4-6 seem to be written for an audience already familiar with the theories and experiments that are discussed. The arguments that various types of causal cognition or alternative views of the nature of concepts can be accommodated within the graphical model framework are compact only because they are rather abstract, and there are times when I found myself wishing Danks had walked the reader through more concrete examples of applications. Some readers may have to spot him the assumption that graphical models provide a good way of accommodating causal cognition, concepts, and decision-making, and proceed to the remaining chapters with the question, "If this is the case, what more general conclusions ought we to draw?"

Danks's answer -- that positing a cognitive architecture based in graphical models allows us to "unify the mind" -- is an important and interesting claim, but may need to be set in context, particularly for readers of this journal who are not specialists in the relevant areas of cognitive science and philosophy of mind.

Types of Unity and Disunity of Mind

There are quite a few philosophical issues that are framed in terms of "unity" and "disunity of mind". Some of these -- debates about the unities of science, knowledge, and reason -- are not so much about the nature of the mind or its cognitive architecture as about whether the objects of cognition (things like beliefs and theories) can be made mutually consistent, reduced to a common denominator, or made into a single super-theory of everything. I have taken to referring to these collectively as questions about unity of understanding. These are not really Danks's targets. Danks mentions the view that scientific theories are idealized (p. 33), but does not take a position on whether this is a general feature of models or explore its consequences for semantics, epistemology, truth, or (more directly germane to topics he does explore) the extent to which theories or models can be taken to constrain one another. I would have liked to see further exploration of this, but it would have taken the book far from its primary agenda.

There are also a variety of kinds of questions about whether "the mind" (as opposed to understanding) is "unified". Danks mentions what may be the oldest of these -- the argument Plato gives in the Phaedo (which is repeated and further developed by Descartes) that the soul is a simple (i.e., indivisible, partless) substance. If Plato ever actually endorsed this view, he seems to have abandoned it in the Republic, where he describes what has come to be known as the tripartite soul, and indeed actually refers to reason, the appetites, and thumos as "parts" of the soul. I see the principal reason for this change in the recognition of a tension between the existence of multiple psychological faculties (an observation about cognitive architecture) and the intuition that the mind or soul or self "must" in some sense be a single thing. Descartes, who had a slightly longer list of faculties (intellect, imagination, sensation, will, memory, emotion), went to some lengths to make a case that the faculties are not things (i.e., substances) at all, but modes (activities or capacities) of a single type of substance. I suspect that Danks is no more inclined to traditional substantival metaphysics than I am. But it is interesting that some form of this tension between the recognition of multiple faculties and an intuition that the mind must be an indivisible unity persists in less metaphysically-inclined philosophers like Kant, who strenuously (and perhaps implausibly) resisted the conclusion that Sensibility and Understanding should be viewed as truly separate faculties. The bundle theory in Hume's Treatise and Dennett's Pandaemonium model stand out as a radical and striking exceptions to the philosophical impulse to find a way to preserve the assumption that the mind is some sense a simple unity in the face of apparent evidence to the contrary in the form of multiple faculties (or, more recently, modules).

A different kind of issue about unity also arises once one begins to enumerate multiple faculties. The list of faculties Descartes inherited from the medievals included "intellect" (a faculty for propositional representations involving the application of concepts and suitable for reasoning) and "imagination" (a faculty for imagistic thought, used both in perception and in free fancy). These seem to require distinct types of representational media: one for language-like thoughts and one for "images". Perhaps some of the other traditional faculties -- sensation, will, emotion -- are not truly representational at all. But even so, they would have to each operate on their own principles, distinct from those of Intellect and Imagination. In contemporary language, recognizing such a list of faculties seems to require that we view the mind as having a composite cognitive architecture, with distinct units that have different functions, and operate on information represented in different ways using different types of procedures. Against this background, we can see certain familiar projects in the history of philosophy as attempts to "unify" the mind by reducing the number of distinct forms of representations and procedures. Empiricists like Locke and Hume claimed that all ideas are ultimately images or associative constructions out of images. Computationalists claim that all cognitive processes are symbolic (though not necessarily symbols structured like sentences or propositions -- bitmaps are symbolic as well). Eliminativists opine that, in the end, "folk psychological" and even computational descriptions will prove to be inadequate to the task of providing satisfactory descriptions and explanations, and that we will find a common descriptive base only in the language and theories of neuroscience. (Though of course theory pluralism in the sciences raises questions about whether even that would provide a unified account of the mind.)

Danks's Unifying Project

Danks's unification project is not quite so broad, but is in the tradition of finding a type of mental representation that is a common denominator to a diverse set of cognitive activities. He is not concerned with sensation, emotion, motor control, or various phenomena studied by neuroscience that might be considered "subcognitive", like sensory pre-processing. There is, of course, a good deal of terminological disparity about just which phenomena should be counted as "mental" or "cognitive"; but Danks is probably wise to steer clear of controversies that risk ending up merely as disagreements about how we shall use particular words. If I had to characterize the domain he is interested in, I would guess that it would be something like "the phenomena that are studied in contemporary cognitive psychology": things like categorization, reasoning, planning, decision-making, and various subcategories of learning and memory. This is already a fairly broad class of cognitive phenomena, and has the advantage of having a wealth of experimental data to which theories can be held accountable.

Danks is, moreover, working within an explanatory framework that is committed to representations and processes, and it is ultimately crucial to his account that these be independent of one another. Given these assumptions, the available theoretical options differ with respect to what kinds of representations and processes they posit, and how they are related. For many philosophers, the kind of representation that comes to mind are sentence-like representations with propositional content, and the kinds of processes ones that involve application of argument forms. The first generation of AI tried this approach, but it quickly ran into a number of obstacles: a proliferation of individual representations, unrealistic computational time for serial processing, explosion of the search space for problem solving, inability to assess relevance. Second generation AI attempted to address this by exploring representational structures that more compactly encode complex understanding of particular semantic relations and problem domains, such as semantic networks (Collins and Quinlan 1969), frames (Minsky 1974), and scripts (Schank and Abelson 1977). These often employed some type of node and link structure, and I tend to use the term 'model' in a generic sense for such structures. Graphical models are a subset of these, typified by particular formal constraints. One of the chief advantages of this general approach is that inferential possibilities that mirror semantic inferences that we readily make often simply pop out of the right combination of model structures and procedures, suggesting that much of our fast, intuitive everyday reasoning is actually underwritten by mental models. And if this is the case, philosophical accounts of understanding that assume that the basic representational units are word-sized concepts and sentence-sized beliefs or judgments are missing something crucial about our cognitive architecture: that in some sense the basic unit of understanding is actually a mental model of a domain of some size, which is larger and semantically richer than a belief, sentence or proposition, but considerably smaller than the holist's comprehensive web of concepts, beliefs, and inferential dispositions. (I would say, in addition, that models generate a space of possible propositional representations using the concepts involved in the models, and in this sense models are prior to language-like propositional representations.)

A model-based account of cognition probably has to posit a great many models of different content domains. And in this sense, model-based approaches would seem to lead to a type of disunity claim in the form of a model-pluralism, which I have dubbed "Cognitive Pluralism" (Horst 2007, 2011). And if you assume, as I do, that models are optimized individually to be good-enough for particular epistemic and practical purposes, there is every reason to expect that different models will sometimes license different conclusions, thus generating inconsistencies that one might count as a form of epistemic disunity.

But this is not the kind of "unification" that Danks is after. His claim, rather, is that cognition is "unified" in the sense that understanding is all (or mostly, or at least to a significant extent) encoded in a particular type of model, the graphical model. Graphical models are a single type of data structure that is flexible enough to be able to be used for many types of representational and computational problems, a claim which Chapters 4-6 are devoted to making plausible through treatment of a significant though non-exhaustive list of cases. There are (presumably) many individual models, and we are continually updating them, but they are all of the same formal type, and a single model may be operated upon by many different reasoning processes. And this is very different from a modular architecture in which each module has its own proprietary types of representations which are operated upon only by a distinctive set of processes, even if modules also produce outputs into a "central cognition" system with its own set of domain-general processes.

Questions Going Forward

There are two questions that I found myself left with upon completing the book. The first is just how general the claim that cognition is based in graphical models is supposed to be. The language of "unifying the mind" might suggest that Danks is claiming that all cognition is based in graphical models. But often he makes more guarded versions of the unifying claim, such as that "large swaths of human cognitive activity" (p. 151) or "many aspects of cognition" (p. 175) can be regarded as graphical models. Indeed, in at least one case, he says that graphical models are not suitable. (pp. 93-98) Clearly, Danks regards his theory as accountable to empirical data. In this light, we should probably assume that his view is that it is ultimately an empirical question whether any particular type of cognitive activity can be explained as an operation upon a graphical model, in which case his unification claim should be regarded as a kind of working hypothesis for which he has already provided partial justification.

The second question is about just what we are committed to if we say that cognitive processes operate over representational structures that "are" graphical models. Danks acknowledges the worry that this might be interpreted as little more than a claim for the utility of a particular mathematical description, though he seems to favor a more realist interpretation of the status of graphical models as representations. Indeed, some passing remarks on representations suggest that he may favor a somewhat more robust "representation realism" than I would.

I propose a particular cognitive architecture in which many of our cognitive representations are well modeled by graphical models. This account is committed to the realism of these representations but is largely agnostic with regard to realism about the processes (though there must be suitable processes that can use the information encoded in the representations). More specifically, the account proposes that there are persistent objects in the mind that subserve a wide range of cognitive processes, but where the precise processing method might, but need not, be identical in all domains or all contexts. The account thus places both upward constraints on accounts of human behavior (e.g., about the effects of learning context on subsequent choices) and downward constraints on neural accounts (e.g., there should be some persistent neural object, state, or disposition that is the representation). (p. 37, emphases added)

This passage starts out sounding like an endorsement of a realism about representations as "persistent objects", which conjures images of the model-based equivalent of symbolic representations of data or stored programs in computer memory. But the last sentence softens this interpretation by including persistent dispositions among the possible realizers of mental models. This ambiguity is repeated in a later chapter. Danks seems to criticize connectionist models for lacking persistent representations because their equivalent of concept-activations appear only evanescently in the hidden layers:

Connectionist networks contain no persistent mental objects that could play the role of representations; cognition instead involves the distributed transformation of distributed information without any explicit or symbolic representation of entities and properties in the mind and the world. Of course, there is a sense in which the hidden unit activation levels at some particular moment do "represent" a particular dog in that moment, but this form of "representation" is radically different from that assumed by cognitive architectures based on discrete, persistent symbols. . . . At the same time, it certainly seems as though we do have persistent cognitive representations, and so something also seems to be wrong with the connectionist picture. (p. 181)

However, two pages later, he seems to want to leave open the possibility that models are, as we might say, functionally emergent from the behavior of the brain:

representation realism is best understood as a set of commitments about future behavior in different tasks that engage the same putative representation. I deliberately did not add any commitment that we have the ability to point toward particular objects in our brains, since we know too little about how the brain both represents and processes information about the world. As a result, my representation realism requires only that people behave in systematic (and systematically interpretable) ways that are best explained as operations on graphical models. That is, I commit myself to a realism about representations that is entirely consistent with them being neutrally distributed or emergent, as long as the distributed representations are appropriately stable across tasks and environments. (p. 183)

I tend to favor the broader interpretation, not only because I agree with Danks that a theory cast at the level of cognitive architecture should leave questions of implementation open (though constrained), but also because I think that what a model-based theory commits us to is something on the order of "having a model" (being able to think in ways corresponding to a model-based description), without any necessary commitment to there being "entities" (except perhaps in the broadest and most abstract sense) that are models.

This is an interesting and engaging book. There are sections that will be hard going for readers not already familiar with such things as Bayesian nets and Markov assumptions. But the exposition of graphical models and the unification claim can be understood without this background, and Danks has provided one of the few book-length philosophical examinations of a model-based approach to cognition, and this fact in itself is enough to make it an important contribution.


Collins, Allan M., and M. R. Quinlan. 1969. Retrieval Time From Semantic Memory. Journal of Verbal Learning and Verbal Behavior 8 (2): 240-247.

Dreyfus, Hubert. 1979. What Computers Can't Do. New York: MIT Press.

Horst, Steven. 2007. Beyond Reduction: Philosophy of Mind and Post-Reductionist Philosophy of Science. New York: Oxford University Press.

---. 2011. Laws, Mind and Free Will. Cambridge, MA: MIT Press.

Minsky, Marvin. 1974. A Framework for Representing Knowledge. http://web.media.mit.edu/%7Eminsky/papers/Frames/frames.html.

Rosch, Eleanor. 1978. Principles of Categorization. In Cognition and Categorization, edited by E. Rosch and B. B. Lloyd. Hillsdale: Lawrence Erlbaum Associates.

Schank, Roger, and Robert P. Abelson. 1977. Scripts, Plans, Goals and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale, NJ: Erlbaum.

Smith, Edward E., and Douglas L. Medin. 1981. Categories and Concepts. Cambridge, MA: Harvard University Press.