The Border Between Seeing and Thinking

At its best, philosophy of mind is informed by, and informs, the sciences of the mind. But getting the balance right is tricky. Too little science, and philosophy risks losing contact with reality. Too much science, and it risks devolving into journalism.

No one has hit the sweet spot more consistently than Ned Block. It should thus come as no surprise that his first monograph, The Border Between Seeing and Thinking, is a model for bringing philosophy and the mind sciences into dialogue.

At the heart of the book is a simple question: what distinguishes seeing from thinking? Or more generally: what distinguishes perception, including not just seeing, but also hearing, touching, and so on, from cognition, which includes thinking, reasoning, and decision-making? To answer this question, Block begins with a commonsense understanding of the perception–cognition border. He then identifies scientific indicators of the border. Finally, he uses those indicators to isolate constitutive features of the border. Crucially, this process allows for revision. The final constitutive account needn’t align with the initial commonsense account.

Among scientific indicators of the border, Block emphasizes adaptation. Aristotle noted that if you watch a rapidly flowing river for a while and then look away, things at rest will appear to move in the opposite direction. Likewise, if you stare at something red and then look at a white wall, the wall will look green. These aftereffects are caused by perceptual adaptation. Block argues that insofar as adaptation occurs in cognition at all, it occurs in a very different way—for example, without being sensitive to spatial or retinal location. Thus, adaptation can empirically distinguish perception from cognition.

Block nicely puts adaptation to work to help determine which “high-level” contents—contents that go beyond those that perception is widely agreed to represent, such as color, shape, and motion—are represented in perception. For example, if you stare at a large number of dots for 30 seconds, new displays of dots will appear less numerous than they would otherwise. Such visual adaptation, Block argues, is evidence that we see numbers. Ditto for causation, emotional expressions, and gender. By contrast, although you can immediately tell upon looking that a necklace is expensive or that an action is immoral, you do not adapt to expensiveness or immorality and so you do not see these properties. They’re represented in cognition, not perception.

While adaptation is a useful tool, Block denies that it is constitutive of perception. We can imagine perceivers that lack it. Still, it helps to refine our grasp of the border’s extension to better ground a constitutive account.

In offering constitutive accounts of the border between perception and cognition, philosophers have appealed to a range of properties, including phenomenology, the will, modularity, stimulus-dependence, representational content, and representational format. Block favors the last of these. While he is not the first to argue that perceptual and cognitive representations are distinguished by their format—Fred Dretske’s (1981) contention that perception is analog and cognition digital is one prominent progenitor—Block offers the fullest and most empirically grounded version of this account to date.

For Block, a mental representation is propositional if it can be used in reasoning. He takes this to require logical structure and then argues that perception is nonpropositional because perceptual representations lack logical structure. For example, there is no disjunction or negation in perception. You cannot perceive something as either red or green, nor as not red. Because concepts are elements of propositional representations, it follows that perception is also nonconceptual.

Block reinforces the conclusion that perception is nonconceptual with an illuminating discussion of color perception in human infants. There is evidence that, prior to eleven months, infants often do not notice color. For example, if a red object moves behind an occluder and then a green object emerges, infants exhibit no evidence of surprise. The amount of time they spend looking is the same whether the object that emerges is the same color as the object that went in or not. By contrast, if the object changes shape or kind—for example, if a disk becomes a triangle or a duck becomes a truck—they do look longer. That might seem to suggest that young infants don’t perceive color. But when shown a red disk against a uniformly green background, infants will fixate on the disk. Block concludes that infants see colors without activating color concepts. Thus, the content of their color perception isn’t determined by their color concepts. Infant color perception is nonconceptual. And since, Block argues, adult color perception is preserved from infancy, adult color perception is nonconceptual too.

The final format property that Block takes to be constitutive of perception but not cognition is iconicity, which he defines in terms of analog tracking and mirroring. According to Block, the “key difference” between iconic and discursive representations “comes in with the role of degrees of difference” (222). What makes the tracking and mirroring analog, and thus iconic, is that one thing that’s degreed tracks and mirrors something else that is degreed. In a mercury thermometer, the height of the mercury is degreed and it tracks and mirrors the temperature, which is also degreed. On Block’s view, perception works like that. One reason to think that perception is constitutively iconic, according to Block, is that visual imagery famously evinces evidence of analog tracking and mirroring, and there is reason to think that perception and imagery involve representations of the same kind.

We thus reach Block’s constitutive account: whereas perception is constitutively nonpropositional, nonconceptual, and iconic (NNI), cognition is constitutively none of those things. Paradigmatically, it is the opposite: propositional, conceptual, and discursive (PCD). This account is intended to informatively capture the nature of the very same border that the scientific indicators, such as adaptation, helped us to get a handle on in the first place.

While Block’s primary aim is to draw the border between perception and cognition, in the final chapter of the book he returns to a debate that has occupied him for the past 30 years: whether phenomenal consciousness overflows access consciousness—or very roughly, whether there is more to an experience than how the subject can cognitively use it. Drawing on his earlier discussion of color perception in infants, Block argues that since infants consciously perceive colors but cannot conceptualize colors, phenomenal consciousness overflows access consciousness, just as he has long maintained.

Block says that the “intended contribution” of his book “is not that perception is nonconceptual, nonpropositional and iconic, but the elaboration of what that view comes to, [and] engagement with the evidence for and against it” (2). Given this aim, the book is indisputably a success. Whether one ultimately accepts Block’s theses or not, the book clarifies the issues and, above all else, creatively shows how empirical work can bear on them. Still, I have some reservations about the theses.

Let’s start with Block’s account of the perception–cognition border. Here there are really two questions. Is he right that perception is constitutively NNI? And is he right that cognition is paradigmatically PCD?

Beginning with the second question, it’s important to be clear that Block is not saying that cognition is constitutively PCD. He allows that “perceptual materials” that are NNI can show up in cognition inside what he calls a “cognitive envelope.” For example, you can visualize your desk being repositioned while you’re trying to imagine how best to rearrange the furniture in your office. But in cases like this Block thinks that the representations comprise a “heterogeneous mixture” of discernible components, like oil and vinegar in a salad dressing. There’s the NNI visual image, and there’s the PCD representations accompanying it.

I worry, however, that admitting this much means giving up on the project of locating a joint in nature between perception and cognition. For if NNI representations can show up on either side of the border, then it’s hard to see how Block has used representational format to explain what the border consists in. When an NNI visual image shows up in cognition, what makes it the case that it is in cognition? Perhaps Block would answer: because it is in working memory. But then the real distinction between perception and cognition is whatever explains the difference between perception and working memory. And that can’t be format since working memory representations can be NNI.^[1]

The worry here is perhaps easier to see when we consider human infants and nonhuman animals. Infants and animals do not just perceive. They also reason, deliberate, plan, evaluate, and decide—capacities that Block associates with cognition. It is doubtful, however, that their cognition is primarily PCD. For one thing, they display very limited logical abilities, which Block takes as markers of propositional thought. For another, their cognition makes extensive use of analog magnitude representations (Beck 2015), which are iconic by Block’s own lights (224). This suggests that even the claim that cognition is “paradigmatically” PCD may be a stretch.

But maybe Block isn’t interested in distinguishing perception from cognition after all. Sometimes he writes as though he only wants to elucidate some features that are constitutive of perception, whether or not they distinguish perception from cognition. But this more modest project is hard to square with his professed interest in “a joint in nature between perception and cognition resting on differences in format” (1). Another possibility is that he’s only interested in distinguishing perception from one species of cognition—namely, propositional thought. But that would make his claim that cognition is paradigmatically PCD almost trivial, as well as make it hard to see why he denies that cognition is constitutively PCD. After all, something like visual imagery simply wouldn’t be part of “cognition” on this construal. Still, either of these more modest projects would be valuable in its own right and would appear to leave the vast majority of the positive claims and arguments in the book intact.

That brings us back to the first question: Is Block right that perception is constitutively NNI? Block takes object files to comprise the primary challenge. Object files are visual representations responsible for tracking and representing objects as they move and change properties. As I watch my three children zigzag around the playground, I use my object file representations to keep tabs on them. Green and Quilty-Dunn (2021) have argued that object files are discursive, at least in part. If object files are also perceptual, then we have a counterexample to Block’s claim that perceptual representations are constitutively NNI. Block denies that they are always perceptual, however. Sometimes they show up in working memory, which is part of cognition. And according to Block, they have discursive elements only when they show up in working memory and acquire “conceptualized remnants of perception in a cognitive envelope” (254). Whether or not one finds Block’s reply compelling here—Green (2023) and Quilty-Dunn (forthcoming) do not—I think there may be a more general concern.

Recall that for Block perception is iconic because it involves analog tracking and mirroring, like a mercury thermometer representing temperature. I’m sympathetic. In fact, I’ve argued that the perception of magnitudes often is analog in just this way (Beck 2019). When you perceptually represent luminance, sound wave intensity, temperature, weight, distance, number, or duration, you are representing a quantity that comes in degrees. Each can take on many values that can be compared in terms of being more or less. And the way you do that is by using some magnitude in your brain, such as the firing rate of a population of neurons, that is also degreed.

But that can’t be the whole story.

Or at any rate, that can’t be whole story unless perception only represents magnitudes and other things that are degreed. That’s because you can’t have an analog representation of something that is not degreed—not if what you mean by analog is one thing that’s degreed tracking and mirroring another thing that’s degreed. So, if perception represents things that are not degreed, its representations can’t be entirely analog, and thus can’t be entirely iconic in Block’s sense.^[2]

What might perception represent that isn’t degreed? It’s hard to say with certainty, but here are some candidates, partly inspired by Block’s own scientific indicators: social categories like face, agent, happy, sad, angry, male, and female; determinable shapes like triangle and straight edge; categories associated with gist perception such as forest, desert, and city; individuals like Ned Block and my car; phonemes like /da/ and /ga/; relations like on top of and inside; and fundamental categories like object and cause. I think Block would maintain that perception represents most, if not all, of these things. So the question is how he can handle them.

One strategy is to argue that they really are degreed. For example, Block might point out that some of these categories (though probably not all) can be characterized by multi-dimensional similarity structures. There’s such a thing as a “face space,” for example, which measures how similar one face is to another. But while a face space measures similarity among individual faces, the category face is not itself degreed. Moreover, concepts in thought can be characterized by multi-dimensional similarity structures. So, being characterized by a similarity structure cannot be sufficient for being analog.

Another strategy is suggested by Block’s discussion of the determinacy of icons. A photograph of the Empire State Building doesn’t require a discursive caption in order to represent the Empire State Building. Likewise, a perceptual icon can inherit singular content from its functional role. So, a degreed vehicle that represents something degreed, and is thus analog, might also represent something non-degreed because of its functional role. This is certainly possible. But I see no reason to think that it’s true in general.

Thus, while I agree with Block that much of perception is analog, and so iconic in his sense, I’m not persuaded that it’s entirely iconic, or constitutively NNI.

Finally, let’s consider Block’s new argument for phenomenal overflow, which borrows from his argument that perception is nonconceptual. Recall that the latter argument leans on the premise that color concepts aren’t activated during color perception. Block reasons: because color concepts aren’t activated, they cannot play a role in determining the content of color perception; hence, color perception is nonconceptual. As Block observes repeatedly (268, 291), this lack of color concept activation is consistent with color concept possession. Infants might have color concepts but not activate them to reason about object individuation. And although Block doesn’t say so, there is some reason to expect just that. After all, shape is a much better cue to object identity than color. It’s easy to individuate objects in a black-and-white photo or even a line drawing. By contrast, it would be very difficult to individuate objects by relying solely on their color. It would hardly be shocking, then, if infants were predisposed to privilege shape over color for object individuation. So, the evidence Block reviews doesn’t show that infants lack color concepts. In fact, as Block notes, infants can be trained in just two to three sessions to notice changes in color (280). If infants lacked color concepts—if they had to construct them on the fly—the rapidity of this learning would be hard to explain.

So far, so good. The problem is that by the time Block gets around to arguing for phenomenal overflow a few chapters later, he has slidden from the premise that infants do not activate color concepts to the premise that they lack color concepts: “The child’s nonconceptual color perception is conscious but not available to the global workspace because of the child’s lack of color concepts” (424, emphasis added). Here, Block argues that for a perceptual representation to be access conscious the perceiver must be able to conceptualize it; and to conceptualize it, the perceiver needs to possess the relevant concepts. No concepts, no conceptualization; no conceptualization, no cognitive access. Or as he puts it, “The infants’ color perceptions are not ‘accessible to reasoning and reporting processes’ because the concepts required for access are missing” (421, emphasis added). It’s crucial for this argument that the concepts are missing. If children had color concepts but simply failed to activate them to reason about object individuation, it wouldn’t follow that the outputs of color perception were not accessible to cognition. You can’t establish the phenomenal overflow of color perception from a failure to activate color concepts in reasoning about object individuation—no more than you can establish the phenomenal overflow of letter perception from the fact that I can stare at my scrabble tiles without noticing the seven-letter word they can make.

Block’s groundbreaking book is teeming with insights on topics that I have had to gloss over for lack of space, including whether perceptual contents are singular or existential, whether perceptual attribution or perceptual discrimination is primary, why Bayesianism about perception should be construed instrumentally, how to think about object files, and the extent to which perception is modular. It also has the virtue of being open access; you can download the book for free from OUP’s website. In fact, that is the only way to access its many helpful figures in color.

If you are looking for a clearly written, innovative, and penetrating treatment of issues that have dominated philosophy of perception in recent years—one that is unrivaled in its ability to demonstrate the relevance of empirical work—I cannot recommend this book highly enough.

ACKNOWLEDGMENTS

Thanks to the members of York’s Philosophy of Perception Reading Group for working through the book with me and to Brandon Ashby, Sam Clarke, Brian Huss, Taylor MacNicholas, Cassandra Williams, and especially Kevin Lande for feedback on this review. My work was supported by Social Sciences and Humanities Research Council of Canada Insight Grant 435-2022-0976, and the Vision: Science to Applications Program, thanks in part to funding from the Canada First Research Excellence Fund.

REFERENCES

Beck, J. (2015). Analogue Magnitude Representations: A Philosophical Introduction. The British Journal for the Philosophy of Science, 66(4), 829–855.

Beck, J. (2018). Marking the Perception–Cognition Boundary: The Criterion of Stimulus-Dependence. Australasian Journal of Philosophy, 96(2), 319–334.

Beck, J. (2019). Perception Is Analog: The Argument from Weber’s Law. The Journal of Philosophy, 116(6), 319–349.

Beck, J. (forthcoming). Between Perception and Thought. Philosophy and Phenomenological Research.

Dretske, F.I. (1981). Knowledge and the Flow of Information. Cambridge, MA: MIT Press.

Green, E.J. & Quilty-Dunn, J. (2021). What Is an Object File? British Journal for the Philosophy of Science, 72(3), 665–699.

Green, E.J. (2023). The Perception–Cognition Border: Architecture or Format? In B.P. McLaughlin & J. Cohen (eds.), Contemporary Debates in Philosophy of Mind. Oxford: Blackwell, 469–493.

Gross, S. (forthcoming). Iconicity, 2nd-Order Isomorphism, and Perceptual Categorization. Philosophy and Phenomenological Research.

Quilty-Dunn, J. (forthcoming). Remnants of Perception: Comments on Block. Philosophy and Phenomenological Research.