A Footnote on the Imagery Debate: Pylyshyn responds to Kosslyn; Kosslyn Replies
ZENON PYLYSHYN [4.11.07]
The following occurs in "What Shape Are A German Shepard's Ears", A Talk With Stephen M. Kosslyn on Edge [7.15.02].
Apart from speculating that my views derive from the fact that I do not have mental images (I have vivid imagery and, being an academic who lives his life in his head, I use them all the time) and even that I fail to get the jokes in images (god knows where these irrelevant ideas come from but they deserve to be in gossip magazines rather than in a public document)! Kosslyn repeats (again and again) the old saw that I believe that images are epiphenomenal and play no role in thought. This gives away an assumption that Kosslyn shares with many people who support the picture-theory of mental imagery: the assumption that the subjective impression we have in mental imagery in itself constitutes a theory. If you claim for some X that X is "epiphenomenal" then you had better be prepared to explain exactly what you mean by X. A vague allusion to a subjective experience cannot be either epiphenomenal or not. Nobody could claim that images are epiphenomenal until they tell us what they think images are, beyond being a subjective experience. Kosslyn has a view about what the experience of having an image comes to: To have an image is to have in your head something that /looks like/ what you are imaging. He calls this relation of "looking like" /depiction/. Since it can't literally look like the dynamic 3D world the best we can hope for is that it is a picture of the world. Now this sort of object is not epiphenomenal, it is just nonexistent because the theory behind it is false.
I have in many writings shown that this assumption is not just a metaphor or a comfortable way of speaking, but it is an essential assumption upon which everything else Kosslyn says about imagery depends — including what one expects to find in the brain to support that view. If you believe in this idea strongly enough there is almost no limit to the sorts of neuroscience findings that you take to be support for this view — including the mere increased metabolic activity in the visual areas of the brain that may accompany imagining. As I have argued at length, even if one found actual pictures laid out in visual cortex this would not support the picture theory, That's because a layout of activity in visual cortex is the wrong kind of thing -- the wrong kind of pattern activity to underwrite mental imagery. Unlike mental images, such patterns (which, by the way, have not been observed beyond a general increase in activity in V1) would have to be two-dimensional, retinotopic (i.e., only a few degrees of visual angle in width), constantly changing, and so on — just as are the patterns put there by vision.
The rush to find neuroscience evidence to support the picture theory is so strong that Kosslyn even argues that low-level psychophysical properties, such as the oblique effect (whereby the resolution threshold for bars is better when the bars are horizontal or vertical than when they are at an oblique angle) occur with mental imagery and are easily explained by the fact that there are more cells sensitive to vertical and horizontal orientations than oblique orientations in visual cortex — an account that may well explain the oblique effect the case of vision. But it could not possibly explain the effect in imagery (assuming that there really is such an effect — it is not easy to do psychophysical experiments with images in which findings are uncontaminated by what the observers know would happen if they actually observed the world). The patterns, according to the picture theory, are projected onto the visual cortex. But the orientation-sensitive cells in the visual cortex are sensitive to the orientation of lines projected on the retina, not the orientation of activation patterns on the surface of the cortex. They are sensitive to retinal orientation because of the way photoreceptive retinal cells are connected, forming a template that is more sensitive to one orientation that another. Unless mental images are projected onto the retina, the numerical distribution of different orientation-sensitive cells in the visual cortex cannot explain any psychophysical effects found with mental images. But so desperate are those who seek a neurophysiological explanation of the bankrupt picture theory of mental imagery that they take almost any neuroscience finding as support. Contrary to Kosslyn's claim (quoted above) that I have "great disdain for neuroscience", what I have disdain for — not only in neuroscience but anywhere else in research — is poor reasoning and irrationally held beliefs. If Kosslyn does not have an adequate argument for his views he should at least keep from speculating on my motives and refrain from advertising his ideas about what I like or don't like in public documents.
And for a work that puts this debate in perspective along with an analysis of vision:
ZENON PYLYSHYN is Board of Governors Professor of Cognitive Science, Center for Cognitive Science and Department of Psychology, Rutgers University. His latest book is Seeing and Visualizing: It's not what you think.
STEPHEN KOSSLYN [4.13.07]
I apologize if my comments offended Zenon Pylyshyn. I was asked questions in the course of the interview, and responded honestly and directly on the basis of what I knew (or thought I knew, based in part on what I recalled Zenon's telling me, some 25 years ago — but I'll be the first to acknowledge that memories blur and morph with time, and I could have misremembered his comments).
Return to Stephen M. Kosslyn's "What Shape Are A German Shepard's Ears"