SHAPE ARE A GERMAN SHEPHERD'S EARS?: A TALK WITH STEPHEN M. KOSSLYN
is a gigantic project yet to be done that will have the effect of
rooting psychology in natural science. Once this is accomplished,
you'll be able to go from phenomenology. . . to information processing.
. . to the brain. . . down through the workings of the neurons, including
the biochemistry, all the way to the biophysics and the way that genes
are up-regulated and down-regulated.
This is going to happen; I have no doubt at all. When it does were
going to have a much better understanding of human nature than is
otherwise going to be possible.
battle coming over the future of smart mobs concerns media cartels
and government agencies are seeking to reimpose the regime of the
broadcast era in which the customers of technology will be deprived
of the power to create and left only with the power to consume. That
power struggle is what the battles over file-sharing, copy protection,
regulation of the radio spectrum are about.
Are the populations of tomorrow going to be users, like the PC owners
and website creators who turned technology to widespread innovation?
Or will they be consumers, constrained from innovation and locked
into the technology and business models of the most powerful entrenched
SHAPE ARE A GERMAN SHEPHERD'S EARS?: A TALK WITH STEPHEN
Stephen Kosslyn received tenure at Harvard, none of his
colleagues in the Psychology Department had scholarly
interests that overlapped with his, since most people
were doing mathematical psychology. Prior to Harvard,
during his time at Johns Hopkins, Kosslyn had become very
interested in the brain and computation, which was the
beginning of cognitive neuroscience. There weren't too
many people thinking about such matters at that point.
Over time, many of his senior colleagues in the Psychology
Department at Harvard retired or left, so he found myself
in the position of being chair of several search committees,
where he could nudge the program in a direction that,
he believes, turned out to be a very good idea. Kosslyn
chaired the committee that hired Dan Schachter, Patrick
Cavanaugh, Ken Nakayama, and Alfonso Caramazza. "I
tried to get Pinker, but failed on that one for
now," he says. "Most recently I chaired the
committee that brought in Susan Carey and Liz Spelke.
The department's gotten strong now. It's got a cohesive,
underlying theme, which means that there is the potential
The Department is currently oriented towards cognitive
neuroscience. "Right now," according to Kosslyn,
"its not very computational, which is a weakness.
Computation is the language of information processing,
not English, French, or any other natural language because
theres no reason to expect the kinds of concepts
and distinctions captured in natural language to be appropriate
for characterizing what's going on in the brain. It's
different than the objects we encounter in our daily lives.
Although we don't have the right version of a computational
language yetone thats tailored for this particular
machine rather than a Von Neumann machinecomputation
is clearly going to be the language."
M. KOSSLYN, a Professor of Psychology at Harvard University,
has published over 200 papers on the nature of visual
mental imagery. He has received numerous honors, including
the National Academy of Sciences Initiatives in Research
Award and the Prix Jean-Louis Signoret, and was elected
to the American Academy of Arts and Sciences and the
Society of Experimental Psychologists. His books include
Image and Mind; Ghosts in the Mind's Machine; Elements
of Graph Design; Wet Mind: The New Cognitive Neuroscience;
Image and Brain: The Resolution of the Imagery Debate;
and Psychology: The Brain, the Person, the World.
Kosslyn is a Fellow of the American Psychological Association,
the American Psychological Society, and the American
Association for the Advancement of Science, and has
served on several National Research Council committees
to advise the government on new technologies. He is
also co-founder of the Journal of Cognitive Neuroscience.
STEPHEN KOSSLYN: For the last 30 years Ive
been obsessed with a question: What shape are a
German Shepherds ears? Of course, I'm not
literally interested in that question, since if
I were I could just go out and look at dogs; Im
really interested in how people answer the question
from memory. Most people report that they visualize
the dogs head and mentally "look at"
its ears. But what does it mean to visualize something?
What does it mean to "look at it in your mind"?
It's a bit absurd, because there can't be a little
man in there that is actually looking at a picture.
If there were, there would have to be a little man
inside that man's head, and so forth, and it doesn't
make any sense.
For many years we tried to collect objective evidence
to show that when you have the experience of visualizing,
theres actually something pictorial in your
head. It turned out that the best way to approach
this was by turning to the brain. There are parts
of the brain that are physically organized such
that when you look at something, a corresponding
pattern is physically laid out on the cortex. Even
the first visual area in the processing stream is
often activated during visual imagery -- even if
your eyes are closed when you visualize. Moreover,
the way it's activated depends on what youre
visualizing. If you visualize something thats
vertical, you find activation along the so-called
vertical meridian ; if its horizontal, the
activation flips over on its side. Its absolutely
amazing. Similarly, visualizing objects at different
sizes changes the pattern of activation in ways
very much like what occurs if you are actually seeing
objects at the corresponding sizes.
But Ive been working on this for over 30 years
now and I want to move on. Instead of trying just
to establish that there actually are mental images
and that these images are bona fide representations
that have a functional role in processing systems,
I want to ask: So what? Who cares? Why should my
mother be interested in this kind of thing?
Lately Ive been working on something that
I'm tentatively calling the "Reality Simulation
Principle." It is built on my lab's findings
that about two-thirds of the same brain areas are
involved in visual mental imagery and visual perception.
This finding occurs even when the tasks seem very
different on the surface (for example, visualizing
an upper case letter in a grid and deciding whether
an X mark would fall on the letter if it were actually
in the grid versus deciding whether a spoken name
is appropriate for a picture). This is a huge amount
of overlap, which leads us to suspect that an object
seen in a mental image can have the same impact
on the mind and body that the actual object would
have. My notion is that once the brain systems are
engaged, they don't know where the impetus came
from. This means that they can produce the same
effects whether you activated it endogenously (from
information in memory) or exogenously (from looking
The "Reality Simulation Principle" describes
how to use mental images as stand-ins for actual
objectsto manipulate yourself, basically.
It is useful to understand it in conjunction with
what I call the GITI cycle, which stands for Generate,
Inspect, Transform, Inspect. If mental images can
simulate or stand in for actual objects and scenes,
you can generate the image, inspect what youve
got, transform it, and inspect the result. This
can be done iteratively, meaning that you can use
imagery to take advantage of the "Reality Simulation
Principle"to do all sorts of good things for
What kinds of good things am I talking about? Memory
is one obvious example. From the work of Alan Paivio
and countless others, we know that youre able
to remember objects better than pictures of objects,
and pictures of objects better than words. It also
turns out that if you visualize the objects named
by words you do better than you would otherwise.
Consequently, we're interested now in things like
hypnosis. We can hypnotize you, have you visualize
an object, and imagine that its actually a
three-dimensional object, appearing in glorious
vivid detail. In this case, your memory would be
boosted even further.
Mental practice is another candidate. Neuroscientists
such as Marc Jeannerod and Jean Decety have shown
that imagining doing something recruits most of
the brain mechanisms that would guide the corresponding
actual movements. And people in sports psychology
have shown that by imagining that youre engaging
in some activity youll actually get better
at doing it. This process involves generating an
image, inspecting the image, transforming it by
imagining your movements, seeing what the result
would be, and then cycling through again. The next
time through you can change the image as a function
of the result you saw. If you imagine youre
playing golf, for example, and your ball doesnt
get in the hole, you can imagine what would happen
if you whacked it a little more softly. Mental practice
clearly works. By understanding how mechanisms of
imagery works we can actually optimize this mental
The "Reality Simulation Principle" can
also be used to acquire self-knowledge. Try this
one out. Imagine its dusk, youre walking
alone, and youre late. You start to walk faster
and then notice a short-cut through an alley. Its
getting a little dark, but you really dont
want to be late, so you start to go towards it,
and you notice that there are three guys lingering
near the mouth of the alley. Now think about a first
scenario: The three guys look like theyre
20 years old, are wearing long droopy shorts, dirty
t-shirts, baseball caps that are on backwards, and
are smoking cigarettes. As you get close, they stop
talking, and all three heads swivel and fixate on
you and start tracking you. How do you feel?
Now try the same thing, except instead of those
three guys, make them three balding middle-age,
overweight accountants wearing suits. Theyre
standing there smoking cigarettes, and their heads
swivel as they track you. How do you feel now?
You can start simulating the effects of different
attributes. For example, what if the guys are black
or Latino teenagers. How do you feel? If you can
start actually sorting out your own emotional landscape
by running these kinds of mental simulations, you
may, in fact, discover certain things about yourself
that may be surprising.
Some people who confront their own beliefs may think
theyve got some racial issues, and it may
turn out theyre actually class issues. Make
those middle-aged accountants black and see how
you feel. Those kinds of simulations can help produce
self-knowledge, and can help a person to improve
his emotional intelligence.
You can also manipulate your body. Its obvious
that if you have a sexual fantasy you manipulate
your body by imagery. Also, if you imagine something
scaryan anticipated encounter with an authority
figure or a walk along a narrow path in the mountains
that is starting to crumbleyour palms will
sweat and your heart beat will change. Its
clear that mental imagery can affect the body, but
it turns out that it may be more interesting than
that. For example, one of the things were
studying now is how to change your hormonal landscape
by manipulating your images.
There's something called the victory effect, where
if you're a male and you win some sort of contest
your testosterone goes up afterwards. If you lose,
it goes down. This is not a surprise. It also turns
out that if you watch your favorite team win your
testosterone will go up. If your team loses, itll
go down. This even works if youre watching
chess, so its not about being aroused. In
fact, it works for the chess players and for the
people who are watching chess games.
Why is this interesting? With men it turns out that
spatial abilities vary as a function of testosterone
levels. In the fall, males testosterone levels
are relatively high. They go down thereafter, and
then they pick up again. Much research suggests
that the relation between testosterone levels and
spatial abilities is a U-shaped function; your spatial
abilities are not as good if you have too much testosterone
or too little testosterone. As you get older, both
testosterone levels and spatial abilities drop.
There is a lot of evidence that there is a connection
between the two. The question is, can we manipulate
ones spatial abilities by having you run simulations
of watching yourself win or lose? If "Reality
Simulation Principle" is correct, manipulating
your own testosterone levels would in turn affect
your spatial abilities. This is work in progress
in my lab, in collaboration with Peter Ellison and
Carole Hooven; stay tuned.
My point is that you can use "Reality Simulation
Principle" in lot of different ways, including
some ways that are not intuitively obvious, such
as manipulating your hormonal landscape. Mental
imagery is also important in creativity and problem
solving. Einstein reported that most of his thinking
was done with images prior to any kind of verbal
or mathematical statement. We know quite a bit now
about how to use images in the service of solving
problems and being creative. In fact, Ron Finke
has written a couple of unusually creative books
on this topic.
Other people have also claimed that you can even
manipulate your health by using what I'm calling
the "Reality Simulation Principle". Im
a little skeptical about this. We may look at that
eventually, but not right now. It's certainly the
case that you can manipulate the placebo effect
to some extent, but the medical effects of "Reality
Simulation Principle" are probably not huge.
It's not going to cure cancer.
My premise has always been that the mind is what
the brain does. Of course, thats a little
too glib; really the mind is what the cortex does,
since the brain does things like respiration that
are not mental. If this is the case, then the question
becomes, how do we understand information processing
in the brain?
Switching topics, let's return to the role of computation
in all of this. The computer is convenient because
it allows us to think about how events at different
levels of analysis can interact. This is one of
the deepest questions in psychology, and probably
science in general. It's really a mystery. How is
it that semantics and the meaning of things dictate
a sequence of events in this wet machine? The wet
machine itself has neurons, each of which have an
average of ten thousand connections. Sure it's complicated,
but ultimately you can understand the whole thing
in terms of chemistry and physics.
But how does this machine produce semantically interpretable,
coherent sequences of activity, and allow these
activities to be modulated by the semantics of what
it registers from the world? When you say something
to me, its obviously not just sound patterns,
since the content influences what my brain is doing.
How Im going to respond is a consequence of
what my brain did to produce the output.
Lets think for a moment about physical events
such as the status of bytes in the computer. Each
bit in each sequence of 8 bits is either on or off.
You can physically describe the nature of this machine
and the hardware, but you can also think about representation:
What does that pattern of physical activity stand
for, and represent in its absence? You can think
about interpreted rule-based systems, where the
representations have an impact on other parts of
a system, causing other representations to be formed,
or combined, or operated on in various ways, and
outputs to be generated. In this regard it's useful
to think about computation in a computer to describe
how the mind works, even though its a wrong
metaphor for the brain.
A computer is based on a Von Neumann architecture,
where you have a strict separation between memory
and the central processing unit. This means that
there is a strict separation between operations
and representations, which sit passively in memory.
The central processing unit is essentially a switching
device that uses instructions to dictate what its
going to do, both in terms of how it interprets
successive sets of instructions and what it does
with the representations. The very idea of representation
depends on how the CPU is set. The exact same pattern
of bytes can represent a number, a letter, or part
of a picture depending on how its being interpreted.
Once an operation is performed, the results go back
into memory, and serve as input for additional processes.
The computer is useful as a way of thinking about
all of this, but its not going to turn out
to be a model of how the brain works; the brain
doesnt work like this at all.
The critical thing about the computer -- in thinking
about computation as a model for understanding the
brain -- is that it really lets us think about how
advanced the mutual interaction of different levels
of analysis is. Its a wonderfulmystery. How
can an idea arise from wet stuff? How can an idea
influence whats going on with the wet stuff?
Here, the analogy really works with the computer.
We really can think about the notion of representation
in the computer, and how it dictates the sequence
of physical events through the organization of instructions.
Even though I try to track developments in computational
ideas, I'm not known as somebody who espouses the
computational model of mind. Nor am I considered
a neuroscientist. In fact, as far as I can see,
Im not known as a card-carrying member of
any particular approach or subfield. Ive always
been on the fringe. When I was a graduate student
I stumbled onto the basic phenomenon I've been studying
for 30-odd years now. In my first year of graduate
school at Stanfordthis was 1970the idea
of semantic memory was really hot. Collins and Quillian
had published a simulation model in 1969 in which
they claimed that information is stored in long-term
memory in the most efficient way possible. (This
makes no sense for the brain, by the way, since
storage space is apparently not an issue although
it is in a computer.) They posited that memories
are organized into hierarchies in which you store
information in as general a representation as possible.
For example, for animals, you've got a representation
of animals, and then birds, mammals, reptiles, etc.
And then under birds you have canaries, robins,
etc. The notion was that you store properties as
high up in the hierarchy as you can rather than
redundantly duplicating them. For example, birds
eat, but so do lizards and dogs, so we store this
property higher, up with the concept of animals.
You tag the exceptions in a lower level.
One way to test this theory was to look at response
times. If you give somebody a statement like "A
canary can sing," that information should be
stored right in the same place and "canary"
and "sing" should be bound together. But
if you ask him, "Can a canary eat?" he
should have to traverse the network to find a connection
between the two (assuming that "eat" is
stored up with "animal"). It should take
a little longer, and it does! Unfortunately for
the model, distance in a semantic net turned out
not to be crucial. My first year project at Stanford
showed that the response time was just due to how
closely associated the terms were, not distance
in a net.
One of the experiments was particularly interesting.
In one item I asked people to verify the statement:
"A flea can bitetrue or false?"
Two people in a row said false, and afterwards I
asked them why. One said that he "looked for"
a mouth, and couldnt find one. The other said
he "looked for" teeth and couldnt
"see" any. This idea of "looking
for" and "seeing" didnt fit
in at all with Collins and Quillian's network-based
computer model, so I started thinking about it.
My idea was that maybe imagery has something to
do with this. I telephoned everybody whom Id
tested already, and asked them if they had tended
to visualize when they were answering the question.
Roughly half said they did and about half said they
didnt. I simply plotted the data separately
for the two groups. What was dictating the response
time of the people who said they didnt use
imagery was how associated the properties were with
the objects. For the people who used imagery, that
had nothing to do itit was how big the properties
I immediately designed an experiment where I pitted
the two characteristics against each other. For
example, I asked people, "True or false?: A
mouse has a back," which is a trait that is
big but not highly associated. I also asked whether
it has whiskers, which is small and highly associated,
or wings, which is not true. I found that if I instructed
people to visualize, the critical thing was how
big the properties were. The bigger they were the
faster the responses were. If I asked them not to
visualize, but to answer intuitively as fast as
they could, the pattern reversed. In this case,
the response speed depended on how associated the
traits were, not how big they were.
The next question was how to think about these results.
Fortuitously, at the same time I was doing these
experiments I was taking a programming class. This
was in the days when you used punch cards. You had
to go to the computer center, submit your stack
of cards, and stand around looking at a monitor,
waiting for your job to come up and see whether
it bombed, which you could tell by how long it was
running. At the end they gave you a big printout.
One of the exercises in the class was to program
a set of little modules that generated geometric
shapes, like triangles and squares and circles,
and to adjust how big they were and where they were
positioned. You had to do things like make a Christmas
tree by recursively calling the same routine that
generated a triangle, and plotting the triangle
at different sizes in different positions, overlapping
them to produce the design.
As I was doing this, it suddenly occurred to me
that this is an interesting model of mental imagery.
We could think of imagery as having four main components:
Its got a deep representation, which is an
abstract representation in long-term memory; its
got a surface representation, which is like a display
in a cathode ray tube; its got generative
processes between the two , so the surface geometry
is reconstructed in the surface image on the basis
of the deep representation; and, finally, it's got
interpretative processes that run off the surface
image, interpreting the patterns as representing
objects, parts or characteristics.
This metaphor was neat, and led me to conduct a
lot of fruitful research. But it had the drawback
that no matter how hard you hit somebody in the
head, youre not going to hear the sound of
breaking glassthere's no screen in there.
Even if there were, would just be be back to that
problem of the little man looking at the screen.
This immediately led me to start thinking about
how to program a system where there are arrays that
function as a surface image and points that are
positioned in space depicting the pattern that represents
an object. And then you have something much more
abstract that's operated on to produce that.
One of the real virtues of thinking by analogy to
the computer is that it focuses you on the idea
of processing systems -- not just isolated representations
or processes, but sets of them working together.
Nobody had ever tried to work out in detail what
a processing system that uses images would look
like. In fact, the few detailed models of imagery
that existed all focused on very specific, artificial
tasks and tried to model them using standard list-structures
-- there were no images in the models of imagery.
We decided to take seriously the idea that perhaps
mental images aren't represented the same way as
language; perhaps they really are images. Steve
Schwartz and I built a series of simulation models
that showed such an approach is not only possible,
but allows you to account for much data. We published
our first paper on this in 77, and another
in 78. I also wrote a book on it in 1980 called
Image in the Mind, where I worked this out in much
more detail than anyone ever cared about. As far
as I can tell it had almost no impact. I remember
asking one my professors at Stanford about it, and
he thought the book was too detailed, and that for
somebody to start working on the topic now theyd
have to look at it, think about it and get into
it, and it was just too much trouble. Psychologists
generally don't like having to work with a really
detailed theoretical framework, and that was basically
the end of it. I have a mild frontal lobe disorder
that leads me to perseverate, and thus I've continued
to work out the theory and do experiments anyway.
My 1994 book on imagery is a direct outgrowth of
the earlier work, but now maps it into the brain.
The Europeans (especially the French) and Japanese
seem interested, if not the Americans.
That said, I should note that lately there are signs
that interest in mental imagery might be picking
up again. This might be a result of another round
in my old debate with Zenon Pylyshyn. Hes
a good friend of Jerry Fodor, but unlike Fodor,
Pylyshyn has maintained forever that the experience
of mental images is like heat thrown off by a light
bulb when youre reading: It's epiphenomenal,
it plays no functional role in the process. Pylyshyn
believes that mental images are just language-like
representations and that its an illusion that
theres something different about them. He
published his first paper in 1973. Jim Pomerantz
and I replied to it in 1977 and the debate has just
been rolling along ever since. Pylyshyn has great
distain for neuroscience, to put it mildly. He thinks
it's useless, and has no bearing at all on the mind.
I really dont know what brings him to this
conclusion. I suspect its because he is one
of the few (less than 2 percent of the population)
people who does not experience imagery. He apparently
doesn't even "get" jokes that depend on
images. He also probably rejects the very idea of
imagery on the basis of of his intuitions about
computation, based on Von Neumann architecture.
He's clearly aware that computers dont need
pictorial depictive representations. His intuitions
about the mind may be similar. But this is all speculation.
Pylyshyn is not only against theories that are rooted
in neural mechanisms (he thinks theories of the
logical structure of language should be a model
for all other types of theories really!),
he's also against neural network computational models.
I've probably published eight to ten papers using
network models. At one point in my career I did
work on the nature of spatial relations. I had the
idea that there are actually two ways to represent
relations among objects. One is what I call categorical,
where a category defines an equivalence class. Some
examples of this would be "left of," "right
of," "above," "below,"
"inside," "outside," etc. If
you are sitting across from me, from your point
of view, this fist is to the right of this open
palm, and that is true for all these different positions
[moving his hand about, always to the right of the
vertical axis created by his fist]. "Right
of" defines a category, and even though I move
my hand around, all of these positions are treated
This is useful for doing something like recognizing
a hand, since the categorical spatial relations
among my fingers, palm, digits, and joints do not
change. That's handy for recognizing objects because
if you store a literal picture in memory, an open-palm
gesture might match well, but if I make another
gesture with my hand, say clenching it, this would
not match well -- so you want something more abstract.
Categorical spatial relations are useful computationally
for that problem, but theyre not useful at
all for reaching or navigating. Just knowing that
a fist is to the left of this palm wont allow
me to touch it precisely; Ive got to know
its exact location in space. If Im walking
around the room knowing the tables in front
of me, "in front of" is a categorical
relation and thus is true for an infinite number
of positions relative to it. This is not good enough
for navigating. Thus, I posit a second type of spatial
relation, which I call coordinate: Relative to some
origin, the metric distance and direction is specified.
In my lab we have shown that the left cerebral hemisphere
is better at encoding categorical spatial relations,
which makes sense because categories are often language-based.
On the other hand (hemisphere), the right hemisphere
is better at encoding coordinate spatial relations.
This is true in normal people, it's true when we
and others have done neuroimaging, and it's true
when you look at deficit sin patients who have brain
lesions. We've constructed a whole raft of neural
network models that showed that, in fact, if you
split a modela networkinto two separate
streams (one for each type of representation), it
does better than if you have a single system trying
to make both categorical and coordinate representations.
The point is not so much that the hemispheres are
different, but rather that the brain relies on two
distinct ways to code spatial relations. This claim
caused a mini-controversy. I'm delighted to see
that in a recent issue of the Journal of Cognitive
Neuroscience researchers (who I don't know) tested
over a hundred people after they "turned off"
one hemisphere at a time for medical reasons, and
showed that with challenging tasks where you have
to make categorical versus coordinate spatial relations
judgments, the laterality effects I predicted worked
beautifully. If it was too easy it didn't work,
which also fits perfectly with our modeling and
previous experiments-- so it looks like this controversy
has been settled (but experience has taught me that
those are famous last words .).
This is really just one little corner of what I
do, and ultimately is related to my imagery work.
Ive always argued that imagery has to be understood
in a system that includes language-like propositional
representations as well as depictive representations.
I dont think of the mind as purely imaginal.
That can't be true. It's got to depend on coordinating
many different types of representations that interact
in intricate and interesting ways. The distinction
between the two types of representations invites
a further distinction between different forms of
imagery, which make use of the different sorts of
spatial relations. And in fact we have evidence
for such a distinction. One clear conclusion from
all this: "Imagery" isn't just "one
Getting back to computational theorizing in psychology
per se. From my perspectiveand maybe Im
missing somethingcomputational theorizing
has reached a plateau. Thats not to say there
isnt progress, but its incremental and
is currently within a paradigm that was set perhaps
ten years ago. I dont see any revolutionary
work out there. Right now, the connectionists are
probably the leaders in computational theorizing
relevant to the brain. David Rummelhart did terrific
work. Terry Sejnowski is excellent, as is Jay McClelland.
These are people who have been at it for years.
I don't see too much really new on the horizon.
In terms of interesting theorizing, Dan Dennett
and Steve Pinker and their colleagues are trying
to cash out the evolutionary psychology program.
Instead of trying to think about behaviors as being
the products of evolution, they are thinking about
how the modular structure of information processing
in the brain is a consequence of evolution. That's
an interesting program. My objection is that this
enterprise is not particularly empirical. Science
is the process of finding things out. You've got
to go out and do studies to find things out. It's
very helpful to have theories as a base from which
you can direct your attention to issues and questions,
but then youre got to go do the actual research.
If you asked me to explain the direction of mind
science writ large, I'd say that what youre
going to see is a bridging between cognitive neurosciencewhere
the mind is conceived of as what the brain doesand
genetics. Those are the two really hot areas right
now, and theres a giant gulf between them.
I was recently writing a introductory psychology
textbook chapter on intelligence, and read a lot
of behavioral genetics. I was really struck by the
fact that these guys are trying to bridge the gap
from genes to behavior in one fell swoop, and theyre
not doing that good a job at it. They're not doing
that well in linkage studies that try to connect
variability in a behavior with variability in different
types of alleles. Sometimes they manage 2% of the
variance. It occurred to me that theyre leaving
out the middle man. They want to think in terms
of the model: genes > behavior. But it
would be much better to think in terms of: genes
> brain, and then brain > behavior.
Genes influence behavior and cognition via what
they do to the brain. Thinking about this has gotten
me very interested in genetics, but not in the sense
that genetics is a blueprint. Most genes functioning
in the adult brain seem to be up-regulated and down-regulated
by circumstances. They turn on and off.
Heres an example developed by Steve Hyman
that can serve as a metaphor: If you want to build
muscles you lift weights. If the weight is heavy
enough its going to damage the muscles. That
damage creates a chemical cascade and reaches into
the nuclei of your muscle cells, and turns on genes
that make proteins and build up muscle fibers. Those
genes are only turned on in response to some environmental
challenge. Thats why youve got to keep
lifting heavier and heavier weights. The phrase,
"No pain no gain," is literally true in
this case. Interaction with the environment turns
on certain genes which otherwise wouldnt be
turned on; in fact, they will be turned off if certain
challenges arent being faced. The same is
true in the brain. Growing new dendritic spines,
or even replenishing neurotransmitters, is linked
to genes that are being turned on and turned off
in response to what the brain is doing, which in
turn is responding to environmental challenges.
I'm really interested in how genes allow the brain
to respond to the tasks at hand. When genes are
turned on and off, this affects what neurons are
doing; which then, of course, affects how blood
is allocated; in turn, affecting cognition and behavior.
There is a gigantic project yet to be done that
will have the effect of rooting psychology in the
rest of natural science. Once this is accomplished,
you'll be able to go from phenomenologythings
like mental imageryto information processingthinking
about things you can model on the computerto
the brainthinking about how a particular kind
of information processing arises from this particular
brain we havedown through the workings of
the neurons, including the biochemistry, all the
way to the biophysics and the way that genes are
up-regulated and down-regulated.
This is going to happen; I have no doubt at all.
When it does were going to have a much better
understanding of human nature than is otherwise
going to be possible. If you want to understand
evolution, the residue of evolution is the genes.
Why not study the genes if you want to understand
the reasons behind the brain's organization? There
are reasons we have those genes rather than other
onesthats where the evolutionary story
comes in. But my particular brain or your particular
brain is the way it is not only because of the particular
genes we have, but also because of the way the environment
up-regulated or down-regulated those genes during
development, sculpting our brains certain ways,
and the ways our genes respond to environmental
and endogenous challenges. All of this is empirically
tractable. The tools are available, the questions
are clear, and we know what sort of answers to seek.
Time to get cracking!
RHEINGOLD: SMART MOBS
1999 and 2000, Howard Rheingold started noticing people
using mobile media in novel ways. In Tokyo, he accompanied
flocks of teenagers as they converged on public places,
coordinated by text messages. In Helsinki, he joined like-minded
Finns who share the same downtown physical clubhouse,
virtual community, and mobile-messaging media. He learned
that the demonstrators in the 1999 anti-WTO protests used
dynamically updated websites, cell-phones, and "swarming"
tactics in the "battle of Seattle," and that a million
Filipino citizens toppled President Estrada in 2000 through
public demonstrations organized by salvos of text messages.
Drivers in the UK used mobile communications to spontaneously
self organize demonstrations against rising petrol prices.
He began to see how these events were connected. He calls
these new uses of mobile media "smart mobs."
For nearly two years, Rheingold visited hotspots around
the world where smart mob technologies and societies were
erupting. He had some idea of how to look for early signs
of momentous changes, having chronicled and forecast the
PC revolution in 1985 and the Internet explosion in 1993.
He is now sees a third wave of change underway in the
first decade of the 21st century, as the combination of
mobile communication and the Internet makes it possible
for people to cooperate in ways never before possible.
RHEINGOLD has evolved from a modest, quiet, thoughtful
working writer/editor into the flamboyant Howard "always
ten years ahead of his time" Rheingold. From his dazzling
hand-designed shoes to his vividly colored suits and
his TV ads for Kinko's, he has invented his own character-spokesman,
communications expert, celebrity, lecturer, writer,
thinker, wise man, one of the first people to recognize
the potential of a new medium for human communication.
As Kevin Kelly says: "Anywhere Rheingold goes,
I'll be there behind him."
is the author of Virtual Reality, and The
Virtual Community, and was the editor of Whole
Earth Review and the Millennium Whole Earth Catalog.
His new book is Smart Mobs: The Next Social Revolution.
mobs use mobile media and computer networks to organize
collective actions, from swarms of techo-savvy youth in
urban Asia and Scandinavia to citizen revolts on the streets
of Seattle, Manila, and Caracas. Wireless community networks,
webloggers, buyers and sellers on eBay are early indicators
of smart mobs that will emerge in the coming decade. Communication
and computing technologies capable of amplifying human cooperation
already appear to be both beneficial and destructive, used
by some to support democracy and by others to coordinate
terrorist attacks. Already, governments have fallen, subcultures
have blossomed, new industries have been born and older
industries have launched counterattacks.
There are both dangers and opportunities posed by this emerging
phenomenon. Smart mob devices, industries, norms, and social
consequences are in their earliest stages of development,
but they are evolving rapidly. Current political and social
conflicts over how smart mob technologies will be designed
and regulated pose questions about the way we will all live
for decades to come.
A number of new technologies make smart mobs possible and
the pieces of the puzzle are all around us now, but haven't
joined together yet. Wireless Internet nodes in cafes, hotels,
and neighborhoods are part of it. The radio chips designed
to replace barcodes on manufactured objects are part of
it. Millions of people who lend their computers to the search
for extraterrestrial intelligence are part of it. The reputation
systems used on eBay and Slashdot, and the peer to peer
capabilities demonstrated by Napster point to other pieces
of the puzzle.
Some mobile telephones are already equipped with location-detection
devices and digital cameras. Some inexpensive mobile devices
already read barcodes and send and receive messages to radio-frequency
identity tags. Some furnish wireless, always-on Internet
connections. Large numbers of people in industrial nations
will soon have a device with them most of the time that
will enable them to link objects, places and people to online
content and processes. Point your device at a street sign,
announce where you want to go, and follow the animated map
beamed to the box in your palm; or point at a book in a
store and see what the Times and your neighborhood reading
group have to say about it. Click on a restaurant and warn
your friends that the service has deteriorated.
The big battle coming over the future of smart mobs concerns
media cartels and government agencies are seeking to reimpose
the regime of the broadcast era in which the customers of
technology will be deprived of the power to create and left
only with the power to consume. That power struggle is what
the battles over file-sharing, copy protection, regulation
of the radio spectrum are about.
Are the populations of tomorrow going to be users, like
the PC owners and website creators who turned technology
to widespread innovation? Or will they be consumers, constrained
from innovation and locked into the technology and business
models of the most powerful entrenched interests?
Telephone companies and cable operators, with enormous investments
in old technologies, are moving to control who can build
enterprises on the Internet, and the kinds of enterprises
they can create. The expensive auctions of radio spectrum
for next-generation "3G" mobile communications are threatened
by the emergence of radically more cost effective technologies
in the form of grassroots wireless networks.
The entire 1920s scheme for regulating the use of the electromagnetic
spectrum is thrown into question by the invention of "cognitive
radios" and other wireless technologies that put power into
the hands of user communities rather than central broadcasters.
Five Hollywood movie studios and the four giant companies
that dominate the global recording industry say they are
trying to protect intellectual property, but are backing
legislation and "protection devices" that will lock down
computers and the Internet into a pay-for-play model in
which only the largest players will be allowed to create
or distribute content or services online, permitted to create
new kinds of computers, or empowered to invent things like
Although the recording industry succeeded in shutting down
Napster, and the legal arguments were about the theft of
copyrighted music, the technical significance of peer-to-peer
resource sharing is far greater than even the future of
the music industry. Seventy million people used Napster
within the first months of its existence. When tens of millions
of people pool their computing power, many things become
Seti@home uses the idle processing power of millions of
PCs to search for life in outer space and other CPU-sharing
"distributed computing" networks help search for new medicines,
understand the immune system, crack codes, predict the weather.
Wireless networks show that communication bandwidth can
be pooled. Combining the data storage, computation, and
communication power of millions of PCs makes possible entirely
new kinds of science, business, and social enterprise, based
on the emergent power of millions of individuals.
Combine wearable computing, wireless communications, and
peer-to-peer resource sharing, and all the people in a building
or a crowd walking down the street can join into ad-hoc
As influential as the Internet has been, it has been, for
the most part, confined to computers on desktops. Mobile
communication and pervasive computing technologies are permeating
every part of our professional and personal lives with Internet-enabled
capabilities. Just as the microprocessor and the television
screen combine into an entirely new technology with its
own capabilities, the personal computer, and millions of
computers linked through the global telecommunication network
constitute an entirely new technology with its own capabilities,
the Internet, the marriage of the mobile telephone and the
Internet will result in far more than email or stock quotes
in your pocket - the mobile Internet in a computation-pervaded
environment will constitute an entirely new medium with
its own properties.
Will the architecture and regulation of the emerging wireless
Internet be dictated by and empower a few large, highly
centralized institutions such as corporations and governments,
or will it favor the cooperative innovations of millions
of citizens - the way the architecture and regulation of
the wired Internet made the Web possible?
The people who make up smart mobs cooperate in ways never
before possible because they carry devices that possess
both communication and computing capabilities. Their mobile
devices connect them with other information devices in the
environment as well as with other people's telephones.
Dirt-cheap microprocessors embedded in everything from box
tops to shoes are beginning to permeate furniture, buildings,
neighborhoods, products with invisible intercommunicating
smartifacts. When they connect the tangible objects and
places of our daily lives with the Internet, handheld communication
media mutate into wearable remote control devices for the
The cost, size, and performing power of computers, video
displays, and wireless communications are moving from the
computer industry into the fashion industry, as wearable
computers embedded in clothing become cost-effective. Ultimately,
with peer-to-peer methodologies, reputation systems that
mediate trust between strangers, and ad-hoc broadband networks,
wearable devices will be desired, purchased, and used as
much for their social capabilities as for their utility
as information appliances.
There are the dangers as well as opportunities concerning
smart mobs. I used the word "mob" deliberately because of
its dark resonances. Humans have used our talents for cooperation
to organize atrocities. Technologies that enable cooperation
are not inherently pathological: unlike nuclear bombs or
land mines, smart mob technologies have the potential for
being used for good as well as evil.
Nevertheless, years before the September 11, 2001 attacks,
commentator Thomas Friedman prophetically referred to "superempowered
individuals" such as Osama Ben Laden who use modern technologies
and networked organizations to execute acts of terrorism.
RAND corporation analysts have pointed out that the Russian
mafia and Colombian narcotics trafficking enterprises use
"netwar" methods combining communication networks, social
networks, and networked forms of organization.
On the other hand, when cooperation breaks out, civilizations
advance and the lives of citizens improve. This is the big
opportunity of smart mobs. Language, the alphabet, cities,
the printing press did not eliminate poverty or injustice,
but they did make it possible for groups of people to create
cooperative enterprises such as science and democracy that
increased the health, welfare, and liberty of many.
Just as medicine only became an effective weapon against
illness when science furnished useful knowledge about the
nature of diseases, the most effective use of communication
and computer technologies could emerge from new scientific
understandings of human cooperation. The most powerful opportunities
for human progress are rooted not in electronics but in
understandings of social practices. Sociologists, political
scientists, evolutionary biologists, even nuclear warfare
strategists have contributed the first clues that an interdisciplinary
science of cooperation might be emerging.
Mobile communications and pervasive computing have the potential
for magnifying cooperation far more powerfully than previous
technologies; coupled with new knowledge about the social
dynamics of collective action, smart mob technologies could
make possible improvements in the way billions of people