Manovich, Lev The Engineering of Vision from Constructivism to Computers

Lev Manovich

The Engineering of Vision from Constructivism to Computers

INTRODUCTION 1

CHAPTER 1. VISUAL ATOMISM

1. I

NTRODUCTION

2. E

XPERIMENTAL

SYCHOLOGY AND THE

CIENCE OF

3. V

ISUAL

TOMISM

ODE FOR

ASS

OMMUNICATION

4. V

ISUAL

TOMISM AND THE

IND

-B

ODY

ROBLEM

5. V

ISUAL

SPERANTO

6. C

ONCLUSION

CHAPTER 2. I SEE, THEREFORE I THINK

1. I

NTRODUCTION

2. I S

, T

HEREFORE

I T

HINK

3. F

IRST

IGNS OF

EVOLT

: V

ENN AND

ALTON

4. "T

O TEACH THE WORKER TO THINK DIALECTICALLY

." 54

5. F

REUD

HEORY OF

ISUAL

EASONING

6. T

ISE OF THE

IAGRAM

7. V

ISUAL

ECHNOLOGIES AND THE

IND

8. A

NALOG

NGINE

CHAPTER 3. MAPPING SPACE

1. V

ISUAL

OMINALISM

2. "T

HE MOST IMPORTANT EVENT OF THE

ENAISSANCE

." 99

3. R

ADAR

: S

EEING

ITHOUT

YES

115

4. 3-D C

OMPUTER

RAPHICS

: I

NTERACTIVE

ERSPECTIVALISM

125

5. C

OMPUTER

ISION

UTOMATION OF

IGHT

132

6. C

ONCLUSION

146

CHAPTER 4: THE ENGINEERING OF VISION FROM INKHUK TO MIT

150

1. N

RTISTS BUT

NGINEERS

150

2. I

NFORMATION

HEORY

NGINEER

NALYZES

OMMUNICATION

156

3. T

NFLUENCE OF

NFORMATION

HEORY OR THE

DEOLOGY OF THE

ODE

162

4. F

ROM

UMAN

OTOR

UMAN

NFORMATION

ROCESSING

" 173

5. C

OMMUNICATION

NGINEER

NALYZES

UMAN

ISION

177

6. H

UMAN

NGINEERING

186

7. C

ONCLUSION

THE

ABOR OF

ERCEPTION

193

CONCLUSION 196

BIBLIOGRAPHY 199

FIGURES 209

Introduction

The dissertation presents a history of modern ideas about vision. I believe that vision is

not a timeless concept; rather, each period understands vision differently depending on how it is

used. In the twentieth century, vision acquired new roles as the medium of mass communication

and the instrument of labor, and, as any other productive tool, it was subjected to engineering,

rationalization and automation. Such new disciplines as applied experimental psychology and

cognitive science, communication engineering and film, robotics, and advertising design

continue to search for ways to utilize vision productively. In the process, they generate new

knowledge about vision, at the same time reducing it to a few disjoined and limited models. The

dissertation chapters follow the development of four such models: vision as a code, vision as a

means of logical reasoning, vision as a way to capture spatial information, and vision as

information processing.

Let us consider a few definitions of vision that are representative of entire research paradigms

and that were unthinkable before the middle of this century.

David Marr's Vision, published in 1980, summarized a decade of investigations on

human perception carried out at the MIT Artificial Intelligence Laboratory. This book has been

the most influential account of the computational approach to vision, shared by computer

scientists and psychologists. It opens with this statement:

What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what
is where by looking. In other words, vision is the process of discovering from images what is
present in the world, and where it is.

David Marr, Vision (New York: W.H. Freeman and Company, 1982), 3.

There is nothing "plain" about this definition of vision; it is functional and pragmatic. Visual

perception is reduced to a number of computational processes for the recovery of limited

information about the world: the identity of objects and their positions. In fact, this is the only

kind of information which may be required for a robot or an automatic missile to perform its

task. Marr projects these goals of machine vision onto human vision. Here, vision is reduced to

the common denominator shared by humans and low level organisms: to detect an obstacle, a

predator, a prey.

In a special 1984 issue of Cognition (the leading periodical of cognitive science) devoted

to visual representations, Steven Pinker outlined the understanding of vision commonly held by

cognitive psychologists:

Certain abstract problems could be best solved by translating their entities into imagined objects,
transforming them using available image transformations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.

In this definition, vision is valued merely for its topological properties -- the ability to represent

such relations as inclusion, proximity, and relative positions. According to many cognitive

scientists, these properties make it a more efficient medium for problem-solving and abstract

thinking than language. And since the mind itself is imagined as an exemplary model of

computational efficiency that evolved through evolution, scientists postulate the centrality of

visual (read: topological) representations for various mental processes. To think productively and

without waste, we might dispense with language and instead think through images, submerging

ourselves into the silent movie theater of our minds.

Here is another model of vision that narrows its definition to a particular productive

property. In a 1951 overview of applied experimental psychology, the field which is today

known as human factors, Paul Fitts writes:

Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 66.

Each sense modality has certain inherent advantages and disadvantages for the detection and
analysis of different kinds of information. Audition is more nearly a continuous sense than vision;
vision is basically selective and intermittent. As a consequence, audition is well adapted for the
detection of warning stimuli that may arise at any moment from one of a variety of sources,
whereas vision is well suited to the selection of and concentration on particular stimuli to the
exclusion of others.

What is the purpose of this research into the relative properties of the senses? In contrast to a

manual worker, the job of an operator in a modern human-machine system, be it an airplane

cockpit, radar display or an automated production line, is primarily perceptual in nature. His or

her work is to monitor the displays and to detect those signals that require intervention. Starting

in World War II, experimental psychologists have collaborated with engineers in the design of

displays and perceptual strategies for their operators. One of the important questions in this

research has been the relative advantages of visual, auditory and tactile displays for the detection

and analysis of different kinds of signals. Consequently, Fitts describes vision as a sense which is

most reliable and efficient for the constant surveillance of a single source -- it is "basically

selective and intermittent." According to this model, in order to use vision most productively,

one's eyes must be literally "glued to the screen."

In each of these models, vision is understood in terms of its efficiency in performing

specific tasks: recovery of three-dimensional information, logical reasoning, detection of signals.

In fact, there is no single meaning of "vision" which is shared among them -- "vision" as such

does not exist. The only thing that unites these modern approaches to vision is the quest for the

efficient, reliable, and effective instruments of labor. In this respect, vision is just a set of

separate tools which can be employed by either a human or a machine to get work done.

It is this emphasis on the productive uses of vision that distinguishes my project from a

number of recent histories of vision. In 1988 Hal Foster edited the anthology Vision and

Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1314.

Visuality which brought together the groundbreaking essays united by the common theme of

historicising modern vision, that is, rejecting it as a natural or a cultural constant.

In the last few

years, each of the five contributors -- Martin Jay, Jonathan Crary, Rosalind Krauss, Norman

Bryson, and Jacqueline Rose -- have developed their essays into books which today largely

define the direction of the discourse on the visual in humanities.

At the center of this discourse is the experience of a seeing subject: how is what one sees

influenced by psychic structures, by gender and power positions, and by the constraints of the

body; how is one's subjectivity determined by the experience of seeing and being seen? At its

boundaries are works of art and other cultural representations, which are interpreted as the reified

traces of this changing experience.

The accounts of these writers greatly expanded the horizons of our knowledge about

modern vision. Yet, what remained unexamined is the employment of vision for work. Indeed, it

is remarkable that the grounding philosophical and psychoanalytic models of vision, influential

in this discourse, are theorized from the experience of a "vacationer." For Sartre, it was the

person wandering in the deserted Parisian park; for Lacan, it was the young man (Lacan

himself), "being observed" by a shining sardine can, while he was in a boat with some fishermen

near a small port; for Freud, it was a young woman confined to isolation in a Viennese apartment

or sent to a resort. Not only are these subjects far from a modern workplace, but they are also

away from the prototypical perceptual spaces of modernity: the city street, the movie theater, the

shopping arcade.

Hal Foster, ed., Vision and Visuality (Seattle: Bay Press, 1988).

Martin Jay, Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought

(Berkeley: The University of California Press, 1993); Jonathan Crary, Techniques of the
Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press,
1990); Rosalind Krauss, The Optical Unconscious (Cambridge: The MIT Press, 1993); Norman
Bryson, Looking at the Overlooked: Four Essays on Still Life Painting (Cambridge: Harvard
University Press, 1990); Jacqueline Rose, Sexuality in the Field of Vision (London: Verso,
1986).

It is precisely the experience of these spaces that gave rise to a very different kind of

theorizing of vision in the writings of Walter Benjamin. Scrutinizing these new spaces of

modernity, Benjamin was among the first to notice the contiguity between the perceptual

experiences in the workplace and outside of it:

Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless, today's
pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus technology has
subjected the human sensorium to a complex kind of training. There came a day when a new and
urgent need for stimuli was met by the film. In a film, perception in the form of shocks was
established as a formal principle. That which determines the rhythm of production on a conveyer
belt is the basis of the rhythm of reception in the film.

Here Benjamin already notices that vision became a functional and pragmatic activity rather than

a disinterested or contemplative one. The eye is trained to keep pace with the rhythm of

industrial production at the factory and to navigate through the complex visual semiosphere

beyond the factory gates.

Among the contributors to Vision and Visuality, Jonathan Crary is the most sensitive to

the primacy of work for the history of vision in the modern period. In Techniques of the

Observer, he examines how such nineteenth century disciplines as physiology and

psychophysics, and such techniques as optical apparatuses and illusions prepared vision for its

later productive deployment. This far-reaching archeology is extremely revealing, yet, because

the account ends in the middle of the nineteenth century, it can outline the future deployment of

vision only in very broad terms: as participating in the process of the rationalization of labor and

as adapting to the new flux of dematerialized images and signs. In order to understand what

really happened to vision in the twentieth century, how it was actually deployed and investigated,

Crary's Techniques of the Observer is necessary but not sufficient. This century has seen the

emergence of disciplines and techniques of vision that simply cannot be traced to the period

Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt

(New York: Schochen Books, 1969), 175.

analyzed by Crary. In order to understand their current directions, we need to consider social

phenomena of more recent origin such as automation, computerization, and "the information

revolution." Most importantly, what could not have been predicted from Crary's account is the

radical shift in the nature of labor itself. Benjamin described the rhythm of perceptual shocks of

the assembly line worker of the industrial era, who was nevertheless paid to perform physical

work: lifting, hammering, filing, and molding. Today, however, in the post-industrial era, the

same worker is paid for "perceiving" -- watching, detecting, scanning, and monitoring.

This dissertation is an archeology of the currently prominent disciplines and techniques

of vision -- computer vision and cognitive science, virtual reality and other new human-computer

interfaces. In this archeology, I try to uncover the recently accumulated layers -- 1920s, 1940s,

1960s. Rather than postulating a single paradigm shift, I describe a number of distinct

approaches to the rationalization of vision, which developed in parallel. Each chapter

investigates one of these approaches.

The first chapter is concerned with the model of vision as code of communication.

Investigations of the psychological effects of basic colors and elemental forms, conducted in

experimental psychology since the second half of the nineteenth century, made possible the idea

of a rational visual language composed from simple elements. This idea acquired new

importance in the 1920s, when artists found themselves in the role of the designers of mass

communication. In Soviet Russia, artists collaborated with psychologists in order to create a fully

rationalized visual code of mass communication where each visual element is capable of

communicating a meaning, producing an emotion or causing a behavioral response.

Visual reasoning is the topic of the second chapter. While classical philosophy

considered vision unsuitable for reasoning, in the modern period this attitude is reversed. Freud's

model as defined in The Interpretation of Dreams, Galton's composite photography, Eisenstein's

intellectual montage, and the adaptation of graphic diagrams in logic and in popular visual

culture are all symptoms of this emerging understanding of vision: as the most efficient medium

for logical thinking. With the centrality of cognition for the post-industrial workplace, the notion

of visual reasoning acquired new importance. Consequently, the work on the models, techniques,

and technologies of visual reasoning, which was previously pursued by a few individuals, now

became a matter of systematic research on the industrial scale in such fields as cognitive

psychology and scientific visualization.

The third chapter addresses the role of vision as a way to capture spatial information.

Historically, this role has been played by the techniques of perspectival representation. Between

1940 and 1960, these techniques were automated (computer vision, computer graphics) and also

expanded beyond the realm of the visible with radar and other remote sensing instruments. The

automation of these techniques necessitated a new paradigm of vision research, which defined

vision as a computational system for the reconstruction of object positions in three-dimensional

space.

The last chapter takes up the understanding of vision in terms of information processing,

which emerged after World War II. In the post-industrial society, vision became the principal

instrument of labor, the most productive organ of an operator in a human-machine system. In

order to monitor and improve the system's productivity, engineers and experimental

psychologists have adopted a common framework of information theory for both human and

machine components.

Today, the number of new disciplines which study vision continues to expand: image

science, computer graphics, image processing, computer vision, research on human-computer

interfaces and so on. Why? Modern society relies on vision not only as a means for the

production of subjectivity but first of all as a means of economic production. If the first use of

vision has been extensively theorized, the second so far remained largely unexamined. I hope

that this dissertation will begin filling this missing gap in histories of vision.

Chapter 1. Visual Atomism

1. Introduction

Let us compare two statements both related to the question of visual effectiveness but made in

different periods.

Charles Henry was a French writer who is mainly remembered today for his theory of

scientific aesthetics which greatly influenced Georges Seurat and Paul Signac.

Writing in the

1880s, he advocated that aesthetic responses to simple perceptual elements (which for him were

brightness, color, and line) need to be studied scientifically:

Art pursues the expression of the physiognomy of things, and aesthetics studies the conditions in
which these things are satisfying; that is, when they are represented gay or sad, agreeable or
disagreeable, beautiful or ugly...aesthetic things for us are reduced to forms, to colors, and to
sounds.

Half a century later, in 1927, in the article Photography in Advertisement, L‡szl— Moholy-Nagy

states:

A modern engineer, if his goal and the functional purpose of his work are clear, can without any
great effort make a product that is formally adequate and perfect in its economic construction. But
the photographic advertisements of our time are not so easy to define. They don't come with
"user's instructions." Research into the physiological and psychological laws of visual
effectiveness is still far behind the times, compared to the study of the physical laws.

Both Henry and Moholy-Nagy argue that the creation of visual artifacts should be rationalized

and grounded in scientific psychological knowledge. However, the impetus for this

rationalization is fundamentally different. For Henry, the function of visual artifacts is artistic

See JosŽ ArgŸelles, Charles Henry and the Formation of a Psychophysical Aesthetics

(Chicago: University of Chicago Press, 1972).

Qtd. in Paul C. Vitz and Arnold B. Glimcher, Modern Art and Modern Science. The Parallel

Analysis of Vision (New York: Praeger, 1984), 87. My analysis of experimental psychology and
scientific aesthetics in this chapter is indebted to this groundbreaking book.

L‡szl— Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era, ed.

Christopher Phillips (New York: Aperture, 1989), 87.

expression; for Moholy-Nagy, it is to induce effects in the viewer. For Henry, the artifacts are

situated in the realm of the aesthetic, producing pleasure or displeasure, divorced from practical

life; Moholy-Nagy is concerned with everything practical -- changing beliefs, attitudes, behavior.

Henry calls for the rationalization of beauty while Moholy-Nagy wants to rationalize the process

of communication. He believes that modern designers have to grasp "the laws of visual

effectiveness" to be able to control the viewer's response -- precisely and predictably. And

finally, while Henry's "scientific aesthetics" deals with the artistic expression and aesthetic

response of an individual spectator, Moholy-Nagy's "laws of visual effectiveness" are to be

employed by the newly emerged institutions of mass communication, such as advertising.

In this chapter I will discuss one solution to the problem of the rationalization of vision

advanced by the 1920s artistic avant-garde. Investigations of the psychological effects of basic

colors and elemental forms, conducted by psychologists since the second half of the nineteenth

century, made possible the idea of a rational visual language composed from simple elements --

"atoms" of visual communication. This idea was already pursued in the nineteenth century by

such artists as Seurat and such theoreticians as Henry. However, in the 1920s, when artists found

themselves in the role of the designers of mass communication, the idea of an atomistic visual

language acquired new urgency and importance. The comparison between the statements of

Henry and Moholy-Nagy makes it clear that it was no longer a question of scientific aesthetics;

of a work of art producing an aesthetic effect in an individual spectator. Now it became the

question of rationalizing mass communication -- the question of economic and political

importance. Therefore, Moholy-Nagy writes about the need to precisely "engineer" photographic

advertisements, and Bauhaus welcomes experimental psychologists. The trend reaches its

extreme in Soviet Russia, where artists thought that they could control not just the consumer

habits of segments of society (as in the West), but the consciousness of the whole country. Here

the program of research into the "atoms" of visual communication became most systematic,

translating into the establishment of a number of psychological laboratories at various art

institutions.

Their goal was to create a fully rationalized visual language of mass

communication where each visual element is capable of communicating a meaning, producing an

emotion or causing a behavioral response.

These institutions included INKhUK (Institute of Artistic Culture, Moscow 1920-24);

GAKhN (State Academy of Artistic Sciences, Moscow 1921-30); VKhUTEMAS (State High
Art-Technical Studios).

2. Experimental Psychology and the Science of Art

As other nineteenth century sciences, experimental psychology approached its subject, the

human mind, by postulating the existence of further indivisible elements, the combination of

which would account for perceptual or mental experience. If chemistry and physics postulated

the levels of molecules and atoms and so forth, and if biology saw the emergence of the concepts

of cell and chromosome, experimental psychology applied the same reductive logic to the human

mind. Psychologists divided sensory consciousness into different modalities: vision, hearing, and

tactility and proceeded to enumerate the elementary sensations of each.

The first psychological laboratory was founded by Wilhelm Wundt at the University of

Leipzig in 1879. The general program of Wundt's laboratory was to define the fundamental

elements of each modality and to formulate the laws according to which these elements are

combined. In 1896 former student of Wundt, E.B. Titchener, who brought experimental

psychology to the U.S., proposed that there are 32,800 visual sensations and 11,600 auditory

sensory elements, each just slightly distinct from the rest. Titchener summarized his research

program as follows: "Give me my elements, and let me bring them together under the

psychophysical conditions of mentality at large, and I will guarantee to show you the adult mind,

as a structure, with no omissions and no superfluity."

This analysis of visual experience into its elements seemed to have no practical utility. In

fact, the new discipline of experimental psychology developed within philosophy, both

intellectually and institutionally. In German universities of the 1870s, more lecture courses were

devoted to psychology than any other branch of philosophy except logic.

Wundt, formerly a

Julian Hochberg, "Sensation and Perception," in The First Century of Experimental

Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, 1979), 94.

Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century of

Experimental Psychology, 25.

Kurt Danziger, Constructing the Subject: Historical Origins of Psychological Research

(Cambridge: Cambridge University Press, 1990), 211.

professor of philosophy, was quite content with the allocation of psychology to philosophy and

had no interest in its practical applications.

Yet, even before the discipline of experimental psychology was recognized officially, it

had already found its first eager users -- artists and aestheticians. This was not simply a question

of creating works of art which directly illustrated the new scientific theory, in this case,

psychologist's atomistic theory of perception -- such as the works of Impressionists and

Neoimpressionists which represented reality with dots and dashes, the painterly equivalents of

sensory elements. Rather than illustrating an atomistic theory of perception, some art theorists

and artists realized that they could adopt its approach as a basis of a truly scientific aesthetic and

art.

What made this idea possible? As experimental psychologists split visual experience into

separate aspects (color, form, depth, motion) and subjected these aspects to a systematic

investigation, from the outset they had been asking two questions. First, what physiological and

psychological mechanisms are responsible for the perception of each aspect of vision? Secondly,

what is the psychological effect on the subject of the stimuli, such as elemental colors or forms?

It is this second question which was of great interest to aestheticians and artists. By relying on

elemental colors or forms with known psychological effect, artists could now predict and

anticipate viewers' emotions. Conversely, aestheticians could now develop an objective aesthetic,

grounded in the scientific laws of the emotional effects of pictorial elements.

As an example of this research, consider an aspect of visual experience such as form --

"visual configuration considered as distinct from such things as color and pictorial content and

from the representation of objects."

While some psychologists were analyzing human

perception of simple forms such as squares, circles, and straight lines of different orientations,

Ibid., 39.

Vitz and Glimcher, Modern Art and Modern Science, 144.

others were investigating their psychological effects. In the 1870s a French professor of art,

Charles Blanc, developed a theory of the intrinsic psychological significance of vertical, straight,

and oblique lines.

Blanc popularized the ideas of the Dutch painter and theorist Humbert de

Superville who around 1830 first claimed that simple lines conveyed emotions by having an

inherent psychological response (fig. 1):

Blanc argued that the straight line is a "symbol of unity" and the curved line of "variety." Of the
straight lines, the most important is the vertical -- since man stands perpendicular to the horizon
and since he is bilaterally symmetric. The horizontal line is next in importance and then the two
obliques. Blanc discusses three types of human faces characterized by three orientations and by
the associated effects of happiness, rest and sadness.

In the 1880s, Charles Henry further advanced Blanc's ideas, developing the "aesthetic

protractor" to measure the harmony of line angles (fig. 2). Henry's device was designed to

measure "whether the angles between lines radiating in different directions from a single point

are harmonious."

It is significant that the first laboratory research on the aesthetics of simple forms was

conducted in 1876 by Gustav Fechner -- the founder of mathematical methods for the

measurement of sensations which became the basis for experimental psychology. Later, at the

turn of the century, many proto-Gestalt psychologists started to investigate the aesthetics of

simple forms, isolating preferences for squares, rectangles, ellipses, and triangles as well as

different lines.

Their publications became filled with "abstract" pictures consisting of simple

forms.

This psychological research into the effects of simple forms influenced Seurat, Signac,

Wassily Kandinsky, Paul Klee, and Piet Mondrian, among others. For instance, Seurat, who was

Ibid., 170.

Ibid., 169.

Ibid., 170.

Ibid., 155.

familiar with the works of Blanc and Henry, advanced a similar theory of the intrinsic

psychological effects of lines of different orientation (fig 3).

In another example, Kandinsky,

in Point and Line to Plane, advocated "microscopic" analysis of three basic elements of form

(point, line, and plane) claiming that there exists reliable emotional responses to simple visual

configurations.

Equally telling of Kandinsky's program are the titles of the articles he

published in 1919: "Small Articles About Big Questions. I. About Point," and "II. About

Line."

At first, artists would embed simple lines or forms in their otherwise representational

compositions. A typical case is Seurat who based the orientations of lines in his major paintings,

such as Le Cirque (1890-91) and La Parade (1887-88), on the theory which he derived from

Blanc: "Gaiety...of line, lines above the horizontal; calmness...the horizontal...Sadness...of line,

downward directions."

Gradually, however, artists gave up representation altogether and began

to compose works which would consist solely of the simple elements already studied by

psychologists. It is, therefore, not accidental that the paintings of Mondrian, Klee, and Kandinsky

look remarkably similar to the visual stimuli already widely used by psychologists in previous

decades. They are also experiments, a result of a systematic investigation into what Kandinsky

called "the science of art," the science which would allow the reliable communication of any

emotional experience.

Proposing scientifically based aesthetics, Henry wrote:

Ibid., 170-71.

Wassily Kandinsky, (1926), Point and Line to Plane (New York: Solomon R. Guggenheim

Foundation, 1947).

Yu. A. Molok, "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel

Florensky's 'dictionary of symbols.' A few margins), Sovetskoe Iskusstvoznanie 26 (1990): 328.

Qtd. in Vitz and Glimcher, 170.

That which science can and must do is to expand the agreeable within us and outside of us, and
from this point of view its social function is immense in this time of oppression and blind
conflicts. It ought to spare the artist hesitations and useless attempts in assigning or indicating the
way in which he can find ever more rich aesthetic elements; it ought to furnish the critic a rapid
means of discerning the ugly, so often informulable, however much it is felt.

Henry equates an artist with a scientist whose vocation is to provide the viewers with the means

of escaping the pressures and the unpleasant experiences of everyday life through the beauty of

art. The artist is the therapist of a society, and because of the importance of this role, he ought to

follow a scientific method in order to make his works most effective in helping the viewers to

forget the "oppression and blind conflicts" of modern existence.

Moholy-Nagy, in the already quoted statement from the 1920s, also compares the artist to

an applied scientist -- an engineer. The artist, however, is no longer removed from the realm of

everyday life. Just as the modern engineer shapes every aspect of material reality -- the machines

for living (architecture), the machines for transportation (airplanes, cars, trains), the machines to

produce other machines (the tools of industrial production) -- an artist has the crucial role of

shaping the psychical reality, controlling people's emotions and ideas through advertisements,

publicity, posters, and other visual propaganda.

Moholy-Nagy's appeal to designers and photographers brings forth the new purpose of

the research into the elemental units of vision -- the need to optimize the process of mass

communication. This new purpose became crystallized at the moment when modernist artists

claimed the position of designers of mass propaganda in Soviet Russia in the 1920s. At this

moment the two lines of inquiry -- artistic exploration of the visual elements and research in

experimental psychology -- explicitly converge in Soviet discourses and institutions. In a number

of Soviet art institutes of the 1920s such as INKhUK (Institute of Artistic Culture, Moscow

1920-24), GAKhN (State Academy of Artistic Sciences, Moscow 1921-30), and VKhUTEMAS

(State High Art-Technical Studios), El Lissitsky, Alexander Rodchenko, Osip Brik, and others

Qtd. in Ibid., 87-88.

collaborated with experimental psychologists to investigate the effectiveness of visual elements

and their combinations.

The paradigm of research into the elements of visual communication and their effects can be

conveniently traced through the history of INKhUK, which during the four years of its existence

united the key groups of avant-garde artists and theoreticians most committed to this research.

In the first year of INKhUK's existence the group led by Kandinsky was the most active.

His program called for the cataloging of basic elements of art and to describe their emotional

effects.

Following Kandinsky's proposals, INKhUK established its own psychophysical

laboratory in the Fall of 1920 but Kandinsky and his supporters were forced to leave a few

months later.

Kandinsky then became vice-president of the newly formed GAKhN and the

head of its largest physical-psychological division. The method of Kandinsky's division was

characterized as "the observation of and experimentation with the content and processes of

artistic creation and reception as well the work of art in all its complexity, understood as a

Other Soviet research institutes of the 1920s whose programs included the development of a

scientific approach to art and art history include: The State Institute of Artistic Culture (headed
by Kasimir Malevich, Leningrad 1923-27); The Institute of Literature, Art and Language, a part
of Communist Academy (1918-36); The State Institute for the History of the Arts (Leningrad
1921-31); The Institute of Art History, a part of Moscow University; The Russian Association of
Research Institutes of Social Sciences (Moscow 1924-31).

Wassily Kandinsky, (1920), "Programma raboty Instituta khudozhestvennoy kultury (The

Program of the Institute of Artistic Culture)," in Sovetskoe iskusstvo za 15 let. Materialy i
dokumentatsiia (Fifteen years of Soviet art: materials and documentation), ed I. Matsa (Moscow,
IZOGIZ: 1933), 126-139.

The work of INKhUK during first year of its existence has been documented in S.O. Khan-

Magomedov, "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920" (INKhUK:
appearance, formation and first period of its work, 1920), Sovetskoe Iskusstvoznanie (1981):
332-368.

3. Visual Atomism: a Code for Mass Communication

psycho-physical object."

The members of the academy included art historians, practicing

artists, physicists, psychologists, and physiologists.

Meanwhile, Alexander Rodchenko became the new president of INKhUK. Rodchenko

and his supporters formulated a new program in total opposition to Kandinsky. Against

Kandinsky's "subjective" psychologism they emphasized the materiality of objects. They also

opposed Kandinsky's "individualistic" composition in favor of construction and promoted the

making of objects rather than the creation of art.

However, the project of a science of visual

elements was not abandoned in practice, only the emphasis shifted from the "subjective" analysis

of individual aesthetic reactions to the "objective" analysis of the elements as manifested in

material objects. INKhUK members, who started to teach at the newly formed VKhUTEMAS

(State High Art-Technical Studios), had the opportunity to put these ideas into practice.

For

instance, in his course on graphics Rodchenko would start his students with systematic exercises

to combine simple forms (circle, triangle, and square) within a given format in order to test the

visual effectiveness of various combinations (fig. 4).

In its third stage INKhUK became the center of activity of the promoters of the idea of

production art -- Osip Brik, Boris Arvatov, Nikolai Tarabukin, and others.

In 1923, this group

founded the journal LEF (The Journal of the Left Front of Art, Moscow 1923-25) the goal of

which was to unite artists of the Left dedicated to radical revolutionary activity in the face of

what they perceived as the new flourishing of "bourgeois" art made possible by the NEP.

P.C. Kogan, "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its

journal), Iskusstvo 1, no. 1 (1923): 9.

A number of early constructivist manifestos are now available in English translation in

Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York: Rizzoli
International Publications, 1990), 61-82.

It was not accidental that the artists referred to their activities at INKhUK as "laboratory

work." "Laboratory" connoted scientific labor, rather than artistic experimentation. "Institut
Khudozestvennoy Kultury" (The Institute of Artistic Culture), Russkoe Iskusstvo 2-3 (1923): 85.

A.H. Lavrent'ev, "Propedevticheskaya distsiplina 'Grafika.' Vkhutemas. 1920-22 gody" (The

discipline 'Graphics.' VKhUTEMAS. 1920-22), Tekhnicheskaya Estetika 7 (1984): 19.

"Institut Khudozestvennoy Kultury," 86.

What should this radical revolutionary activity be? Western art historians, critics, and

artists glorified the constructivists' desire to give up the traditional roles of artists and instead

submerge themselves in industry, to leave the artist's studio for the factory -- in other words, to

become industrial designers.

What received practically no attention was another role which

constructivists and many other Soviet artists were claiming for themselves in the 1920s -- the

role of theoreticians and practitioners of mass visual communication, be it propaganda or

advertisement. And it was in this role, particularly as advertising designers, that VKhUTEMAS

artists, as well as their counterparts from the Bauhaus, were able to accomplish the transition

"from an easel to a machine" (to quote the title of a book by Tarabukin, one of the theoreticians

of production art).

These machines were not a lathe or an engine but a printing press, a film

camera, a light projector, a radio transmitter -- the machines of mass communication rather than

mass production.

Left artists were united in rejecting art in favor of mass propaganda. In the first issue of

LEF Sergei Tretyakov writes: "Revolution put forward new practical tasks [for artists] --

affecting mass psyche, organizing the will of the class"; "The work of futurism is parallel to the

work of communism -- futurism is fighting for the same dynamic organization of personality

without which the movement toward communism is impossible."

For LEF, this task of the

revolutionary education of the masses required a scientific method and in the same article

Tretyakov called for the creation of a "science of art as a means of emotional and organizing

In 1923, Lunacharsky wrote: "They [constructivists] play at being engineers, but they don't

know as much of the essence of machinery as a savage." Qtd. in Brandon Taylor, Art and
Literature under the Bolsheviks (London: Pluto Press, 1991), 1: 177.

Between 1923 and 1925 Rodchenko designed advertising for many Soviet government

agencies; Lissitsky, who was sent to Germany to publicize the art of the first Socialist state,
created brilliant designs for many German firms; so did Moholy-Nagy and other Bauhaus
faculty.

Sergei Tretyakov, "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to?

(Perspectives of futurism)], LEF 1 (1923): 197, 203.

influence on psyche, in connection with the task of class struggle."

In fact, artists should

become as scientific in their work as Marxists are in their political activity. To do so, they should

rationalize the creative process and put art education on a scientific basis.

This idea was most

systematically promoted by Proletcult (Organization of the Representatives of Proletarian

Culture). Thus, Tarabukin organized the artist's workshop aiming to bring the principles of

Taylorism and Fordism into artistic practice: "We are analyzing processes of work, experiment

with compressing work periods, aim to rationalize work and leisure. In its practice the workshop

tries to get away from bohemian habits so common to artists."

Artists should also rationalize

viewers' reactions by developing a science which would enable them to produce predictable and

measurable effects in the viewer. Thus, the author of a 1927 article which summarized the

activities of the Art Section of Leningrad's Proletcult writes:

Artist-communist, artist-revolutionary came to understand the necessity to perfectly master his
weapons to be able to strike without a miss. The striving to conscious mastery was put forward as
the main task ("it is necessary to know in order to produce"). Artists of Proletcult were sure that at
the basis of art's effect on perception are to be found completely objective laws. From this came
the task to master the "mechanics" of artistic work, in order to orient it toward new utilitarian tasks
of revolutionary art.

This dual goal of rationalization was symbolized by the popular metaphor of artist as

"psycho-engineer," frequently encountered in the discourse of constructivism, Proletcult and

Ibid., 199.

LEF required the same from poets and writers who could make their work scientific by

relying on the advances made by Russian Formalists in studying the devices of literature. For
LEF, these devices became the tools of verbal propaganda. In the first issue of LEF Osip Brik
writes: "OPOYAZ studies the laws of poetic production... OPOYAZ will help the art of the
proletariat, but not with vague conversations about the 'proletariat's spirit' and 'communist
consciousness,' but with precise technical knowledge of the devices of contemporary poetic
work." Osip Brik, "T.n. 'formalnyi metod'" (So called 'formal method'), LEF 1 (1923): 214-215.
The meaningless experimental poetry of Futurists which originally made Formalists recognize
the "independent value of the poetic sound" now became full of meaning; the techniques of
"poetic instrumentation" were now seen as the techniques of organizing the consciousness of the
masses.

I. Matsa, Sovetskoe iskusstvo za 15 let, 279.

Ibid., 281.

LEF. Aleksey Gastev, a leading figure of Proletcult as well as of the Soviet Taylorist movement

who in 1921 organized the Central Institute of Labor coined the phrase "social engineering" to

describe the process of the radical reconstruction of the human psyche.

Tretyakov applied the

metaphor to art: "Along with man of science the art worker should become psycho-engineer,

psycho-constructor."

The crucial question then becomes how to rationalize visual effectivity. In principle,

different approaches are possible: to create particular conditions of reception; to limit the number

of available images in the public sphere (this was done later in the period of Socialist Realism);

and finally, to create a particular code, a new visual language guaranteed to communicate

effectively and efficiently to mass audiences (often, in the case of Soviet Russia of the 1920s,

illiterate). It is the last solution which was explored by Leftist artists such as Rodchenko or

Lissitsky.

Why did the notion of a visual language consisting of simple abstract forms become

attractive? First, abstract geometric forms were neither associated with bourgeois figurative

representation nor did they look like the "subjective" abstract improvisations of Kandinsky.

Secondly, as Igor Golomstock points out, abstraction was justified by emphasizing its stylistic

origin in truly popular decorative and religious art (for instance, icons).

Thirdly, simple forms

were proclaimed to be efficient in a situation of mass communication. Thus, Lissitsky argued:

"The most unambiguous and immediately recognizable forms are geometric forms. No one will

confuse a rectangle with a circle, or a circle with a triangle."

Finally and most importantly,

such forms were imagined to be perfectly suited for the purpose of controlling the effect in the

viewer -- both the idea and the imagery being prepared by decades of investigations in the

Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 26.

Tretyakov, "Otkuda i kuda?," 202.

Golomstock, Totalitarian Art, 26.

Qtd. in Golomstock, Totalitarian Art, 24.

psychology of perception and in scientific aesthetics. The tradition of research into the

effectiveness of visual elements transformed into the dream of mass "psychophysical culture"

advanced in Soviet Russia in the 1920s.

The most striking record of this dream is the paper "Engineerism" which was presented at

one of INKhUK's meetings. The paper starts with an analysis of modernism as a reduction of

visual experience to its elements: "In the final stage of its development art has renounced

representation and has moved toward visual sensations as such."

The trend continues with

abstract art, the aim of which, according to the author, is "the organization of visual perceptions."

Bourgeois art cannot go beyond this stage of analysis, but in the first proletarian state, this

analytic work will become the basis of a new visual culture of social control: "Visual sensations

as such will concertedly shape the human being as an organized unit, with the tempo of

something which belongs only to its own time."

From this perspective, we also should not be surprised to find in the same period the

recurrent attempts to establish various graphic alphabets and dictionaries. The poet Velemir

Khlebnikov in his 1919 article "Artists of the World" postulated two types of a universal visual

code -- graphic signs and color equivalents. In the already mentioned INKhUK program,

Kandinsky proposed to create a full dictionary of colors, forms and their combinations with

descriptions of the psychological effect of each element. Pavel Florensky together with his

students at VKhUTEMAS started to compose "Symbolarium" ("The Dictionary of Symbols") in

1922.

This "international and ahistorical dictionary" should have contained all cultural

meanings of every graphic symbol. In the introductory article of "Symbolarium" Florensky

regretted that while whole academies were busy preparing and preserving the dictionaries of

Allison, Art Into Life, 79.

Ibid., 81.

P.A. Florensky, (1922), "Symbolarium (Slovar simvolov). Predislovie" [Symbolarium (The

dictionary of symbols). Introduction], Trudy po znakovym sistemam V (Tartu: Uchebnye zapiski
Tarturskogo Universiteta 284, 1971): 521-527.

national languages, no work yet has been done in composing a similar dictionary of ideograms --

the "universal language of mankind."

Similar projects of a universal graphic alphabet were

advanced in the same years by Ivan Puni, Petr Miturich, and other Russian artists.

To summarize: early Soviet attempts to theorize and put into practice a scientifically grounded

visual language of mass communication bring together artistic exploration of visual elements and

research in experimental psychology. While the desire of Soviet artists to emulate "scientific"

Marxism and to adopt Taylorism and Fordism were already sufficient reasons to pursue the

science of elements of art, the ultimate justification for such a science was provided by

experimental psychology and scientific aesthetics (Blanc, Henry, and others). Their

investigations of the psychological effects of basic colors and simple forms prepared the idea of

a language of mass communication composed from simple elements. In theory, this model called

to enumerate visual elements and to describe their effects in order to compose dictionaries of

"visual effectiveness." In practice, it translated into the creation of works composed from distinct

and simple abstract forms.

From 1921 to 1929 the psychological studies of viewers' responses to visual forms were

actively conducted in the central art institutes. In 1926 GAKhN created a psychophysiology

laboratory, a psychophysics department, and laboratories for experimental aesthetics and art

theory.

In order to arrive at the general laws of the effectiveness of visual forms the

investigators have systematically analyzed the basic dimensions of volume, line, color, and

Ibid., 527, 523.

Molok, "'Slovar simvolov' Pavla Florenskogo," 322.

E.L. Beliajeva, Arkhitekturno-prostranstvennaya sreda goroda kak obyekt zritelnogo

vospriyatiya (Architectural and spacial city environment as an object of visual perception)
(Moscow: Stroyizdat, 1977), 13.

4. Visual Atomism and the Mind-Body Problem

texture. It was hoped that the results of the analysis would put visual communication on a

scientific basis. Jack Chen, a Communist British artist trained in the Soviet Union during the

1920s, wrote about this program of art production based on psychological research:

Art must be a science, an industry. Pictures and sculptures should be constructed according to
exact scientific principles after colors and forms had been classified according to their human
reaction values...A picture according to the Left was really nothing but a "machine" for generating
certain predetermined human reactions. Artists should be engineers of form and color.

There does not appear to be a disagreement among the Left as to the function of art -- to

create predetermined viewer responses. But how was this to be achieved? In other words,

assuming that we have isolated the elements of a visual code, how should these elements affect

the viewer? Do they induce emotions, do they communicate ideas or do they control behavior in

a more direct way?

To answer this question, Soviet artists of the 1920s relied heavily on the psychological

theories of the day.

It is possible to discern two distinct trends in how psychology was used. In

the first trend, autonomy was granted to the human psyche. The second trend privileged

physiology, considering the mind a neurological organ not different from the rest of the

organism. This trend was represented by such influential scientists as Bekhterev and Pavlov,

with their studies of physiology, reflexology, and conditioning.

The latter model was also prevalent in Soviet Marxist psychology in the 1920s. In 1924 it

was challenged by Lev Vygotsky, a young psychologist and a former literary critic. Recognized

today as the forerunner of cognitive psychology, he argued that the higher cognitive functions

are qualitatively different from the lower physiological processes and require different

Jack Chen, Soviet Art and Artists (London: The Pilot Press, Ltd., 1945), 58.

For the history of Soviet psychology in the 1920s, I have consulted David Joravsky, Russian

Psychology: A Critical History (Cambridge, Mass.: Basil Blackwell, 1989); Alex Kozulin,
Psychology in Utopia: Toward a Social History of Soviet Psychology (Cambridge: The MIT
Press, 1984).

investigative approaches. In his famous address to the Second Russian Psychoneurological

Congress, Vygotsky declared human consciousness to be a fundamental problem in the

psychology of behavior, claiming that it cannot be understood through the study of reflexes.

Significantly, the turning point of Vygotsky's career from literary critic to psychologist was his

1924 dissertation Psychology of Art. Through the analysis of Hamlet Vygotsky attempted to

demonstrate that art is a product of the capacities of the specifically human mind -- imagination,

emotion, and symbolic representation.

Crucial for Vygotsky's theory was the study of the formal organization of works of art:

...every work of art is considered to be a system of stimuli, which are organized consciously and
intentionally in order to cause an aesthetic reaction. This way we reconstruct the structure of the
reaction while analyzing the structure of the stimuli...The general direction of this method can be
summarized in the following formula: from the form of the art work through the functional
analysis of its elements and structure to the reconstruction of the aesthetic reaction and the
discovery of its general laws.

In Vygotsky's formula the given was the existing works of art and the unknown was the laws of

aesthetic reaction. As we have already seen, for the artists, designers, and film directors of the

time the formula was reversed. If for the psychologist Vygotsky, visual works represented a

reservoir of knowledge about the human mind, artists, on the contrary, were hoping to utilize the

objective psychological knowledge in order to create visual works which would produce pre-

determined responses in the viewer.

Contemporary psychologists -- the supporters of Pavlov and Bekhterev on the one hand,

and younger psychologists such as Vygotsky and A. Luria on the other hand, argued over

whether the mind could be reduced to the laws of physiology. Similarly, the artists in theory and

in practice drew on these two alternative psychological models. Sergei Eisenstein's attempts to

ground his filmmaking methods in different psychological theories epitomizes the two models.

In one of his latest written works Eisenstein summed up his work in film:

Lev Vygotsky, Psikhologija Iskusstva (Psychology of art) (Moscow: 1968), 39-41.

I never intended to "reflect" the existing reality. I had one task -- using the means of its influence
-- to affect the feelings and thoughts, to influence the psyche and to shape the consciousness of
the viewer in the desired, required, and chosen direction.

Although Eisenstein's goal remained the same, the means of achieving the desired effect were

conceived of differently throughout his life.

On the one hand, Eisenstein developed the concept of "intellectual montage," privileging

a purely intellectual response. The goal of cinema, in this view, was thought to induce

dialectical reasoning. The structure of the film itself was conceived of as dialectical thinking in

visual form with montage being the means of representing the dialectical process through the

contrast and juxtaposition of images. This preoccupation of Eisenstein with cinema as the

visualization of the work of the intellect culminated in his project to create a screen adaptation of

Marx's Capital.

On the other hand, Eisenstein's second important concept developed in the Montage of

Attractions takes the viewers' physiological reaction as a point of departure.

Eisenstein based

his concept on the psychological theories of Ludwig Klages and William James.

According to

Klages, in a human being emotional states are expressed through bodily movements. Klages also

insisted that human expressivity is characterized by a unique quality -- the muscular contractions

of one person are involuntarily repeated by the observer. James' theory was related to Klages' but

causally reversed. He postulated that emotions were the effect of muscular contractions -- one

does not cry because he is sad, but one becomes sad due to crying. Eisenstein combined the two

theories: the emotional state of the actor translates into his muscular movements; these

Qtd. in Yu. A. Vasilieva, "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the

concept of 'aggressive art'), Kinovedcheskije Zapiski 3 (1989): 207.

I will return to Eisenstein's "intellectual montage" in chapter 2.

Sergei Eisenstein, "Montazh atraktsionov" (Montage of attractions), LEF 3 (1923): 70-75.

Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu

psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin':
between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 178-80.

movements are involuntarily repeated by the viewer causing him to experience similar emotions.

The important issue, then, was the training of actors in simulating precise gestures and facial

expressions in order to produce a desired emotional response in the viewer. Eisenstein proposes

to develop a collection of emotional stimuli which would be strung together in a film --

"montage of attractions." The film becomes a script of the emotional responses of viewers.

The two concepts -- the intellectual montage and montage of attractions -- are the rare

summits in Eisenstein's intellectual career where the opposing tendencies of his thought

concerning the modes of cinematic influence on the viewer can be clearly distinguished. Most of

the time they seem to have been intertwined, appearing simultaneously in his changing

conception of cinematic effectivity. Similarly, in the cultural imagination of the 1920s, the two

tendencies appear to be running side by side. Is there a mind, a specifically human autonomous

cognitive apparatus, with operational laws not reducible to simple reflexes? Or can human

behavior be controlled by affecting the organism through a set of predictable physiological

stimuli?

This second position received its extreme elaboration in the theory developed by

Emmanuel Enchmen, an early Socialist revolutionary.

Enchmen's "theory of new biology" was

based on Pavlovian reflexology and proposed a purely proletarian culture based directly on

conditioned reflexes, without the necessity of conventional language or thought. The new

proletarian language would consist of conditioned grunts and gestures and replace the verbal

culture of bourgeois society.

All in all, the co-existence of the two schools in Soviet psychology lead to the co-

existence of the two kinds of models of artistic affectivity: affecting consciousness by

communicating ideas and bypassing consciousness altogether. This dichotomy is reflected, for

instance, in the following statement from "New Artistic Program of Proletcult" published in

Joravsky, Russian Psychology, 212.

1923: "Art should become the necessary part of everyday life both in its active-representational

forms (poster, advertisement, agitational theater, cinema) and in its material-organizational forms

(psychophysical culture, organization of mass happenings, processions and demonstrations,

construction of things)."

The opposition "representational" -- "organizing" reflects two views

of affectivity: affecting the mind and affecting the body.

These different models of affectivity were also used to understand the roles of visual

elements which were being "isolated" and "purified" in the laboratories of the 1920s. The

elements could have affected the viewer in more than one way. No longer just catalysts of basic

emotional states, as in the tradition of experimental psychology and scientific aesthetics, they

were now re-interpreted as capable of carrying complex messages addressed to the mind, and as

simple signals designed to trigger physiological and behavioral responses.

I have discussed different models of affectivity which underlined the research into the elements

of a visual code. But what kind of code can be formed from such elements?

Semiotician Luis Prieto distinguished between codes with and without articulation.

code with articulation consists of a number of elements combined to produce messages. The

examples are human language, playing cards, and maritime flag codes. The crucial feature of any

of these codes is that they allow for the generation of a large or even infinite number of messages

from a small set of elements. For instance, in the case of human language, its speaker can

generate an infinite number of sentences. In contrast, in codes without articulation there are as

many meanings as elements. For example, in the language of flowers, each flower signifies a

Matsa, Sovetskoe iskusstvo za 15 let, 261.

Luis Prieto, Messages et signaux (Paris, PUF, 1966).

5. Visual Esperanto

distinct meaning (for instance,"red rose" = "victory is yours"); a combination of flowers does not

create any new meanings.

In view of this distinction, the research into visual elements could justify both a code with

articulation as well as a code without articulation. In other words, it is possible to understand the

elements in two ways: as carrying a complete message (a verbal meaning, an emotion or a

particular behavioral response), or as just a part of the message, like a word in a sentence,

combined with other words through grammar.

The artistic tradition of visual atomism, from Blanc to Kandinsky, appears to favor the

first interpretation. In their writings, Blanc, Henry, and Seurat correlated the orientation of

straight lines with different emotions such as gaiety or sadness. Kandinsky's famous

questionnaire, compiled in 1920 while he was active in INKhUK, also indicates that he

considered the elements to directly correspond to emotions or verbal meanings: "How do you see

a triangle -- do you feel that it is moving, where, do you see it as more witty than square; is the

sensation from a square similar to the sensation from a lemon; what is a canary's song more like -

- a triangle or a square."

A code without an articulation requires as many signs as there are potential messages and

therefore is not efficient. Yet, it is this inefficiency which makes it ultimately suitable for new

communicative situations faced by artists and designers in the earlier decades of this century.

Traffic signs, the labels on instrument panels, trademarks had to be unambiguous and instantly

recognizable by an untrained eye. Not surprisingly, the designers in every field have adopted

simple abstract forms. Thus, in 1909, when Mondrian and Kandinsky were moving toward their

first abstractions, the first International Conference on the Regulation of Automobile Traffic took

Qtd. in Khan-Magomedov, "INKhUK," 346.

place in Paris. Following its recommendations, in the 1910s-20s, universal pictograms signifying

main road situations were developed and employed.

For the Soviet "psycho-engineers," the notion of a visual code without articulation, which

correlates each message with a separate sign, had a particular attraction. The behaviorist

psychologies of Pavlov and Bekhterev popularized the notion of directly controlling behavior

through conditioned reflexes. Simple visual forms could have acted as such stimuli, controlling

"the proletarian zombie" as the lights and whistles already controlled a dog in Pavlov's

laboratory. In this interpretation, the dictionary of visual elements would contain not a set of

meanings or emotions but a set of commands -- previously learned responses to stimuli. This is

what the author of the "Engineerism" paper seems to have in mind when he writes of the future

where "visual sensations as such will concertedly shape the human being as an organized unit."

In other words, just as the motorists in the 1920s were beginning to memorize the international

traffic signs, learning how to respond to them automatically (today, this is the closest most of us

come to experience conditioning through a visual code), artists took this as a paradigm for

controlling the masses through visual communication.

Yet, while suitable for communication of basic emotional states or a set of behavioral

stimuli, the code without articulation is not practical for communicating an arbitrary large

number of messages. If visual language is to compete with a verbal language, and even to replace

it, visual forms have to be interpreted as equivalents of letters or words. They also have to be

supplemented by grammar, the rules of how these visual "words" are combined to form

sentences.

Indeed, contemporary debates on whether such animals as dolphins or primates can use

human language center around the attempts to teach them to use visual signs in this way. It is

A.S. Sardarov, "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic

sign), Tekhnicheskaya Estetika 9 (1984): 14-19.

easy to train animals to respond to visual forms as distinct messages: a square means "take food,"

a circle means "jump," and so on. The question is whether animals can string such symbols

together to form new messages, thus treating the individual symbols as words rather than as

complete messages. At least a few researchers have claimed that they have taught some animals

to do so, thus teaching them a visual code with articulation.

What about humans? The idea of replacing particular verbal language by a universal

visual language -- more efficient, capable of reaching new masses of immigrants (U.S.) or

"proletarians of the whole world" (U.S.S.R.) was often expressed in the 1920s. Film, in

particular, was seen as the prime candidate for a visual Esperanto. In a 1921 interview D.W.

Griffith said: "A picture is the universal symbol, and a picture that moves is a universal

language. Moving pictures, someone suggests, 'might have saved the situation when the Tower

of Babel was built.'"

Miriam Hansen writes about the metaphor of film as visual Esperanto in

the U.S.:

Griffith participated in the widespread celebration of film as a new "universal language" which
accompanied its formation as an institution. Used by journalists, intellectuals, social workers,
clergy, producers, and industrial apologists alike, the metaphor of film as a universal language
drew on a variety of discourses (Enlightenment, nineteenth-century positivism, Protestant
millennialism, the Esperanto movement, and the growing advertising industry) and oscillated
accordingly between utopian and totalitarian implications.

However, it was across the Atlantic, in the U.S.S.R., where the comparison between film and

verbal language was explored most systematically by Victor Shklovsky and other literary

theorists associated with the so-called formalist school as well as by film directors and theorists

such as Eisenstein, Vsevolod Pudovkin, and Lev Kuleshov. In their writings we encounter

Qtd. in Miriam Hansen, "The Hieroglyph and the Whore: D.W. Griffith's Intolerance," The

South Atlantic Quarterly 88, no. 2 (1989): 363.

Ibid., 362.

frequent comparisons of a shot and a word, or a sequence and a sentence as well as references to

a "grammar of film."

Can we find similar kinds of analogies proposed for the elements of a single image? Just

as the artists were comfortable with different mutually exclusive models of affectivity, they were

equally at ease with two contradictory models of a code. If the visual elements were imagined to

be capable of carrying complete messages, they were also thought of as the elements of a

message. Thus, art historian S.O. Khan-Magomedov writes about the INKhUK program: "The

point was to discover certain primary elements of artistic expressiveness, in themselves without

signification and understood as an alphabet of artistic-compositional system."

What we don't find in the 1920s, however, is equally serious thinking about the grammar

of a still image. It was only in the 1960s, when the analogy between a verbal and a visual

language was taken with new rigor in structural semiotics, that coherent theorizing about the

grammar of visual elements began to appear.

The understanding of vision in modernity is characterized by attempts to think of vision as a

language. This is reflected in the popularity of the term "visual language," this term which is so

familiar to us today that we tend not to ask about its historical specificity. What this term points

Paradoxically, the classical Hollywood film style, which matured by the 1920s, did become

the universal visual language of the twentieth century, while the films of Soviet avant-garde
directors, who were most systematic in theorizing film as a language, were rejected by mass
audiences and continue to play art houses, as they did in the 1920s.

Khan-Magomedov, "INKhUK," 343.

The best critical review of visual semiotics available in English is Gšran Sonesson, Pictorial

Concepts. Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual
World (Lund, Sweden: Lund University Press, 1989).

6. Conclusion

to, in my view, is the new social role of vision as the medium of mass communication which it

acquired in the earlier decades of this century.

The attempts to rationalize vision in this role, to conceive of it as a code which can

function independently of verbal language and which can be effective in the new communicative

situations created by modernization have centered around the quest for the elements, the "atoms"

of visual communication. This quest reaches its culmination in the work of a number of art

institutes in Soviet Russia in the 1920s, where artists collaborate with experimental psychologists

to isolate these elements and study their effectiveness.

The search for the essential elements of each art which preoccupied early twentieth

century culture is usually seen in the context of artistic modernism. This search is interpreted in

the context of the rhetoric of the purity of medium. Indeed, especially in the 1910s-20s, artists

tried to reduce every medium to its unique qualities. To do so, they gave up representation and

concentrated on the material elements thought to be unique to each medium. Poets, such as

Russian futurists, were experimenting with sounds; filmmakers proposed that the essence of

cinema was movement and temporal rhythm (French film theory of the 1920s)

or montage

(Kuleshov's group in Russia); and painters were exploring pure colors and geometric forms. The

following statement made in 1924 by Jean Epstein, a French avant-garde filmmaker and

theoretician, is typical of modernist rhetoric of purity; countless statements like it appeared on

the pages of avant-garde publications of the time:

For every art builds its forbidden city, its own exclusive domain, autonomous, specific and hostile
to anything that does not belong. Astonishing to relate, literature must first and foremost be
literary; the theater, theatrical; painting, pictorial; and the cinema, cinematic. Painting today is
freeing itself from many of its representational and narrative concerns...And any literature worthy
of the name turns its back on those twists and turns of plot which lead to the detective's discovery

See especially Jean Epstein, "On Certain Characteristics of PhotogŽnie," in French Film

Theory and Criticism, ed. Richard Abel (Princeton: University of Princeton Press, 1988), 1: 314-
318; Germaine Dulac, "Aesthetics, Obstacles, Integral CinŽgraphie," in French Film Theory and
Criticism, 1: 389-397.

of the lost treasure...The cinema must seek to become, gradually and in the end uniquely,
cinematic; to employ, in other words, only photogenic elements.

Yet, as this chapter has suggested, more was at stake in the quest for the visual "table of

elements" than just the demand for purity of medium. While many artists searched for "ever

more rich aesthetic elements" (Henry), trying to make their art into a "forbidden city" (Epstein),

others welcomed the opportunity to take their art to the streets. For these artists, the science of

visual elements has provided hope for the creation of a fully rationalized visual language of mass

communication.

The dream to rationalize communication was not unique to this project -- the proposals

for a universal language go back to the seventeenth century. Descartes suggested that the lexicon

of a universal language should be composed of primitive elements. By systematically combining

these elements, one can generate "an infinity of different words."

In the early eighteenth

century Leibniz outlined a language in which grammatical and logical structure would coincide,

thus making possible the automation of thinking. The basic elements of his ideal language were

characters representing unambiguously a limited number of elementary concepts. Leibniz called

the inventory of these concepts "the alphabet of human thought." Through the algebra of thought

primitive concepts would be combined to form any complex idea.

The proposal of a universal

language composed from graphic symbols and therefore suitable only for writing followed soon

with the first pure pasigraphy ("writing for all") invented by Joseph de Maimieux in 1797.

However, until the twentieth century these remained isolated philosophical ideas advanced by

single individuals. With the rise of mass communications, the fantasy of a universal rational

language attains new significance, extending into the realm of vision. Thus, what we now

Ibid., 314-15.

Qtd. in Winfried Nšth, Handbook of Semiotics (Bloomington and Indianapolis: Indiana

University Press, 1990), 272.

Ibid., 274.

Ibid., 269.

witness are systematic and extensive attempts to establish fully rationalized visual languages of

mass communication. At the same time, the studies of behavior and conditioning in physiology,

behaviorism and reflexology, lead to new interpretations of the functioning of the elements of a

visual Esperanto. Not only would they be capable of communicating ideas thus affecting the

intellect, but they would also directly control behavior bypassing the mind altogether, thus

making possible an international "psychophysical culture."

In the sixth Meditation Descartes defined man "as a thing that thinks," to whom reasoning comes

naturally. Imagination, on the other hand, requires a special effort and is in no way necessary for

a human being. Distinguishing between imagination and pure intellection, Descartes

demonstrated the inferiority of vision to reasoning:

For example, when I imagine a triangle I not only conceive (intelligo) that it is a figure
comprehended by three lines, but at the same time also I look upon (intueor) these three lines as
present by the power and internal application of my mind (acie mentis), and this is what I call
imagining. But if I desire to think of a chiliagon, I indeed rightly conceive that it is a figure
composed of a thousand sides, as easily as I conceive that a triangle is a figure composed of only
three sides; but I cannot imagine the thousand sides of a chiliagon as I do the three sides of a
triangle, nor, so to speak, view them as present [with the eyes of my mind].

Indeed, how can humans possibly reason through visual representations, if the latter are not

capable of representing general concepts to begin with? In Treatise Concerning the Principles of

Human Knowledge, George Berkeley insisted, for instance, that it is impossible to have a mental

image of an idea, such as "man," as a generality; it is only possible to visualize a tall or a short

man, but not man as such. Thinking deals with generalities and does not tolerate particular

things. If a philosopher tries to reason about the nature of "man," any image of a particular man

would mislead the reasoning process.

The philosophical arguments against visual reasoning still continue in this century, but

they are out of step with the modern productive use of vision: not only the concepts of "visual

RenŽ Descartes, "Meditations on the First Philosophy," in The Rationalists, trans. John Veitch

(Garden City, NY: Doubleday & Company, 1960), 160-161.

Rudolf Arnheim, "A Plea for Visual Thinking," in New Essays on the Psychology of Art (Los

Angeles and London: University of California Press, 1986), 136.

Chapter 2. I See, Therefore I Think

1. Introduction

thinking," "thinking through images," etc. have become respectable and commonplace,

but

they are also put into practice daily through new techniques and technologies of visual

representation. While Descartes thought that it was impossible (and, at any rate, unwise) to

imagine the thousands of sides of a chiliagon, today scientists stare at computer monitors, where

computer imaging brings before their eyes not only chiliagons but geometric objects of n-

dimensions, objects composed not from thousands but from millions of sides, objects

representing weather systems, diffraction patterns, atomic surfaces, and other processes

inconceivable to the bare imagination (fig. 5).

"Scientific visualization" is the name of the new

field which claims to transform the nature of scientific reasoning in biology, chemistry, and even

mathematics.

Likewise, Berkeley's insistence that images cannot convey "man" as a generality has been

superseded. Since the 1960s, numerous space stations sent to the Moon, Venus, Mars or simply

into the infinity of the Universe have carried images destined to communicate to intelligent

beings (should they happen to find the stations) the basics of human civilization and the "essence

of man" in a "universal language of images." Long after the sun explodes and man disappears,

space stations will continue to fly on their trajectories, carrying such images as the famous

drawing by Leonardo da Vinci of an "ideal man." Intelligent (or perhaps, not so intelligent)

beings will eventually discover these images, learning from them if not about human civilization

in general, then at least about the artistic tastes of administrators of space programs, assuring

Leonardo of inter-galactic fame.

The images of general or abstract concepts can be encountered not only on space

missions but on Earth, everyday and everywhere, in numerous logotypes and designs (for

As witnessed, for instance, by such titles as Thinking Eye (Paul Klee) or Visual Thinking

(Rudolf Arnheim).

Examples of scientific visualizations presented at the Showcase at SIGGRAPH 1992.

SIGGRAPH '92 Final Program (New York: The Association for Computing Machinery, 1992),
33-40.

instance, in symbols which distinguish men's from women's lavatories), or in the representation

of unimaginable data sets. However, these examples do not tell us anything about the specificity

of the new function of vision in modernity. After all, for centuries pictures have been employed

as visualizations of the holy stories. Michael Baxandall cites a late thirteenth-century text which

summarizes this purpose of images:

Know that there were three reasons for the institution of images in churches. First, for the
instruction of simple people, because they are instructed by them as if by books. Second, so that
the mystery of the incarnation and the examples of the Saints may be more active in our memory
through being presented daily to our eyes. Third, to excite feelings of devotion, these being
aroused more effectively by things seen than by things heard.

Therefore, what is different about the modern function of vision is not simply the use of images

to represent abstract concepts concretely. What is different is the idea of reasoning through

images, the idea unthinkable for Descartes or Berkeley. Thus, the field of scientific visualization

is based on the assumption that visual observation of computer generated images of data sets and

processes leads to scientific breakthroughs otherwise impossible. Richard Mark Friedhoff and

William Benson propose that "using a computer simulation...even a beginning student of

chemistry might be able to deduce benzene's structure" (a reference to Friedrich KebulŽ's 1865

discovery of the molecular stucture of benzene supposedly made after he saw a dream involving

a snake biting its own tale).

Similarly, Jaron Lanier, one of the most visible

researchers/promoters of virtual reality, maintains that virtual reality will lead to the age of "post-

symbolic communication," the communication of ideas and arguments solely through images.

Finally, a prominent cognitive psychologist Philip Johnson-Laird claims that everyday logical

Michael Baxandall, Painting and Experience in Fifteenth Century Italy (Oxford: Oxford

University Press, 1972), 41.

Richard Mark Friedhoff and William Benson, The Second Computer Revolution:

Visualization (W.H. Freeman and Company, 1991), 13.

See, for instance, Timothy Druckrey, "Revenge of the Nerds. An Interview with Jaron

Lanier," Afterimage (May 1991): 5-9.

reasoning takes place through images: when we are engaged in reasoning, we construct a mental

model of a situation which represents its essential features in the form of topological relations.

Before it became possible to send Leonardo's images to remote corners of the Universe to

represent "humanity," before scientists accepted that computer-aided visualization would

dramatically speed up scientific reasoning, before psychologists could claim that reasoning in

humans is a matter of manipulating mental images, a profound change in the cultural attitude

toward vision had to take place. This change occurred between the 1870s and the 1920s.

Logical thinking entails the representation of abstract ideas and, also, of the logical

relations between them. During this period, Galton, Venn, Freud, Eisenstein, and others put

vision to use for both purposes. More importantly, for the first time, we can find in their work the

explicit justifications for the very notion of reasoning through vision.

In 1877 Galton's composite portraits visualized universal human types; in 1880, John

Venn made public a method for solving problems in logic by using graphic diagrams (section 3).

In The Interpretation of Dreams, published in 1900, Sigmund Freud developed a systematic

theory of how abstract notions and logical forms can be visualized (section 5). In the late 1920s

Eisenstein proposed that film can be used "to encourage and direct the whole thought process"

(section 4).

These different models of how vision can be used in reasoning represent the first stage in

the reversal of attitude towards the inadequacy of vision. The second stage arrived when, in post-

industrial society, the concern with the efficiency of the mind overwhelmed the concern with the

efficiency of the body. Now, the questions of how information can be coded, stored, retrieved,

and processed more efficiently give direction to the study of the mind, including the use of visual

Philip Johnson-Laird, Mental Models: Towards a Cognitive Science of Language, Inference,

and Consciousness (Cambridge: Cambridge University Press, 1983).

representations in mental processes. Cognitive scientists, among others, begin to approach vision

from this new perspective, and they postulate the centrality of visual representations for human

reasoning, because of their efficiency (sections 6 and 7).

To be able to appreciate the novelty of the modern paradigm of visual reasoning, I will begin by

discussing the classic philosophical positions on the relationship between vision and reason.

Many philosophers thought vision to be unsuitable for reasoning, first of all because throughout

the history of philosophy, reason was closely associated with logic. If, following its

formalization by George Boole and Gottlob Frege in the nineteenth century, logic has developed

into a sophisticated branch of mathematics, today seemingly removed from our everyday

discourse, originally it was conceived by Aristotle as a systematization of principles according to

which rational debates should be conducted.

Therefore, it was Aristotle who originated the

tradition of discussing human reasoning and logic interchangeably. Locke, for instance defines

reason as "that faculty whereby man is supposed to be distinguished from beasts, and wherein it

is evident he much surpasses them." Claiming that the highest order of reason consists in "the

discovering and finding out of proofs," Locke writes:

Sense and intuition reach but a very little way. The greatest part of our knowledge depends upon
deductions and intermediate ideas: and in those cases where we are fain to substitute assent instead
of knowledge, and take propositions for true without being certain they are so, we have need to
find out, examine, and compare the grounds of their probability.

Antony Flew, ed., A Dictionary of Philosophy (London: Pan Books Ltd., 1984), 208-212.

John Locke, An Essay Concerning Human Understanding, ed. A.S. Pringle-Pattison (Oxford:

Clarendon Press, 1924), Book IV, Ch. 17.

2. I See, Therefore I Think

The equation of reason with logic persisted late into the twentieth century. Piaget

assumed that the final stage in the development of human thought processes was a kind of formal

logic analysis. Equally, Levi-Strauss's binary oppositions and Greimas's semiotic square are

based on stripped down formal logic. This logic is postulated as the structure of the "native

mind" (Levi-Strauss) or the structure of all signification (Greimas).

Only in the last decades

have cognitive psychologists discovered that human judgments do not follow statistical

probabilities and tend to ignore information about the prior probability of an event;

that rather

than having some uniform "laws of thought," humans employ a variety of heterogeneous

heuristics; and that in general the normative rules of logic do not describe human thinking.

However, the idea of a separation between logic as a normative or prescriptive discipline and

human reasoning, which can be described and studied empirically and which does not follow any

strict rules of logic, is quite recent, and would be unimaginable for Aristotle or Locke.

Therefore, in classical philosophy to reason is to carry out logical operations. How then

can one possibly reason through images? How can one represent visually "all," "some," "any" or

What is different about reason as imagined by Levi-Strauss and Greimas in contrast to Locke

is its new industrial efficiency: the concept of binary oppositions was brought by Jakobson from
MIT, the birthplace of information theory which demonstrated that binary code is the most
economical of all. According to Geoffrey Sampson, "what led Jakobson to the hypothesis that all
parameters are 'binary' seems to have been the mathematical notion that a transmission-code is
more efficient when it uses only independent binary choices." Geoffrey Sampson, Schools of
Linguistics (Stanford, CA: Stanford University Press, 1980), 250.

Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic

Press, 1983), 190.

Michael I. Posner, and Gordon L. Shulman, "Cognitive Science," in The First Century of

Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 393.

A recent dictionary of philosophy still equates thinking with logic: "The mental activity of (a)

theoretical contemplation directed toward some object with a view to reaching a propositional
conclusion; or (b) practical deliberation directed toward some object with a view of reaching a
decision to act." Flew, A Dictionary of Philosophy, 352. Emphasis mine - L.M. An encyclopedia
of philosophy is equally categorical: "particular thoughts have some kind of logical form; they
may be categorical, hypothetical, disjunctive, universal, particular, and the like." Paul Edwards,
ed., Encyclopedia of Philosophy (New York: Macmillan, 1967), 8: 101.

other logical constants? How to represent truth and falsity? How to distinguish particular from

general?

Reason, of course, cannot proceed without the raw materials, collected for it by vision.

However, the role of vision is limited to the passive and unintelligent collection of data

(sensations), later to be interpreted by reason. Experimental psychology, emerging from

empiricist philosophy in the nineteenth century, gave additional support to this conceptual

hierarchy by providing a topological scheme: the outside world enters through the filters of the

eyes and then is carried through visual pathways inside the head, to the brain, the site of "higher

mental faculties." Even today, psychological textbooks are organized around the same hierarchy,

inherited from empiricist philosophy: they start with chapters on sensation (vision and other

senses) and end with cognition (thinking, memory, text comprehension, etc.)

These are some of the ways in which philosophy has demoted vision in relation to reason: it can't

represent logical forms; it can't generalize; its only job is to collect the raw material reason

needs.

However, this philosophical tradition is counterbalanced and upset by other traditions.

For, while subordinating vision to reason, philosophers also often claimed that reason in fact was

born of vision. Moreover, it can be said that they could only conceive of reason in visual terms.

For instance, while in terms of the temporality of everyday mental functioning, vision

serves reason, its task being simply the collection of data for reason to work with, in terms of a

much longer temporality of phylogenetic development of the human race, vision is often

postulated as reason's origin. In particular, Locke, Herder, Vico, Nietzsche, and others claimed

that abstract categories of language evolved from concrete visual (and also bodily) experience.

Locke: "I doubt not but, if we could trace them [words] to their sources, we should find, in all

In his Downcast Eyes, unavailable to me at the time of this writing, Martin Jay discusses the

role of vision in modern French philosophy. Martin Jay, Downcast Eyes: the Denigration of
Vision in Twentieth-century French Thought (Berkeley: The University of California Press,
1993).

languages, the names which stand for things that fall under our senses to have had their first rise

from sensible ideas."

Nietzsche: "Everything which makes man stand out in relief against the

animal depends on this faculty of volatilizing the concrete metaphors into a schema and therefore

resolving perception into an idea."

Vico: "It is noteworthy that in all languages the greater part

of the expressions relating to inanimate things are formed by metaphor from the human body and

its parts and from the human senses and passions."

Vico's ideas, in particular, have been recently revived in the work of the linguist George

Lakoff. Lakoff argues that most, if not all, abstract semantic concepts and abstract human

reasoning are based on metaphorical mapping of spatial concepts (such as inside-outside, part-

whole, and others) which are inherently meaningful to human beings since they have bodies.

Like Vico before him, Lakoff challenges three traditional hierarchies -- between concrete

and abstract thought, between vision and reasoning, and between literal and metaphorical

meanings. In contrast to the objectivist-computational paradigm, which underlies much of

Anglo-American philosophy of language and cognitive science, in which thinking is

conceptualized as the algorithmic manipulation of symbols meaningless in themselves, Lakoff

claims that in thinking we project the structures of our sensory and bodily experience to abstract

Qtd. in Tzvetan Todorov, Theories of the Symbol (Ithaca: Cornell University Press, 1984),

237.

Friedrich Nietzsche, "On Truth and Falsity in their Ultramoral Sense," in Early Greek

Philosophy and Other Essays (New York: The Macmillan Company, 1924), 181.

Thomas Goddard Bergin and Max Harold Fisch, The New Science of Giambattista Vico

(Ithaca: Cornell University Press, 1968), 129. Discussing the metaphorical origin of abstract
concepts, Vico goes on to claim that the common assumption "that prose speech is proper
speech, and poetic speech improper" reverses the true history of human mind: "first nations"
have spoken and thought in poetry, i.e. through the metaphorical tropes (131).

situations.

The semantics of natural language is built upon the meaningful "image-schemes"

which are the generalizations of what Lakoff calls our "basic human experience." We live inside

our bodies; our bodies consist of parts which are linked. Therefore, we conceptualize all

situations in terms of containers (for instance, an enormous number of expressions use in and

out), part-whole relations (divorce is splitting up) and links (we make connections, break

relations). For Lakoff, metaphors such as these are not the exception but the norm in language,

George Lakoff, "Cognitive Linguistics,"

Versus 44/45 (1986): 119-154.

which involves the extension of the bodily and sensory experience to the domain of abstract

concepts. Because spatial (more precisely, topological) relations are an important part of our

experience, spatial metaphors are at the core of the semantics of human languages; reaching

goals, moving out, walking over an enemy, etc. Not only does the semantics of natural languages

turn out to be based on spatial metaphors but abstract reason itself is the manipulation of

spatially represented information: "natural language reasoning makes use of at least some

unconscious and automatic image-based processes such as superimposing images, scanning

them, focusing on part of them, etc."

In fact, reason is a special case of more general mental

operations which are needed for perception: "Abstract reasoning is a special case of image-based

reasoning. Image-based reasoning is fundamental and abstract reasoning is image-based

reasoning under a metaphorical projection to an abstract domain."

With this claim, Lakoff transcends the earlier views of Locke, Vico, and Nietzsche

according to whom abstract reason phylogenetically evolved from vision. It is one thing to

suggest that abstract concepts have evolved from "sensible ideas" (Locke) or perceptions

(Nietzsche). It is quite different to start with the assumption, as Lakoff does, that abstract ideas

are nothing but a limited set of visual (more precisely, topological) forms (such as inside-outside,

part-whole) extended into the abstract domain.

Here the modern obsession with the visualization of reasoning leads to the most dramatic

break with the traditional conceptualization of the relation between reason and vision. Modern

writers such as Lakoff or Johnson-Laird, claim is that there is only one kind of cognitive

operations, which reason and vision share. These operations are fundamental, original to vision:

reason borrows these operations; it does not invent any of its own.

Ibid., 149.

George Lakoff, "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"

Cognitive Linguistics 1, no. 1 (1990), 65. Perhaps even this very brief presentation of Lakoff's
ideas already makes apparent a substantial problem of his theory -- the rather unquestioned
notion of "basic human experience."

Johnson-Laird has proposed that when we are engaged in reasoning, we construct a

mental model of a situation which represents its essential features in the form of topological

relations.

The conclusion then simply can be "read off" from the representation. For instance,

how can we see the truth of a modus ponens: "If X is an A, and if all A's are B's, then X is a B"?

We construct a mental model: X is inside A; A is inside B. If the model is mentally "scanned,"

the right conclusion is invariably arrived at. According to Johnson-Laird, we don't have to

Philip Johnson-Laird, Mental Models:

Towards a Cognitive Science of Language, Inference, and Consciousness (Cambridge:

Cambridge University Press, 1983).

assume the existence of any special rational logical competence unique to humans. Logical

reasoning is just one consequence of vision.

But do these views of Lakoff and Johnson-Laird really represent a break? Perhaps this

revolt against the condemnation of vision in the philosophical tradition was prepared by this

tradition itself? For, while declaring vision unsuitable for reason, philosophers conceived of

reason in visual terms all along: witness the persistence of the metaphors of eye, light, mirror,

revelation and unveiling, from Plato to Descartes, Hegel, Heideger, and Lacan as standing in for

logos, knowledge, and truth.

"Idea" comes from the Greek verb "to see" and is frequently linked to ancient optics and

theories of perception. According to Aristotle, in thinking ideas function to substitute for the

absent objects as their images: "The reasoning mind thinks its ideas in the form of images; and as

the mind determines the objects it should pursue or avoid in terms of these images, even in the

For gender implications of this tradition,

see Evelyn Fox Keller and Christine R. Grontkowski, "The Mind's Eye," in Discovering Reality:
Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy of
Science, ed. Sandra Harding and Merrill B. Hintikka (London: D. Reidel Publishing Company):
207-224. For a discussion of Lacan's concept of the mirror stage in relation to the Western
philosophical tradition with its concepts of knowledge and subjectivity grounded in vision see
"The Statue Man," in M. Borch-Jakobson, The Absolute Master (Stanford: Stanford University
Press, 1991).

absence of sensation, so it is simulated to action when occupied with them."

For Berkeley and

Hume thinking involved a sequential series of images; these images are tied to certain habits of

the mind to move from one image to another. Reason was equated with an eye which compared

all objects of thought laid out before it. As summarized by Jonathan Crary,

But whether it is Berkeley's divine signs of God arrayed on a diaphanous plane, Locke's sensations
'imprinted' on a white page, or Leibniz's elastic screen, the eighteenth century observer confronts a
unified space of order...on which contents of the world can be studied and compared, known in
terms of a multitude of relationships. In Rorty's words, "It is as if the tabula rasa were perpetually
under the gaze of the unblinking Eye of the Mind..."

Qtd. in Jean Matter Mandler and George Mandler, eds., Thinking: From Association to

Gestalt (New York: John Wiley & Sons, Inc., 1964), 9.

Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth

Century (Cambridge: The MIT Press, 1990), 55.

It was Heidegger who, in What is Called Thinking?, insisted on the visual basis of

Western philosophy (both epistemology and ontology) from Plato to Hegel most strongly.

"The Greeks...conceived knowledge as a kind of seeing and viewing...Because Being means

presence and permanence, 'seeing' is especially apt to serve as an explanation for the grasping of

Martin Heidegger, What is Called

Thinking? (New York: Harper & Row, Publishers,

1968).

what is present and what is permanent."

For Plato, Being is idea, present as outward

appearance. However, already in Plato the seeds of the split between object and subject are

implanted: Being is presence but at the same time is what man brings before his eyes. From this

moment on, idea is gradually transformed into perception and representation, that, according to

Heidegger, characterizes the Cartesian cogito. Descartes laid a new foundation in privileging the

thinking subject, but the certitude of the cogito derives from its visibility: as Heidegger explains,

cogitatio means Vor-stellung, i.e. "the bringing-before-itself and what-is-brought-before-itself

and made 'visible' in the widest sense."

Indeed, Cartesian rationality is based on the notion of a subject which is outside himself,

observing himself contemplating the subject. This is the price Descartes pays for his

condemnation of images and imagination in relation to reason: he can only conceive of reason in

visual terms. Before "I think therefore I am" comes "I see therefore I think."

Qtd. in Borch-Jakobson, The Absolute Master,

53.

Qtd. in Ibid., 54.

Like Siamese twins, vision and reason are inseparable in classical philosophy, being connected

in a complex network of dependencies, only a few of which have been discussed above. On the

one hand, vision is but a servant of reason, unable to represent logical relations and abstract

concepts and therefore, clearly unsuitable for reasoning. On the other hand, vision achieves its

revenge, consistently slipping in as the phylogenetic origin of reason and as the basis of

metaphors in which operations of reason are conceived.

Regardless of whether Heidegger's brilliant account of some of these dependencies

represents a break with this classical tradition or is still a part of it, he clearly lives in a different

age. Vision and reason are no longer thought of as antagonists; on the contrary, "visual

reasoning" is now taken for granted and even promoted as a new highly efficient means of

decision making and/or propaganda, suitable for the age of mass mobilization, mass education,

industrialized science, and bureaucracy. Heidegger is a contemporary of Eisenstein who, in line

with Stalin's plans to industrialize Russia overnight, dreams of turning workers into dialectical

thinkers by passing them through the darkness of special incubators of cognitive skills -- movie

houses, thus shortening the duration of the learning process to hours. Heidegger is also a

contemporary of Benjamin who wants to teach German workers dialectical thinking even

without any external aids (such as film) -- by simply juxtaposing images, in montage-like

fashion, in the mind. Finally, Heidegger's younger compatriot, Rudolf Arnheim, becomes the

most outspoken champion of thinking in images, arguing that artistic skills should become the

3. First Signs of Revolt: Venn and Galton

basis of all education, since all, even the most abstract reasoning, involves representation of

information in visual form.

100

The signs of the arrival of the new age of visual reasoning start to appear in the last

decades of the nineeteenth century. In 1880 English logician John Venn published an article "On

the Diagrammatic and Mechanical Representation of Propositions and Reasoning."

101

The

100

See

Arnheim, "A Plea for Visual

Thinking."

101

Martin Gardner, Logical Machines and Diagrams, 2nd ed. (Chicago: The University of

Chicago Press, 1982), 32.

article presented a method to use graphic diagrams to represent complex logical relations and it

soon became the standard for logicians. Venn's method not only made it possible to illustrate the

rules of inferential logic in visual forms; but much more importantly, now one could reason

automatically by simply drawing pictures (fig. 6). Consider, for instance, the following

inference: "If all A is contained in B, and if C is contained in A, then C must also be contained in

B." Its validity becomes obvious once the terms are represented in a diagrammatic form.

Likewise, such inferences as "All men are mortal. Socrates is a man. Therefore, Socrates is

mortal" can be arrived at through similar diagrams. Images thought to be unsuitable for

representing logical relations and thus unsuitable for reasoning turn out to be its perfect indeed

its natural medium since they can represent topological relations with sufficient precision and

complexity.

If Venn's diagrams showed that vision can be used to arrive at logical relations, Galton's

composite photographs proved that abstract concepts can be represented visually. Sir Francis

Galton, a statistician and a cousin of Charles Darwin, a founder of eugenics (a project of social

betterment through planned breeding), and the author of highly influential psychological texts,

pioneered in 1877 a procedure of making composite photographs which proliferated widely in

the next three decades.

102

Fabricated by a process of successive registration and exposure of

portraits onto a single plate, Galton's composites were thought to constitute true statistic

averages, representing human types -- a criminal, a prostitute, an Englishman, a Jew, and others

(fig. 7). Galton wrote about his composite pictures that they are "much more than averages; they

are rather the equivalents of those large statistical tables whose totals, divided by the number of

classes and entered on the bottom line, are the averages. They are real generalizations, because

they include the whole of the material under consideration."

103

102

Allan Sekula, "The Body and the Archive," October 39 (1987): 40.

103

Qtd. in Ibid., 47.

Galton not only claimed that "the ideal faces obtained by the method of composite

portraiture appear to have a great deal in common with...so-called abstract ideas" but in fact he

proposed to rename abstract ideas "cumulative ideas." In contrast to the human mind, "a most

imperfect apparatus for the elaboration of general ideas," Galton championed his composite

photographs, which, being mechanical and precise, were much more reliable for arriving at

abstract representations.

104

With his photographs, Galton not only proposed that universals may be represented

through generic images; he actually objectified and materialized them. Plato's ideas were given

concrete form: they could now be touched, copied, fabricated, multiplied, distributed, etc.

If Venn's and Galton's inventions overturned the traditional claims about the impossibility

of visual reasoning, they did that not through new philosophical arguments about vision but first

of all by inventing new ways to use it. The latter invented a new visual technology: representing

universals through composite photographs. The former introduced a new visual technique, a new

way to employ visual representations, better yet, a new graphic sign system -- diagrams which

signify not any concrete objects, but logical forms.

105

In effect, Venn's technique and Galton's technology, which gave logical forms and

abstract ideas material tangible form, externalized reasoning. What before was a mental process,

a uniquely individual state, now became part of a public sphere. Unobservable and interior

processes and representations were taken out of individual heads and put outside -- as drawings,

104

Qtd. in Ibid., 51.

105

I should clarify the distinction between the terms visual techniques and visual technologies

as they are used in this chapter. By "techniques" of visual representation, I mean particular
graphic and/or pictorial signifying systems which do not depend in any crucial way on any one
medium or technological base. For example, Euler's diagrams can be drawn on paper, etched on
metal, generated with computer graphics, etc. The term "visual technologies" refers to certain
representational devices, which do depend on the particular technological configuration and
would be hard or impossible to achieve without it. The examples include dissolve or traveling
shots in film, which are only possible given a sequence of images in time generated by a film
projector, video tape player or computer. Galton's composite photography is such visual
technology, dependent, both in its design and its artifacts, on particular photographic technology.

photographs and other visual forms. Now they could be discussed in public, employed in

teaching and propaganda, standardized, and mass-distributed. What was private became public.

What was unique became mass-produced. What was hidden in an individual's mind became

shared.

We should not be surprised, then, that the revolutionary new medium, the medium of mass

society par excellence -- cinema -- was immediately proclaimed to be the machine for the

externalization of private mental functions and states. In 1916 Hugo MŸnsterberg, a Professor of

Psychology at Harvard University and one of the founders of the fields of industrial and applied

psychology, published The Film: A Psychological Study, today canonized as one of the earliest

4. "To teach the worker to think dialectically."

theoretical treatments of cinema.

106

According to MŸnsterberg, the essence of the new medium

lies in its ability to reproduce, or "objectify" various mental functions on the screen: "The

photoplay obeys the laws of the mind rather than those of the outer world."

107

In contrast to the

theater, where the action is constrained by the limitations of physical reality, film is free to shape

arbitrarily its material, closely approximating flashes of memory, the flights of imagination, and

106

Hugo MŸnsterberg, The Photoplay: A

Psychological Study (New York: D. Aplleton & Co., 1916).

107

Ibid., 41.

other mental acts. For instance, while in theater events have to follow each other corresponding

to the progression of time, in film the action can suddenly jump back and forth, just as in an act

of imagination.

MŸnsterberg was not content to point out the analogy between film and mental life; in an

astounding analysis, he correlated the main cinematic techniques to different mental functions

such as attention and memory, one-to-one. For example, in the close-up, "everything which our

mind wants to disregard has been suddenly banished from our sight and has disappeared,"

analogous to how our attention selects a particular object from the environment. Similarly, the

"cut-back" technique objectifies the mental function of memory.

"In both cases," MŸnsterberg wrote, "the act which in the ordinary theater would go on

in our mind alone is here in the photography projected into the pictures themselves."

108

The

psychological laboratory became indistinguishable from the movie house; the textbook of

experimental psychology -- from the cinematographer's manual. The mind was projected on the

screen; the inside became the outside.

MŸnsterberg admired the power of film to externalize the functions of consciousness.

The next logical step was taken by a German psychologist Kurt Levin who, in 1924-25, was the

first to use film in his experiments. He wrote that "fiction film attempts to objectify certain

psychological processes for the viewer. Psychological (scientific) film studies to what extent

these psychological processes can be objectified."

109

Soviet psychologist A.P. Luria, who

planned to establish a psychological laboratory in Moscow in cooperation with the State Film

Academy, acquainted Levin with Eisenstein, who attended the shooting of one of Levin's films

and advised him.

110

108

Ibid., 41.

109

Qtd. in Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu

psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin':
between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 187.

110

Ibid., 177.

The figure of Eisenstein is particularly important because it reveals the historical

connection between the interest in visual reasoning and the rise of mass communication, of

which film was a major vehicle. The emergence of new mass societies in the earlier part of this

century dictated the necessity to communicate ideological concepts to mass populations which

were often illiterate.

111

In the 1920s Eisenstein boldly conceived a screen adaptation of Marx's

Capital as a way to efficiently bring about the political enlightenment of Russian audiences,

especially the peasants who would not sit through a political lecture but, attracted by the

"novelty" of a movie projector, would come to see movies, regardless of what was shown.

112

Unprecedented as his project was, its radicalism lay not only in the decision to visualize the

abstract notions and logic of Capital but in the method employed, which, according to Eisenstein,

would directly provoke dialectical thinking in audiences.

113

Jacques Aumont concludes that for

Eisenstein, "the object privileged in Marx's work is not a theoretical one, like any of the key

concepts from Capital. It is at another level entirely that Eisenstein selects his true object -- the

Marxist method itself."

114

Thus it was not simply a matter of the modern redeployment of the

directions of the 1492 sermon: "...Images of the Virgin and the Saints were introduced...on

account of the ignorance of simple people, so that those who are not able to read the scriptures

can yet learn by seeing the sacraments of our salvation and faith in pictures."

115

The viewers of

Capital were not only to learn the scriptures of the new atheistic religion; they were to learn the

process of reasoning.

111

For instance, according to the 1926 census, out of every 1,000 citizens of the U.S.S.R., only

445 were literate. Peter Kenez, The Birth of the Propaganda State. Soviet Methods of Mass
Mobilization, 1917-1929 (Cambridge: Cambridge University Press, 1985), 157.

112

Kenez, The Birth of the Propaganda State, 220.

113

The pioneering work of Annette Michelson was important in bringing my attention to

Eisestein's Capital project. Annette Michelson, "Reading Eisenstein Reading 'Capital'," October
2 (1976): 27-38; October 3 (1977): 82-88.

114

Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and

Indiana University Press, 1987), 163.

115

Qtd. in Baxandall, Painting and Experience in Fifteenth Century Italy, 41.

It is significant that the most categorical statement by Eisenstein on the possibility of

"filmic reasoning," reasoning through images, appears in the context of his discussion of the

sequence known as For God and Country from October (1929):

Maintaining the denotation of "God," the images increasingly disagree with our concept of God,
inevitably leading to individual conclusions about the nature of all deities...a chain of images
attempted to achieve a purely intellectual resolution, resulting from a conflict between a
preconception and a gradual discrediting of its purposeful steps.
Step by step...power is accumulated behind a process that can be formally identified with that of a
logical deduction...The conventional descriptive form for film leads to the formal possibility of a
kind of filmic reasoning. While conventional film directs emotions, this suggests an opportunity to
encourage and direct the whole thought process as well.

116

Far from simply representing God or deities, as they did for centuries, here images serve a totally

new function -- to provoke and direct reasoning, reasoning of a particular kind -- "Marxist

dialectics." In accordance with its principles, as canonized by the official Soviet philosophy,

Eisenstein wants to present the viewer with the visual equivalents of thesis and anti-thesis so that

the viewer can then proceed to arrive at synthesis, i.e. the correct conclusion, pre-programmed by

Eisenstein.

"The content of CAPITAL (its aim) is now formulated: to teach the worker to think

dialectically," Eisenstein writes enthusiastically in April of 1928.

117

Schooled by the film,

viewers would become self-sufficient thinkers, learning the skill of "Communist decoding of the

world," each a walking camera, snapping pictures of visual thesis and anti-thesis, the brain

automatically executing cognitive operations of montage, thinking through images, efficiently

and effectively.

Eisenstein claims the radical novelty of his concept of "filmic reasoning":

116

Sergei Eisenstein, "A Dialectical Approach to Film Form," in Film Form: Essays in Film

Theory, ed. Jay Leyda (New York: Harcourt Brace and World, 1949), 62. Emphasis mine -- L.M.

117

Sergei Eisenstein, "Notes for a Film of 'Capital,'" trans. Maciej Sliwowski, Jay Leuda, and

Annette Michelson, October 2 (1976): 10.

The proclamation that I'm going to make a movie of Marx's Das Kapital is not a publicity stunt. I
believe that the films of the future will be found going in this direction (or else they'll be filming
things like The Idea of Christianity from the bourgeois point of view!) In any case, they will have
to do with philosophy...the field is absolutely untouched. Tabula rasa.

118

Yet, Einstein's theory was not an isolated development. Many in the artistic left of the 1920s

shared a similar belief in the cognitive power of new visual forms such as montage. In the late

1920s Alexander Rodchenko promoted the use of montage sequences in graphic design and, like

Eisenstein, he saw montage as being equivalent to "dialectical" reasoning. In this formulation, an

individual image corresponded to a single concept, and thinking was thought to be provoked

when a number of images were juxtaposed in a series.

119

Walter Benjamin's notion of

"dialectical seeing," central in his unfinished Passagen-Werk project, also depends on montage,

but within a single frame, so to speak. "Dialectical seeing" was conceived by Benjamin as a way

to grasp the forms of the present by looking to the past and to the future in the same instant,

juxtaposing them in the same mental image.

120

118

Michelson, "Reading Eisenstein," October 2: 28.

119

While at first Rodchenko practiced juxtaposition of many separate photographs and

fragments within the space of a single image, at the end of the 1920s his photomontages became
multi-page layouts composed of a number of more "traditional" photographs, more like a film
montage sequence.

120

On Benjamin's concept of "dialectical seeing" see

Susan Buck-Morss, The Dialectics of

Venn, Galton, MŸnsterberg, Eisenstein. Logical diagrams, composite photographs, cinematic

devices of close-up, cut-back, and montage. Different, in effect, as they are, these developments

are symptoms of a single social imaginary at work: to make reasoning more efficient and at the

same time more controllable by externalizing it and rendering it visible. This imaginary can be

glimpsed in the new techniques of visual representation, such as Venn's diagrams. It also governs

the development and the promotion of new visual technologies, such as Galton's composites or

Eisenstein's dialectical montage. Finally, this imagination can be also read in theoretical

speculations about how abstract concepts and logical forms can be represented as images in

principle, such as in Freud's psychoanalysis.

Is there a single person whose work brings into focus the modern preoccupation with

visual reasoning -- whether in the form of new theoretical models, new techniques of visual

representation or new visual technologies? It is not Venn, or Galton, or MŸnsterberg. This

person must be Sigmund Freud. He is not only more explicit in his defense of the possibility of

visual reasoning; he develops a systematic theory of how abstract notions and logical forms can

Seeing: Walter Benjamin and the Arcades Project (Cambridge: The MIT Press, 1989).

be visualized. This theory becomes one of the cornerstones of a new science, devoted, we should

note, to externalization, bringing into the open, into public view the most intimate, private, and

unobservable psychic phenomena -- the unconscious ideas, unreachable for consciousness. In

this, the new science goes well beyond MŸnsterberg's notion of the externalization of memory

and attention by the techniques of film -- after all, these mental functions can be voluntarily

controlled and commanded by consciousness. Indeed, MŸnsterberg and his fellow experimental

psychologists at the turn of the century scrutinized the mind through the technique of

introspection -- which, by default, limited them to the study of only those mental acts and

functions that could be voluntarily controlled and consciously observed. Instead, the new science

postulates the existence of unobservable, uncontrollable psychic forces, which, lying deep below

the level of consciousness, nevertheless control it.

The latter hypothesis, it is often said, constitutes the most dramatic attack on Cartesian

rationality launched in this century. However, it should be noted for the record that the new

science also gives rationality and reason even more power than before -- and not only because it

tamed the unconscious for the consciousness but also because it gained another ground for

reason -- vision. Having postulated the existence of subterranean unconscious ideas and

unconscious reasoning, this science immediately proposes a technique by which these

phenomena can be brought to the surface, and rendered visible and harmless. It claims that

unconscious ideas, groups of ideas, and even arguments are transformed into the form of dream-

images. By analyzing these images, an analyst can recover these ideas and arguments.

5. Freud's Theory of Visual Reasoning

Freud's The Interpretation of Dreams, published in the first year of this century, inaugurated the

rise of visual reasoning.

121

This work exemplifies a new understanding of vision -- as not

inferior to reason but, on the contrary, as its equal, as a medium quite capable of representing

abstract ideas and logical arguments. According to Freud, dreams can translate into visual form

such concepts as "a high-placed official" (Freud 378) or "superfluous" (441). They can also

represent logical connection (349), causality, negation (361, 372) or contradiction (470). In short,

121

All subsequent refererences in this chapter to Freud, unless overwize indicated, are to

Sigmund Freud, The Interpretation of

Dreams (New York: Avon Books, 1965).

the whole range of reasoning operations can be encoded in visual images. With this theory,

vision is completely subsumed into the realm of logical thinking.

According to Freud, the mind's contents, ideas and logical arguments, are securely hidden

from its owner, from the psychoanalyst and from the public, lurking, invisible and unreachable,

in the archaic cave of the unconscious. They travel through the darkness of association networks,

like newly formed urban masses, riding to and from work in the underground subways, which in

the end of the nineteenth century were rapidly constructed in major metropolitan centers. Like

these urban masses, always a danger, potentially lurking in the darkness of the labyrinths of city

streets at night, ideas are formed, dissolved and rearranged in the total darkness, which the

limited "spotlight of attention" (Wundt) cannot reach.

This, of course, cannot be tolerated. Streets are streamlined and, even more importantly,

well lit. Indeed, the turn of the century is often called "the age of electricity" due to its obsession

with electrical light. But does not this light provide the best way of fighting invisibility and

assuring externalization? Similarly, ideas and reasoning need to be externalized, brought into the

open, rendered visible. Luckily, Freud assures us, the psyche already possesses a process which

transforms the invisible contents of the unconscious into images within our reach. All the

psychoanalyst needs to do is to properly interpret these dream images by tracing them back to

their underlying ideas.

So how can dreams represent ideas visually? In the section "Consideration of

Representability," after he establishes the requirement for visualization -- "considerations of

representability in the peculiar psychical material of which dreams make use -- for the most part,

that is, representability in visual images," (379) Freud provides a number of examples of how

dreams transform abstract concepts and thoughts into pictures.

122

Here are a few of them:

122

Mieke Bal's seminar on "Interpreting Culture" at the University of Rochester brought my

attention to this point. For the original discussion of considerations of representability see
Samuel Weber, The Legend of Freud (Minneapolis: University of Minnesota Press, 1982).

"A high-placed official" is represented by a high tower (378).

"Working my way through" is represented by "a long knife under a cake, as though to lift

out a slice" (380).

"Superfluous" is represented by water everywhere -- "overflowing," "flowing over"

(441).

Legs (part of the body) are represented through pillars or columns (382).

Is there a single logic these examples follow? Yes, and we have already encountered it in Lakoff.

Lakoff, as we have seen, claims that abstract concepts and abstract reasoning take shape in the

process of extending the structures of human sensory and bodily experience to abstract

situations; and that even the most abstract categories are metaphors, based on such topological

relations as part-whole and inside-outside. To use Freud's examples, such concepts as "a high-

placed official" or "working my may through" are meaningful in the first place because they

involve the conceptualization of abstract situations ("position of people in society"; "solving a

problem") in spatial terms.

Lakoff postulates that behind all seemingly abstract concepts of natural language lie

concrete "image-schemes." If we accept Lakoff's theory for a moment and read Freud's examples

through its terms, it appears that in dreams abstract concepts and ideas turn back into the "image-

schemes" from which they evolved in the first place. Thus, "a high-placed official" becomes

again a concrete spatial representation, "high tower."

Of course, the point is not that Lakoff's theory "explains" Freud or that Freud

"anticipates" Lakoff; rather, it is that both implicitly rely on the notion which has always figured

centrally in the speculations about the origin of language and reason (including, as we have seen,

of such thinkers as Locke, Herder, Vico, and Nietzsche): that abstract concepts have evolved

from concrete ones, the latter directly connected with perceptual experience. Freud, however,

adds a new and important notion: regression. If in dreams abstract concepts and abstract thinking

itself return to their "origin" as images and "image-based reasoning" (Lakoff's term) this

transformation would be in perfect correspondence with the regressive character of dreams.

The notion of regression is the key for understanding Freud's theory of visual thinking.

Not only does it explain why in dreams abstract concepts return to their visual origins, it also

partly clarifies why dreams invariably rely on visual means of representation at all: "what is

older in time is more primitive in form and in psychical topography lies nearer to the perceptual

end" (587). Regression to the visual mode in dreams, then, operates in three ways: regression to

an older representational mode (formal regression); topographic regression (to the perceptual

system); and temporal regression to the older (in the evolutionist sense) psychical structures.

There is also another, fourth meaning of regression to the visual mode: regression to the visual

scenes witnessed in childhood ("primal scene"). Therefore, the visual means of representation

employed by dreams are characterized by Freud as an archaic, "primitive mode of representation

and expression" (587). Vision, in Freud's diagnosis, is seen as the primordial, original language

of mankind.

Should we be surprised that, according to Freud, Vienna's inhabitants circa 1900, modern

and respectable by day, at night totally lose their modernity, shed away progress, and instead

immerse themselves in archaic visual illusions? No, if we recall the view, frequently voiced in

the earlier part of the twentieth century, that the new visual forms of mass culture represent a

return, indeed, a regression to humanity's archaic stage. Most frequently, it was cinema which

was seen (for instance, by Nabokov) as a time machine, where the metropolitan masses, stripped

of their individuality, subjectivity, or even simply basic interiority, in total darkness (like that of

prehistoric caves, even darker than the cave described by Plato), gave up the light of

Enlightenment, Civilization and Reason, and instead were chanting in unison...a collective body,

mesmerized by visual forms, moving before them on the screen, made possible through the

magic of the cinematograph.

As his contemporaries, Freud believes that the visual cinematograph of dream-images

represents the regression to the original language of mankind -- the language of pictures;

however, he does not think that this entails the loss of rationality, logic, and reason. Freud not

only claims, as we have just seen, that the dream-work has the means to represent singular ideas,

even abstract ones, but he also confidently describes the mechanisms by which the dream-work

visualizes the logical relations between dream-thoughts.

"What representation do dreams provide for 'if,' 'because,' 'just as,' 'although,' 'either-or,'

and all the other conjunctions without which we cannot understand sentences or speeches?," asks

Freud. His initial answer is negative: "In the first resort our answer must be that dreams have no

means at their disposal for representing these logical relations between the dream-thoughts"

(347). But astonishingly, contradicting his own statement, Freud then proceeds to elaborate the

mechanisms by which the whole range of logical relations can be represented in images:

Logical connection is represented by simultaneity in time (349).

Spatial proximity is used to represent logical relation: "Whenever they [dreams] show us

two elements close together, this guarantees that there is some specially intimate connection

between what corresponds to them among the dream-thoughts" (349).

Causality is reproduced by a time sequence (349) or by a metamorphosis.

An alternative ("either-or") may be represented by dividing the dream into two pieces of

equal length (353).

Negation is usually discarded (353) -- but later Freud gives examples to the contrary

(361, 372).

Similarity ("just as") is the "favorite" of dreams -- "this relation, unlike any other, is

capable of being represented in dreams in a variety of ways" (354), including "composition" -- the

creation of composites. In principle, composition is similar to metamorphosis: "The psychical

process of constructing composite images in dreams is evidently the same as when we imagine or

portray a centaur or a dragon in waking life" (359).

"The temporal repetition of an act is regularly shown in dreams by the numerical

multiplication of an object" (407).

Contradiction is represented by absurdity (470).

Freud does not explicitly connect his descriptions of the dream-work's mechanisms of the

visualization of logical relations with his view of the dream-work as regression to archaic mental

structures. However, his characterization of these mechanisms only makes sense if we assume

that, like everything connected with the dream-work, they owe their character to regression. The

connection is provided in the work of another writer, Lucien LŽvy-Bruhl, a leading French

anthropologist and Freud's contemporary. Like Freud, LŽvy-Bruhl was concerned with the

nature of archaic mental structures. But while Freud located these structures in the unconscious

of every modern civilized individual, LŽvy-Bruhl claimed to have found them in the mental

language of present-day "savages."

In a number of books LŽvy-Bruhl advanced the notion of "primitive" logic which

enjoyed wide popularity in the first decades of the twentieth century. According to LŽvy-Bruhl,

the thinking of primitive people follows a logic quite different from our own "normal" logic.

First of all, primitive people confuse temporal succession with causality. If they witness the two

events following each other, they assume that the two are causally related. Thus, LŽvy-Bruhl

recounts that if one of them sees a snake fall down from the tree in front of him and learns soon

thereafter that his son is dead, he will relate these two facts.

123

Second, primitive people believe

that if two objects have been physically connected, they continue to influence each other when

separated. For instance, some primitive people are careful to hide a babies' milk-teeth -- for if

somebody finds them, then the new teeth will grow improperly. Summing up this feature of

primitive logic, LŽvy-Bruhl writes: "Primitive people confuse the antecedent with the cause.

This would be a very common error in reasoning that is known as post hoc, ergo propter hoc

sophism."

124

These "laws" of "primitive thinking," as described by LŽvy-Bruhl, are the same ones that

Freud claims govern the representation of the relations between dream-thoughts. Savages think

that two events following each other are causally related (LŽvy-Bruhl) -- dreams represent

causality by temporal sequence (Freud). For savages, two objects physically connected are also

logically related -- dreams signify a logical connection by the spatial closeness of elements or by

the simultaneity of time (Freud, 349). Moreover, as "savage mind," the dream-work finds

similarity everywhere, seeing logical relations where there are none: "One and one only of these

logical relations is very highly favored by the mechanism of dream-formation; namely, the

relation of similarity, consonance or approximation -- the relation of 'just as.' This relation,

unlike any other, is capable of being represented in dreams in a variety of ways" (354). The

visual language of the dream-work -- the archaic language of the psyche, as described by Freud -

- is the same as the primordial "mental language" still at work in modern savages -- as claimed

by LŽvy-Bruhl.

123

Lucien LŽvy-Bruhl, Les functions mentales dans les sociŽtŽs primitives (Paris: 1910), 72.

124

Ibid., 73.

6. The Rise of the Diagram

Freud does not just insist that abstract ideas and logic can be visualized or provide a theoretical

model of how this can be done in principle. He practices what he preaches. What makes The

Interpretation of Dreams a truly modern text, compatible with the age of illustrated magazines,

color printing, scientific documentaries, dialectical montage, and other techniques and

technologies of visual communication is not the abundance of illustrations or figures (in fact,

there are only a couple). Freud does not have to include illustrations, because many of his

theories, and indeed his overall theory of the psyche (the famous topographic model), are based

on simple visual models, easy to grasp and easy to remember. And this is what makes his text

modern. The reader is invited to disregard the complexities of real anatomical data and instead to

conceptualize the psyche "as resembling a compound microscope or a photographic apparatus, or

something of the kind" (574). According to Freud, this apparatus is made up of just a few

components (called by Freud agencies or systems), which are arranged in spatial order (574-

583). This "spatial analogy" (653) allows Freud to visualize any psychic process as a simple

movement between a few parts of the model.

It is as though Freud senses that the construction of such models is very effective in

pushing the reader to "see" the self-evidence of the appropriate conclusions. Thus, after

presenting the typographic model of the psyche, Freud uses it to explain the newly introduced

concept of regression: "I believe that the name 'regression' is of help to us in so far as it connects

a fact that was already known to us with our schematic picture, in which the mental apparatus

was given a sense or direction. And it is at this point that picture begins to repay us for having

constructed it. For an examination of it, without any further reflection, reveals a further

characteristic of dream-formation" (582. Emphasis mine -- L.M.)

To clarify the logical relation between the conscious and the unconscious, Freud again

relies on the same technique of constructing a visual diagram. From the point of view of the

ontogenetic and phylogenetic explanation of the formation of the psychical apparatus, primary

processes, as their name implies, "are present in the mental apparatus from the first" (642) while

secondary processes appear only later and never achieve full domination. Thus, concludes Freud,

the primary processes (and corresponding to them the system of the unconscious) constitute the

essence of psychical life: "The unconscious is the true psychical reality" (651). To emphasize

this point and to make sure the reader remembers it, Freud "visualizes" it in the form of a simple

image of a sphere within a larger sphere: "The unconscious is the larger sphere, which includes

within it the smaller sphere of the conscious" (651). He then "reads off" his conclusion from this

visual model: "Everything conscious has an unconscious preliminary stage" (651).

Freud's skillful use of visual models, analogous to Venn's diagrams, to represent the

logical relations between the terms in his theories is unusual in that Freud painstakingly

describes a visual model verbally, rather than directly presenting the reader (or should we say,

the viewer?) of his text with a picture. It is as though Freud fully realized the power of diagrams

to represent complex concepts, models and arguments but, circa 1900, he was still hesitant to

"vulgarize" his scientific opus with images.

In 1933 Freud finally includes a real diagram in The New Introductory Lectures on

Psychoanalysis to illustrate the relation between two triads of concepts: id, ego, and superego,

and unconscious, preconscious, and conscious. Freud's diagram, which effectively signifies a

complex set of relations by just a few lines, is representative of a whole family of new techniques

of visual representation, new to the modern period (fig. 8). These techniques have risen to fill

what can be called the cognitive needs of modernization: to quickly communicate more and more

complicated relations between various entities, to manipulate huge sets of data (whether in

science -- which became more and more the problem of managing collected data -- or in the

bureaucratic management of people), to depict in manageable forms endless hierarchies of

relations -- in short, to represent reasoning visually and to reason with the help of vision. These

techniques of visual representation comprise what semioticians call the graphic sign systems --

signifying systems, employing a particular set of graphic conventions to represent particular

kinds of objects.

125

We have already encountered one of these new graphic systems -- Venn's diagrams.

These diagrams rely on the unique property of vision, which, along with touch, as Rudolf

Arnheim points out, is "the only sensory medium that conveys such spatial properties as

inclusion, overlap, parallelism, size, etc., with some precision."

126

This property makes possible

the precise representation of the logical relations between classes in graphic form. But, while in

principle visual images are capable of representing spatial properties, not every image can

effectively do so. For instance, continuously varying tonal gradients are useless, whereas discrete

elements are effective. And these elements, to be easily readable, must have enough visual

difference among them. Therefore, Venn's diagrams are typically drawn using simple lines. The

classes are represented through circles (a representation in the form of rectangles is possible, but

it would be ambiguous, leading the viewer to believe that the shapes of figures code some

additional logical relation). If circles overlap, the common area (representing exclusive "and") is

clearly distinguished through shading or color.

In short, we are dealing with a distinct graphic system, employing particular kinds of

graphic marks, optimized for the representation of particular references -- a logic of classes.

This graphic system accentuates one property of images (the ability to represent spatial

properties such as inclusion) at the expense of other properties, for instance, the ability to

represent continuously varying quantities through continuously varying tone or color.

Venn's diagrams is an example of a specialized graphic sign system. Their use is limited

to a narrow professional domain of logic. In the same period, another graphic sign system for

visualization of logical relations comes into use (fig. 9). In contrast to Venn's diagrams, the

125

The most comprehensive study of graphic sign systems is Jacques Bertin, Semiology of

Graphics (Madison, WI: The University of Wisconsin Press, 1983).

126

Arnheim, "A Plea for Visual Thinking," 143.

graphic marks of this sign system are now everywhere, seemingly the very air of which the

contemporary visual semiosphere is composed.

What I have in mind is the gradual emergence of greatly simplified graphic marks which

came to form a notation, a sort of an erector construction set, whose parts can be used to quickly

assemble a visualization of a network of relations, a representation of a process or a scheme of

dependencies. This construction set includes boxes, arrows, vectors, solid and dotted lines, and

so on. Arrows can denote such relations as "belongs to," "relates to," "is a member of a class of,"

"causes," etc. Boxes, on the other hand (or simply any area separated by lines) can represent a

physical object, the state of a system, a stage in a process or any abstract entity. Freud's drawing

circa 1933, where a few encircled areas together with dotted and solid lines represent a complex

relationship between psychical agencies, illustrates the power of this graphic system, which I

will call diagrammatic.

My comparison of the diagrammatic graphic system with a construction set is not

accidental. A construction set includes some parts with more or less fixed functions (for instance,

wheels for moving models), while others (bars with holes) can be used for a variety of purposes -

- to hold other parts, to represent parts of a model, to be stationary or to move. Similarly, while

some elements of the diagrammatic system have relatively fixed meanings (for instance, wavy

brackets), others, such as arrows or dotted lines, can have numerous meanings, depending on

their particular arrangement.

This leads us to the second feature. In diagrammatic language, the same few symbols can

represent anything: a box, for instance, may stand for an idea, a theory, a building, 200 buildings,

a citizen, an army, or a state of a physical system. Unlike the users of verbal language who learn

particular semantics, the users of the diagrammatic system learn the general conventions of

syntax. It is enough to know that a box represents an entity -- regardless of what the entity may

be. In this, diagrammatic language, used to represent a set of entities and the relations between

them, functions exactly like the symbolic language of logic: a properly executed logic diagram is

isomorphic with a logical statement.

127

But this power is achieved through the standardization

of graphical conventions themselves.

A diagrammatic system is a metalanguage, whose symbols represent not any particular

ideas or objects, but rather the abstract relations evoked in any process of reasoning: "a part of;"

"leads to;" "associated with." It is not accidental, therefore, that this language of arrows and

boxes is what recent generations of computer programs, aiming to help reasoning or to imitate it

(outliners, hypertext, project management tools, idea generators, and the like) provide their users

with. The mental construction set became the graphic toolbox, always just a mouse click away.

This computer implementation of already existing conventions of visual culture is accompanied

by a further standardization of graphic symbols. This standardization is justified by the

argument, popularized since the adaptation of Macintosh user interface, that it helps the user to

recognize familiar symbols and commands regardless of the program she is using (fig. 10).

The standardization of diagrammatic symbols went hand in hand with their

simplification. While drawings were used for centuries to represent schematic information rather

than simply to symbolize particular objects or to record their shapes, modernization is

accompanied by the simplification of graphic symbols -- boxes, arrows, and the like. (This was

also accompanied by the streamlining of typography.) Now, it becomes much easier to "read off"

logical relations represented by the diagrams, since there are fewer graphical marks to scan.

When do the symbols of diagrammatic language, as it exists today, first appear?

Obviously, we would not find a sharp "break" in graphic paradigms, with a new graphic system

appearing overnight. According to Ernst Gombrich, the arrow does not acquire its meaning as

pointer or vector until the eighteenth century. And even then, he could only find it in a couple of

specialized professional contexts: it appears in a French treatise on Hydraulic Architecture of

127

Gardner, Logic Machines and Diagrams, 28.

1737 and on some maps, where it was used to show the direction of the river's flow.

128

Similarly, we can find other abstract graphic marks employed for centuries in graphic systems

used in particular professions -- for instance, engineering drawing. Gradually, however, these

symbols leave these professional contexts and enter visual culture at large.

Finally, what is the relation between the development of the diagrammatic graphic

system and the overall movement in modern visual culture towards the simplification,

streamlining, and standardization of all visual forms?

In the opening chapter of Techniques of the Observer, Jonathan Crary points out an

inherent contradiction present in most standard accounts of modernism. On the one hand, these

accounts postulate the emergence of new radical models of perception and representation,

summed up in the notion of abstraction, whose first signs appear with Manet, impressionism or

post-impressionism and which fully blossoms with Kandinsky, Kupka, Mondrian, and others. On

the other hand, it is also postulated that in the same historical period new popular visual forms

developed (first photography and then cinema), further solidifying realism. As Crary notes,

128

Ernst Gombrich, "Pictorial Instructions," in Images and Understanding, ed. Barlow Horace,

Colin Blakemore and Miranda Weston-Smith (Cambridge: Cambridge University Press,

1990)

, 28.

"Classical space is overturned, so it seems, on one hand, but it persists on the other. This

conceptual division leads to the erroneous notion that something called realism dominated

popular representational practices, while experiments and innovations occurred in a distinct (if

often permeable) arena of modernist art making."

129

Crary's way to solve this contradiction is to claim, throughout his book, that the so-called

realistic forms of popular visual culture, based on photography, and abstract art are both

grounded in the emergence of "subjective vision" in the early nineteenth century, in which the

link between representational signs and external referents is severed. "I argue," writes Crary,

"that some of the most pervasive means of producing 'realistic' effects in mass visual culture,

such as stereoscope, were in fact based on a radical abstraction and reconstruction of optical

experience, thus demanding a reconsideration of what 'realism' means in the nineteenth

century."

130

What his anti-realist argument leaves out, however, is any consideration of the

representational power of new visual technologies such as photography. With photography,

based on mechanized perspective, the ability to establish a "reciprocal, metrical relationship

between the shapes of objects as definitely located in space and their pictorial representations"

(William Ivins) became greatly amplified. In Ivins' words, photography is "a form of

picturemaking that is not only precisely duplicable but one in which geometrical perspective is

so inherent that today the camera is used as a surveying and measuring instrument."

131

Rather

than being anti-realistic, photography became the perfect technology of instrumental realism

(Allan Sekula's term), literally allowing its users to have physical power over the represented

subjects and objects. As Sekula points out in his important account of the instrumental uses of

photography at the turn of the century, criminal identification photographs, for instance, "are

129

Crary, Techniques of the Observer, 4.

130

Ibid., 9.

131

William M. Ivins, On the Rationalization of Sight (New York: Da Capo Press, Inc., 1975),

12.

designed quite literally to facilitate the arrest of their referent" and, similarly, military

reconnaissance photographs make possible identification and destruction of the objects they

capture.

132

A different way to solve the contradiction, noted by Crary, between the opposition

abstraction/high art versus realism/popular visual culture is to point out that in fact the

development of abstract visual forms was not limited to modernist art but equally took over the

realm of the popular -- first of all, in the adaptation of what I have called here diagrammatic

visual language. But in order to see this it is necessary to include in the discussion more than

images from the realms of art, or entertainment or leisure -- the realms to which the problematic

of high-low still limits itself, and whose borders in Crary's argument are represented by

modernist painting, on the one side, and photographs, stereoscopes, phenakistiscopes and other

popular nineteenth century optical toys, on the other. Rather, it is necessary to consider the whole

range of images employed in modern culture -- images employed not only for leisure but also for

work; images through which, literally speaking, modern work gets done: aerial surveillance

photographs and criminal identification shots, engineering and architectural drawings and plans,

maps and typography, instrumentation displays, and scientific imagery. And, last but not least,

we should consider the abstract graphic marks of diagrammatic visual language, designed to

facilitate cognitive and communicative workloads of modernity, in the office, at school, in the

control room, in the boardroom, and on the assembly line -- which is today replaced by equally

monotonous rows of computer terminals, outputting not Model T, but bits of information; this

steel and coal of the information -- or shall we better say, cognitive age.

Information age, knowledge economy, post-industrial society. All these terms come to designate

the changing nature of society after World War II: the gradual displacement of manual labor by

132

Allan Sekula, "The Body and the Archive," 7.

cognitive labor. With the centrality of cognition for the post-industrial workplace, the notion of

visual reasoning acquired new importance. Consequently, the work on the models, techniques,

and technologies of visual reasoning, which was previously pursued by a few isolated individuals

(Galton, Freud, Eisenstein), has now become a matter of systematic research on the industrial

scale in such fields as cognitive psychology, scientific visualization, and virtual reality. I will

consider this research in the next two sections.

What is the relation between the psychological theories of the mind and their contemporary

visual technologies? The recurrent claims that new visual technologies externalize and objectify

reasoning, and that they can be used to augment or control it, are based on the assumption of the

isomorphism of mental representations and operations with external visual effects such as

dissolves, composite images, and edited sequences. This assumption, which I so far have not

questioned, on a closer examination appears to be highly problematic. Whatever mental

representations and operations really are, the mind surely does not contain pictures, photographs

or film clips which some mental homunculus looks at. The external images presented to the mind

are not magically transplanted inside it as ready-made ideas and arguments. Regardless of what

visual forms can be presented before the eye -- diagrams, photographs, film images -- as any

other visual input, they are subjected to the complicated processing by the nervous system which

constructs its own internal representations.

Yet, the assumption of such an isomorphism continues to persist in modern thinking

about vision, ignited by every new round of visualization technology: photography, film,

computer animation, and virtual reality. Consider the claims which surround the new field of

scientific visualization -- visualization of data sets, their relationships and their dynamic behavior

using computer graphics. Richard Mark Friedhoff and

7. Visual Technologies and the Mind

William Benson proclaim that computer visualization techniques constitute the second computer

revolution because they act as the direct "extension of preconscious visual processes."

133

They

assume that the images on a computer screen do not simply function as an aid for reasoning but

that they are equivalent to the mental representations the mind may construct while thinking --

and this is the source of their power.

Or consider the technology, which, even more so than scientific visualization, is seen as

capable of completely objectifying, better yet, transparently merging with mental processes --

virtual reality (VR). Again, the descriptions of its capabilities do not distinguish between internal

mental functions, events and processes, and externally presented images. This is how, according

to Jaron Lanier, VR can take over human memory: "You can play back your memory through

time and classify your memories in various ways. You'd be able to run back through the

experiential places you've been in order to be able to find people, tools."

134

Lanier also claims

that VR will lead to the age of "post-symbolic communication," communication without

language or any other symbols. Indeed, why should there be any need for linguistic symbols, if

everybody, rather than being locked into a "prison-house of language" (Jameson), will happily

live in the ultimate nightmare of democracy -- the single mental space which is shared by

everybody, and where every communicative act is always ideal (Habermas). This is Lanier's

example of how post-symbolic communication will function: "you can make a cup that someone

else can pick when there wasn't a cup before, without having to use a picture of the word

"cup."

135

Here, as with the earlier technology of film, the fantasy of objectifying and

augmenting consciousness, extending the powers of reason, goes hand in hand with the desire to

see in technology a return to the primitive happy age of pre-language, pre-misunderstanding.

133

Friedhoff and Benson, The Second Computer Revolution: Visualization, 13.

134

Druckrey, "Revenge of the Nerds," 9.

135

Ibid., 6.

Locked in virtual reality caves, with language taken away, we will communicate through

gestures, body movements, and grimaces, like our primitive ancestors...

136

What can one make of this apparently unsound, yet irresistible, assumption of

isomorphism between the mental process of reasoning and external, technologically generated

visual forms, haunting us at least since the end of the nineteenth century? The conflation of

outside and inside is, of course, symptomatic of the desire to project the inside onto the outside,

to make it objective and public. But is this all? To really understand the persistance of this

assumption, we should turn to the history of ideas about the nature of mental processes.

It is well known that technologies have historically provided and continue to provide the

models according to which people imagine the mind. In the seventeenth century it was the clock,

in the nineteenth -- the motor, in the second half of the twentieth -- digital computers. More

precisely, the paradigms were provided not by the technologies themselves, but by theories

which made them possible. To take the last paradigm as an example, cognitive psychology, born

in the 1950s and gaining prominence ever since, approaches the mind as an information-

processing system, as software which runs on the hardware of the brain. But what gave cognitive

psychology its epistemological basis was not the new technology itself (computers), but the

information theory accompanying it. It was this paradigm which substituted the discussions of

mind and brain by the notion of "human information processing." And before information

136

At SIGGRAPH 1992, the premier annual conference on Computer Graphics and Interactive

Techniques in the U.S., attended by nearly 30,000 people, about a dozen VR exhibits always had
long lines of visitors. However, the lines to two of these exhibits were much longer than any of
the others. One was called Dome, the other -- Virtual Reality Cave; in both cases, to see the
show, the visitors had to go inside some cave-line structures. It did not matter that one of the
exhibits featured a scientific visualization display, of interest only to specialists. Clearly, the fact
that in order to see the spectacle one had to go inside a dark, cave-like space, different from
normal space, provided enough attraction at the end of this century, just as it did at its beginning,
when millions flocked into the dark caves of movie theaters.

theory, theories of the mind were influenced by thermodynamics (as in Freud) and mechanics

(Hobbes).

137

It can be also claimed, however, that human imagination about the mind's operations is

limited by the current visual technologies. Lakoff's assertion that "natural reasoning makes use of

at least some unconscious and automatic image-based processes such as superimposing images,

scanning them, focusing on part of them" (1986: 149) as well as Johnson-Laird's proposal that

logical reasoning is a matter of scanning visual models, would have been impossible before

television and later, computer graphics. These visual technologies made operations on images

such as scanning, focusing, and superimposition seem natural. Even more telling are the models

of cognitive psychologists who, in the last two decades, have systematically scrutinized the role

played by mental images in reasoning. These models, which define mental images in terms of

such characteristics as spatial resolution, speed of access, basic graphic operations (rotation,

translation, copy), seem to be describing first of all computer imaging systems. Psychologists

argue among themselves which imaging systems better resemble mental processes, but they do

not doubt the basic metaphor. As Paul Virilio notes, "now the virtual images of the computer

screen seem to confirm not only the existence of certain forms of representation but, more

immediately, the objective presence of mental images."

138

137

While this theory is well known and widely accepted, other facts suggest that, at least,

sometimes, the influence runs in the opposite direction -- biological and psychological theories
of body and mind providing paradigms for theories of mechanisms. For instance, it appears that
Norbert Wiener's cybernetics was inspired by the concept of homeostasis developed in biology:
"Physiologist Walter B. Cannon viewed the animal body as a self-regulating machine. Building
on the work done by Claude Bernard in the nineteenth century, Cannon developed the concept of
'homeostasis' -- the process by which the body maintains itself in a state of internal equilibrium.
Cannon's ideas were well known to Norbert Wiener. In fact, Cannon's Wisdom of the Body
(1932) may be read as sort of an introduction to Wiener's Cybernetics (1948)." Charles Eames
and Ray Eames, A Computer Perspective: Background to the Computer Age (Cambridge:
Harvard University Press, 1990), 99. Another example is provided by the turn of Artificial
Intelligence (AI) in the 1980s from trying to simulate the disembodied mind to the simulation of
a collective of primitive organisms, having the functionality of insects. Drawing directly on
research in biology, the researchers in AI hope that intelligence will emerge as a product of the
collective behavior of machines simulating simple biological organisms.

138

Paul Virilio, Lost Dimension (New York: Semiotext(e), 1991), 114.

Similarly, in the earlier period, when Freud, in The Interpretation of Dreams, described

the mechanisms by which dream-thoughts and the logical relations between them are represented

in dreams, he and his fellow psychologists relied on available visual technology for their

understanding of the mind. Not surprisingly, Galton's composites, the earliest form of image

processing before digital computers, provided a particularly attractive model. Freud compared

the process of condensation with one of Francis Galton's procedures which became especially

famous: making family portraits by overlaying a different negative image for each member of the

family and then making a single print.

139

Writing in the same decade, the American

psychologist Edward Titchener opened the discussion of the nature of abstract ideas in his

textbook of psychology by noting that "the suggestion has been made that an abstract idea is a

sort of composite photograph, a mental picture which results from the superimposition of many

particular perceptions or ideas, and which therefore shows the common elements distinct and the

individual elements blurred."

140

He then proceeds to consider the pros and cons of this view. We

should not wonder why Titchener, Freud and other psychologists take the comparison for granted

rather than presenting it as a simple metaphor -- contemporary cognitive psychologists also do

not question why their models of the mind are so similar to the computer workstations on which

they are constructed.

Thus, not only reason has always been conceived by philosophers in visual terms; as

psychologists begin to furiously take on the questions philosophers only wondered about,

subjecting mental processes to controlled, scientific, laboratory study, their models begin to

reflect, more and more, the external visual forms made possible by whatever visual technology

dominates the period. As before, vision is the mirror reason looks into.

139

Sigmund Freud, Standard Edition of the Complete Psychological Works (London: Hogarth

Press, 1953), 4: 293.

140

Edward Bradford Titchener, A Beginner's Psychology (New York: The Macmillan

Company, 1915), 114.

Given the reliance of psychological theories of the mind on contemporary visual

technologies, is there any "progress" between the turn of the century and today, except that the

imagination of contemporary psychologists depends on the more sophisticated visual

technologies of computer graphics? In fact, during this century, the assumption of an

isomorphism between the mental and the objective became even more prominent; and

externalization of reasoning has been taken much further, both technologically and theoretically.

On the one hand, the refinement of various medical imagining techniques in the 1980s

made possible an increasingly precise imaging of brain activity, including the visualization of

reasoning -- in a literal sense. It is now possible to ask the experimental subject to concentrate

on solving a problem and to see which parts of the cerebral cortex are active. The question of

whether reasoning in fact depends on the operations normally involved in perception becomes

more and more the question which, according to a number of researchers, can be answered

through experimentation -- it is enough to show that the part of a cortex normally dedicated to

the processing of visual information is activated in the process of reasoning.

141

More importantly, in their theories, many cognitive psychologists have accepted as given

the equivalence between internal mental processes and the operations which can be done with

external, objectively existing visual representations and objects. Consider the debates about the

nature and role of mental imagery, which have constituted one of the most active areas of

research in cognitive psychology in the last two decades.

142

On the one hand, there are those

(such as Zenon Pylyshyn) who argue that mental imagery simply consists of the use of general

thought processes to simulate physical perceptual events. In this view, if the subjects report the

presence of mental imagery during reasoning and problem solving, this is simply a side effect, a

141

Martha Farah, "Is Visual Imagery Really Visual? Overlooked Evidence from

Neuropsychology," Psychological Review 95, no. 3 (1988): 307-317.

142

For a summary of different positions, see Ronald A. Finke, Principles of Mental Imagery

(Cambridge: The MIT Press, 1989).

by-product of real mental computations which do not involve visual representations. On the other

hand, there are those psychologists and neurophysiologists who, through experiments and direct

imaging of brain activity, want to prove that reasoning takes place through the construction and

manipulation of mental images (Alan Pavio, Roger Shepard, Stephen Kosslyn, Martha Farah).

One of the most well-known experiments in defense of the latter view has been done by

Roger Shepard and Jacqueline Metzler of Harvard University.

143

They presented subjects with

143

S. Shepard and J. Metzler, "Mental

Rotations of Three-dimensional Objects," Science 171 (1971): 701-

703.

pairs of perspective line drawings of three-dimensional forms constructed from small cubes. The

subjects' task was to determine whether or not the forms were identical in shape, despite the

difference in orientation. Shepard and Metzler have found that the reaction time was proportional

to the degree of rotation which is required to bring the two objects into a similar position. These

results were taken as proof that in solving the problem, the subjects mentally rotated

representations of three-dimensional objects, and that imagined rotations corresponded to actual

physical rotations of objects: "Imagined rotations and physical transformations exhibit

corresponding dynamic characteristics and are governed by the same laws of motion."

144

Thus, a

mental process was equated with an operation one would perform with real, objectively existing

objects.

Other experiments in defense of the position that many kinds of reasoning involve

manipulation of mental imagery entail the comparison of abstract qualities. When subjects were

asked to recall two animals and to judge which one was larger, the reaction time decreased

proportionally to the difference in estimated size. In another experiment, one group of subjects

was asked to rate animals in intelligence on a scale from one to ten, while another group had to

compare the intelligence of pairs of animals. Again, the reaction time decreased as the distance

in rated intelligence increased. It was concluded that when the subjects tried to discriminate

between two objects, reaction time was shorter the greater the difference between two objects,

regardless of whether the objects were really presented (for instance, two lines of different

length), or were imagined (size of animals), or whether the qualities to be compared were

abstract (intelligence of animals).

145

We shall leave to psychologists the debates whether these and numerous related

experiments indeed prove that internal mental processes involve the manipulation of pictures

144

Finke, Principles of Mental Imagery, 93.

145

John Robert Anderson, "Mental Imagery," in Cognitive Psychology and Its Implications,

(W.H. Freeman and Company, 1980).

similar in their qualities to real images. But for my purpose it is significant in itself that in

imagining what mental processes are like, contemporary psychologists have assumed, without

any reservations, an equivalence between the internal and the external, between the mental

objects and the real ones.

Modernization, visualization, externalization. In order to externalize the internal, the invisible, it

was first equated with the visual. Once this was accomplished, it was simple and only logical to

equate the visual inside with the visual outside, the objectivity and standardization of images

drawn on a classroom blackboard, on the screen of a movie theater, or, most recently, on the

computer terminal.

In this chapter I am concerned with the answer to the problem of how to represent, communicate,

induce or perform reasoning more efficiently. This problem first became important with the

emergence of mass society which required an efficient means to communicate abstract ideas to

large audiences. It became even more important with the shift to post-industrial society when

logical reasoning and other cognitive skills attained central economic importance.

The answer was to adopt vision as a more efficient medium. It can be perceived in

Freud's theoretical speculations of how ideas and logical arguments can be visualized, in the

development of new techniques of visual representation (Venn's pictorial logic, diagrammatic

graphic system), and in the claims made about new visual technologies (Galton, MŸnsterberg,

Eisenstein).

Vision, traditionally considered to be inferior to reason, is now seen as the proper, indeed,

as the ideal medium for it. And this is not only a question of the efficiency

8. Analog Engine

and effectiveness of communication, although, with visual reasoning, these two demands are

fully satisfied. Indeed, in Eisenstein's montage sequences, reasoning is speeded up -- to twenty-

four frames a second. And, with pictorial statistics and diagrammatic visual language, reasoning

becomes more effective: complex conceptual hierarchies are reduced to boxes and arrows, huge

sets of data become marks on paper which can be easily manipulated. But what is equally

important is that visual reasoning fits perfectly with the demand of mass society for

standardization. The subjects have to be standardized, and the means by which they are

standardized need to be standardized as well. Hence the objectification of internal, private mental

processes, including ideas and logical connections, and their equation with external visual forms

which can be easily manipulated, mass produced, and standardized on its own. Whether we are

dealing with the conventions of Venn's diagrams, of signs of diagrammatic sign system, or of

cinematic techniques, the private and individual is translated into the public and becomes

regulated.

This new use of vision as a means of reasoning is accompanied by its rationalization on a

conceptual level. If previously vision was thought to be confused, incapable of forming

generalizations and unsuitable for representing logical forms, now this is all forgotten. Vision is

accepted as being fully capable of representing logical relations (Freud), abstract ideas (Galton),

and even dialectics (Eisenstein).

This equation of vision with logic, accomplished by the beginning of the twentieth

century, is the first stage in vision's rationalization. When industrial society turned into the

society of information, computers became the dominant technology, both economically and

intellectually, and behaviorism (efficiency of the body) was replaced by cognitive psychology

(efficiency of the mind), this rationalization entered into a second stage. Now, with the questions

of how information can be coded, stored, retrieved, and processed more efficiently becoming a

matter of economic survival, cognitive psychologists, together with computer scientists, began to

scrutinize vision from this new perspective. In the process, they came up with very precise

descriptions of what vision is. However, what they describe is not some ahistorical, essential

vision, but vision already conceived in a historically and socially specific way -- as an efficient

computational system.

Why is it that when cognitive psychologists started to wonder about the properties of data

structures involved in human reasoning, vision began to take a central place in their models?

Because every mental process is conceived as a computation, and it is assumed that this

computation has to be efficient. The criterion of efficiency is a very general and pervasive notion

in contemporary cognitive psychology. This criterion is the main factor in deciding which of the

competing models should be favored. The criterion is justified by a reference to biological

evolution -- it is said that in the process of evolution the most efficient method, or the most

efficient representational code had to be adopted. However, the real justification of this criterion,

in my view, is economic, coming from the close connection between cognitive psychology and

its master discipline, to which cognitive psychology ultimately serves -- computer science, and

behind it, the computer industry. Just as computer scientists would not want to build an

inefficient computer, psychologists cannot imagine an inefficient mental process.

Once psychologists realized that images possess some unique properties, which make

them a very efficient medium for computation, it was only natural that they entered into their

theories. How does Johnson-Laird justify his theory of mental models, according to which

logical reasoning involves the construction and manipulation of spatial models, analogous to

Venn's diagrams? A model represented in a dimensional space can be constructed, manipulated

or scanned in any direction, while propositional representations can only be processed in one

direction, making them much less efficient.

Why do some cognitive psychologists argue that human long-term memory stores not

only knowledge in the form of propositions, but also as images? Because if we store appearances

of objects then later we can compute those properties that could not be anticipated when we saw

the objects initially.

146

In contrast to propositional (linguistic) representation, images turn out to

be capital which can be used over and over in the mind's economy.

Finally, why all the debates in cognitive psychology about the role of mental imagery in

reasoning in the last two decades? Because many problems can be solved more efficiently by

analog rather than digital computer, and vision has unique properties that enable it to function as

such a computer:

If images are representations in a medium with certain fixed properties, and can be subjected to
transformations such as rotation and scaling, then imagery could be used as an analog computer, to
solve problems whose entities and relations are isomorphic to objects and spatial relations. That is,
certain abstract problems could be best solved by translating their entities into imagined objects,
transforming them using available image transformations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.

147

What are these "certain properties" which make vision into an analog engine? On the one hand,

images can represent continuously varying information. On the other hand, images can represent

topological relations such as inclusion, "next to," and others. Taken together, these two

properties make vision a very powerful computer, better suited for solving many problems than

language. As Steven Pinker explains, in a visual medium, "it is impossible to represent the fact

that one object is next to another without also committing oneself to which is to the left. (This is

not true for abstract propositional representations, where the two-place predicate 'next to' can be

asserted of a pair of objects with no such commitment.)"

148

In principle, it is always possible to solve analog problems using propositional

representations but this simply would not be as efficient. So efficiency becomes the final reason

why vision, in spite of the arguments of Descartes, Locke or Leibniz, is today equated with

146

Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 65.

147

Ibid., 66.

148

Ibid., 66-67.

reasoning. However, in this equation, vision has little to do with what these philosophers meant

by it -- it became an analog code, a "data structure" more proficient for reasoning than language.

What remained the same, although recast in new terms, is the idea that vision came

before verbal language. If for Freud vision was the original, primordial language of humankind,

today cognitive psychologists speculate that the continuous, analog representational system of

mental images emerged first, followed by the system for propositional representation (language).

I once saw, therefore I think.

The year 1991 saw two events, of different importance and seemingly unrelated. One was the

long awaited publication in English of one of the most influential essays of modern art history --

Chapter 3. Mapping Space

1. Visual Nominalism

Erwin Panofsky's Die Perspektive als 'symbolische Form'.

149

The interest generated around the

re-emergence of this legendary essay, written in 1924-1925,

150

demonstrates that the problem of

149

Erwin Panofsky, Perspective as

Symbolic Form (New York: Zone Books,

1991)

perspective is still felt to be relevant to contemporary culture. Joseph Leo Koerner starts his

review of Perspective as Symbolic Form with the following claim: "Perspective is again the

rage."

151

The second event was the Gulf War, the outcome of which was largely predetermined

by Western superiority in the techniques of perspectival representation.

150

Erwin Panofsky, "Die Perspektive als

'symbolische Form'," Všrtrage der Bibliothek Warburg (Leipzig & Berlin: 1927), 258-330.

151

Joseph Leo Koerner, "The Shock of the View," The New Republic (April 26, 1993): 32.

The images extensively televised during the Gulf War, confirmed Paul Virilio's thesis

that modern warfare has become a matter of the "logistics of perception."

152

True, broadcasts

have included more traditional views of soldiers, planes, and tanks as seen from the outside, by a

reporter's video camera. But what we also saw were not just images of the war, but endless

images through which the war was carried out: video images from an infrared camera mounted

on a plane; video images from a camera installed on a weapon guided by a laser sensor; video in

its role as "battle damage assessment" where a weapon equipped with an imaging device follows

a weapon of destruction and records details of the damage. This was no longer a traditional

reporter's view of a battle. We saw what the soldiers themselves saw: the images that were their

only information about their targets. More often, in a strange case of identification, we witnessed

what was "seen" by a machine, a bomb, or a missile.

Television reports became catalogs of various vision technologies used to plan a battle, to

fight it, and to assess the results. They included satellite imagery of Iraqi territory; diagrams of

Iraqi radar activity over the course of a week; 3-D visualizations of weapons; a 3-D animation

computed from satellite data of a mountain region of Iraqi thought to contain a secret sight of

152

Paul Virilio, War and Cinema: the

Logistics of Perception (London: Verso, 1989).

biological weapons production. We saw U.S. soldiers equipped with night vision devices as well

as U.S. pilots and soldiers training in flight and battle simulators. We also saw military advisors

scrutinizing and interpreting aerial photographs and satellite imagery. We were also shown Iraq's

weak, antiquated resistance against the superior Western technologies of vision, image making,

and image analysis. Iraq fought back with camouflage weapons, decoys, and finally, according to

Western journalists, with oil fires set to confuse U.S. surveillance satellites and planes.

The Gulf War was the combat of surveillance against camouflage, visibility against

invisibility, human eye against computer eye. This warfare was indeed based on the "logistics of

perception," but we can describe its visual techniques even more precisely. Vision was employed

in a limited way as an instrument to capture and represent information about shapes of objects

and their positions in three-dimensional space.

The productive employment of this function of vision can be seen most dramatically in

the example of radar. A radar screen shows nothing but distances between the radar and other

objects in its field of vision. The colors and textures are not important. The guiding laser sensor

of a "smart" missile tracks the same information -- the distance between the missile and the

target. Similarly, it is the layout of a scene, distances between points, dimensions of objects

which are recorded by a satellite photograph.

The multitude of vision technologies which only "see" distances and dimensions are

paralleled by the reduction of human vision to the same single function. Thus, the images from

traditional or infrared video cameras, which contain information about various features of the

visual world, are used by a soldier or a pilot in a much more limited way -- as a source of

information about distances and shapes in a scene. What is the shape of an object on the horizon

-- is it an enemy or a friend? What is the distance between me and the enemy?

Finally, the technologies (such as radar) and techniques (using human sight in a function

of a radar) of obtaining information about shapes and distances in the real world are

complemented by 3-D computer graphics that simulate the real world on the basis of such

information. In their turn, simulated images generated with 3-D computer graphics are used in

flight and battle simulators, where the military eye undergoes its training to learn how to reduce

the world to geometry and topology.

The use of technologies which capture and visualize information about shapes and

distances today extends beyond warfare into all spheres of industry and science. The pilots of

commercial airlines utilize the same radar screens and are trained in the same flight simulators as

Air Force pilots.

153

Engineers and architects use 3-D graphics to visualize their designs. Oil-rich

153

As I write this in early 1993, many companies which yesterday supplied very expensive

simulators to the military are busy converting them into entertainment arcade-based systems. In
fact, one of the first such systems already commercially operating in a number of major cities,
including Chicago and Tokyo -- Battletech Center from Virtual World Entertainment, Inc. -- is
directly modeled on SIMNET (Simulation Network) developed by DARPA (Defense Advanced
Research Projects Agency). SIMNET can be thought of as the first model of cyberspace, the very
first collaborative VR environment. SIMNET consists of a number of individual simulators,
networked together, each containing a copy of the world database and the virtual representation
of all other participants in the conflict such as the Kuwaiti theater of operations. Similarly, a
Battletech Center comprises a networked collection of futuristic cockpit models with VR gear.
For $7 each, seven players can fight each other in a simulated environment. In another example,
in 1992 Lucas Arts has teamed up with Hughes Aircraft, combining the expertise in computer
games of the former with the expertise in building actual flight simulators of the latter, in a joint
venture aimed at theme-park type rides. On the connection between SIMNET and Battletech

Centers, see

Tony Reveaux, "Virtual

Reality Gets Real," New Media (January 1993): 36-41. On VR entertainment systems in the

countries pay a high price for satellite images of their territories to help them discover potential

oil fields. And, in automated factories, tireless robotic arms move parts under the control of

computerized vision systems trained to recognize the shapes of these parts.

In this chapter I will discuss twentieth century automation of the use of vision to record and

visualize geometric and topographic information -- to chart the distance and shape of real and

imaginary scenes and objects. This function of vision complements another function which was

the subject of the previous chapter: the visual representation of abstract ideas and their logical

relations. Here we will deal with the opposite problem -- how to represent a single, concrete

object or concrete scene. How, for instance, to represent not an idea of a chair but a particular

chair with a unique shape and dimensions. These two kinds of referents -- abstract ideas and

arguments, on the one hand, and concrete three-dimensional shapes, on the other hand, can be

related to two long-standing philosophical positions on what constitutes reality. For Plato,

sensible particulars were but pale reflections of Ideas or Forms. Aristotle criticized Plato,

declaring the primary substances were not Ideas but the individual things such as particular men

or animals. These opposing views have continued to be debated in scholastic philosophy, with

Plato's view giving rise to realism and Aristotle's to nominalism.

154

context of location-based entertainment -- arcades and theme parks -- see Richard Cook,
"Serious Entertainment," Computer Graphics World (May 1992): 40-48.

154

Allan Sekula relies on this distinction between the philosophical positions of realism and

nominalism in his discussion of different systems of police photography in the end of the

In scholastic philosophy the terms "realism" and "nominalism" refer to two opposing

ontological positions: realism claiming the primacy of Ideas and nominalism -- the primacy of

individual things.

155

Although in contemporary thought the meanings of "realism" and

"nominalism" are somewhat different, the original meanings are helpful in distinguishing two

fundamental kinds of representations: abstract ideas versus concrete objects.

I have suggested in the previous chapter that the desire to visualize abstract ideas and

logical relations was new to modernity. But it would not be reasonable to propose that the

function of vision which can be called visual nominalism -- to capture the identity of individual

objects and spaces by recording distances and shapes -- originates in the twentieth century as

well. After all, maps were used for many centuries; and the sophisticated graphic language of

engineering drawings was well developed by the nineteenth century. Yet, as this chapter will try

to demonstrate, in this century the automation of visual nominalism, which began with the

Renaissance perspective, entered a new stage. The sign of this automation is the multitude of

new technologies used to capture and visualize three-dimensional reality that have emerged since

nineteenth century.

Allan Sekula, "The

Body and the Archive," October 39 (1987): 3-64.

155

Antony Flew, ed., A Dictionary of Philosophy (London: Pan Books Ltd., 1984), 250, 299.

the middle of the twentieth century, such as radar, infrared imaging, laser sensors, CAT scans,

magnetic resonance imagining, 3-D computer graphics, and computer holography. Finally, since

the early 1960s work has been under way to automate vision completely, to create computer

vision systems that will recognize objects and interpret scenes automatically.

The development of these technologies has been accompanied by massive research into

the general problems of visual nominalism in computer science, experimental psychology, and

neuroscience. New formal mathematical techniques were developed to analyze images as a

source of depth information and, vice versa, to transform this information into realistic images.

The work on the automation of visual nominalism has also lead to new attention to particular

aspects of human perception. A new paradigm for the study of human vision emerged during the

1970s at MIT associated with the name of David Marr.

156

Within this paradigm, the goal of

human perception is taken to be the recognition of shapes, leading researchers to study

algorithms by which the brain "computes" shapes of objects from retinal input in the hope that

these algorithms can then be used by computer vision systems.

This chapter brings together seemingly unrelated visual technologies and techniques:

radar, 3-D computer graphics, and computer vision. I will suggest that they are ultimately related

by the function they all serve -- visual nominalism. Until this century, this function has been

performed by techniques of perspectival representation such as perspective and descriptive

geometry and, later, photography. Rather than thinking of them as artistic techniques, such

writers as William Ivins and Bruno Latour emphasize their significance for the development of

science, modern technology, and the economy. What these techniques allow best is to represent

precisely three-dimensional information on a two-dimensional plane, thus recording

quantitatively and efficiently the knowledge of the surrounding space. New twentieth century

techniques further extend the usefulness of these kinds of representations. Radar allows to obtain

156

David Marr, Vision (New York: W.H. Freeman and Company, 1982).

spatial information without the limitations of visibility (section 3); 3-D computer graphics add

the dimension of interactivity (section 4); and computer vision automates visual nominalism

altogether (section 5).

While there is no history of the burgeoning techniques of visual nominalism in the twentieth

century, the earlier history of these techniques has been studied in detail by art historians. The

problem of representing three-dimensional space upon a two dimensional plane has been a

subject of art historical research from at least the beginning of the century. It is quite remarkable

that the people who are today recognized as the founders of modern art history (Alois Riegl,

Heinrich Wšlfflin, and Erwin Panofsky), all defined the field as the history of the representation

of space. Working within the paradigms of cyclic cultural development and racial topology, they

related the representation of space in art to the spirit of entire epochs, civilizations and races. In

his 1901 Die SpŠtršmische Kunstindustrie, Riegl characterized humankind's cultural

development as the oscillation between two extreme poles, two ways to understand space, which

he called "haptic" and "optic." Haptic perception isolates the object in the field as a discrete

entity, while optic perception unifies objects in a spatial continuum.

157

Riegl's contemporary,

Heinrich Wšlfflin, similarly proposed that the temperament of a period or a nation expresses

itself in a particular mode of seeing and representing space. Wšlfflin's Principles of Art History

(1913) plots the difference between Renaissance and Baroque on five dimensions: linear --

painterly, plane -- recession, closed form -- open form, multiplicity -- unity, and clearness --

unclearness. Finally, another founder of modern art history, Erwin Panofsky, contrasted the

157

Michael Ann Holly, Panofsky and the Foundations of Art History (Ithaca: Cornell University

Press, 1984), 73.

2. "The most important event of the Renaissance."

"aggregate" space of the Greeks with the "systematic" space of the Italian Renaissance in the

essay Perspective as a Symbolic Form (1924-1925).

158

Panofsky established a parallel between

the history of spatial representation and the evolution of abstract thought. The former moves

from the space of individual objects in antiquity to the representation of space as continuous and

systematic in modernity; in Panofsky's neologisms, from "aggregate" space to "systematic"

space. Correspondingly, the evolution of abstract thought progresses from ancient philosophy's

view of the physical universe as discontinuous to the post-Renaissance understanding of space as

infinite, ontologically primal in relation to bodies, homogeneous, and isotropic -- in short, as

"systematic."

Panofsky's 1924-25 essay has been recognized as the first in what became an ever-

growing series of interpretations of perspective.

159

These interpretations related perspective to

every known characteristic of the modern period: economic, social, and philosophical. Some of

these interpretations acquired the character of dogmas, self-evident truths. For instance, the

correlation between Cartesian ideas of rational subjectivity in philosophy and Renaissance

perspective in visual art has been presented as one of the central metaphors in interpreting

158

Panofsky, "Die Perspektive als 'symbolische Form.'

159

The recent very important work on perspective is Hubert Damisch, L'origine de la

perspective (Paris: Flammarion, 1987).

Western modern culture. This concept became so prevalent that in his critical analysis Martin Jay

feels justified to name it "Cartesian perspectivalism."

160

Most interpreters have singled out an aspect of perspective, which then become for them

the core of its symbolic function. For instance, Pierre Francastel, and following him, John

Berger, and many film theorists of the 1960s and 1970s, focused on the mechanical character of

perspective and its capacity to organize the representation of reality around a single point of

view. They theorized a connection between Renaissance perspective and the development of

early capitalism, the emergence of instrumental consciousness which measures everything,

which is concerned with solidity and extension, with numbers and equivalents -- the

consciousness of market and accounting. Representations of cosmic, religious, unmeasurable

space were replaced by the pictures of cultivated, appropriated landscapes or of rooms cluttered

by precisely drawn objects signifying wealth. Perspective presented the world as ready to be

mastered, consumed, colonized -- the world originating in the eye of the spectator.

161

160

Martin Jay, "Scopic Regimes of

Modernity," Vision and Visuality, ed. Hal Foster (Seattle: Bay Press, 1988), 4.

161

Andrew Dudley, Concepts in Film Theory (Oxford: Oxford University Press, 1984), 31.

Other writers foregrounded different characteristics of perspective in their

interpretations. For instance, for Rudolf Arnheim, a pyramidal world portrayed by perspective

signifies a hierarchical conception of human existence.

162

On the other hand, for Fernande

Saint-Martin, perspectival representation which can only show objects at a distance, corresponds

to the "conceptual rather than sensorial, 'ideological' rather than concretely experienced" relation

of a subject to the world.

163

Panofsky's own account of perspective is more subtle than many of the consequent

interpretations it inspired. Panofsky correlates a mode of visual representation with a

philosophical system, but he is careful not to read one as an expression of the other. His

conclusion is open: perspective can be equally interpreted as subjective and objective, since it

presents space as an objective phenomenon but inevitably perceived from a subjective point of

view. Calling perspective "a two-edged sword," Panofsky writes:

Perspective subjects the artistic phenomenon to stable and even mathematically exact rules, but on
the other hand, makes the phenomenon contingent upon human beings, indeed upon the
individual: for these rules refer to the psychological and physical conditions of the visual
impression, and the way they take effect is determined by the freely chosen position of a
subjective "point of view."

164

162

Rudolf Arnheim, Art and Visual Perception. The New Version (Berkeley: University of

California Press, 1974), 295.

163

Fernande Saint-Martin, Semiotics of Visual Language (Bloomington and Indianapolis:

1990), 141.

164

And while modern perspective, as a coherent unity, is contrasted by Panofsky to the modes

of representation in antiquity and the Middle Ages, he also suggests that within the perspective
mode there exist many distinct variations. Working within the same mode of perspective,
different cultures and epochs take its meaning to be different: the Dutch represent "near space,"
Germans favor "oblique space," Italians explore "high space."

While Panofsky's essay had to wait almost six decades before its publication in English,

another equally important essay on perspective, "Obratnaya Perspektiva" (The Inverted

Perspective) still remains untranslated and unknown to Western art historians.

165

In his article

Joseph Leo Koerner briefly refers to the essay and regrets its unavailability in English (this is the

only reference I am aware of).

166

The essay was written by the Russian philosopher, linguist,

mathematician and art historian Father Pavel Florensky, who headed the department of the

Ervin Panofsky, Perspective as Symbolic

Form, 68-70.

165

Pavel Florensky, "Obratnaya Perspektiva (The Inverted Perspective)," Sobranie Sochineniy,

ed. N.A. Struve (Paris: YMCA-PRESS, 1985), 1: 117-192.

166

Koerner, "The Shock of the View," 37-38.

Analysis of Spatial Representation in the Arts at VKhUTEMAS from 1921-1924.

167

Florensky's

essay can be justly considered as the counterpart to Panofsky's work, completed in 1922, two

years before Panofsky's work. Both Panofsky and Florensky interpret any perspectival system as

convention and oppose the view that any particular system is natural.

168

At the same time, their

arguments tend to favor antithetical perspectival systems and world views: linear perspective and

Renaissance individualism in the case of Panofsky; inverted perspective and medieval religious

collectivism in the case of Florensky.

Panofsky narrates the evolution of the representation of space as an inevitable

progression towards the "systematic" yet subjective space of the Renaissance; thus, the

Renaissance perspective is given a privileged status, its final triumph as inevitable as the

progress of history itself. The discussion of medieval representation of space becomes for

167

VKhUTEMAS (State High Art-

Technical Studious) was the leading school of art and design in the USSR in the 1920s and the
center of avant-garde artistic culture.

168

Florensky writes: "Is it true that perspective, as it is claimed by its supporters, expresses the

true nature of things and therefore should be everywhere and always understood as the absolute
condition of artistic truth? Or is it only a scheme, only one among many schemes of
representation, corresponding not to the universal world view but only to one possible
understanding of the world, connected with a particular sensibility and cognizance?" Florensky,
"Obratnaya Perspektiva," 123.

Panofsky nothing more than a brilliant narrative device, a dramatic detour in what otherwise is

presented as a linear course from Antiquity to Renaissance: "When work on certain artistic

problems has advanced so far that further work in the same direction, proceeding from the same

premises, appears to bear fruit, the result is often a great recoil, or perhaps better, a reversal of

direction."

169

In Panofsky's interpretation, in other words, medieval representation of space

serves as a dialectical antithesis between ancient and Renaissance representations. Thus,

Panofsky claims, while it may appear that medieval representation gave up the advance of

Antiquity in repressing three-dimensionality, in fact it approached the Renaissance

understanding of space as infinite and homogeneous: "For if Romanesque painting reduced

bodies and space to surface, in the same way and with the same decisiveness, by these very

means it also managed for the first time to confirm and establish the homogeneity of bodies and

space."

170

Florensky, on the other hand, is mostly interested in the medieval representation of space

as manifested by the spatial constructions of Russian icons between the fourteenth and sixteenth

centuries. According to Florensky, seemingly heterogeneous spatial constructions of icons were

neither a result of the lack of artistic skill nor a step towards the Renaissance perspective.

171

169

Panofsky, Perspective as Symbolic Form, 47.

170

Ibid., 51.

171

The view of medieval art as unsystematic representation of space, against which Florensky

revolted, continues to persist. Thus, Samuel Edgerton writes in 1975: "Unlike the Renaissance
painter depicting his scene in perspective, the medieval artist viewed his world quite
subjectively. He saw each element in his composition separately and independently, and thus

Rather, they are result of a conscious and coherent system for representing reality which

corresponds to a coherent world view. It is not hard to see that for the orthodox and slavophile

Florensky, this world view is superior to Western post-Renaissance individualism. Russian

artists, like the artists of ancient Egypt and China, were aware of perspective but consciously

refused its power, choosing instead "religious objectivity and super-personal metaphysics." For

paid little attention to anything in the way of systematic spatial relationship between objects."

Samuel Edgerton, The Renaissance

Rediscovery of Linear Perspective (New York: Basic Books, 1975), 21.

Florensky, the Western Renaissance represents the beginning of the Fall: "When religious

stability of the world view begins to disintegrate, and the sacred metaphysics of collective

popular consciousness is segmented by individual vision of an individual subject from an

individual point of view, and only in a particular moment of time -- only then perspectivalism,

symptomatic of disunited consciousness, emerges."

172

Florensky's Slavophilism strikes us today as being politically biased; yet, this bias makes

us realize that Panofsky's seemingly more objective account is also biased in its privileging of

what Florensky labeled "bourgeois individualism." While Florensky offered a theoretical critique

of linear perspective, for his compatriots at the time this critique became a practical problem:

what new visual forms shall be suitable for the "first people's state"? Similarly identifying linear

perspective with "bourgeois individualism," El Lissitsky and Kazimir Malevich relied in their

designs on other types of perspectival construction such as parallel projection. In this type of

projection, all projection rays are parallel to each other, instead of originating from a single point

of view. Parallel projection, with its absence of a single point of view, was presented as a more

suitable way to symbolize the vision and knowledge of a collective. Paradoxically, this

"objective" point of view of a collective was expressed in the image (commonly encountered in

172

Florensky, "Obratnaya Perspektiva," 125-126. Justifying the value of reverse perspective as a

symbol of "super-personal metaphysics," Florensky offered a critique of the arguments later to
be found in Panofsky's "Die Perspektive als 'symbolische Form.'" For instance, he precisely
articulates the notion of "systematic" space, which, according to Panofsky, represented the
triumph of the Renaissance and was still absent in Medieval thought: "If we summarize
everything which is said in a formal sense against Medieval art, they are reduced to one
criticism: 'There is no understanding of space.' This criticism means that there is no spatial unity,
no scheme of Euclid's and Kant's space, the latter reduced, within the framework of painting, to
linear perspective and proportionality."

Malevich's writings) of the individual artist looking at the Earth from an infinitely far away point

in outer space, so that the convergence of rays is replaced by a parallel projection.

173

A.G. Rappoport, "El Lissitsky i ego 'Pun-

geometiya'" (El Lissitsky and his 'Pan-geometry'), Sovetskoe Iskusstvoznanie 25 (1989): 118-
119. Four decades later, LANDSAT satellites start bringing back images of Earth objects in
axonometric projections (because they were shot from high orbits) thus fulfilling Malevich's
designs.

The refusal of linear perspective by such artists as Malevich or Lissitsky is usually seen as just

one episode in what by now became a widely accepted narrative -- the shattering and negation of

perspectival space by modernist artists. According to this narrative, perspective was already

dead by the time art historians such as Panofsky had begun writing its history. Such narrative is

announced, for instance, in the very title of Pierre Francastel's Painting and Society. Birth and

Destruction of Plastic Space from the Renaissance to Cubism (1952). The opening section of

The Production of Space by Henri Lefebvre is equally authoritative:

The fact is that around 1910 a certain space was shattered. It was a space of common sense, of
knowledge (savoir), of social practice, or political power...a space, too, of classical perspective and
geometry, developed from the Renaissance onwards on the basis of the Greek tradition (Euclid,
logic) and bodied forth in Western art and philosophy, as in the form of the city and town.

174

Yet, if perspective disappeared from modern art, it survived as one of techniques of the

visual nominalism, a method for precisely representing the three-dimensional world on a two-

dimensional surface. In this role, it extended into many new domains and became the foundation

of new kinds of automated technologies of remote sensing and image synthesis.

To consider perspective in its role as a technique of visual nominalism we should turn to

another interpreter of perspective -- William Ivins and his short 1939 essay On the

Rationalization of Sight.

175

If Panofsky connects the development of perspective with the idea

of infinite space, abstract space existing prior to objects, Ivins on the contrary emphasizes that

perspective allows the creation of precise maps of three-dimensional reality, to record the shapes

of concrete objects and the layout of concrete spaces. It is the tool of a businessman and a

scientist rather than an artist.

In Ivins' definition, perspective is "a practical means for securing a rigorous two-way, or

reciprocal, metrical relationship between the shapes of objects as definitely located in space and

174

Henri Lefebvre, The Production of Space (Oxford: Blackwell Publishers, 1991), 25.

175

William M. Ivins, On the Rationalization of Sight (New York: Da Capo Press, Inc., 1975).

their representations."

176

This definition sums up a number of unique characteristics of

perspectival projection. First, the size of graphic marks in a perspectival image is proportionally

related to the dimensions of represented objects. For instance, a foot-long object would be

represented by a line of three inches, while an object twice as long would become a line of six

inches, and so on. Second, the ratio of sizes between these lines is proportional to the ratio of

sizes between the corresponding objects. Third, the perspectival image does not only record the

shapes and dimensions of objects but also contains information about their distance from one

another (in contrast to parallel projection, for instance). Finally, all these relationships between

the perspectival symbols on the one hand, and between the symbols and the corresponding

objects on the other hand, are controlled by a single rule or, to use the language which would

become common within a decade after the publication of On the Rationalization of Sight, by a

single algorithm. Given the coordinates of the point of view and the projection surface, the

projection of any point in space can be determined by mechanically following a sequence of

steps.

The most important quality of perspective foregrounded in Ivins' definition is the precise

and reciprocal relationship it establishes between objects and their symbols. We can go from

objects to symbols (two-dimensional representations); but we can also go from such symbols to

three-dimensional objects. Using the rules of perspective, an architect could create a visual

representation of an already existing building, change this representation, and the workers would

execute the specified changes in the building itself. Better yet, our architect can design a brand

new building -- not by painstakingly constructing a small model, where each change takes time

to implement -- but by creating a perspectival drawing, where each form is just a line and change

requires only a single movement of an eraser (or a single press of a key on a computer keyboard,

if the architect is working with a CAD system).

176

Ibid., 9.

Bruno Latour extended Ivins' idea by pointing out that a reciprocal relationship made

possible by perspective allows us not only to represent reality but also to control it.

177

Latour

sees perspectival representations as the "most powerful instrument of power," defined as the

ability to mobilize resources across space and time, to manipulate these resources at a distance.

For instance, we cannot measure the sun in space directly, but we only need a small ruler to

177

Bruno Latour, "Visualization and

Cognition: Thinking with Eyes and Hands," Knowledge and Society: Studies in the Sociology of
Culture Past and Present 6 (1986): 1-40.

measure it on a photograph (the perspectival image par excellence).

178

And even if we could fly

around the sun, we would still be better off studying the sun through its representations which we

can bring back from the trip -- because now we have unlimited time to measure, analyze, and

catalog them. We can also represent absent things and plan our movement through space by

working on representations: "One cannot smell or hear or touch Sakhalin Island, but you can

look at the map and determine at which bearing you will see the land when you send the next

fleet."

179

We can move objects from one place to another by simply moving their

representations: "You can see a church in Rome, and carry it with you in London in such a way

as to reconstruct it in London, or you can go back to Rome and amend the picture." Finally, as

Latour points out, "the two ways become a four-lane freeway! Impossible palaces can be drawn

realistically, but it is also possible to draw possible objects as if they were utopian ones." Real

and imagined objects can meet on a flat space of perspectival representation. All in all,

perspective is more than just a sign system, reflecting reality -- it makes possible the

manipulation of reality through the manipulation of its symbols.

178

Ibid., 22.

179

Ibid., 8.

Citing perspective as a perfect example of a new instrument of power and domination,

Latour proposes that the history of science and technology should be understood as a cascade of

such innovations in representation. Yet, his argument is already anticipated by Ivins. Ivins

concludes his essay by stating that the beginning of the rationalization of sight through the

discovery and the development of perspective "was the most important event of the

Renaissance."

180

It was more important than the fall of Constantinople or the Reformation or the

Counter Reformation because it made all other discoveries possible by launching a constantly

accelerating series of innovations.

181

The invention of perspective propelled modern empirical

science, for instance biology, which could now represent forms of nature with mathematical

precision. It also stimulated the rise of modern engineering and manufacturing by making

feasible the distribution of identical designs to far away places.

182

Ivins' approach stands in sharp contrast to the more traditional art historical analyses of

perspective by Panofsky and Francastel. They are concerned with perspective as an artistic form

and do not look beyond its history in art. Ivins, on the contrary, is concerned with visual culture -

- the techniques and technologies of visual representation available to a society at a given

moment and the fundamental role they play in shaping every aspect of society. Because he sees

these techniques and technologies

183

as ultimately determining social development, his approach

is deeply materialistic. Perspective, for Ivins, is not an artistic form reflecting social or

philosophical modernity; it is the very condition which makes modernity and modernization

possible.

184

For instance, he writes: "Many reasons are assigned for the mechanization of life

180

Ivins, On the Rationalization of Sight, 13.

181

Ibid., 7, 12.

182

Ibid., 13.

183

Ivins focuses on two fundamental developments: first, the invention of perspective and its

development into descriptive and perspective geometry; second, the invention of technique for
printing pictures which made possible easy duplication of exact copies of the same image.

184

This is the theme which Latour also greatly extends, proposing "a new kind of materialism"

which will account for social and scientific history through the analysis of available forms of
representation.

and industry during the nineteenth century, but the mathematical development of perspective was

absolutely prerequisite to it."

185

Ivins is also a materialist in the most extreme Marxist sense

because he suggests that the available forms of representation, such as perspective, determine the

development of abstract thought. He points out that as important as the development of

perspective was for picture making, "it is doubtless even more important to general thought,

because the premises on which it is based are implicit in every statement made with its aid."

186

This idea is repeated again at the end of the essay:

The constant extensions of the fields of usefulness of the pictorial symbol that is precisely
duplicable [through printing] and of the grammars of its use [development of perspective and
descriptive geometry] have had a most astonishing effect not only upon knowledge but upon
thought and its basic assumptions or intuitions...Relativity, which now in one form or another runs
throughout contemporary thought and practice, is in large measure a development of ideas that
were evolved through the study and use of projective transformations.

187

It is Ivins' general focus on visual culture, rather than art, that makes him attentive to the

current use of perspective and its continuing development and expansion. While Panofsky or

Francastel write about perspective in the past tense aware of its demise in modern art, Ivins on

the contrary notes "that today there are few sciences and technologies that are not predicated in

one way or another upon this power of invariant pictorial symbolization."

188

Modern designers,

scientists or engineers, of course, do not simply use perspective as it was formulated by Alberti

in the fifteenth century; they use much more sophisticated techniques. According to Ivins, the

rationalization of perspectival sight proceeded in two directions. On the one hand, perspective

became the foundation for the development of the techniques of descriptive and perspective

geometry which became the standard visual language of modern engineers and architects (fig.

11). On the other hand, the photographic technologies automated the creation of perspectival

185

Ivins, On the Rationalization of Sight, 12.

186

Ibid., 9. Emphasis mine - L.M.

187

Ibid., 13.

188

Ibid.

images. Both were accomplishments of the nineteenth century; in fact, both were developed

more or less simultaneously. Indeed, as Ivins points out, Ni pce and Talbot, the founders of

photography, were contemporaries of Monge and Poncelet, decisive figures in the development

of descriptive and perspective geometry.

Writing On Rationalization of Sight between 1936 and 1938, Ivins mentions such examples of

the contemporary use of perspective as aerial photographic surveillance, classification in the

field of archeology, and criminal detection.

189

However, all these applications of perspectival

techniques already existed in the nineteenth century and, by the 1930s, did not represent the

"latest" developments.

While photo reconnaissance was first employed systematically on a mass scale during

World War I, the interest in using photography for aerial surveillance existed since its invention.

FŽlix Tournachon Nadar, one of the most eminent photographers of the nineteenth century,

famous for his portraits, succeeded in exposing a photographic plate at 262 feet over Bi vre,

France in 1858. He was soon approached by the French Army to attempt photo reconnaissance

but rejected the offer. In 1882, unmanned photo balloons were already in the air; a little later,

they were joined by photo rockets both in France and in Germany. The only innovation of World

War I was to combine aerial cameras with a superior flying platform: the airplane.

190

189

Ibid., 12, 13.

190

Beaumont Newhall, Airborne Camera (New York: Hastings House, Publishers, 1969). For

critical histories of photo reconnaissance see Allan Sekula, "The Instrumental Image: Steichen at
War," in Photography against the Grain: Essays and Photo Works, 1973-1983 (Halifax: The
Press of the Nova Scotia College of Art and Design, 1984); Paul Virilio, War and Cinema: the
Logistics of Perception (London: Verso, 1989);

3. Radar: Seeing Without Eyes

In 1858, Albrecht Meydenbauer, a director of the German Government Building Office,

published a proposal to use photographs for scale measurement. His proposal was based on the

existence of a geometrical relationship between the photographic image and the object being

photographed. Why, for instance, climb a facade of a cathedral in order to measure it (as

Meydenbauer had to do, nearly once getting killed) when it is much safer to measure a

photograph? Additionally, wrote Meydenbauer, "some may find it hard to believe, but

Manuel De Landa, "Policing the

Spectrum," in War in the Age of Intelligent Machines (New York: Zone Books,

1991)

experience has proven than one can see, not everything, but many things, better in scale

measurement than on the spot." In 1885 the Royal Prussian Institute for Scale Measurement was

founded and the measurement of photographs of historic monuments became a frequent

practice.

191

In the field of criminal detection, perspectival technology of photography was routinely

employed decades before the time of Ivins' writing. In his 1844 Pencil of Nature William Fox

Talbot, one of photography's founders, comments on a beautiful calotype depicting several

shelves of china: "should a thief afterwards purloin the treasures -- if the mute testimony of the

picture were to be produced against him in court -- it would certainly be evidence of a novel

kind."

192

In 1883, Alphonse Bertillon, director of the Identification Bureau of the Paris

Prefecture of Police, developed a system for classification and identification of criminals that

relied on photographs. By 1893, Bertillon's system was already employed in the United States,

Belgium, Switzerland, Russia, much of South America, Tunisia, the British West Indies, and

Romania.

193

Modern perspective and descriptive geometry were fully developed in the first half of the

nineteenth century. Photography was already utilized by a number of professions for

measurement, identification, and classification by the end of the century. What was the next step

in the "rationalization of sight"?

In fact, while Ivins was writing his essay on perspective, across the Atlantic, in England,

work was already underway to install twenty radar stations on the east and southeast coasts to

provide surveillance of these air approaches. These radar installations turned out to be absolutely

essential in the coming war, allowing for the severely outnumbered Royal Air Force to defeat the

191

Harun Farocki, "Reality Would Have to Begin," Documents 1/2 (1992): 136-146.

192

Qtd. in Sekula, "The Body and the Archive," 6.

193

Sekula, "The Body and the Archive," 25, 35.

Luftwaffe in the Battle of Britain. Radar, the latest technology of visual nominalism, became

Britain's most important weapon.

194

The following anecdote about the "discovery" of radar, regardless of its historical

accuracy, is useful in understanding how radar works.

195

In 1922, two civilian scientists

employed by the Navy set out to conduct a routine communication experiment along the arm of

the Potomac river. As usual, they set a short-wave radio receiver and transmitter along the two

banks of the river. This time, however, a passing steamer interrupted the signal, creating an

electronic echo. The scientists immediately realized that such echoes could be used to deduce the

locations of ships entering a harbor.

This anecdote illustrates the main principle of radar operation. Radar is an acronym for

Radio Detection and Ranging. Like sound waves, radio waves create echoes when they are

interrupted by objects in their path. Radar transmits a radio wave in a particular direction. The

signal reflected back from the objects is picked up by an antenna. The time between the

transmission and the reception of the echo indicates the distance to the object; the direction the

antenna is pointing when the echo is received reveals the object's position in relation to the radar.

Detected objects appear as bright spots on a display watched by a radar operator.

196

Radar exemplifies the further rationalization of visual nominalism. All it sees and all it

shows are the positions of objects, 3-D coordinates of points in space, points which correspond

to submarines, aircraft, birds or missiles. Color, texture, even shape are disregarded. Instead of

Alberti's window, opening onto the full richness of the visible world, a radar operator sees a

194

For the history and technology of radar, I mainly relied on two sources: Echoes of War

(Boston: WGBH Boston, n.d.), videotape; McGraw-Hill Encyclopedia of Science & Technology:
an International Reference Work in Twenty Volumes Including Index (New York: McGraw-Hill,
1992.)

195

Echoes of War (Boston: WGBH Boston, n.d.), videotape.

196

Numerous variations of basic radar technology exist. For instance, in addition to active

radars which send a signal and detect energy reflected by objects there are also passive radars
which do not send a signal themselves. However, all radars have in common the use of
electromagnetic radiation (radio waves) to detect and measure objects in their vicinity.

screen, a dark field with a few bright spots. Here, the function of visual nominalism, which

perspectival image performed along with many other functions (representation of light, color,

shade, texture, etc.), is isolated and abstracted.

Radar serves a single function -- but it performs it more efficiently than any previous

perspectival technique or technology. First, the detection of objects' positions in space is no

longer limited by conditions of visibility. Vision is no longer confined by the distance resolvable

by the human or camera eye, since radar can detect objects hundreds of miles away, and at any

time of day or night. Second, this recording takes place instantly. Previously, military

commanders had to wait until pilots come back from surveillance missions and the film was

developed. The inevitable delay between the time of the observation and the delivery of the

finished image limited its usefulness: since by the time the image was produced, enemy positions

could have changed. Now, the imaging is instantaneous.

With radar, its vision not limited by visibility, its image literally tracking the referent,

rationalization of visual nominalism reaches a new stage. The usefulness of vision and of

imaging are greatly extended.

This expansion is achieved by fundamentally redefining the very foundations of vision

and imaging technologies. While radar came to supplement perspectival representation and

photography in terms of its function, it also represents a fundamental break with older

technologies. Like the photographic camera, radar captures the positions of objects and displays

them on a flat surface. But how it sees has little resemblance to what was commonly meant by

vision as defined by older optical apparatuses -- from human eyes to photography.

Instead of "looking" in a particular direction, radar sees all around, its antenna scanning

the area in a circular sweep. Instead of passively receiving light reflected by objects, radar sends

energy into the environment. Its vision is active; it can be compared to exploration of the

environment by a blind person.

197

Instead of relying, like photography, on the small region of

the electromagnetic spectrum to which our eyes are sensitive, it uses other regions, sending and

receiving waves of different lengths. Vision is no longer limited by the spectral capacity of the

human or camera eye; it is extended to include the whole of electromagnetic spectrum. The

visible becomes a small part of a larger field of sensory exploration of the environment.

Images produced by radar are also fundamentally different than those of previous

imaging technologies, from drawing paper to photographic film. The photographic image is an

imprint corresponding to a single referent or to a limited time of observation. With the invention

of the radar screen, the image surface is no longer static. For the first time it becomes constantly

updatable in real time. Here, with radar, we see for the first time the mass employment of a

fundamentally new kind of display, which soon comes to dominate modern visual culture, found

today in video or computer screen. What is new about such a display is that its image can change

in real time reflecting changes in the referent, be it the position of an object in space (radar), any

change in visible reality (live video) or changing data in computer memory (computer screen).

The radar screen changes, tracking the referent. But while it appears that the element of

time delay always previously present in imaging is eliminated, in fact time enters radar image in

a new way. With older photographic technologies, all parts of an image are exposed

simultaneously. Now the image is produced through sequential scanning, which means that its

contiguous parts in fact correspond to different moments in time. Like in an audio record,

moments in time become circular tracks on a surface.

Along with radar, many other technologies of visual nominalism came into existence following

the advances in electronics and computers during World War II: ultrasonic imaging,

197

There also exist passive radars which have no transmitters but are equipped to measure

signals from targets themselves. The active radar can be thought as a historical continuation of
projectors previously used by the military to artificially light up the scene of a battle to increase
visibility.

multispectral photography, multispectral imaging, infrared, sonar, magnetic resonance imaging,

and so on.

198

As radar, these technologies are effectively used to record distance, position,

layout, shape, and volume. Sonar, for instance, detects objects in the water by using sound

waves. Ultrasonic computer tomography uses sound waves and computer graphics to construct

images of body tissues. Multispectral photography isolates energy reflected from surfaces in a

number of given wavelength bands.

Coming to supplement perspectival techniques and technologies, the new technologies of

visual nominalism are more efficient but also more specialized. If older optical technologies

indiscriminately capture a number of dimensions of visual reality (color, shape, texture, tonality),

now a single dimension can be isolated. More precisely, a single dimension, in this case 3-D

information, is broken into distinct components recorded by different imaging technologies. The

technique of perspective allows one to record both the shape of objects and their relative

positions in relation to each other. Radar only records positions, it is blind to shape. On the other

hand, a laser or ultrasound range finder produces a depth map, treating as shapes both the actual

objects and the empty space between them.

This specialization can be seen as a side effect of the automation of visual nominalism.

This automation requires a priori assumptions about the nature of reality which become

embedded in different technologies. In the case of radar, reality is reduced to a set of geometric

points which have no extension. In the case of a range finder, reality is treated as if a skin, a

membrane, a continuous 3-D form. We may recall here the far-seeing prediction of the effects of

photography published in 1859 by Oliver Wendell Holmes: "Every conceivable object of Nature

and Art will soon scale off its surface for us. Men will hunt all curious, beautiful, grand objects,

198

See McGraw-Hill Encyclopedia of Science & Technology.

as they hunt cattle in South America, for their skins and leave the carcasses as of little worth."

199

Holmes' words seem to come true not only metaphorically (modern culture which values only

images) but also literally, as range finders methodically skin reality.

Along with the separation between technologies that record positions and technologies

that record shapes, another specialization emerges. Depending upon whether the position of the

observer in relation to objects in space is important or not, the technologies of visual nominalism

specialize in creating two different types of representations which can be called viewer-oriented

and viewer-independent.

200

The first kind of representation is exemplified by the radar screen, a

record of distances between enemy targets and the subject -- the point in the center of the screen.

Here, even more so than in the traditional perspective, the world is represented in relation to the

observer who becomes the center of the coordinate system. In the second type of representation,

the distance between the observer and the object is unimportant and in fact, the position of the

observer cannot even be deduced from the image. These are images which employ types of

projection other than perspective: maps, engineering and architectural plans, and aerial and

satellite photography, whether conventional, infrared or multispectral.

A third specialization, a third division of labor brought about by new technologies is that

between vision and imaging, which now become distinct processes. Engineering textbooks and

encyclopedias group many new technologies of visual nominalism under the term "remote

sensing," defined as the gathering and imaging of information without actual physical contact

with the object or area being investigated.

201

This definition clearly separates the two

199

Oliver Wendel Holmes, "The Stereoscope and the Stereograph," in Photography, Essays and

Images: Illustrated Readings in the History of Photography, ed. Beaumont Newhall (New York:
Museum of Modern Art, 1980), 60.

200

This distinction is related to but not synonymous with the distinction between "viewer-

centered" and "object-centered" representations, common since David Marr's Vision (New York:
W.H. Freeman and Company, 1982). It is also related, but again, not equivalent to, the difference
between perspectival and orthographic projections. The first records reality from a single point of
view, the second does not have a single point of view.

201

McGraw-Hill Encyclopedia of Science & Technology, 15: 311.

operations: the gathering of information (vision) and its presentation (imaging). The first

operation may have nothing to do with what is visible to the human eye, but in the second

operation the eye eventually comes into play since the gathered information has to be presented

to the human observer in visual form in order to be useful.

Technologies of remote sensing convert the data obtained by technologically augmented

biological senses (for instance, we can think of radar, which sends waves in order to detect

objects, as an augmented sense of touch) into information perceivable by the human eye. They

map other parts of the electromagnetic spectrum into a small visible part. The invisible (sound

waves, microwaves, ultraviolet waves) is converted into the visible. This is why these

technologies are also often called imaging technologies -- multispectral imaging, ultrasound

imaging, infrared imaging, and so on.

I have emphasized the continuity of function between perspectival drawing, photography, and

newer technologies such as radar and ultrasound imaging. Yet, not only these newer technologies

of visual nominalism serve the same function as the old -- to capture the identity of individual

objects and spaces by recording distances and shapes -- but they rely on the same principle of

perspective. We can justifiably refer to them as perspectival technologies if we understand

perspective as extending beyond the domain of the visible.

In his seminar "Of the Gaze as Object Petit a" Jacques Lacan argued that this is how

perspective should be understood.

202

He starts by reminding us that an image is anything

defined "by a point-by-point correspondence of two unities in space." To obtain an image of

something we do not have to rely on light or to operate in the domain of the visible. Nor do we

have to limit images to 2-D representations of 3-D reality. We can represent an object by another

202

Jacques Lacan, "On the Gaze as Objet Petit a," in The Four Fundamental Concepts of

Psycho-Analysis, ed. Jacques-Alain Miller (New York: W.W. Norton & Company, 1981), 67-
122.

object or represent a 2-D form by another form. All that is required is a rule to establish the

correspondence between the points of the object being imaged and the points on the image.

Similarly, says Lacan, "what is an issue in geometric perspective is simply the mapping

of space, not sight."

203

Perspective is one such rule, a particular method to establish a

correspondence between the object and its image. The method of perspective consists of

connecting a single point in space (usually referred to as subject's point of view) with a number

of points on the object by straight lines; the intersection of these lines with a plane creates an

image. It is coincidental that perspective, whether as a part of the human sight apparatus or as a

part of the photographic apparatus, works through light. Light travels in straight lines, therefore

it can be used to create perspectival images. But one can construct such images without light: "In

Descartes, dioptrics, the action of the eyes, is represented as the conjugated action of two

sticks."

204

As Lacan points out further on in the seminar, this idea that perspective is not limited

to sight alone but functions in other senses as well defines the classical discourse on perception:

"The whole trick, the key presto!, of the classic dialectic around perception, derives from the fact

that it deals with geometric vision, that is to say, with vision in so far as it is situated in a space

that is not in its essence the visual."

205

Lacan's clarification that the principle of perspective is not limited to the visible helps us

understand that the technologies of remote sensing function on the principle of perspective.

Regardless of their lengths, all waves travel in straight lines, and therefore points in space are

connected by straight lines to a point of reception (such as radar antenna) or recording (such as a

photographic camera). Radar, infrared imaging, sonar, and ultrasound are all part of what Lacan

called "geometric vision," perspectival vision extending beyond the visible.

203

Ibid., 86.

204

Ibid., 87.

205

Ibid., 94.

The technologies of remote sensing made it possible to map space without the limitations

of visibility. In the next section I will discuss the technology of 3-D computer graphics, which

further extended the usefulness of perspectival representations by making them interactive.

From the moment of adaptation of perspective, artists and draftsmen have attempted to aid the

laborious manual process of creating perspectival images.

206

Between the sixteenth and the

nineteenth century various "perspectival machines" were constructed. They were used to

construct particularly challenging perspectival images, to illustrate the principles of perspective,

to help students learn how to draw in perspective, to impress artists' clients, or to serve as

intellectual toys. Already in the first decades of the sixteenth century, DŸrer described a number

of such machines.

207

One device is a net in the form of a rectangular grid, stretched between the

artist and the subject. Another uses a string representing a line of sight (fig. 12). The string is

fixed on one end, while the other end is moved successively to key points on the subject. The

point where the string crosses the projection plane, defined by a wooden frame, is recorded by

two crossed strings. For each position, a hinged board attached to the frame is moved and the

point of intersection is marked on its surface. It is hard to claim that such a device, which gave

rise to many variations, made the creation of perspectival images more efficient, however the

images it helped to produce had reassuring mechanical precision. Other major types of

perspectival machines that appeared subsequently included the perspectograph, pantograph,

physionotrace, and optigraph.

206

For a survey of perspectival instruments, see Martin Kemp, The Science of Art (New Haven:

Yale University Press, 1990), 167-220.

207

Ibid., 171-172.

4. 3-D Computer Graphics: Interactive Perspectivalism

Why manually move the string imitating the ray of light from point to point? Along with

perspectival machines a whole range of optical apparatuses was in use, particularly for depicting

landscapes and in conducting topographic surveys. They included versions of camera obscura

from large tents to smaller, easily transportable boxes. After 1800, the artist's ammunition was

strengthened by camera lucida, patented in 1806.

208

Camera lucida utilized a prism with two

reflecting surfaces at 135û. The draftsman carefully positioned his eye to see both the image and

the drawing surface below and traced the outline of the image with a pencil.

Optical apparatuses came closer than previous perspectival devices to the automation of

perspectival imaging. However, the images produced by camera obscura or camera lucida were

only ephemeral and considerable effort was still required to fix these images. A draftsman had to

meticulously trace the image to transform it into the permanent form of a drawing.

With photography, this time-consuming process was finally eliminated. The process of

imaging physical reality, the creation of perspectival representations of real objects was now

mechanized. However, this mechanization did not affect other uses of perspectival

representation. According to Latour, the greatest advantage of perspective over other kinds of

representations is that it establishes a "four-lane freeway" between physical reality and its

representation. We can combine real and imagined objects in a single geometric model and go

back and forth between reality and the model. By the twentieth century, the creation of a

geometric model of both existing and imagined reality still remained a time consuming manual

process, requiring the techniques of perspectival and analytical geometry, pencil, ruler, and

eraser. Similarly, if one wanted to visualize the model in perspective, hours of drafting were

required. And to view the model from another angle, one had to start all over again. The

mechanization and automation of geometrical modeling and display were yet to come.

208

Ibid., 200.

Nothing perhaps symbolizes mechanization as dramatically as the first assembly lines installed

by Henry Ford in U.S. factories in 1913. The assembly line relied on two crucial principles. The

first was the standardization of parts, already employed in the production of military uniforms in

the nineteenth century. The second, newer principle, was the separation of the production process

into a set of repetitive, sequential, and simple activities that could be executed by workers who

did not have to master the entire process and could be easily replaced.

It seemed that mechanical modernity was at its peak. Yet, in the same year the Spanish

inventor Leonardo Torres y Quevedo had already advocated the industrial use of programmed

machines.

209

He pointed out that although automatons existed before, they were never used to

perform useful work:

The ancient automatons...imitate the appearance and movement of living beings, but this has not
much practical interest, and what is wanted is a class of apparatus which leaves out the mere
visible gestures of man and attempts to accomplish the results which a living person obtains, thus
replacing a man by a machine.

210

With mechanization, work is performed by a human but his or her physical labor is augmented

by a machine. Automation takes mechanization one step further: the machine is programmed to

replace the functions of human organs of observation, effort, and decision.

The term "automation" was coined in 1947; and in 1949 Ford began the construction of

the first automated factories. Mass automation was made possible by the development of digital

computers during World War II and thus became synonymous with computerization. A decade

later, the automation of the process of constructing perspectival images of both existent and non-

existent objects and scenes was well underway.

211

By the early 1960s Boeing designers already

209

Charles Eames and Ray Eames, A Computer Perspective: Background to the Computer Age

(Cambridge: Harvard University Press, 1990), 65-67.

210

Qtd. in ibid., 67.

211

I am not aiming here by any means to provide a full account of the history of 3-D computer

graphics or its various uses. I am concerned with computer graphics as one development, among
others, in the general move toward the rationalization of perspectival imaging. For a more

relied on 3-D computer graphics for the simulation of landings on the runway and of pilot

movement in the cockpit (fig 13).

212

By automating perspectival imaging, digital computers completed the process which

began in the Renaissance. This automation became possible because perspectival drawing has

always been a step-by-step procedure, an algorithm involving a series of steps required to project

coordinates of points in 3-D space onto a plane. Before computers the steps of the algorithm

were executed by human draftsmen and artists. With a computer, these steps can be executed

automatically and, therefore, much more efficiently.

The details of the actual perspective-generating algorithm which could be executed by a

computer were published in 1963 by Lawrence G. Roberts, then a graduate student at MIT.

213

The perspective-generating algorithm constructs perspectival images in a manner quite similar to

traditional perspectival techniques. In fact, Roberts had to refer to German textbooks on

perspectival geometry from the early 1800s to get the mathematics of perspective.

214

The

algorithm reduces reality to solid objects, and the objects are further reduced to planes defined by

straight lines. The coordinates of the endpoint of each line are stored in a computer. Also stored

are the parameters of a virtual camera -- the coordinates of a point of view, the direction of sight,

and the position of a projection plane. Given this information, the algorithm generates a

perspectival image of an object, point by point.

comprehensive account of 3-D computer graphics techniques, see J. William Mitchell, The
Reconfigured Eye: Visual Truth in the Post-Photographic Era (Cambridge, The MIT Press,
1992), 117-162.

212

Jasia Reichardt, The Computer in Art (London and New York: Studio Vista and Van

Nostrand Reinhold Company, 1971), 15.

213

L.G. Roberts, Machine Perception of Three-Dimensional Solids, MIT Lincoln Laboratory

TR 315, 1963; L.G. Roberts, Homogeneous Matrix Representations and Manipulation of N-
Dimensional Constructs, MIT Lincoln Laboratory MS 1405, 1965.

214

"Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and

Harvard," in SIGGRAPH '89 Panel Proceedings (New York: The Association for Computing
Machinery, 1989), 72.

The subsequent development of computer graphics can be seen as the struggle to

automate other operations involved in producing perspectival stills and moving images. The

computerization of perspectival construction made possible the automatic generation of a

perspectival image of a geometric model as seen from an arbitrary point of view -- a picture of a

virtual world recorded by a virtual camera. But, just like with the early perspectival machines

described by DŸrer, early computer graphics systems did not really save much time over

traditional methods. To produce a film of a simulated landing, Boeing had to supplement

computer technology with manual labor. As in traditional animation, twenty-four plots were

required for each second of film. These plots were computer-generated and consisted of simple

lines. Each plot was then hand-colored by an artist. Finished plots were filmed, again manually,

on an animation stand.

215

Gradually, throughout the 1970s and the 1980s, the coloring stage

was automated as well. Many algorithms were developed to add the full set of depth cues to a

synthetic image -- hidden line and hidden surface removal, shading, texture, atmospheric

perspective, shadows, reflections, and so on.

216

Today, these algorithms make possible the simulation of almost any object in such a way

that its computer image is indistinguishable from the photograph. Yet, this by itself does not

represent a radical break with older techniques of visual nominalism. A painter can paint the

215

This mixture of automated and pre-industrial labor is characteristic of the early uses of

computers for the production of images. In 1955 the psychologist Attneave was the first to
construct an image which was to become one of the icons of the age of digital visuality --
random squares pattern. A pattern consisted of a grid made from small squares colored black or
white. A computer generated table of random numbers has been used to determine the colors of
the square -- odd number for one color, even number for another. Using this procedure, two
research assistants manually filled in 19,600 squares of the pattern. Paul Vitz and Arnold B.
Glimcher, Modern Art and Modern Science (New York: Praeger Publishers, 1984), 234. Later,
many artists, such as Harold Cohen, used computers to generate line drawings which they then
colored by hand, transferred to canvas to serve as a foundation for painting, etc.

216

For further discussion of the problem of realism in computer graphics, see Lev Manovich,

"'Real' Wars: Esthetics and Professionalism in Computer Animation," Design Issues 6, no. 1
(Fall 1991): 18-25; Lev Manovich, "Assembling Reality: Myths of Computer Graphics,"
Afterimage 20, no. 2 (September 1992): 12-14.

same image, although it would take much longer. What does represent a radical break is

interactive computer graphics.

In 1962 Ivan Sutherland designed his now legendary Sketchpad program. With

Sketchpad, a human operator could create graphics directly on the computer screen by touching

the screen with a light pen. In the same year, ITEK began marketing its Electronic Drafting

Machine, an apparatus similar to Sketchpad.

217

Although both programs dealt only with 2-D

graphics, they introduced a new paradigm of interactive graphics: by changing something on the

screen, the operator changed the data in the computer's memory.

218

When this paradigm of interactive editing was combined with the algorithms of 3-D

graphics, a radically new way of using perspectival images emerged. This development was

more revolutionary than the automation of perspective construction per se. Indeed, traditional

draftsman could have accomplished what the computer at Boeing was doing -- generating plots

in perspective given 3-D database -- only more slowly. But now it became possible to change the

point of view of a virtual camera and see the corresponding changes in the perspectival image

after a short delay. It also became possible to build and modify 3-D models interactively and

observe the changes on the screen.

The emergence of interactive 3-D computer graphics started the race to eliminate the time

delay between the action of an operator and the displayed results. To be able to move through

virtual world in real time a computer has to generate 30 frames a second. The amount of

217

"Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and

Harvard," 51.

218

In fact, interactive computer graphics technology appeared earlier, although it was not

publicized. Already in the 1950s the Air Force used interactive CRT displays and light pens in
order to more efficiently process information obtained by radar. Both CRT displays and light
pens were designed at Lincoln Laboratory as part of the SAGE project. Using this technology,
Lincoln researchers created a number of computer graphics programs. They included programs
which made posible the display of brain waves (1957), of simulated planet and gravitational
activity (1960), as well as the creation of 2-D drawings (1958). "Retrospectives II: The Early
Years in Computer Graphics at MIT, Lincoln Lab, and Harvard," 42-54.

calculations involved increases proportionally as the model becomes more complex and as more

depth cues are added such as shading and shadows.

In this race for speed, which accelerated in the 1970s as synthetic images began to be

utilized in flight simulators, the algorithms of 3-D graphics were gradually transported from

software into hardware, each algorithm becoming a special computer chip. In 1966 Ivan

Sutherland and his colleagues started research on the first head-mounted display, a prototype of

Virtual Reality -- the ultimate interface for 3-D computer graphics. The research was

cosponsored by ARPA (Advanced Research Projects Agency) and the Office of Naval

Research.

219

Describing the project, Sutherland wrote in 1968:

The fundamental idea behind the three-dimensional display is to present the user with a
perspective image which changes as he moves. The retinal image of the real objects which we see
is, after all, only two-dimensional. Thus if we place suitable two-dimensional images on the
observer's retinas, we can create the illusion that he is seeing a three-dimensional object...The
image presented by the three-dimensional display must change in exactly the way that the image
of a real object would change for similar motions of the user's head.

220

The images seen in 1970 by the first users of Sutherland's system, such as a wireframe

cube or a square room, looked remarkably similar to those utilized by Renaissance artists in their

demonstration of perspective. Demonstrations of perspective always favored such simple

geometrical objects because they most clearly showed the achievement of perspective -- the

creation of a 2-D image of a 3-D object with foreshortening that matches our visual

experience.

221

The difference was that now the perspectival image of a cube was not fixed on a

sheet of paper; it appeared to an observer as floating in the air in front of her, changing its view

219

Howard Rheingold, Virtual Reality (New York: Simon & Schuster, Inc., 1991), 105.

220

Qtd. in ibid., 104.

221

Later, Renaissance artists would deliberately insert checkerboard floors in perfect

perspective in their paintings to impress their clients -- a move repeated in computer graphics,
where such checkerboard background receding towards infinity became a favorite motif.

in correspondence with the slightest movement of the observer's head. Perspectival imaging

became interactive.

In order to achieve this interactivity, special hardware was built. To speed up the

calculations necessary to update the synthetic image 30 times a second, Sutherland's group broke

down the process of 3-D image synthesis into a series of discrete steps. Each step was delegated

to a special electronic circuit: a clipping divider, a matrix multiplier, a vector generator. Later on,

such circuits became specialized computer chips, connected together to achieve real-time, high

resolution, photorealistic 3-D graphics. Silicon Graphics, one of the major manufacturers of

computer graphics hardware, labeled such a system "geometry engine."

The term appropriately symbolizes the second stage of the automation of perspectival

imaging. At the first stage, the photographic camera, with perspective physically built into its

lens, mechanized the process of creating perspectival images of existing objects. Now, with the

perspectival algorithm and other necessary geometric operations embedded in silicon, it becomes

possible to display and interactively manipulate models of non-existent objects as well.

In the next section, I will discuss the final stage in the automation of perspectivalism --

the development of computer vision.

In his papers, published between 1963 and 1965, Roberts formalized the mathematics necessary

for generating and modifying perspective views of geometric models on the computer. This,

writes William J. Mitchell, was "an event as momentous, in its way, as Brunelleschi's perspective

5. Computer Vision: Automation of Sight

demonstration."

222

However, Roberts developed techniques of 3-D computer graphics having in

mind not the automation of perspectival imaging but another, much more daring goal -- "to have

222

Mitchell, The Reconfigured Eye

, 118.

the machine recognize and understand photographs of three dimensional objects."

223

Thus, the

two fields were born simultaneously: 3-D computer graphics and computer vision, automation of

imaging and of sight.

The field of computer vision can be seen as the culmination of at least centuries-long

histories. The first is the history of mechanical devices designed to aid human perception, such

as Renaissance perspectival machines. This history reaches its final stage with computer vision,

223

"Retrospectives II: The Early Years in

Computer Graphics at MIT, Lincoln Lab, and

Harvard,"

57.

which aims to replace human sight altogether. The second is the history of automata, whose

construction was especially popular in the seventeenth and eighteenth centuries. Yet, despite

similarity in appearance, there is a fundamental difference between Enlightenment automata

which imitated human's or animal's bodily functions and the modern day robots equipped with

computer vision systems, artificial legs, arms, etc. As noted by Leonardo Torres, old automata,

while successfully copying the appearance and movement of living beings, had no economic

value. Indeed, such voice synthesis machines as Wolgang von Kempelen's 1778 device which

directly imitated the functioning of the oral cavity or AbbŽ Mical's T tes Parlantes (1783)

operated by a technician hiding offstage and pressing a key on a keyboard were used only for

entertainment.

224

When in 1913 Torres called for automata that would "accomplish the results

which a living person obtains, thus replacing a man by a machine" he was expressing a

fundamentally new idea of using automata for productive labor. A few years later, the brother of

the Czech writer Karel Capek coined the word robot from the Czech word robota, which means

"forced labor."

225

Capek's play R.U.R. (1921) and Fritz Lang's Metropolis (1927) clearly

demonstrate this new association of automata with physical industrial labor.

Therefore, it would be erroneous to conclude that, with computer vision, twentieth

century technology simply added the sense of sight to eighteenth century mechanical statues. But

even to see computer vision as the continuation of Torres', Capek's or Lang's ideas about

industrial automation which replaces manual labor would not be fully accurate. The idea of

224

Remko Scha, "Virtual Voices," MediaMatic 7, no. 1 (1992): 33. Scha describes two

fundamental approaches taken by the developers of voice imitating machines: the genetic method
which imitates the physiological processes that generate speech sounds in the human body and
the gennematic method which is based on the analysis and reconstruction of speech sounds
themselves without considering the way in which the human body produces them. While the
field of computer vision, and other fields of artificial intelligence, first clearly followed
gennematic method, in the 1980s, with the growing popularity of neural networks, there was a
shift towards the genetic method -- direct imitation of the physiology of the visual system. In a
number of laboratories, scientists begin to build artificial eyes which move, focus, and analyze
information exactly like human eyes.

225

Eames and Eames, A Computer Perspective, 100.

computer vision became possible and the economic means to realize this idea became available

only when automation entered its second stage after World War II. The attention turned from the

automation of the body to the automation of the mind, from physical to mental labor. This shift

to the automation of mental functions such as vision, hearing, reasoning, problem solving is

exemplified by the very names of the two new fields that emerged during the 1950s and 1960s --

artificial intelligence and cognitive psychology. The latter gradually replacing behaviorism, the

dominant psychology of the "Fordism" era. The emergence of the field of computer vision is a

part of this cognitive revolution, a revolution which was financed by the military escalation of

the Cold War.

226

This connection is solidified in the very term "artificial intelligence" which

may refer simultaneously to two meanings of "intelligence": reason, the ability to learn or

understand, and information concerning an enemy or a possible enemy or an area. Artificial

intelligence: artificial reason to analyze collected information, collected intelligence.

In the 1950s, faced with the enormous task of gathering and analyzing written,

photographic, and radar information about the enemy, the CIA and the NSA (National Security

Agency) began to fund the first artificial intelligence projects. One of the earliest projects was a

Program for Mechanical Translation, initiated in the early 1950s in the attempt to automate the

monitoring of Soviet communications and media.

227

The work on mechanical translation was

probably the major cause of many subsequent developments in modern linguistics, its move

towards formalization; it can be discerned in Noam Chomsky's early theory which, by

postulating the existence of language universals in the domain of grammar, implied that

translation between arbitrary human languages could be automated. The same work on

mechanical translation was also one of the catalysts in the development of the field of pattern

recognition, the precursor to computer vision. Pattern recognition is concerned with

226

De Landa, "Policing the Spectrum," 194-203.

227

Ibid., 214.

automatically detecting and identifying predetermined patterns in the flow of information. A

typical example is character recognition, the first stage in the process of automating translation.

Pattern recognition was also used in the U.S. for the monitoring of Soviet radio and telephone

communication. Instead of listening to every transmission, an operator would be alerted if

computer picked up certain words in the conversation.

As a "logistics of perception" came to dominate modern warfare and surveillance and as

the space race began, image processing became another major new field of research. Image

processing comprises techniques to improve images for human or computer interpretation.

228

1964, the space program for the first time used image processing to correct distortions in the

228

The first paper on image processing was published in 1955.

L.S.G. Kovasznay, and H.M. Joseph,

"Image Processing," Proceedings of IRE 43 (1955): 560-570.

pictures of the Moon introduced by a on-board television camera of Ranger 7.

229

In 1961, the

National Photographic Interpretation Center (NPIC) was created to produce photoanalysis for the

rest of the U.S. intelligence community and, as Manual De Landa points out, by the end of the

next decade computers "were routinely used to correct for distortions made by satellite's imaging

sensors and by atmospheric effects, sharpen out-of-focus images, bring multicolored single

images out of several pictures taken in different spectral bands, extract particular features while

diminishing or eliminating their backgrounds altogether..." De Landa also notes that computer

analysis of photographic imagery also became the only way to deal with the pure volume of

intelligence being gathered: "It became apparent during the 1970s that there is no hope of

keeping up with the millions of images that poured into NPIC...by simply looking at them the

way they had been looked at in World War II. The computers therefore also had to be taught to

compare new imagery of a given scene with old imagery, ignoring what had not changed and

calling the interpreter's attention to what had."

230

229

Rafael C. Gonzalez, and Paul Wintz,

Digital Image Processing (Reading, Mass.: Addison-Wesley Publishing Company, 1977), 2.

230

Qtd. in De Landa, "Policing the Spectrum," 200.

The techniques of image processing, which can automatically increase an image's

contrast, remove the effects of blur, extract edges, record differences between two images, and so

on, greatly eased the job of human photoanalysts. And the combining of image processing with

pattern recognition made it possible in some cases to delegate the analysis of photographs to a

computer. For instance, the technique of pattern matching used to recognize printed characters

can also be used to recognize objects in a satellite photograph. In both cases, the image is treated

as consisting of two-dimensional forms. The contours of the forms are extracted from the image

are then compared to templates stored in computer memory. If a contour found in the image

matches a particular template, the computer signals that a corresponding object is present in a

photograph.

A general computer vision program has to be able to recognize not just two-dimensional

but three-dimensional objects in a scene taken from an arbitrary angle.

231

Only then it can be

used to recognize an enemy's tank, to guide an automatic missile towards its target or to control a

robotic arm on the factory floor. The problem with using simple template matching is that "a

two-dimensional representation of a two-dimensional object is substantially like the object, but a

two-dimensional representation of a three-dimensional object introduces a perspective projection

that makes the representation ambiguous with respect to the object."

232

While pattern

recognition was working for images of two-dimensional objects, such as letters or chromosomes,

a different approach was required to "see" in 3-D.

Roberts' 1965 paper "Machine Recognition of Three-dimensional Solids" is considered to

be the first attempt at solving the general task of automatically recognizing three-dimensional

231

Within the field of computer vision, a scene is defined as a collection of three-dimensional

objects depicted in an input picture. David McArthur, "Computer Vision and Perceptual
Psychology," Psychological Bulletin 92, no. 2 (1982): 284.

232

Paul R. Cohen and Edward A. Feigenbaum, eds., The Handbook of Artificial Intelligence

(Los Atlos, CA: William Kaufmann, Inc., 1982), 3: 139.

objects.

233

His program was designed to understand the artificial world composed solely of

polyhedral blocks -- a reduction of reality to geometry that would have pleased CŽzanne. Using

image processing techniques, a photograph of a scene was first converted into a line drawing.

Next, the techniques of 3-D computer graphics were used:

Roberts' program had access to three-dimensional models of objects: a cube, a rectangular solid, a
wedge, and a hexagonal prism. They were represented by coordinates (x, y, z) of their vertices. A

233

L.G. Roberts, "Machine perception of

three-dimensional solids," Optical and Electo Optical Information Processing, ed. J.T. Tippett

(Cambridge: The MIT Press, 1965)

program recognized these objects in a line drawing of the scene. A candidate model was selected
on the basis of simple features such as a number of vertices. Then the selected model was rotated,
scaled, projected, and matched with the input line drawing. If the match was good, the object was
recognized, as were its position and size. Roberts' program could handle even a composite object
made of multiple primitive shapes; it subtracted parts of a line drawing from the drawing as they
were recognized, and the remaining portions were analyzed further.

234

Was this enough to completely automate human vision? This depends upon how we

define vision. The chapter on computer vision in The Handbook of Artificial Intelligence (1982)

opens with the following definition: "Vision is the information-processing task of understanding

a scene from its projected images."

235

But what does "understanding a scene" mean? With

computer vision research financed by the military-industrial complex, the definition of

understanding becomes highly pragmatic. In the best tradition of the pragmatism of James and

Pierce, cognition is equated with action. The computer can be said to "understand" a scene if it

can act on it -- move objects, assemble details, destroy targets. Thus, in the field of computer

vision "understanding a scene" implies two goals. First, it means the identification of various

objects represented in an image. Second, it means reconstruction of three-dimensional space

from the image. A robot, for instance, need not only recognize particular objects, but it has to

construct a representation of the surrounding environment to plan its movements. Similarly, a

missile not only has to identify a target but also to determine the position of this target in three-

dimensional space.

It can be seen that Roberts' program simultaneously fulfilled both goals. His program

exemplified the approach taken by most computer vision researchers in the following two

decades. A typical program first reconstructs the three-dimensional scene from the input image

and then matches the reconstructed objects to the models stored in memory. If the match is good,

the program can be said to recognize the object, while simultaneously recording its position (fig.

14).

234

Cohen and Feigenbaum, The Handbook of Artificial Intelligence, 3: 129.

235

Ibid., 127.

A computer vision program thus acts like a blind person who "sees" objects (i.e.,

identifies them) by reading their shapes through touch. As for a blind person, understanding the

world and the recognition of shapes are locked together; they cannot be accomplished

independently of one another. With computer vision, visual nominalism, which, after all, is just a

particular use of vision, attains unprecedented importance. Now, to see means to see shapes and

distances. Visual nominalism and vision itself have become equated.

I have presented the new twentieth century technologies of visual nominalism as building upon

perspective, extending its powers in space and time. Technologies of remote sensing, such as

radar, extended perspective beyond the realm of the visible. 3-D computer graphics speeded up

and automated design and perspectival display of the models of both real and imagined objects.

With the emergence of the field of computer vision, perspectival sight reaches its

apotheosis and at the same time begins its retreat. At first computer vision researchers believed

that they could invert the perspective and reconstruct the represented scene from a single

perspectival image. Eventually, they realized that it is often easier to bypass perspectival images

altogether and use other means as a source of three-dimensional information.

Latour points out that with the invention of perspective it became possible to represent

absent things and plan our movement through space by working on representations. To quote

him again, "one cannot smell or hear or touch Sakhalin island, but you can look at the map and

determine at which bearing you will see the land when you send the next fleet."

236

Roberts'

program extended these abilities even further. Now the computer could acquire full knowledge

of the three-dimensional world from a single perspectival image! And because the program

determined the exact position and orientation of objects in a scene, it became possible to see the

reconstructed scene from another viewpoint. It also became possible to predict how the scene

236

Latour, "Visualisation and Cognition," 8.

would look from an arbitrary viewpoint.

237

Finally, it also became possible to guide

automatically the movement of a robot through the scene.

Roberts' program worked using only a single photograph -- but only because it was

presented with a highly artificial scene and because it "knew" what it could expect to see.

Roberts limited the world which his program could recognize to simple polyhedral blocks. The

shapes of possible blocks were stored in a computer. Others simplified the task even further by

painting all objects in a scene the same color.

However, given an arbitrary scene, composed from arbitrary surfaces of arbitrary color

and lighted in an arbitrary way, it is very difficult to reconstruct the scene correctly from a single

perspectival image. The image is "underdetermined." First, numerous spatial layouts can give

rise to the same two-dimensional image. Second, "the appearance of an object is influenced by

its surface material, the atmospheric conditions, the angle of the light source, the ambient light,

the camera angle and characteristics, and so on," and all of these different factors are collapsed

together in the image.

238

Third, perspective, as any other type of projection, does not preserve

many geometric properties of a scene. Parallel lines turn into convergent lines; all angles change;

equal lines appear unequal. All this makes it very difficult for a computer to determine which

lines belong to a single object.

Thus, perspective, which until now served as a foundation of new technologies of visual

nominalism, becomes the drawback which needs to be overcome. Perspective, this first step

towards the rationalization of sight (Ivins) has eventually become a limit to its total

rationalization -- the development of computer vision.

The realization of the ambiguities inherent in a perspectival image itself came after years

of vision research. In trying to compensate for these ambiguities, laboratories began to scrutinize

237

Cohen and and Feigenbaum, The Handbook of Artificial Intelligence, 3: 141.

238

Ibid., 128.

the formal structure of a perspectival image with a degree of attention unprecedented in the

history of perspective. In 1968 Adolpho Guzman classified the types of junctions that appear in

line representations after he realized that a junction type can be used to deduce whether regions

of either side of a junction line were part of the same object.

239

In 1986 David Lowe presented

a method to calculate the probability that a particular regularity in an image (for instance, parallel

lines) reflects the physical layout of the scene rather than being an accident due to a particular

viewpoint.

240

All other sources of depth information such as shading, shadows or texture

gradients were also systematically studied and described mathematically.

Despite these advances, a single perspectival image remained too ambiguous a source of

information for practical computer vision systems. An alternative has been to use more than one

image at a time. Computer stereo systems employ two cameras which, like human eyes, are

positioned a distance apart. If the common feature can be identified in both images, then the

position of an object can be simply determined through geometric calculations. Other systems

use a series of continuous images recorded by a video camera.

But why struggle with the ambiguity of perspectival images at all? Instead of inferring

three-dimensional structure from a two-dimensional representation, it is possible to measure

depth directly by employing various remote sensing technologies. In addition to video cameras,

modern vision systems also utilize a whole range of different range finders such as lasers or

ultrasound.

241

Range finders are devices which can directly produce a three-dimensional map of

an object. The same basic principle employed in radar is used: the time required for an electro-

magnetic wave to reach an object and reflect back is proportional to the distance to the object.

But while radar reduces an object to a single point and in fact is blind to close-by objects, a range

239

Ibid., 131.

240

David Lowe, Three-Dimensional Object Recognition from Single Two-Dimensional Images,

Robotics Report 62 (New York: Courant Institute of Mathematical Sciences, New York
University, 1986).

241

Cohen and Feigenbaum, The Handbook of Artificial Intelligence, 3: 254-259.

finder operates at small distances. By systematically scanning the surface of an object, it directly

produces a "depth map," a record of an object's shape which can be then matched to geometric

models stored in computer memory thus bypassing the perspectival image altogether (fig. 15).

What began as separate developments -- to automate the recording of three-dimensional

information, the generation of perspectival views, and the identification of objects -- eventually

converged. Remote sensing, 3-D computer graphics, and computer vision form a closed circle.

As we have seen, algorithms of 3-D computer graphics were first developed by Roberts in order

to solve the general problem of computer vision. From that point on, the two fields developed in

parallel. The field of computer graphics was gradually learning to simulate more and more

"realistic" images of reality, adding shading, shadows and texture to initial wireframe drawings.

Meanwhile, the field of computer vision was learning to deal with shading, shadows, and texture

as sources of depth information.

In effect, the goal of computer vision is to undo what 3-D computer graphics aims to

achieve. The former tries to reconstruct a scene from its photographic image, to deduce viewer

independent information about the objects: their shapes and dimensions. The goal is to "produce

an object centered representation that is independent of the particular details of the viewing angle

or photographic process."

242

The aim of 3-D computer graphics is exactly the opposite: given

the objective information about the scene (shapes and positions of objects, their materials,

direction of light) to produce its image which is virtually indistinguishable from a photograph.

At the foundation of this mutual dependence is the shared geometrical model of reality as

a set of surfaces having a particular orientation, position, and curvature. The circle is completed

by the technologies of remote sensing that, by producing depth maps of real scenes, can

242

David Peat, Artificial Intelligence: How Machines Think (New York: Baen Enterprises,

1985), 230.

automatically reduce them to geometric models. Portable shape acquisition cameras are already

being developed that, instead of taking snapshots of a scene, will automatically generate

geometric models which then can be used for computer vision or 3-D computer graphics.

243

Technologies of remote sensing can automatically create a depth map of reality. Using

such depth map as input, 3-D computer graphics can be used to generate an image which, in its

turn, can act as input to a computer vision system. Computer vision then can reconstruct

"objective" information from an image, a model which can act as a source for further 2-D

computer graphics synthesis. A series of mirrors, constructions and reconstructions are triggered

by each other, with speed only limited by the processing power of "geometry engines."

In this chapter I discussed the twentieth century automation of the function of vision which I

have called visual nominalism -- capturing the information about shapes and distances and

representing this information in two-dimensional displays. Radar and other technologies of

remote sensing extend the capacities of human vision beyond the limits of visibility, making it

possible to track positions of objects in real time. Techniques of 3-D computer graphics allow the

interactive display and construction of geometric models of both real and non-existent objects.

Finally, computer vision automates the recognition of objects and the reconstruction of three-

dimensional scenes from their images.

In this discussion, the concept of perspective played a special role for a number of

reasons. I relied on the familiarity of this concept to explain the principles of operations of newer

twentieth century technologies of visual nominalism. I have also relied on Ivins' account of

perspective and its extension by Latour to suggest that the Renaissance's adaptation of

243

Rheingold, Virtual Reality, 229, 252.

6. Conclusion

perspective represented the first step in the automation of visual nominalism. While other

cultures used sophisticated methods of space mapping, the importance of perspective lies not in

its representational superiority but in its algorithmic character. This algorithmic character

enabled the gradual development of visual languages of perspective and descriptive geometry

and, in parallel, of perspectival machines and technologies, from a simple net described by

DŸrer to photography and radar. And when digital computers made possible mass automation in

general, automation of perspectival vision and imaging followed soon. It is this automation of

vision and imaging which gave the West superiority in the "logistics of perception," and

contributed to its victory in the Gulf War.

A two-dimensional representation containing information about geometry and

topography allows one to control reality across space and time. The advantage of perspective is

that it makes it possible to present this information in an easily readable form within a single

image.

In the West, until this century, perspective formed the foundation of techniques and

technologies of visual nominalism. New twentieth century technologies of visual nominalism

extend perspective, utilizing to the extreme its inherent qualities such as the algorithmic

character and the reciprocal relationship it establishes between reality and representation. For

instance, the perspective algorithm, a foundation of both computer graphics and computer vision,

is used to generate perspectival views given a geometric model and to deduce the model given a

perspectival view. In another example, technologies of remote sensing, such as radar, extend

perspectival vision beyond the visible, relying on the fact that "what is an issue in geometric

perspective is simply the mapping of space, not sight" (Lacan). Yet, while giving rise to new

technologies of "geometric vision," perspective also becomes a limit to the final automation of

sight -- recognition of objects by a computer. Finally, it is displaced from its privileged role,

becoming just one among other techniques of space mapping and visualization.

In order to automate sight, to replace human vision with machine vision, a new

understanding of vision became necessary. David Marr's Vision (1980) has been the most

influential account of this paradigm, shared by computer scientists and psychologists. The book

opens with this statement:

244

There is nothing "plain" about this definition of vision. Marr projects the goals of computer

vision onto human vision: the identification of objects represented in an image and the

reconstruction of their positions in three-dimensional space. The larger part of the book is

devoted to the discussion of algorithms by which the human nervous system may accomplish this

reconstruction.

In this way, nature was redefined as the most useful instrument in its own replacement.

What David Marr, and many others have assumed, is that the key to successful computer vision

systems lies in emulating the algorithms hidden in human nervous system. If only these

algorithms, created by nature during millions of years of evolution, could be understood, the way

to replace human vision by computer vision will be open.

The automation of vision is a part of the overall process of industrial automation after

World War II. Automation affects the modern understanding of vision in two crucial ways. On

the one hand, automation entails the replacement of human cognitive functions by a computer,

such as the substitution of vision by computer vision. Here, visual nominalism becomes the

dominant paradigm for the study of vision. On the other hand, the automation also involves the

integration of human and machine in new man-machine systems, such as radar installations or

pilot cockpits. The need to describe the performance of human and machine components in the

244

Marr, Vision, 3.

same terms leads to the emergence of another paradigm -- vision as information processing --

which will be discussed in the following chapter.

Writing in 1927, L‡szl— Moholy-Nagy makes a comparison between engineering and design:

245

In the opening chapter, where this quotation already appeared, I have traced one line of research

into "the physiological and psychological laws of visual effectiveness," suggesting that

investigations of the psychological effects of basic colors and elemental forms, conducted in

experimental psychology since the second half of the nineteenth century made possible the idea

of a rational visual language composed from simple elements. In theory, this model called for the

enumeration of visual elements and the description of their effects in order to compose

dictionaries of "visual effectiveness." In practice, it translated into visual works which at first

only contained and then gradually became fully composed of simple abstract forms. Such theory

and practice were already pursued in the nineteenth and early twentieth centuries by a number of

artists (Seurat, Signac, Kandinsky, Klee, Mondrian) and critics interested in "scientific

aesthetics" (Blanc, Henry). However, in the 1920s, when artists found themselves in the roles of

the designers of mass communication, the idea of a scientifically grounded visual language

acquired new urgency and importance. It was no longer a question of scientific aesthetics, of a

work of art producing an aesthetic effect in an individual spectator. Now it became a question of

rationalizing mass communication -- a question of economic and political importance. The

design of mass communication was too important to be left to an artist's intuition. Therefore,

245

L‡szl— Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era,

ed. Christopher Phillips (New York: Aperture, 1989), 87.

Chapter 4: The Engineering of Vision from INKhUK to MIT

1. Not Artists but Engineers

Moholy-Nagy wrote about the need to precisely "engineer" photographic advertisements, and the

Bauhaus welcomed experimental psychologists. The trend reached its extreme in Soviet Russia,

where artists thought that they could control not just the consumer habits of segments of society

(as in the West), but the consciousness of the whole country --"affecting mass psyche, organizing

the will of the class" (Tretyakov). Here the discourse of engineering visual communication

became most systematic, translating into the establishment of a number of psychological

laboratories at various art institutions, such as VKhUTEMAS, GAKhN, and the State Institute of

Artistic Culture. Their goal was to create a fully rationalized visual language of mass

communication where each visual element is capable of communicating a meaning, producing an

emotion or causing a behavioral response. In the words of a paper entitled "Engineerism"

presented at INKhUK: "visual sensations as such will concertedly shape the human being as an

organized unit."

246

This is one development, one trajectory -- from Blanc's theories and Fechner's

psychophysical experiments to INKhUK and VKhUTEMAS -- where the dream of controlling

the masses through perception was systematically pursued. This trajectory leads from Wundt's

psychological laboratory to the "laboratory" explorations of constructivists and rationalists; and

from the laboratory to the sites of mass communication -- a movie hall, a printed page, an

architectural space, a stadium, a city street with posters, advertisements, and placards.

In this concluding chapter I will follow the next part of this trajectory, the next stage in

the rationalization of visual communication: from the avant-garde's fantasies of psychophysical

control of the masses to information theory, engineering psychology, and human-machine

interfaces. The visual language advocated by Rodchenko or Lissitsky -- elemental abstract forms,

each having a predictable effect -- seemed the most effective solution to the problem of

246

Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York:

Rizzoli International Publications, 1990), 81.

rationalizing vision. But after World War II, a new kind of rationalization became necessary. A

new paradigm was now needed to encompass not only human communication but also

communication between a human and a machine, the two linked in a human-machine system. It

is the emergence of this paradigm which is the subject of this chapter.

The new paradigm, called information theory, emerged in the 1920s in response to the

growth of modern telecommunications. The theory was originally developed by engineers in

order to measure the efficiency of communication systems, such as telephone, radio, and

television. However, in the late 1940s, the theory left its engineering context and was embraced

by aestheticians, linguists, social scientists, and semioticians. Most importantly, it was taken up

by psychologists faced with the practical task of designing human-machine systems which

increasingly came to dominate both the battlefield and the workplace after World War II (radar

screen, aircraft controls, computer terminals of the automated factory). As human vision became

the main channel of communication between human and machine, it also came to be understood

in terms of information theory which now provided a new quantitative model of a human

observer. Thus the dreams of the avant-garde for an engineering approach to visual

communication were fulfilled.

The avant-garde was concern with rationalizing vision, with creating visual communication

according to scientific principles. In this avant-garde artists likened themselves to modern

engineers, as can be seen from Moholy-Nagy's statement. But what is a modern engineer?

The engineering profession emerged in the 1870s as the liaison between science and

industry. The job of an engineer was to put any industrial or corporate process on a scientific

basis in order to achieve the maximum output with a minimum investment of time, materials,

and energy. In other words, the job of an engineer was to make a process efficient, regardless of

what industry s/he worked in. A 1933 book on engineering defines it in this way: "Engineering is

the science and art of efficient dealing with materials and forces...it involves the most

economical design and execution of a vast number of important undertakings, assuring, when

properly performed, the most advantageous combination of accuracy, safety, durability, speed,

simplicity, efficiency and economy possible for the design and service."

247

The avant-garde

artists aimed to apply these principles of engineering to the field of mass visual communication.

In 1921, during a heated discussion at INKhUK (Institute of Artistic Culture) over how to

turn their fellow artists into "constructivists," who would utilize modern engineering methods for

construction -- be it the construction of consumer goods or mass psyche, industrial design or

mass propaganda -- participants expressed doubts about whether traditional artists could

accomplish this transition from "art into life." Boris Arvatov suggested that accomplished artists

should become political activists, and the young ones would have to go to the polytechnic.

248

More categorical was the opinion of Sternberg, who, after asking the same question, "What are

artists to do?," answered it in these words, "They're good for nothing, they should be dealt with

the way the Cheka deals with counterrevolutionaries."

249

A few minutes later in the discussion

he suggested a more humane solution, the solution which was repeatedly put into practice in

regard to Soviet non-conformist artists decades later -- their confinement to a mental institution.

"All the artists who conduct youth groups, and who are going to continue their work and perhaps

teaching the way they have in the past, we should really stuff a hospital with these artists."

250

Arvatov's request that artists should get their training in an engineering school, and

Sternberg's warning that those who refuse to comply would be shot, illustrate the grave

seriousness with which many Soviet avant-garde artists of the 1920s took the ideal of

engineering as a model for their own practice. Consider the term constructivism itself. In

247

Qtd. in Theodore Hoover and Charles Fish, The Engineering Profession (Stanford: Stanford

University Press, 1941), 416. Emphasis mine -- L.M.

248

Allison, Art Into Life, 74.

249

Ibid.

250

Ibid., 77.

Russian, konstruktor (the one who constructs) is a synonym for engineer. This allowed a number

of artists to call themselves konstruktors and to refer to their movement as constructivism. The

term conveniently emphasized that, like engineers, they would construct utilitarian objects or

propagandistic works on the basis of scientific, rational principles, in contrast to traditional

artists who were said to rely on inspiration. Translated into English, the word constructivism

loses its close association with engineering, leading to a somewhat different interpretation of the

entire movement.

In the early 1920s, constructivist artists competed to be the best at embodying the ideal of

an engineer. Vertov announces his desire to become "kinok-engineer, controlling cameras over

distance."

251

Eisenstein, who, like Sternberg, graduated from an engineering school before the

1917 Revolution, compares the theatrical apparatus (actors' performances, sound effects,

lighting) to the machinery (orudie obrabotki) which a director should use to form the viewer -- as

a milling machine forms machinery parts.

252

Later, he likens the effect of a film to the work of

an engine -- a significant analogy since engineering was historically identified with the design of

engines.

253

Tatlin photographs himself next to his already legendary design for the monument to

the Third International. The monument, with its two intertwining lattice spirals 400 meters high,

looks like an Eiffel Tower put at an angle, eternally moving into the sky, a symbol appropriate to

signify the idea of permanent revolution, the (future) industrial might of the new state and, at the

same time, the engineering skills of its designer. Finally, Lissitsky and his students design

another triumph to engineerism -- an inclined lattice girder entitled "Lenin Tower."

251

Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 143.

252

Sergei Eisenstein, "Montazh atraktsionov (Montage of attractions)," LEF 3 (1923): 71.

253

A book on the engineering profession published in 1941 quotes these dictionary definitions

of engineering: "engineering: originally, the art of making engines; in its modern and extended
sense, the art and science by which mechanical properties of matter are made useful to man in
structures and machines;" "engineering: the art and science of making, building, or using engines
and machines, or of designing and constructing public works or the like requiring special
knowledge of materials, machinery, and the laws of mechanics." Hoover and Fish, The
Engineering Profession, 416.

Here is another device: similar to traditional portraits where the identity of a person was

established by depicting the instruments of his or her craft, constructivists' portraits and self-

portraits prominently feature engineering instruments. Lissitsky's self portrait: on the background

of graphed paper -- his face and through it -- a hand holding a compass, its leg touching a

perfectly drawn square. The title of this photomontage is Konstruktor. Even more dramatic is

another Lissitsky's montage, a portrait of Tatlin (fig. 16). Tatlin is standing on a chair, holding a

long pole -- a ruler. By his feet are mathematical symbols (an integral, a square root), a curve

crossing axis lines. Behind him is a blackboard with more symbols. And, a monstrosity: a huge

compass, widely open, growing straight from Tatlin's eye. A man-compass -- measuring,

calculating, constructing.

Constructing what? "My reaction to the participation of the artist in production is

positive," declared architect Vesnin during one INKhUK discussion, "but an artist must look

after his own affairs, and his affair is the [psycho-physiological] effect of form on [human]

consciousness." In other words, Vesnin is saying: let us leave the engineering of machines and

mass-produced objects to real engineers. Our business, the business of designers is the

engineering of perception, the engineering of visual communication -- the creation of

architecture, mass spectacles, posters, designs, photomontages, which would produce calculated

bodily responses, emotions, and meanings in the viewers. "The modern engineer has created

brilliant objects: the bridge, the steam engine, the airplane, the crane,...the modern artist must

create objects that are equal to them in power, intensity, and potential in the context of their

psycho-physiological impact as an organizing element in man's consciousness."

254

The job of an

artist is to engineer vision, and to engineer vision means to affect the viewer with engineering

precision, predictability, and effectiveness.

254

Qtd. in Anatole Senkevitch, Jr., "The Sources and Ideals of Constructivism in Soviet

Architecture," in Art Into Life: Russian Constructivism 1914-1932, ed. Nicholas H. Allison
(New York: Rizzoli International Publications, 1990), 173-174.

In this statement from the early 1920s, Vesnin envisions an engineer as the designer of

physical objects -- the bridge, the steam engine, the airplane, the crane. But already in the same

decade a new kind of engineer appears, whose job is to design not objects but communication

systems -- telephone, radio, television. A new paradigm appears as well -- information theory

which becomes the scientific basis of communication engineering. In the next section, I will

follow its emergence.

In 1922 Moholy-Nagy created his famous "Telephone Pictures." Here is his account:

In 1922 I ordered by telephone from a sign painter five paintings in porcelain enamel. I had the
factory's color chart before me and I sketched my paintings on graph paper. At the other end of the
telephone the factory supervisor had the same kind of paper, divided into squares. He took the
dictated shapes in correct position. (It was like playing chess by correspondence.)

255

There is no conclusive proof that this event actually took place. At any rate, the story helped

Moholy-Nagy get a teaching job at the Bauhaus the next year, where he could fully pursue his

vision of using the latest communication systems as artistic tools. His colleague wrote of the

Bauhaus atmosphere: "There is incessant talk of cinema, optics, mechanics, projection and

continuous motion...Is this the atmosphere in which painters like Klee and some others of us can

go on developing? Klee was quite depressed yesterday when talking about Moholy."

256

The story of "Telephone Pictures" became one of the myths of modern art, or rather, of

the modern struggle against the Romantic notion of art: to refuse subjectivity, the unique touch

255

L‡szl— Moholy-Nagy, The New Vision (New York: Wittenborn, 1947), 79. The three

existing accounts of this episode are discussed in Eduardo Kac, "Aspects of the Aesthetics of
Telecommunications," in SIGGRAPH '92 Visual Proceedings, ed. John Grimes and Gray Lorig
(New York: The Association for Computing Machinery, 1992), 52-53.

256

Qtd. in Otto Stelzer, "Moholy-Nagy and His Vision," in L‡szl— Moholy-Nagy, Painting,

Photography, Film (1925; reprint, Cambridge: The MIT Press, 1973), 146.

2. Information Theory: an Engineer Analyzes Communication

of the artist's hand and the artist's eye (exemplified in Bauhaus by Klee) and to embrace the

mass-produced, the industrial, the mechanical, electrical, electronic, and digital -- from

Duchamp's ready-made to Op Art, Kinetic Art, copy art, telecommunication art, fax art,

computer art -- in short, the tradition of industrial art, throughout the twentieth century feeding

on the latest technical developments in communication and imaging. And Moholy-Nagy

intended the story this way, as a myth, a beginning more important than Duchamp's ready-

mades. Indeed, not just art as chess, as a logical game (as with Duchamp), but -- art as "playing

chess by correspondence," a communication over distance between a sender and a receiver who

do not occupy the same physical space.

Let us analyze Moholy-Nagy's story from the point of view of telecommunication -- the

communication over distance. It mentions three different communication systems. First, the most

ancient: communication with the help of signs, such as traffic or street signs (Moholy-Nagy

orders his pictures at the sign factory). Second, more modern: the postal service. Third, still a

recent development for the 1920s: the telephone. As communication systems developed, they

made possible the transmission of more information over longer distances in a shorter period of

time. The effectiveness of road signs was limited by the distance at which they were visible, and

the amount of information they carried was minimal. With the postal service, this distance limit

was overcome, but a significant time delay remained. This delay was finally eliminated by the

telephone.

In the story, Moholy-Nagy communicated his design to the factory supervisor over the

telephone, dictating coordinates and colors of squares. He was able to do this only by fitting his

design to a pre-existing factory grid, treating the grid's squares as minimal units of his

composition. (Moholy-Nagy, in other words, fully embraced digital visuality and computer art

long before digital computers.) But could he, in 1922, transmit this design as a picture -- and in

real time? And what if the picture was an arbitrary image, for instance a photograph of a real

scene, rather than a by-product of the communication code?

The rapid transmission of photographic pictures was just becoming possible, utilizing the

same method employed by Moholy-Nagy -- digital representation. In 1921, digitized newspaper

photographs began to be regularly sent between London and New York over a submarine cable

in less than three hours. On one end, a picture was segmented using a grid, and the tone of each

cell was coded; on another end, the received code would drive a special printer fitted with

typefaces simulating different tones.

257

A few years later, Moholy-Nagy included the examples of this "wireless telegraphed

photography" in his Painting, Photography, Film, proudly putting them at the very end of the

photographic section of the book to represent the latest achievement in mechanized tele-seeing

(fig. 17). Yet, this system was not fast enough for the age of "the film; the electric sign,

simultaneity of sensorily perceptible events."

258

The speed of wireless telegraphed photography

was limited by the channel capacity of an older communication system -- telegraphy. Images

contained much more information than telegraphic cable was originally designed to carry, so

speed had to be sacrificed. Throughout the 1920s, engineers in Russia, the U.S., England, and

Germany were working on a new system to transmit images in real time called television --

vision over distance.

The explosive development of modern communication systems -- telegraphy, telephone,

radio, and television, along with their growing economic importance -- urged the creation of a

general communication theory.

259

With telephone, radio, and television becoming the

foundations of whole new industries in the earlier decades of the century, the question of

257

M. D. McFarlane, "Digital Pictures Fifty Years Ago," Proc. IEEE 60, no. 7 (1972).

258

Moholy-Nagy, Painting, Photography, Film, 39.

259

For a history of the developments leading to the creation of mathematical theory of

communication, see Colin Cherry, On Human Communication, 2nd ed. (Cambridge: The MIT
Press, 1968), 41-52.

efficiency came to the foreground. Engineers needed a quantitative theory which could define

and measure the commodity that was communicated over telegraph, telephone, and radio

networks. Quantitative measures were also required in order to compare the capacities of various

communication systems (for instance, wire versus radio transmission) and to evaluate their

performance.

The distant precursor to modern communication theory can be found in the centuries-old

interest in inventing secret codes for military and diplomatic communication. But with the

appearance of modern telecommunications, the problem of designing codes to make the

deciphering of messages more difficult for the enemy was joined by a new problem, that of

designing codes which could make everyday communication more efficient. Already by 1825,

telegraphists designated certain commonly used messages, such as commercial expressions and

greetings, by single numbers, thus making coded messages shorter. The famous dot-dash code,

introduced by S.F.B. Morse in 1832, associated the most commonly used letters with the shortest

dot-dash symbols, thus making coded messages use even fewer symbols.

260

The problem of how to devise the most efficient methods of encoding, transmitting, and

decoding signals was greatly intensified by the development of television in the 1920s.

Television, the "Far Seer," required the simultaneous transmission of a much greater amount of

information than in previous communication technologies: "masses of information had to be read

off at high speed at the camera end, transmitted, and reassembled in the receiver."

261

The great

channel capacity required for television spurred new theoretical studies, and by the end of the

1920s, the fundamental ideas of the modern mathematical theory of communication (which also

came to be called information theory or statistical communication theory) were formulated.

(Significantly, it was the great promise and also the great difficulty in the development of a

260

Cherry, On Human Communication, 36-37.

261

Ibid., 42.

system for visual telecommunication -- television -- which was decisive in the rise of the new

paradigm of a quantitative approach to communication.)

The essential ideas of information theory were presented in 1927 at the International

Congress of Telegraphy and Telephony by R.V.L. Hartley of the Bell Telephone Laboratories in

a paper entitled "Transmission of Information." Hartley proposed "to set up a quantitative

measure whereby the capacities of various systems to transmit information can be compared"

and to "discuss its application to systems of telegraphy, telephony, picture transmission and

television over both wire and radio paths."

262

Hartley's theory conceptualized the transmission

of a message as the successive selection of signs from a set, known to both the sender and the

receiver. If the receiver knows beforehand which of the symbols will be transmitted, no new

knowledge (no information) is communicated. The amount of information is proportional to the

uncertainty about which sign can be received. If a particular message is made up from N signs

which can be chosen from a total set of S signs, then there are S

distinct possibilities. Hartley

proposed that the "quantity of information" (H) of such a message is H=N log

S. Hartley also

established one of the basic laws of information theory: the total amount of information a system

can transmit "is proportional to the product of the frequency range which it transmits by the time

available for transmission."

263

In other words, the more signals a system transmits

simultaneously (frequency range, or bandwidth) and the longer the time available for

transmission, the greater its information capacity.

With Hartley's paper the two fundamental tools of information theory were now in place:

a quantitative measure of information and an equation specifying the information capacity of a

communication system. It may seem that the theory defined information in an abstract and

counter-intuitive way by associating it with the freedom to make a choice (on the part of the

262

R.V.L. Hartley, "Transmission of Information," Bell System Tech. Journal 7 (1928): 535.

263

Ibid., 554.

sender) and with uncertainty (on the part of the receiver). Yet, the definition of information is

quite pragmatic from the point of view of an engineer concerned with making a communication

system efficient. What the measure of information (H) defines is the minimum number of binary

choices which enables the receiver to reconstruct the message, or conversely, which is required

to transmit it.

264

In other words, "it is a measure of the minimal effort by which message can be

transmitted."

265

The development of information theory was further motivated by the work of scientists

and engineers on communication and electronic systems during World War II, the most

important of which was radar. In its modern form, the theory was formulated by Claude

Shannon; in 1949, his papers, together with a non-technical overview by Warren Weaver, were

published in what became the bible of the coming information age -- Mathematical Theory of

Communication.

266

In his contribution, Weaver extended Shannon's model of information transmission,

created to describe physical systems of telecommunications, such as telegraphy or radio, to a

more general model of any communication situation (fig. 18). To emphasize this generality, his

description of the model deliberately juxtaposes examples of telecommunication systems and

human communication:

The information source selects a desired message out of a set of possible messages...The

selected message may consist of written or spoken words, or of pictures, music, etc.
The

transmitter changes this message into the signal which is actually sent over a

communication channel from the transmitter to the receiver. In the case of telephony, the channel

264

Each binary choice constitutes one bit of information. Weaver writes: "if one has available

say 16 alternative messages among which he is equally free to choose, then since 16=2

so that

log

16=4, one says that this situation is characterized by 4 bits of information." Claude E.

Shannon and Warren Weaver, The Mathematical Theory of Communication (Urbana: The
University of Illinois Press, 1949), 100-101.

265

D. Gabor, "A Summary of Communication Theory," in Proceedings of a Symposium on

Applications of Communication Theory, London, 1952, ed. Willis Jackson (London: Butterworth
Scientific Publications, 1953), 2.

266

Shannon and Weaver, The Mathematical Theory of Communication.

is a wire, the signal a varying electrical current on this wire; the transmitter is the set of devices
(telephone transmitter, etc.) which change the sound pressure of the voice into the varying
electrical current. In telegraphy, the transmitter codes written words into sequences of interrupted
currents of varying lengths (dots, dashes, spaces). In oral speech, the information source is the
brain, the transmitter is the voice mechanism producing the varying sound pressure (the signal)
which is transmitted through the air (the channel). In radio, the channel is simply space (or the
aether, if any one still prefers that antiquated and misleading word), and the signal is the
electromagnetic wave which is transmitted.
The

receiver is a sort of inverse transmitter, changing the transmitted signal back into a

message, and handing this message on to the destination. When I talk to you, my brain is the
information source, yours the destination; my vocal system is the transmitter, and your ear and the
associated eighth nerve is the receiver.

In the process of being transmitted, it is unfortunately characteristic that certain things are

added to the signal which were not intended by the information source. These unwanted additions
may be distortions of sound (in telephony, for example), or static (in radio), or distortions in shape
or shading of picture (television), or errors in transmission (telegraphy or facsimile) etc. All of
these unwanted changes in the transmitted signal are called noise.

267

It is not accidental that the principal examples in this description -- telegraphy, telephony, and

oral speech -- all have to do with written or verbal communication. By the middle of the century,

both the engineer and the public still associated electronic communication with telephone and

radio, and this is why the example of human communication which naturally comes to Weaver is

speech. It would be few decades before electronic communication would become synonymous

with the communication of images (television, teleconferencing, multimedia, teleoperators,

ISDN, cyberspace); before the terms of information theory, such as channel, bit, bandwidth will

enter the speech of journalists -- telling us everyday about yet another development in visual

telecommunication -- as well as the speech of policymakers, endlessly debating who will control

these channels and how wide their bandwidth will be.

The Mathematical Theory of Communication appeared in 1949 and within years the theory was

taken up by linguists, psychologists, social scientists, and even art historians. Thus, Roman

267

Ibid., 98-99. Emphasis in the original.

3. The Influence of Information Theory or the Ideology of the Code

Jakobson, who as a professor at MIT was close to the source of the paradigm, adopted the model

for linguistics and then semiotics.

268

Even quicker to embrace the paradigm were experimental

psychologists.

269

As information theory was transformed from a tool of the communication engineer into a

broad intellectual paradigm, three crucial developments took place. First, the theory was

extended to understand not just communication within an electronic system, but also

communication between humans as well as between humans and machines. Second, the theory

was adopted to describe the meaning and effect of communication -- its semantic and pragmatic

aspects. Third, the basic assumption of the theory, that communication is a one-way process, was

extended to theorize social communication. In this section, I will first consider these three

developments in turn.

The first development is connected with the figure of Norbert Wiener, a professor of

mathematics at MIT who supervised Shannon's research. In 1948 Wiener, published Cybernetics

or Control and Communication in the Animal and the Machine which proposed that seemingly

different entities -- a machine, an animal, a human, an animal collective, a human collective --

can be conceived of as communication systems. Wiener also equated communication and

control, thus opening a way to think of social, political, and economic problems in information-

cybernetic terms. He justified this equation in this way:

When I communicate with another person, I impart a message to him, and when he communicates
back with me he returns a related message which contains information primarily accessible to him

268

Roman Jakobson, (1956), "Metalanguage as a Linguistic Problem," Presidential address

delivered at the Annual Meeting of the Linguistic Society of America, 1956, published in The
Framework of Language (Ann Arbour: Michigan Studies in the Humanities, 1980): 81-92;
Roman Jakobson, (1960), "Closing Statement: Linguistics and Poetics," in Semiotics. An
Introductory Anthology, ed. Robert Innis (Bloomington: Indiana University Press, 1985).

269

G.A. Miller, Language and Communication (New York: McGraw-Hill Book Co., Inc.,

1951); D.E. Broadbent, Perception and Communication (Oxford: Pergamon Press, 1958); H.
Quastler, ed., Information Theory in Psychology: Problems and Methods (Glencoe, IL.: Free
Press, 1955); F. Attneave, Applications of Information Theory to Psychology (New York: Holt,
1959).

and not to me. When I control the actions of another person, I communicate a message to him, and
although this message is in the imperative mood, the technique of communication does not differ
from that of a message of fact. Furthermore, if my control is to be effective I must take cognizance
of any messages from him which may indicate that the order is understood and has been
obeyed.

270

In this explication Wiener talks about human communication, but his realization that the

"problems of control engineering and of communication engineering were inseparable" emerged

during the war when he worked on anti-aircraft artillery systems. Wiener realized that the key to

successful control lies in feedback -- the modification of behavior of a system (be it human or a

machine) by taking into account the results of the actual (rather than predicted) performance. The

results of the performance are communicated back to the control mechanism, making it adjust

the system's behavior. A prototypical example of a feedback mechanism is a self-guided shell:

"When we desire a motion to follow a given pattern, the difference between this pattern and the

actually performed motion is used as a new input to cause the part regulated to move in such a

way as to bring its motion closer to that given by the pattern."

271

The authors of information theory always insisted that the theory was only concerned

with the conditions for the correct and efficient transmission of signals constituting the message,

and not with its meaning, interpretation, or effect. Hartley points out that "in estimating the

capacity of the physical system to transmit we should ignore the question of interpretation."

272

Shannon also begins by insisting that "semantic aspects of communication are irrelevant to the

engineering problem."

273

Finally, Weaver starts his essay included in The Mathematical Theory

of Communication by distinguishing between the three levels of a communication problem:

technical (how accurately the signs can be transmitted), semantic (how accurately signs convey

270

Norbert Wiener, (1950), The Human Use of Human Beings. Cybernetics and Society (New

York: Avon Books, 1967), 24-25.

271

Norbert Wiener, Cybernetics. Of Control and Communication in the Animal and the

Machine (New York: John Wiley & Sons, Inc., 1948), 13.

272

Hartley, "Transmission of Information," 538.

273

Shannon and Weaver, The Mathematical Theory of Communication, 3.

the desired meaning), and pragmatic (how effective the message is in affecting the behavior of

the receiver). Weaver refers to these levels as A, B, and C.

274

He next emphasizes that the

mathematical theory of communication as developed by Shannon applies only to the first level,

yet also immediately suggests that it is deeply relevant for the second and third. Impressed by the

fact that the technical aspect of a communication process is finally controlled by a precise

mathematical theory, Weaver clearly hopes that the theory can somehow be extended to handle

the elusive questions of semantics and pragmatics with the same quantitative precision.

Throughout the essay he promises a detailed discussion of how this can be done. Weaver never

keeps his promise; instead, in a number of places he suddenly announces that perhaps no

modifications will be necessary. The announcements appear abruptly at the end of sentences,

explosions in what is otherwise the dry rhetoric of a scientific discourse. "It is the purpose of this

concluding section to review the situation, and to see to what extent and in what terms the

original section was justified in indicating that the progress made at Level A is capable of

contributing to levels B and C, was justified in indicating that the interrelation of the three levels

is so considerable that one's final conclusion may be that the separation into the three levels is

really artificial and undesirable." "It is almost certainly true that a consideration of

communication on levels B and C will require additions to the schematic diagram on page 97

[fig. 18], but it seems equally likely that what is required are minor additions, and no real

revision."

275

The temptation to extend information theory to handle the meaning and effect of

communication proved too great to resist. This was understandable given the historical context of

the Cold War during which the theory matured and attracted public attention. This context is not

hard to see in Weaver's descriptions of different levels of communication: "The semantic

274

Ibid., 96.

275

Ibid., 114, 115. Emphasis mine -- L.M.

problem has wide ramifications if one thinks of communication in general. Consider, for

example, the meaning to a Russian of a U.S. newsreel picture." "The problem of effectiveness

involves...all the psychological and emotional aspects of propaganda theory."

276

Even more

revealing is the opening paragraph of the essay, where within a few lines we are transported from

artistic communication, painting, theater, and ballet, to the theater of the new cybernetic warfare,

which emerged during World War II and now protected the U.S. against a Soviet attack:

The word communication will be used in a very broad sense to include all of the procedures by
which one mind may affect another. This, of course, involves not only written and oral speech, but
also music, the pictorial arts, the theatre, the ballet, and in fact all human behavior. In some
connections it may be desirable to use a still broader definition of communication, namely, one
which would include the procedures by means of which one mechanism (say automatic equipment
to track an airplane and to compute its probable future positions) affects another mechanism (say a
guided missile chasing this airplane).

277

In 1952 Y. Bar-Hillel, who was working in the field of automatic language translation (which,

as I mentioned in the previous chapter, also developed during Cold War with the promise to

automate the surveillance and analysis of Soviet military communication and media), extended

information theory to define and measure "semantic information" of the propositions in a

language system. In the same years, the Shannon-Weaver model (fig. 18) was adopted in the

U.S. by the growing field of mass communication studies as the theoretical foundation of the

field. Weaver's hope that the model, originally designed to deal solely with the technical aspects

of telecommunication, could be applied to describe semantic and pragmatic levels as well, was

fully realized: the model was adopted to describe the process of mass communication: sender

(film studio, TV station, publisher) -- message (a film, a television program, a newspaper story) -

- receiver (film viewer, television viewer, newspaper reader).

278

276

Ibid., 97.

277

Ibid., 95.

278

Wilbur Schramm, "How Communication Works," in The Process and Effects of Mass

Communication (Urbana: University of Illinois Press: 1954): 3-26.

In this transfer, the basic postulate of the information theory became an ideology.

Shannon begins his exposition of information theory by stating that "the fundamental problem of

communication is that of reproducing at one point either exactly or approximately a message

selected at another point."

279

American scholars of mass communication applied the same

postulate to the semantic level. Even a textbook on mass communication published in the 1970s,

still states: "if the meaning of the destination is isomorphic with the meaning of the source which

originated the act, then communication can be said to have taken place."

280

With such a

definition, any discrepancy between the codes of a sender and a receiver becomes undesirable

"noise." As Stuart Hall pointed out, the ideology lies in assuming that encoding and decoding

codes are or should be the same: "What are called 'distortions' or 'misunderstandings' arise

precisely from the lack of equivalence between the two sides in the communication

exchange."

281

Thus, while American social scientists contrasted American democracy with

Soviet totalitarianism, they simultaneously adopted the theoretical model according to which

mass communication was synonymous with following the prescribed meaning. In short,

communication was defined as control.

Despite the fact that modern mass communication crucially depends on the visual channel,

American scholars of mass communication never seriously confronted its visual aspects. Others,

however, have tried to use information theory to understand visual communication. I will next

279

Shannon and Weaver, The Mathematical Theory of Communication, 3. Shannon includes

"approximately" because one of the main concerns of a communication engineer is to increase
the efficiency of a communication system by making it transmit only the information which is
absolutely essential for the correct identification of a message by the receiver. For instance, the
telephone does not carry the whole range of frequencies of the human voice but only those
sufficient for understanding what is said.

280

Melvin L. De Fleur and Sandra Ball-Rokeach, Theories of Mass Communication (New

York: Longman, 1975), 127.

281

Stuart Hall, "Encoding/Decoding," in Culture, Media, Language, ed. Stuart Hall et al.

(Hatchinson, 1980), 131.

discuss these attempts which took place in the late 1950s and in the 1960s in semiotics,

experimental aesthetics, and art history.

In the early 1960s Roland Barthes published The Photographic Message and Rhetoric of

the Image where he applied a semiotic apparatus to the analysis of mass media images, in his

case, publicity photographs.

282

However, it may be surprising to discover that The Photographic

Message opens by formulating the situation of mass communication in terms of information

theory, exactly in the same way as it was done in communication studies:

The press photograph is a message. Considered overall this message is formed by a source of
emission, a channel of transmission and a point of reception. The source of emission is the staff of
the newspaper, the group of technicians certain of whom take the photo, some of whom choose,
compose and treat it, while others, finally, give it a title, a caption and a commentary. The point of
reception is the public which reads the paper. As for the channel of transmission, this is the
newspaper itself...

Barthes, however, was not going to propose, as communication researchers did, that the

system of mass communication also contains a feedback mechanism -- the right of readers to

write letters to newspapers with complaints or suggestions -- thus assuring that mass

communication truly serves the purposes of Democracy and Free Speech, rather than

unquestioned ideology. In fact, Barthes' whole purpose in his articles on the semiotics of mass

media images was to reveal the mechanisms by which images communicate or subvert an

ideological meaning. Communicate or subvert: Barthes' position oscillates between these two

mutually exclusive positions. On the one hand, Barthes proposes in The Rhetoric of the Image

that the specificity of a photograph's codes is what allows it to naturalize the ideological message

in a particularly transparent manner; on the other hand, in the same article he also claims that an

image is essentially polysemous and therefore a caption is required to anchor a single meaning.

In a later article The Third Meaning Barthes makes an even stronger claim for the subversive

282

Roland Barthes, (1961), "The Photographic Message," in Image, Music, Text, ed. Stephen

Heath (New York: Hill and Wang, 1977); "Rhetoric of the Image," (1964), in Image, Music,
Text.

nature of the visual code in relation to both ideology and interpretation by suggesting that a film

still contains an excess, an "obtuse meaning" not reducible to its "informational" or "symbolic"

meanings.

283

The code. More than any other, it was this particular concept of information theory that

entered semiotics in the 1960s. And for semioticians concerned with visual signs, it became the

main preoccupation of their research: is there a unique visual code, an autonomous "language of

pictures," and if so, what is its specificity? Now, after we have followed the development of

information theory, we can see the historical specificity of approaching vision as a code by

Barthes, J.M. Floch, Felix ThŸrlemann, the members of the Groupe Mu, Fernande Saint-Martin,

and other semioticians.

284

The notion of the code, as information theory itself, has emerged in

response to the practical problem faced by engineers: how to make telecommunication systems

more efficient and how to measure this efficiency. Encoding is one of the principal ways to

achieve this efficiency. Morse code condensed transmitted messages by associating the shortest

combination of dots and dashes with the most frequently used letters. Later, communication

engineers were able to achieve even greater economy by encoding only as much information as

necessary for the receiver to decode the message with sufficient accuracy.

285

When carried over

to visual semiotics, the notion of the code retained its original engineering meaning, its

association with the issue of efficiency. Indeed, to ask what kind of information can be obtained

through seeing and/or represented as images; how visual codes differ from other kinds of codes;

whether visual representations have any unique qualities in terms of their effect on the viewer --

in short, to propose the uniqueness of the visual code or to deny this uniqueness is already to

283

Roland Barthes, (1970), "The Third Meaning," in Heath, Image, Music, Text.

284

For a critical discussion of visual semiotics, see Gšran Sonesson, Pictorial Concepts.

Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World
(Lund, Sweden: Lund University Press, 1989).

285

A principal contribution of Shannon was his famous Sampling Theorem which specified the

absolute minimum of the signal which needs to be retained before information is irreversibly
lost.

conceive of vision as productive, as potentially a special way of transmitting information, to

conceive of it as a communication engineer would.

286

The semioticians did not go as far as to apply the mathematical apparatus of information

theory to the analysis of visual signs. However, the field of experimental aesthetics, which, from

its origins in the nineteenth century always tried to reduce the effect of an image to a

mathematical formula, welcomed the quantitative aspect of the theory. But how to apply a theory

developed for the purpose of engineering communication to a very different question of aesthetic

pleasure? Experimental psychologists interested in aesthetics assumed that a measure of

information in an image could represent its aesthetic effect. In the 1960s the German aesthetician

Max Bense developed a theory of "information aesthetics" and tried to measure the

"information" contained in such classical images as Rembrandt's etchings. It was not at all

obvious, however, how to measure the "information" in an etching or a painting, so other

researchers went on to construct simple abstract patterns where "information" was easier to

quantify. They reasoned that since the amount of information in a message is directly

proportional to its unpredictability and inversely proportional to its redundancy, a pattern made

from small squares, where the tone of each square is determined randomly, would be the most

unpredictable and therefore would contain the most information. On the contrary, in a pattern

like Mondrian's or Malevich's painting there is much more redundancy (more predictability) and

consequently, the amount of information is low. In a number of experiments, subjects rated their

"aesthetic pleasure" when looking at different patterns (fig. 19); psychologists then tried to

286

This carrying over of assumptions of information theory into semiotics reaches its extreme in

Jurij Lotman's Structure of the Artistic Text (1970). Lotman was strongly influenced by
cybernetics which in the 1960s was more popular in the Soviet Union than it ever was in the
West and which became as important for Soviet semioticians as Saussure or Peirce. (In part, it
was used to justify semiotics in the eyes of the authorities by presenting it as a part of
cybernetics, concerned with the study of different artificial languages, from computer languages
to the languages of art.) Lotman claimed that the artistic text is a message coded in a particular
way and that art is the most economical way to transmit information because of the unique
features of the artistic code. Jurij Lotman, (1970), The Structure of the Artistic Text (Ann
Arbour: Department of Slavic Languages and Literatures, The University of Michigan, 1977).

determine the level of visual organization that was biologically most pleasurable.

287

Psychologists, turned aestheticians. Aestheticians, turned communication engineers.

Finally, one of the most influential texts of post World War II art history is also heavily

indebted to information theory.

288

In Art and Illusion (1960) Ernst Gombrich provides a

revealing analysis of Greek vases and mosaics as precursors to modern digital codes; he

compares the reading of pictures to the decipherment of codes; and he uses various concepts of

information theory throughout the book.

289

However, Gombrich does more than simply pay

tribute to a fashionable theory. During the war, Gombrich was employed for six years by the

British Broadcasting Corporation to listen for radio transmissions.

290

He was confronted daily

with the central problem of modern information theory -- the identification of signal in the

presence of noise. As he himself admits, it was this experience which led to his deep interest in

information theory as a paradigm for understanding perception.

Rather than being accidental, the paradigm of information theory plays a central role in

Art and Illusion. One of Gombrich's key concepts is "the beholder's share": the importance of

inference in interpreting the image. According to Gombrich, a representational image is

essentially incomplete.

291

As a representation of three-dimensional space, a two-dimensional

287

See, for instance, D.E. Berlyne, "Novelty, Complexity, and Hedonic Value," Perception and

Psychophysics 8 (1970): 279-86; D. Dorfman and H. McKenna, "Pattern Preference as a
Function of Pattern Uncertainty," Canadian Journal of Psychology 20 (1966): 143-53; P.C. Vitz,
"Preference for Different Amounts of Visual Complexity," Behavioral Science 2 (1966): 105-14.
It is interesting to compare this work in experimental aesthetics with Lotman's view of art as
communication (see footnote 278). Lotman proposed that a unique feature of art is that it carries
more information than any other kind of communication, while experimental aestheticians
proceeded under the assumption that there is some optimal level between the complete lack of
information and the complete saturation by it which characterizes the most pleasurable works of
art.

288

I am grateful to Michael Ann Holly and Norman Bryson who brought to my attention the

relevance of Art and Illusion to the problematic of this chapter.

289

E.H. Gombrich, Art and Illusion (Princeton: Princeton University Press, 1960), 39-40, 88,

205.

290

Ibid., 204.

291

Ibid., 208, 211.

image is always ambiguous, potentially corresponding to numerous spatial configurations. It is

equally ambiguous as a representation of other aspects of visual reality, substituting a few brush

strokes for a field of grass, a few spots of color for a human form. The job of a beholder is to

supplement the partial information in the image with her or his own projection. In other words,

Gombrich understands visual communication in terms of information theory: as the decoding of

a signal in the presence of noise. According to information theory, this decoding is always

probabilistic: there can never be the absolute certainty than an original signal is correctly

reconstructed. Similarly, Gombrich describes interpretation as the process of forming and

rejecting hypotheses, a process which is never finished.

For an engineer, this uncertainty of communication is a problem to be solved. Gombrich,

however, interprets it as the source of our aesthetic pleasure. We derive pleasure from art

precisely because in trying to "identify a signal" we have to use our imagination. There is an

unexpected agreement here between Gombrich and experimental aestheticians of the 1960s, such

as Bense. Both start from the assumption that the aesthetic pleasure is proportional to the amount

of information in a message, i.e. its unpredictability. As Gombrich explains it, "the greater the

probability of a symbol's occurrence in any given situation, the smaller will be its information

content."

292

Because of their ambiguity, representational images in Western art carry significant

information -- and, correspondingly, aesthetic pleasure.

Mass communications, visual semiotics, and aesthetics are just some of the fields that adopted

concepts of information theory in order to understand visual communication. The avant-garde of

the 1920s, which dreamed of engineering visual communication, would welcome all these

applications of information theory. The theory, which was developed by a communication

engineer in response to the growth of television, was now used to rationalize the idea that mass

292

Ibid., 205.

communication is synonymous with control, and that to understand is to decode an intended

meaning (mass communication studies). The theory, designed to measure only the efficiency of

telecommunications networks, was now adopted to describe all other levels of communication as

well. Visual semiotics and experimental aesthetics extended information theory to the levels of

meaning and effect, the former addressing the specificity of visual signification, the latter

attempting to quantify the effect of an image by measuring its "information" content.

However, it was the field of experimental psychology that was most transformed by

information theory. After World War II, two crucial developments took place. The first is that

psychologists, by adopting the approach of information theory, essentially became engineers

who approach human vision as a communication system, studying its channel capacity,

bandwidth, and other engineering characteristics. The second is that work, more and more,

became a matter of the mental processing of visually presented information. And this made

experimental psychology, for the first time since the discipline emerged, practically important; in

fact, as important as communication engineering. In the next three sections we will see how

these developments came about and what their impact was on modern understanding of vision.

Why was it that even though the essential ideas of information theory were articulated in the

1920s, it was only in the late 1940s, after the publication of The Mathematical Theory of

Communication, that the theory was suddenly widely accepted in so many fields? Is there a

relationship between this acceptance and the social and economic changes which were taking

place in modern society after World War II? The best way to answer these questions is to

compare the role of information theory in the second half of the twentieth century to the role

4. From "Human Motor" to "Human Information Processing"

previously occupied by thermodynamics.

First, a historical parallel. Hermann von Helmholtz formulated the universal law of the

conservation of energy in 1847. Exactly a century later, the theory of information in its modern

form was developed by Shannon and Wiener. Helmholtz was instrumental in turning the theory

of thermodynamics into a more general paradigm which treated the work of nature, the work of

machines, and human labor on the same terms. Similarly, Wiener extrapolated the concepts of

information theory and the techniques of control engineering into the science of cybernetics

which described the organization and behavior of human, machine, and society using a single set

of terms as well -- information, feedback, and entropy.

The concept of entropy, in fact, directly connects thermodynamics and information

theory. In 1929 the physicist Leo Szilard identified entropy with information, and the measure of

information with the negative of the measure of entropy.

293

Entropy is the central notion of

thermodynamics, referring to the tendency of a closed system to change from an organized,

differentiated, and less probable state to a chaotic, undifferentiated, and more probable state.

Since entropy is the measure of randomness, i.e. the unpredictability of a thermodynamic system,

and since information similarly measures the unpredictability of a message, the equations for

entropy and information are the same. As Weaver explains, "for a communication source one can

say, just as he would also say it of a thermodynamic ensemble, 'This situation is highly

organized, it is not characterized by a large degree of randomness or of choice -- that is to say,

the information (or the entropy) is low.'"

294

But beyond these connections, there is a more crucial parallel. Just as thermodynamics

provided the model for understanding human work in the latter nineteenth and first half of the

293

Howard Resnikov, The Illusion of Reality (New York: Springer-Verlag New York Inc.,

1989), 5.

294

Shannon and Weaver, The Mathematical Theory of Communication, 103.

twentieth century, information theory has been crucial for a new post-industrial model of work,

which gradually emerged after 1940.

In The Human Motor: Energy, Fatigue and the Origins of Modernity Anson Rabinbach

demonstrated how the scientific ideas of thermodynamics, formulated in the middle of the

nineteenth century, became central for the conception of work in modernity. Helmholtz, who

discovered the law of the conservation of energy, promoted this law as the universal principle

which equally applies to nature, machines, and humans. Helmholtz "portrayed the movements of

the planets, the forces of nature, the productive force of machines, and of course, human labor

power as examples of the principle of conservation of energy."

295

All work was understood as

the expenditure of energy, with a crucial consequence of redefining human labor as labor power,

the expenditure of the energy of a body. Thus a worker was redefined as a "human motor." This,

in turn, lead to the emergence, towards the end of the century, of the movement which

Rabinbach calls the European science of work, "the search for the precise laws of muscles,

nerves, and the efficient expenditure of energy centered on the physiology of labor."

296

manual labor, the energy stored in the body where it was accumulated through the intake of food,

sleep, and rest is transferred into muscular force -- hammerer striking a blow, filer filing a

machine part, and so on. Therefore, psychologists, physiologists and industrial experts searched

for methods to maximize both the accumulation of a worker's energy (through proper nutrition,

shorter working hours, appropriate breaks) and its expenditure in labor. Just as an engineer

designing an engine was concerned with the most efficient transfer of fuel energy into

movement, European work experts aimed to maximize worker efficiency and to eliminate

possible waste. Central to the quest for the efficiency of the human motor was the struggle

295

Anson Rabinbach, The Human Motor: Energy, Fatigue, and the Origins of Modernity (Basic

Books, Inc., 1990), 3.

296

Ibid., 10.

against fatigue, understood as the equivalent of entropy. "As entropy revealed the loss of energy

involved in any transfer of force, so fatigue revealed the loss of energy in the conservation of

Kraft to socially useful production. As energy was the transcendental, 'objective' force in nature,

fatigue became the objective nemesis of a society founded on labor power."

297

The European science of work may appear to be very similar to the American scientific

management movement pioneered by Frederick Winslow Taylor, a former engineer turned

management consultant. As a part of his program, Taylor aimed to minimize and standardize the

time required by a worker to perform each operation. He employed the method of time studies

whereby the best workers were timed and the results became the norm to be followed by the

rest.

298

Later, Frank and Lilian Gilberts (he -- an engineer, she -- a psychologist) popularized

another method of motion study.

299

They argued that maximizing worker productivity is best

achieved by the elimination of unnecessary movements and making the necessary more efficient.

Although both time and motion studies and the European science of work were concerned with

the efficiency of manual work, there was a fundamental difference between the two

approaches.

300

Taylorism aimed for maximum productivity, and had no concern for the

exhaustion and deterioration of the human motor. In contrast, European scientists aimed for

optimum productivity, and therefore were concerned not only with the rationalization of the

workplace, but also with the workers' health, nutrition, safety, and the optimal length of a

workday. In short, Taylorism had no reservations about replacing one exhausted human motor

with another -- the philosophy which in the U.S. seems to go hand in hand with the emerging

ethics of the consumer society and with immigration policies which assured the constant supply

297

Ibid., 68.

298

Frederick Winslow Taylor, The Principles of Scientific Managment (New York, 1967).

299

William R. Spriegel and Clark E. Myers, eds., The Writings of the Gilbreths (Homewood,

IL., 1953). I am grateful to Lisa Cartwright for introducing me to the work of the Gilbreths and
its relevance for the history of modern vision.

300

Rabinbach, The Human Motor, 117, 277.

of a cheap labor force. Europeans, on the other hand, were committed to caring for and servicing

the human motor. The two paradigms converged after World War I, when European

industrialists partly adopted the more brutal, but ultimately more effective Taylorist methods,

while U.S. managment experts became more sensitive to workers' physiology and psychology.

Taylorism reduced the worker's body to a mechanical machine and had no concern for

her or his mind. Indeed, as Marta Braun points out, Taylorism aimed to systematically rob the

worker of any degree of independence or even understanding of the overall work process by

"separating responsibility for the execution of work from its planning or conception."

301

This

disdain for the mind was shared by behaviorism, which matured at the same time as the

European science of work and Taylorism, and which equally well characterizes the imaginary of

hard-edged social engineering of the first half of the twentieth century. In 1913, J.B. Watson, the

founder of behaviorism, explicitly defined it as the science of social control: "Psychology as the

behaviorist views it is a purely objective experimental branch of natural science. Its theoretical

goal is the prediction and control of behavior."

302

Behaviorism approached the human subject as

an input-output system of stimulus and response to be controlled through conditioning.

Concerned with controlling the body, it almost completely suppressed any studies of perceptual

or mental processes between 1920 and 1950 in the U.S. It was a psychology well suited for

controlling the subject already reduced to the brainless human motor.

301

Marta Braun, Picturing Time: the Work of Etienne-Jules Marey (1830-1904) (Chicago: The

University of Chicago Press, 1992), 337.

302

Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century

of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 27.

5. Communication Engineer Analyzes Human Vision

After World War II, the references to time and motion studies disappear. And by 1960, the

dominance of behaviorism is over as well. Industrial society became post-industrial society.

In this shift, the concepts of manual labor, production of goods, and fatigue were replaced

by new concepts of cognitive labor, information processing, and noise. Taylorism, Gilberts'

motion studies, and behaviorism gave way to engineering psychology, "human information

processing," and cognitive science. In short, with the transformation of industrial society into

post-industrial society, the disciplines of the efficiency of the body were replaced by the

disciplines concerned with the efficiency of the new instrument of labor -- the mind.

One obvious and perhaps the earliest sign of this transition was the speed with which

information theory was disseminated into so many fields by the end of the 1940s and early

1950s. But, along with being a symptom of post-industrial society which was yet to fully

manifest itself, information theory also provided actual techniques for defining and studying the

human subject as an "information processing unit," and its visual apparatus as a crucial part of

"human information processing." Information theory therefore is central for understanding the

new ideas and techniques of vision that emerged after the World War II.

The significance of the adaptation of information theory as a new model of the human

observer, which took place in the 1950s, can be best seen through the following analogy:

Industrial society: manual labor, energy, fatigue.

Post-industrial society: mental labor, information processing, noise.

1. Manual versus mental labor. The point is not whether corporeal labor was indeed

universally displaced by mental labor: this is different from country to country, from industry to

industry. What is important is that the obsession with the rationalization of corporeal work

(Taylorism, European science of work, psychotechnics) disappeared, displaced by new obsession

with the rationalization of the mind (cognitive psychology, artificial intelligence, cognitive

engineering). Regardless of the percentage of the work force that still may be engaged in manual

labor, society is no longer concerned with spending more intellectual resources to perfect

workers' movements. What comes under scrutiny since the 1950s, when cognitive psychology

begins to displace then dominant behaviorism, are mental functions: perception, attention, text

comprehension, memory, and problem solving.

This replacement of manual work by cognitive work is directly related to automation.

Already in 1961, in an influential study of automation in French industry, Pierre Naville and his

fellow sociologists had described the transition from the "work of the laborer to the work of

communication," work which became primarily "cognitive or semiotic."

303

In his summary of

this study Rabinbach writes, "The appearance of the cerebral worker whose material and product

is 'information' is emblematic of the vast distance traversed between the worker who surveys

complex technologies of communication and the 'man-beef' of Taylor."

304

It is important to note

that automation does not lead to the replacement of human by machine. Rather, the worker's role

becomes one of monitoring and regulation: watching displays, analyzing incoming information,

making decisions, and operating controls. And it is the corresponding human functions of

perception, attention, memory, and problem solving which become the subject of research by

new cognitive sciences.

What Taylor's scientific management was for the age of industrialization, cognitive

sciences are for the age of automation. In the 1940s, Herbert Simon worked on theories of

management, the field of research originated by Taylor. Having recognized the increasing

importance of mental skills in the corporate workplace, Simon became one of the pioneers of

cognitive science with his work on automatic reasoning by computer. In 1964 he wrote that "the

303

Qtd. in Rabinbach, The Human Motor, 298.

304

Ibid., 298.

bulk of productive wealth consists of programs...stored in human minds."

305

Another pioneer of

cognitive science was Jerome Bruner. Reflecting back on his work in the 1950s, he noted in

1983: "It seems plain to me now that the 'cognitive revolution'...was a response to the

technological demands of the 'post-industrial revolution.' You cannot properly conceive of

managing a complex world of information without a workable concept of mind."

306

2. Just as thermodynamics in the nineteenth century provided a model for the laboring

body, information theory provided a new model for the mental processes, becoming particularly

influential in the research on human senses, including vision.

In 1958 the English psychologist D.E. Broadbent published Perception and

Communication, which was the first attempt to understand human sensory-motor performance as

information processing, or, as he put it, to describe it "in terms originally developed for

telephone engineering."

307

This statement may at first remind us of the comparison between

nerves and telegraphy, frequently evoked by nineteenth century physiologists and psychologists,

including Helmholtz.

308

Yet, the approach of Broadbent and other psychologists who followed

him was fundamentally new. It was less concerned with human physiology than with the limits

of human performance understood as a communication system. In other words, just as a modern

telephone engineer is concerned with the capacity of the telephone system, a psychologist is

concerned with the amount of visual or auditory information that can be adequately processed by

a human system; sensed, recognized, stored in memory, recalled, acted upon. What for

Helmholtz was a metaphor became a systematic program of research a century later.

305

Qtd. in Douglas Noble, "Mental Materiel: The Militarization of Learning and Intelligence in

U.S. Education," in Cyborg Worlds: the Military Information Society, ed. Les Levidov and
Kevin Robins (London: Free Association Books, 1989), 34.

306

Qtd. in Ibid., 34-35.

307

Broadbent, Perception and Communication, 36.

308

Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth

Century (Cambridge: The MIT Press, 1990), 93-94.

For a communication engineer, information theory provides both a standard set of terms

to specify quantitatively the performance of a communication system and a set of principles

which specify the limits of this performance. For instance, the capacity of a system depends on

the code used. Morse code makes transmitted messages shorter by associating the most

frequently used letters with the shortest dot--dash symbols. The time saved in transmitting,

however, is offset by the additional time now required to encode and decode the message.

Another example is the principle, already established by Hartley, that the maximum amount of

information a system can transmit is the product of its bandwidth and the time available for

transmission. In the 1920s, it was possible to transmit images over telegraph wires -- a small

bandwidth system -- by sacrificing time.

In addition to the adoption of the terms of information theory, such as bandwidth,

information capacity, and filter, psychologists also applied its principles to understanding the

limits of human performance. One of Broadbent's contributions was the idea that the nervous

system cannot process all of the sensory stimulation which may be available and that there is "a

filter at the entrance to the nervous system which will pass some classes of stimuli but not

others"

309

(fig. 20). In other words, because of its limited bandwidth, a nervous system can only

process a certain amount of information in a given unit of time. How much can be processed

depends on the nature of this information since different human senses such as vision and

hearing, now understood as separate communication channels, have different information

capacities.

Once filtered through the senses, the information is further processed through a series of

stages. Each stage is understood as a specific communication system with its own capacities

determined by its codes and bandwidth. Figure 21, taken from 1986 Handbook of Perception and

309

Broadbent, Perception and Communication, 42.

Performance, summarizes the current knowledge of the temporal capacities of different stages of

sensory and mental processing.

So just as the productivity of a laboring body was limited by the available energy, it is

now assumed that the productivity of the mind is limited by its information-processing capacity.

In fact, the notion of a limited reservoir of the energy in the body has found a direct correlatation

in the notion of there being a limited amount of human information-processing resources which

can be used in different ways to deal with the incoming information. For instance, the subject

may concentrate all his/her resources on processing visual information, thus sacrificing his or her

attending to auditory stimuli.

3. Fatigue versus noise. Just as fatigue is the opposite of energy, the limit to the body's

productivity, noise, always present in a communication system, whether natural or man-made, is

the limit to perfect communication and, consequently, the limit to faultless human information

processing. Noise is the source of error. And error is the worst enemy of the human-machine

system in post-industrial cybernetic society.

The late nineteenth century science of work came about as a struggle against fatigue, this

"subversive element in the human motor, the body's fifth column...fatigue was the physiological

limit of even the most perfectly executed work, the horizon of the metaphor of the human

motor."

310

As physiologists and psychologists were preoccupied with fighting the body's

fatigue, cognitive psychologists became preoccupied with preventing human cognitive and

perceptual errors. A textbook on human factors begins a chapter on error by optimistically

asking: "Are errors a basic part of human nature that cannot be changed, or is there some way to

eliminate or reduce error and the damage errors cause?" The experts on human factors classified

310

Rabinbach, The Human Motor, 118.

various types of human errors, came up with mathematical formulas to calculate human

reliability, and compiled tables of Human Error Probability (HEP) for different tasks.

311

What are the sources of human error? In contrast to a manual worker of the industrial

age, an operator in a human-machine system is primarily engaged in the observation of displays

which present information in real time about the changing status of a system or an environment,

real or virtual: a radar screen tracking a surrounding space (see chapter 3); a computer screen

updating the prices of stocks; a video screen of a computer game presenting an imaginary

battlefield; a control panel of an automobile showing its speed, etc.

312

From time to time, some

information causes an operator to make a decision and to intervene in the system's operation: tell

the computer to track an enemy bomber noticed on the radar screen; buy or sell a stock; press a

joystick; change gears. It is not essential that in some situations these interventions may be

required every second (a pilot engaged with an enemy, a computer game player, a financial

analysis monitoring stock prices), while in others they are needed very rarely (a technician

monitoring an automated plant, power station, a nuclear reactor; a radar operator monitoring a

radar screen, waiting for potential enemy planes). This is why a human-machine system is

defined as "an equipment system, in which at least one of the components is a human being who

interacts with or intervenes in the operation of the machine components of the system from time

to time."

313

In a famous passage Walter Benjamin characterized modern experience as a constant

periodic rhythm of perceptual shocks; the experience shared by an assembly line worker, by a

311

Barry H. Kantowitz and Robert D. Sorkin, Human Factors: Understanding People-System

Relationships (John Wiley & Sons: 1983), 30-57.

312

A 1965 textbook on human-machine systems calls an automobile "a first rate example of a

true man-machine system...a highly complex system in which the operator plays a commanding
role or actively intervenes in the system from time to time." Alphonse Chapanis, Man-Machine
Engineering (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 16.

313

Chapanis, Man-Machine Engineering, 16. Emphasis mine -- L.M.

pedestrian, and by a film viewer.

314

This experience, characteristic of modernization, finds a

direct continuation in one type of cybernetic workplace: the constant overwhelming amount of

information; the constant cascade of cognitive shocks which require immediate interventions (a

pilot engaged with an enemy, a player of a computer game). Now, however, these shocks arrive

exclusively through the visual channel (dials, computer screen, head-mounted display). Thus of

the roles mentioned by Benjamin, it is the film viewer rather than the assembly line worker who

directly anticipates the experience of an operator in this type of human-machine situation.

However, there is also a second type of work experience, new to post-industrial society: work as

waiting for something to happen. A radar operator waiting for a tiny dot to appear on the screen;

a technician monitoring an automated plant, power station, or nuclear reactor, knowing that a

software bug will eventually manifest itself, making a pointer on one of numerous dials shoot

into the red...

If a manual worker is eventually overcome by fatigue, an operator monitoring displays in

situations where nothing may happen for hours eventually loses vigilance. In terms of the

information processing approach, when a signal occurs infrequently and randomly over time, an

operator is more likely to miss the signal.

315

This is one source of human error. Another source

314

"Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless,

today's pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus
technology has subjected the human sensorium to a complex kind of training. There came a day
when a new and urgent need for stimuli was met by the film. In film, perception in the form of
shocks was established as a formal principle. That which determines the rhythm of production on
a conveyer belt is the basis of the rhythm of reception of film." Walter Benjamin, "On Some
Motives in Baudelaire," in Illuminations, ed. Hannah Arendt (New York: Schochen Books,
1969), 175.

315

In the extreme case of sensory deprivation the subject becomes totally disoriented,

experiences illusions and even faints. It is interesting that in the 1950s, when automation was
making this new work-as-waiting-for-signal commonplace, experimental psychologists began to
conduct experiments in artificial sensory deprivation. In 1953 L.A. Riggs and his colleagues
discovered that if the image is stabilized on the retina, thus eliminating miniature eye movements
normally accompanying vision, the image vanishes. In 1954-57, W. Heron constructed an
experimental cubicle to study the effects of total perceptual isolation. Subjects, deprived of all
visual, auditory, and tactile stimuli, experienced extreme boredom, restlessness and, eventually,
visual hallucinations. L.A. Riggs, et al., "The Disappearance of Steadily Fixated Visual Tests

is visual illusions, limits to the reliability of human vision, which preoccupied experimental

psychologists from the early days of the discipline. Still another source is the inherent limitations

of a human as a decision maker. When cognitive psychologists began to study human reasoning,

they made a grave discovery: human judgments do not follow statistical probabilities and ignore

information about the prior probability of an event.

316

So even if the operator detects a signal in

time, s/he may still make a wrong decision.

Along with providing a model that allowed psychologists to conceive of human vision as a

communication system where performance may be described in quantitative terms, information

theory also provided a general concept to specify an inherent limit to this performance -- noise.

The incorporation of the concept of noise is what distinguishes modern information theory, as

developed by Shannon, Wiener, Kolmogoroff, and others, from the original formulations by

Hartley. In any real telecommunication system noise is always present due to the random motion

of electrons in all electrical conductors. Therefore, in practice, the signal at the receiving end is

always to some degree different from the signal originally sent. One of Shannon's principal

contributions was an equation for determining the maximum capacity of a communication

channel to transfer information in the presence of noise. As Colin Cherry notes, "This channel

capacity concept is somewhat analogous to the notion of conservation of energy; it is a definite

limit which no practical system can exceed, and its principal value is that it provides a standard

against which efficiencies may be assessed."

317

Just as entropy limits the performance of any thermodynamic system (such as a motor),

leading to some of the energy being lost unproductively, noise sets the limit on the efficiency of

Objects," J. Optical Soc. America 43 (1953): 495; W. Heron, "The Pathology of Boredom,"
Scientific American 196 (1957): 52-56.

316

Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic

Press, 1983), 190.

317

Cherry, On Human Communication, 52.

communication. And just as attempts to rationalize manual performance stumbled across the

problem of fatigue, the "physiological limit" to the efficiency of work, psychologists recognized

noise as the inherent limit in the performance of a human information processing system. If the

source of noise in a telecommunication apparatus is the random motion of electrons, the source

of noise in the nervous system is the random firing of neurons. Always present, these random

firings are assumed to generate a small but constantly present level of neural activity which

interferes with human information processing. The observer may miss a signal (for instance, a

flash of light in a psychophysical experiment or a dot on a radar screen in a real situation) or, on

the contrary, detect a signal where none was present; or s/he may make a mistake in estimating

the signal's strength or its position, and so on.

In fact, the noise in man-made components and biological noise become combined. Due

to the noise in its electronic parts, radar may display a dot when nothing is present or the location

of the dot may be inaccurate. Additional noise is added by the imperfections of a display, such as

glare, inadequate contrast, or placement. Finally, neural noise may cause the radar operator to

miss the dot, to misjudge its size and, consequently, either to miss an enemy plane or to send an

intercepting weapon towards a friendly aircraft. In this way, the overall performance of a human-

machine system, such as a radar installation, is equally limited by the performance of its

electronic and human parts.

A radar operator? A human-machine system? Was not I talking about information theory as a

paradigm for experimental psychology, a purely academic discipline? A discipline whose

practitioners, according to William Estes, "developed the tradition and the reputation of being

6. Human Engineering

among the purest of pure scientists" during its first decades?

318

Practitioners who for decades

never left the seclusion of the university laboratory while their colleagues -- physiologists,

fatigue experts, behaviorists -- seemed to hold the knowledge necessary to maximize the

productivity of industrial society?

The fact that after 1950 experimental psychologists displaced their colleagues in

importance, of course, fits in with the larger shift from industrial to post-industrial society and

the new image of work: visual and mental processing of information rather than corporeal labor.

The problem with such global explanations is that they explain things too well. It is easy to

connect the interest of experimental psychologists in human information processing to the shift

to post-industrial society. In fact, it is as easy as to connect the very emergence of experimental

psychology in the nineteenth century to the processes of modernization, industrialization, or

rationalization. For instance, discussing Fechner's psychophysics, developed in the 1850s,

Jonathan Crary writes that Fechner's equations were "a means of rendering a perceiver

manageable, predictable, productive and above all consonant with other areas of

rationalization."

319

Similarly, in his discussion of reaction time experiments, which became

standard in experimental psychology by the 1890s, Didier Deleule points out that "the measured

time is only the sign of the living machine's capacity for increasing its productivity."

320

Both Crary and Deleule are right in seeing the social utility of experimental psychology.

It should be also noted, however, that this social utility remained unfulfilled for many decades. In

fact, the enormous amount of knowledge accumulated by psychologists did not find its

systematic application until World War II.

318

William Estes, "Experimental Psychology: an Overview," in The First Century of

Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 629.

319

Crary, Techniques of the Observer, 147.

320

Didier Deleule, "The Living Machine: Psychology as Organology," in Incorporations, ed.

Jonathan Crary and Sanford Kwinter (New York: Zone Books, 1992), 215.

The gradual expansion of the practical applications of experimental psychology provides

a precise map of the new occupations and new conditions of modern experience which call for

perceptual skills. During World War I, England, Germany, and France utilized experimental

psychologists to design and administer tests for aviation pilots, aeronautical, and airplane

observers, hydrophone operators, and submarine "listeners-in."

321

During peacetime, a number

of psychologists published papers on the readability of written text and of highway signs and on

the visibility of lights at sea.

322

However, in the industrial world which conceived of the worker

as a human motor and was largely concerned with the productivity of manual rather than

perceptual labor, these studies were an exception rather than the mainstream rule.

It was World War II which finally put to use the expertise of experimental psychologists.

Why did this happen? The first textbook on applied experimental psychology (1949) opens by

describing the recent origins of the field:

For years experimental psychologists have worked diligently in academic laboratories studying
man's capacities to perceive, to work, and to learn. Only very slowly, however, have the facts and
methods which they have assembled been put to use in everyday life. A particularly glaring gap in
modern technology, both industrial and military, is the lack of human engineering -- engineering
of machines for human use and engineering of human tasks for operating machines. Motion-and-
time engineers have been at work on many of these problems, but the experimental psychologist is
also needed for his fundamental knowledge of human capacities and his methods of measuring
human performance.

The recent war put the spotlight on this gap. The war needed, and produced, many

complex machines, and it taxed the resources of both the designer and operator in making them
practical for human use. The war also brought together psychologists, physiologists, physicists,
design engineers, and motion-and-time engineers to solve some of these problems. Though much
of their work began too late to do any real good, it has continued on a rather large scale into the
piece.

Today, there are many groups busy with research on man-machine problems. They use

different names to describe the work in its various aspects: biotechnology, biomechanics,
psychoacoustics, human engineering, and systems research. Other names may be appropriate and
may appear in the future. In casting about for a title for this book, we tried to select one that would
describe the subject matter without the restrictive connotations attaching to some of the names

321

Morris Viteles, Industrial Psychology (New York: W.W. Norton & Company, Inc., 1932),

43.

322

Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1287-
1340.

mentioned above. Applied Experimental Psychology seems best to fill these requirements, because
the traditional data and subject of experimental psychology are fundamental to this field.

323

Already before the war, experimental psychologists assisted in selecting military personnel for

such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a

much greater number of pilots, radar operators and other similar personnel became needed. The

emphasis was shifted, therefore, from selecting personnel with particularly good perceptual and

motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match

the sensory capacities of an average person.

324

And it was the field of experimental psychology

that possessed the knowledge about the sensory capacities of an average, statistical person: how

visibility and acuity vary between day and night; how the ability to distinguish colors and

brightness vary with illumination or distance; what the smallest amount of light is which can be

reliably noticed; and so on.

325

All this data was now utilized for designing better displays and

controls of the first modern human-machine systems such as high-speed aircrafts or radar

installations.

The development of these new human-machine systems during the war pushed human

perceptual and mental performance to the limit and this was the second reason why experimental

psychologists were called in. The performance of a human-machine system was limited by

human information capacity to process information. In the words of the authors of Applied

Experimental Psychology,

We can make a machine that will do almost anything, given enough time and enough engineers.
But man has limits to his developments, at least as far as we can see it. When we think how much
a single radar can do in a small fraction of a second, and then realize by comparison that even the
simplest form of reaction for a human being requires about a fifth of a second, we realize what we
are up against... The full potential of radar, for example, lagged far behind physical developments

323

Alphonse Chapanis, Wendell R. Garner, and Clifford T. Morgan, Applied Experimental

Psychology (New York: John Wiley & Sons, Inc., 1949), v.

324

Ibid., 8.

325

Estes, "Experimental Psychology: an Overview," 630.

because human operators could not master the complex operation of this machine system. We had
to worry about such things as a new kind of visual signal -- very small and not very bright.

326

Considering that the authors described the work of time-and-motion engineers as directly leading

to applied experimental psychology, this rhetoric can be expected. Taylor was impatient with the

limitations of the body; now there was a similar impatience with the limitations of human

information processing. With Taylor, it was the question of the speed of muscular movements;

now, it became the question of reaction time: the minimum time in milliseconds required for an

operator to detect a signal, to identify it, to press a control.

In order to measure normal human sensory capacities, experimental psychologists have

always put subjects in, so to speak, boundary conditions. They measured sensory thresholds,

such as the least amount of light which can be detected. They also measured just noticeable

differences (j.n.d.), the smallest difference between two stimuli which can be detected. Finally,

they measured reaction times, the measure which became the main tool to deduce the time taken

by different mental processes. In order to measure these characteristics, a number of standard

experiments were designed, and they remained largely unchanged from the times of Weber,

Fechner, and Wundt. In a detection experiment, the task of an observer is to detect the presence

of barely visible stimuli, for instance a tiny light briefly flashed in the dark (did I see

something?). In an identification experiment, the task is to identify which of possible stimuli was

presented, for instance which of two colors (which one did I see?). In a recognition experiment,

the task is to not only detect something, but to recognize what it is, for instance: what was the

shape that briefly appeared (what did I see?)

During World War II, the radar operator, the anti-aircraft gunner, the aircraft pilot found

themselves in the same situations in which nineteenth century psychologists put their

experimental subjects. The setups of psychophysical experiments became, in all details, the

326

Chapanis, Applied Experimental Psychology, 7-8.

conditions of military work; the tasks devised by psychologists to study human vision became

the actual tasks faced by the operators of human-machine systems. Like the subject of a detection

experiment, a radar operator scans the radar screen for a barely noticeable dot of light.

327

the subject of an identification experiment, he has to try to guess whether this dot is the same or

different from another dot which from his previous experience he knows to correspond to a

friendly airplane. An anti-aircraft gunner is subjected to a recognition experiment, trying to

identify a plane by its shape. And all of them, especially the pilot, are engaged in a sort of

reaction time experiment.

Thus, nineteenth century psychophysical setups became the military, and soon, civilian

workplaces of post-industrial society; from there, they traveled back into laboratories, leading to

such close interrelations between basic research in experimental psychology and its practical

applications that they were no longer separable.

328

The terms "applied experimental

psychology," "human engineering" and "man-machine engineering" were replaced by another

term standard today -- "human factors." The radar operator who in the 1940s and 1950s was the

prototypical example of a human-machine system, was replaced by the 1980s by a new

prototypical figure, the computer user. Thus, references to "human-machine systems" became

references to "human-computer systems." The same amount of intellectual energy and research

which in the middle of the century went into theorizing the performance of a radar operator and

adapting him and radar display to each other, today goes into the work on computer interfaces. In

327

As Paul Fitts notes in his 1951 overview of engineering psychology, "radar operators are

often forced to search for weak signals at near-threshold levels." Fitts, "Engineering Psychology
and Equipment Design," 1290.

328

For example, a 1947 article in American Psychologist describes the work of Naval Research

Laboratory as following these three directions: "the design of gun fire control and missile control
instruments from the point of view of ease and efficiency of operation; the design and evaluation
of synthetic gunnery and missile control trainers; and basic psychological research." But what is
meant here by "basic research"? We read that "at present, all basic research studies are aimed at
the eye-hand coordination problem involved in target tracking." "Target tracking" is just one
example of a military task which traveled into a psychological laboratory, and gradually become
a standard psychophysical experiment. Franklin Taylor, "Psychology at the Naval Research
Laboratory," American Psychologist 2, no. 3 (1947): 87, 91.

retrospect then, we should recognize the radar operator as the central figure standing at the

origins of post-industrial society, the figure which put directly into motion the new disciplines of

the efficiency of the mind: engineering psychology, human information processing, and

cognitive science.

We are now in a position to see that the ready acceptance of information theory by

psychologists in the 1950s was not simply a part of a "Zeitgeist" of the coming information age.

Rather it was a practical necessity. The 1986 Handbook of Perception and Human Performance

describes the rationale for adopting the information processing approach in psychology in this

way:

Because the language of information processing provides an objective and quantitative way of
describing the basis of human performance, it has proven useful in applications. Indeed, much of
the impetus for the development of this kind of empirical study stemmed from the desire to
integrate description of the human within the overall systems. In the design of cockpits, control
systems for weapons, and other military applications, it has been useful to have descriptions of the
speed, accuracy, and reliability of the human in terms comparable to those used for other
mechanical parts of the system.

329

As psychologists became involved in designing human-machine systems, they needed a language

to characterize human performance quantitatively. But not just any language. If the human is

conceived of as part of a human-machine system (for instance, in the role of a radar operator), it

is convenient to use the same language to describe the performance, capacities, and errors of

both the electronic and human components. Put differently, if the human-machine system is

conceived of as a machine, and its performance is to be characterized in engineering terms, the

same terms need to be applied to its human components if the performance of the overall

machine is to be characterized at all. Since information theory already provided a language to

329

Michael I. Posner, "Section V: Information Processing. Overview," in Handbook of

Perception and Human Performance, ed. Kenneth Boff, Lloyd Kaufman, and James P. Thomas
(New York: John Wiley and Sons, 1986), II: V-6.

characterize the performance of electronic components, it was logical to apply it to the human

component as well.

Modernization brought with it a special discipline concerned with efficiency -- engineering. The

job of an engineer was to ensure maximum performance with a minimum investment of energy,

materials, and time, be it the performance of machines (mechanical engineering), communication

systems (communication engineering) or human bodies (scientific management, time and motion

studies). Inspired by modern engineering, the avant-garde of the 1920s tried to systematically

apply its principles to vision.

To engineer vision meant to eliminate waste, to use minimal material resources. Thus,

constructivist graphic design streamlined typography, eliminating complicated typefaces in favor

of block letters consisting of straight lines; it also eliminated illustrations and "wasteful"

decorations by making type itself the main element of design. The goal: maximum impact with

minimum use of ink.

To engineer vision also meant to minimize the psycho-physical resources required of the viewer.

Vertov writes in his famous 1923 manifesto: "The least advantageous, the least economical

communication of a scene is theatrical communication."

330

In contrast, montage forces the eye to see the

right thing at the right time, thus eliminating the visual waste of theater, ballet, painting, and other

traditional forms. In montage, "camera drags the eyes of a film viewer from hands to legs, from legs to

eyes and the rest in the most advantageous order..."

331

330

Vertov, "Kinoki," 139.

331

Ibid., 139. Emphasis in the original -- L.M.

7. Conclusion: the Labor of Perception

To engineer vision also meant to ensure perception in the shortest possible time. Here as

well, the avant-garde promoted montage as an example of possible economy, in this case

economy of time. Maud Lavin describes the 1930 manifesto of the group of leading German

designers headed by Kurt Schwitters: "Walter Dexel writes that modern man has the right to

expect communications in the shortest possible time. Willi Baumeister points out that

photomontage is efficient, allowing for the quick grasp of several images at once...Similarities

between photomontage and film are often emphasized, with photomontage being considered a

quicker, more efficient medium."

332

Finally, to engineer vision also meant to be able to measure its efficiency, or, to use the language

of a communication engineer, to measure "system performance." Eisenstein, fresh from engineering

school, invented his first theory of artistic communication, the famous "montage of attractions":

"...Laboratory analyses and diagrams...Mendeleev's Periodic Table and the laws of Gay-Lussac and Boyle

Mariotte in the realm of art!...Let us therefore search for the unit which will measure the influence exerted

by art! Science has its "ions," its "electrons," its neutrons." Art will have -- attractions!"

333

In its desire to engineer vision, the avant-garde was ahead of its time. The systematic engineering

of vision took place only after World War II when vision become the major instrument of labor,

the most productive organ of a worker in a human-machine system. To ensure the maximum

performance of such a system, it became necessary to engineer it around the capacities and the

limitations of human vision. It also became necessary to understand vision in a new way: as

information processing.

332

Maud Lavin, "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of

New Advertising Designers," in Montage and Modern Life: 1919-1942, ed. Matthew Teitelbaum
(Cambridge: The MIT Press, 1992), 54.

333

Qtd. in Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and

Indiana University Press, 1987), 41. Emphasis mine -- L.M.

This is the history of the engineering of vision from INKhUK to MIT: from the Soviet art

institute of the 1920s which gathered the avant-garde artists united by the idea of engineering

vision -- to the premier engineering school where much of this engineering later occurred

through the work on information theory, on radar and computer graphics, on new human-

machine interfaces, on algorithms of human vision.

This dissertation begins at the scene of heated debates at the Soviet art institutes in the

early 1920s. It ends with the recent research by cognitive scientists and engineers at MIT. What

justifies this temporal continuity, on the one hand, and disciplinary shift, on the other hand --

from the Soviet artistic avant-garde to one of the premier science institutions in the world, which

today is in the forefront of research on robotics, artificial intelligence, and human-computer

interfaces? What can be in common between Soviet constructivism and American cognitive

science?

Both are the product of the age of engineering. Engineering, the quintessentially modern

profession, appeared in the 1870s. The job of an engineer was to ensure maximum performance -

- of machines, human bodies or communication systems -- with minimum investment of energy,

materials, and time. The engineer was the new specialist in the efficiency of any productive

process, regardless of its nature. And this ideology of "engineerism" became the modern myth,

the religion of a society aspiring to total rationalization.

334

In order to employ vision efficiently

in its new roles, whether as a medium of propaganda and advertising or as the channel of

communication between human and machine, artists, designers, and scientists adopted the

engineering approach as well. It is not surprising that Soviet constructivists, the pioneers of

modern propaganda design, insisted on calling themselves engineers rather than artists.

It is also not surprising that the archeology of twentieth century vision would uncover

numerous similar solutions to the problem of its rationalization in such seemingly distant fields

as film and robotics, graphic design and experimental psychology. For instance, let us compare

two statements: one by a designer, the other by a psychologist. El Lissitsky wrote in the 1920s

regarding his new geometric style of poster design: "The most unambiguous and immediately

334

Max Weber proposed rationalization as an umbrella concept to describe the social processes

of modernity. See, for instance, From Max Weber: Essays in Sociology (New York: Oxford
University Press, 1958).
Max Weber, General Economic History (New York, 1927).

Conclusion

recognizable forms are geometric forms. No one will confuse a rectangle with a circle, or a circle

with a triangle."

335

In 1951, the already mentioned experimental psychologist Paul Fitts echoed

practically the same words in his summary of research on the use of graphic symbols for human-

machine interfaces: "Geometrical figures differ considerably in legibility. Straight lines have

been found to be more legible than curved ones. The triangle, the rectangle, and the square have

been reported to be more easily recognized under conditions of low illumination and in

peripheral vision than circular and hexagonal forms."

336

Both Lissitsky and Fitts were concerned

with the same problem -- reliability and speed in the recognition of visual symbols -- and,

therefore, it is logical that they recommended the same solution.

Whether filmmakers or psychologists, engineers or interface designers, twentieth century

professionals have understood vision as a medium of communication and have attempted to put

its use on a scientific basis. If in the early part of the century this research focused on human

communication (the human as the subject of mass propaganda and mass entertainment), after

World War II, the focus shifted to human-machine interface. Human vision became the key

instrument of post-industrial labor as the channel of communication between human and

machine.

This history can be summarized by three images. The first image: a portrait of Tatlin by

Lissitsky (fig. 16). A compass, extending straight from Tatlin's eye, a metaphor of vision for

work.

The second image, a popular icon in the late 1980s and early 1990s, promoted virtual

reality interface designed at NASA/Ames Human Factors Research Center (fig. 22).

337

Instead

335

Qtd. in Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 24.

336

Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1298.

337

On NASA/Ames virtual reality research in the 1980s, see Scott S. Fisher, "Virtual Interface

Environments," in The Art of Human-Computer Interface Design, ed. Brenda Laurel (Reading,
Mass.: Addison-Wesley Publishing Company, 1990): 423-438.

of the metaphor of the eye-compass, a reality: video monitors strapped to the eyes. The notion of

vision as work is now fully realized: the operator wearing the gear works by mentaly processing

visually presented information. The gear is designed using all the available knowledge

accumulated by experimental psychology about human vision. In the photograph we see the last

leftover from the age of manual labor -- an arm in a DataGlove. It will soon disappear since

through gaze tracking the operator can control the system by merely looking at different points in

virtual space.

What is the third image which stands between these two? Let me quote from the

description of the history of engineering psychology found in an 1965 overview of the field:

The primary emphasis in time-and-motion engineering has been on man as a worker; that is, as a
source of mechanical power. It was not until World War II that a new category of machines
appeared -- machines that made demands not upon the operator's muscular power, but upon his
sensory, perceptual, judgmental, and decision-making abilities. The job of a radar operator, for
example, requires virtually no muscular effort, but makes severe demands on sensory capacity,
vigilance, and decision-making ability. This new class of machines raised some intricate and
unusual questions about human abilities: How much information can a man absorb from a radar
screen?

338

The World War II radar operator is this third image, a historical intermediary between Lissitsky's

man-compass (1920s) and NASA's virtual reality user (1980s). The figure of a radar operator

stands at the gates to the post-industrial society of perceptual labor, at the origins of research on

human-machine interface, and the understanding of human vision as information processing (fig.

23).

"I am a mechanical eye," wrote Dziga Vertov in 1923.

339

338

Alphonse Chapanis, Man-Machine Engineering (Bemont, CA: Wadsworth Publishing

Company, Inc., 1965), 9-10.

339

Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 141.

Allison, Nicholas H., ed. Art Into Life: Russian Constructivism 1914-1932. New York: Rizzoli
International Publications, 1990.

Anderson, John Robert. Cognitive Psychology and Its Implications. W.H. Freeman and
Company, 1980.

ArgŸelles, JosŽ. Charles Henry and the Formation of a Psychophysical Aesthetics. Chicago:
University of Chicago Press, 1972.

Arnheim, Rudolf. "A Plea for Visual Thinking." In New Essays on the Psychology of Art. Los
Angeles and London: University of California Press, 1986.

Arnheim, Rudolf. Art and Visual Perception. The New Version. Berkeley: University of
California Press, 1974.

Attneave, F. Applications of Information Theory to Psychology. New York: Holt, 1959.

Aumont, Jacques. Montage Eisenstein. London and Bloomington: BFI Publishing and Indiana
University Press, 1987.

Barthes, Roland. "Rhetoric of the Image." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.

Barthes, Roland. "The Photographic Message." In Image, Music, Text, edited by Stephen Heath.
New York: Hill and Wang, 1977.

Barthes, Roland. "The Third Meaning." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.

Baxandall, Michael. Painting and Experience in Fifteenth Century Italy. Oxford: Oxford
University Press, 1972.

Beliajeva, E.L. Arkhitekturno-prostranstvennaya sreda soroda kak obyekt zritelnogo vospriyatiya
(Architectural and spacial city environment as an object of visual perception). Moscow:
Stroyizdat, 1977.

Benjamin, Walter. "On Some Motives in Baudelaire." In Illuminations, edited by Hannah
Arendt. New York: Schochen Books, 1969.

Bergin, Thomas Goddard and Max Harold Fisch. The New Science of Giambattista Vico. Ithaca:
Cornell University Press, 1968.

Berlyne, D.E. "Novelty, Complexity, and Hedonic Value." Perception and Psychophysics 8
(1970): 279-86.

Borch-Jakobson, M. The Absolute Master. Stanford: Stanford University Press, 1991.

Braun, Marta. Picturing Time: the Work of Etienne-Jules Marey (1830-1904). Chicago: The
University of Chicago Press, 1992.

Bibliography

Brik, Osip. "T.n. 'formalnyi metod'" (So called 'formal method'). LEF 1 (1923): 214-215.

Broadbent, D.E. Perception and Communication. Oxford: Pergamon Press, 1958.

Bryson, Norman. Looking at the Overlooked: Four Essays on Still Life Painting. Cambridge:
Harvard University Press, 1990.

Buck-Morss, Susan. The Dialectics of Seeing: Walter Benjamin and the Arcades Project.
Cambridge: The MIT Press, 1989.

Bulgakova, Olga. "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu psikhoanalizom
i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin': between
psychoanalysis and structural psychology). Kinovedcheskie Zapiski 2 (1988): 174-191.

Chapanis, Alphonse. Man-Machine Engineering. Bemont, CA: Wadsworth Publishing Company,
Inc., 1965.

Chapanis, Alphonse, Wendell R. Garner, and Clifford T. Morgan. Applied Experimental
Psychology. New York: John Wiley & Sons, Inc., 1949.

Chen, Jack. Soviet Art and Artists. London: The Pilot Press, Ltd., 1945.

Cherry, Colin. On Human Communication. 2nd ed. Cambridge: The MIT Press, 1968.

Cohen, Gillian. The Psychology of Cognition. 2nd ed. (London and New York: Academic Press,
1983).

Cohen, Paul R. and Edward A. Feigenbaum, eds. The Handbook of Artificial Intelligence. Los
Atlos, CA: William Kaufmann, Inc., 1982.

Cook, Richard. "Serious Entertainment." Computer Graphics World (May 1992): 40-48.

Crary, Jonathan. Techniques of the Observer: on Vision and Modernity in the Nineteenth
Century. Cambridge: The MIT Press, 1990.

Damisch, Hubert. L'origine de la perspective. Paris: Flammarion, 1987.

Danziger, Kurt. Constructing the Subject: Historical Origins of Psychological Research.
Cambridge: Cambridge University Press, 1990.

De Fleur, Melvin L. and Sandra Ball-Rokeach. Theories of Mass Communication. New York:
Longman, 1975.

De Landa, Manuel. War in the Age of Intelligent Machines. New York: Zone Books, 1991.

Deleule, Didier. "The Living Machine: Psychology as Organology." In Incorporations, edited by
Jonathan Crary and Sanford Kwinter. New York: Zone Books, 1992.

Descartes, RenŽ. "Meditations on the First Philosophy," In The Rationalists. Garden City, NY:
Doubleday & Company, 1960.

Dorfman, D. and H. McKenna. "Pattern Preference as a Function of Pattern Uncertainty."
Canadian Journal of Psychology 20 (1966): 143-53.

Druckrey, Timothy. "Revenge of the Nerds. An Interview with Jaron Lanier." Afterimage (May
1991): 5-9.

Dudley, Andrew. Concepts in Film Theory. Oxford: Oxford University Press, 1984.

Dulac, Germaine. "Aesthetics, Obstacles, Integral CinŽgraphie." In vol. 1, French Film Theory
and Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988.

Eames, Charles and Ray Eames. A Computer Perspective: Background to the Computer Age.
Cambridge: Harvard University Press, 1990.

Echoes of War. Boston: WGBH Boston, n.d. Videotape.

Edgerton, Samuel. The Renaissance Rediscovery of Linear Perspective. New York: Basic Books,
1975.

Eduardo, Kac. "Aspects of the Aesthetics of Telecommunications." In SIGGRAPH '92 Visual
Proceedings, edited by John Grimes and Gray Lorig. New York: The Association for Computing
Machinery, 1992.

Edwards, Paul, ed. Encyclopedia of Philosophy. New York: Macmillan, 1967.

Eisenstein, Sergei. "A Dialectical Approach to Film Form." In Film Form: Essays in Film
Theory, edited by Jay Leyda. New York: Harcourt Brace and World, 1949.

Eisenstein, Sergei. "Montazh atraktsionov" (Montage of attractions). LEF 3 (1923): 70-75.

Epstein, Jean. "On Certain Characteristics of PhotogŽnie." In vol. 1, French Film Theory and
Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988).

Estes, William. "Experimental Psychology: an Overview." In The First Century of Experimental
Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers,
1979.

Farah, Martha. "Is Visual Imagery Really Visual? Overlooked Evidence from
Neuropsychology." Psychological Review 95, no. 3 (1988): 307-317.

Farocki, Harun. "Reality Would Have to Begin." Documents 1/2 (1992): 136-146.

Finke, Ronald A. Principles of Mental Imagery. Cambridge: The MIT Press, 1989.

Fisher, Scott S. "Virtual Interface Environments." In The Art of Human-Computer Interface
Design, edited by Brenda Laurel. Reading, Mass.: Addison-Wesley Publishing Company, 1990.

Fitts, Paul. "Engineering Psychology and Equipment Design." In Handbook of Experimental
Psychology, edited by S.S. Stevens. New York and London: John Wiley & Sons, Inc., 1951.

Flew, Antony, ed. A Dictionary of Philosophy. London: Pan Books Ltd., 1984.

Florensky, Pavel. "Obratnaya Perspektiva" (The Inverted Perspective). In vol. 1, Sobranie
Sochineniy, ed. N.A. Struve. Paris: YMCA-PRESS, 1985.

Florensky, P.A. "Symbolarium (Slovar simvolov). Predislovie" [Symbolarium (The dictionary of
symbols). Introduction]. Trudy po znakovym sistemam V. Tartu: Uchebnye zapiski Tarturskogo
Universiteta 284, 1971.

Foster, Hal, ed., Vision and Visuality. Seattle: Bay Press, 1988.

Freud, Sigmund. The Interpretation of Dreams. New York: Avon Books, 1965.

Freud, Sigmund. Standard Edition of the Complete Psychological Works. London: Hogarth
Press, 1953.

Friedhoff, Richard Mark and William Benson. The Second Computer Revolution: Visualization.
W.H. Freeman and Company, 1991.

Gabor, D. "A Summary of Communication Theory." In Proceedings of a Symposium on
Applications of Communication Theory, London, 1952, edited by Willis Jackson. London:
Butterworth Scientific Publications, 1953.

Gardner, Martin. Logical Machines and Diagrams. 2nd ed. Chicago: The University of Chicago
Press, 1982.

Golomstock, Igor. Totalitarian Art. New York: HarperCollins Publishers, 1990.

Gombrich, E.H. "Pictorial Instructions." In Images and Understanding, ed. Barlow Horace, Colin
Blakemore and Miranda Weston-Smith. Cambridge: Cambridge University Press, 1990.

Gombrich, E.H. Art and Illusion. Princeton: Princeton University Press, 1960.

Goodman, Nelson. Languages of Art. Indianapolis: Hackett, 1976.

Hall, Stuart. "Encoding/Decoding." In Culture, Media, Language, ed. Stuart Hall et al.
Hatchinson, 1980.

Hansen, Miriam. "The Hieroglyph and the Whore: D.W. Griffith's Intolerance." The South
Atlantic Quarterly 88, no. 2 (1989): 361-392.

Hartley, R.V.L. "Transmission of Information." Bell System Tech. Journal 7 (1928): 535-563.

Hearst, Eliot. "One Hundred Years: Themes and Perspectives." In The First Century of
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
1979.

Heidegger, Martin. What is Called Thinking? New York: Harper & Row, Publishers, 1968.

Heron, W. "The Pathology of Boredom." Scientific American 196 (1957): 52-56.

Hochberg, Julian. "Sensation and Perception." In The First Century of Experimental Psychology,
edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, 1979.

Holly, Michael Ann. Panofsky and the Foundations of Art History. Ithaca: Cornell University
Press, 1984.

Holmes, Oliver Wendel. "The Stereoscope and the Stereograph." In Photography, Essays and
Images: Illustrated Readings in the History of Photography, edited by Beaumont Newhall. New
York: Museum of Modern Art, 1980.

Hoover, Theodore and Charles Fish. The Engineering Profession. Stanford: Stanford University
Press, 1941.

"Institut Khudozestvennoy Kultury" (The Institute of Artistic Culture). Russkoe Iskusstvo 2-3
(1923): 85-88.

Ivins, William M. On the Rationalization of Sight. New York: Da Capo Press, Inc., 1975.

Jakobson, Roman. "Closing Statement: Linguistics and Poetics." In Semiotics. An Introductory
Anthology, edited by Robert Innis. Bloomington: Indiana University Press, 1985.

Jakobson, Roman. "Metalanguage as a Linguistic Problem," Presidential address delivered at the
Annual Meeting of the Linguistic Society of America, 1956. In The Framework of Language.
Ann Arbour: Michigan Studies in the Humanities, 1980.

Jay, Martin. Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought.
Berkeley: The University of California Press, 1993.

Jay, Martin. "Scopic Regimes of Modernity." In Vision and Visuality, ed. Hal Foster. Seattle:
Bay Press, 1988.

Johnson-Laird, Philip. Mental Models: Towards a Cognitive Science of Language, Inference,
and Consciousness. Cambridge: Cambridge University Press, 1983.

Joravsky, David. Russian Psychology: A Critical History. Cambridge, Mass.: Basil Blackwell,
1989.

Kandinsky, Wassily. Point and Line to Plane. New York: Solomon R. Guggenheim Foundation,
1947.

Kandinsky, Wassily. "Programma raboty Instituta khudozhestvennoy kultury" (The program of
the Institute of Artistic Culture). In Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia
(Fifteen years of Soviet art: materials and documentation), ed I. Matsa (Moscow, IZOGIZ:
1933).

Kantowitz, Barry H. and Robert D. Sorkin. Human Factors: Understanding People-System
Relationships. John Wiley & Sons: 1983.

Keller, Evelyn Fox and Christine R. Grontkowski. "The Mind's Eye." In Discovering Reality:
Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy of
Science, edited by Sandra Harding and Merrill B. Hintikka. London: D. Reidel Publishing
Company.

Kemp, Martin. The Science of Art. New Haven: Yale University Press, 1990.

Kenez, Peter. The Birth of the Propaganda State. Soviet Methods of Mass Mobilization, 1917-
1929. Cambridge: Cambridge University Press, 1985.

Khan-Magomedov, S.O. "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920"
(INKhUK: appearance, formation and first period of its work, 1920). Sovetskoe Iskusstvoznanie
(1981): 332-368.

Koerner, Joseph Leo. "The Shock of the View." The New Republic (April 26, 1993): 32-38.

Kogan, P.C. "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its
journal). Iskusstvo 1, no. 1 (1923).

Kovasznay, L.S.G. and H.M. Joseph. "Image Processing." Proceedings of IRE 43 (1955): 560-
570.

Kozulin, Alex. Psychology in Utopia: Toward a Social History of Soviet Psychology.
Cambridge: The MIT Press, 1984.

Krauss, Rosalind. The Optical Unconscious. Cambridge: The MIT Press, 1993.

Lacan, Jacques. "On the Gaze as Objet Petit a." In The Four Fundamental Concepts of Psycho-
Analysis, edited by Jacques-Alain Miller. New York: W.W. Norton & Company, 1981.

Lakoff, George. "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"
Cognitive Linguistics 1, no. 1 (1990): 39-74.

Lakoff, George. "Cognitive Linguistics." Versus 44/45 (1986): 119-154.

Latour, Bruno. "Visualization and Cognition: Thinking with Eyes and Hands." Knowledge and
Society: Studies in the Sociology of Culture Past and Present 6 (1986): 1-40.

Lavin, Maud. "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of New
Advertising Designers." In Montage and Modern Life: 1919-1942, edited by Matthew
Teitelbaum. Cambridge: The MIT Press, 1992.

Lavrent'ev, A.H. "Propedevticheskaya distsiplina 'Grafika.' Vkhutemas. 1920-22 gody" (The
discipline 'Graphics.' VKhUTEMAS. 1920-22). Tekhnicheskaya Estetika 7 (1984): 16-21.

Lefebvre, Henri. The Production of Space. Oxford: Blackwell Publishers, 1991.

LŽvy-Bruhl, Lucien. Les functions mentales dans les sociŽtŽs primitives. Paris: 1910.

Locke, John. An Essay Concerning Human Understanding. Edited by A.S. Pringle-Pattison.
Oxford: Clarendon Press, 1924.

Lotman, Jurij. The Structure of the Artistic Text. Ann Arbour: Department of Slavic Languages
and Literatures, The University of Michigan, 1977.

Lowe, David. Three-Dimensional Object Recognition from Single Two-Dimensional Images.
Robotics Report 62. New York: Courant Institute of Mathematical Sciences, New York
University, 1986.

Mandler, Jean Matter and George Mandler, eds. Thinking: From Association to Gestalt. New
York: John Wiley & Sons, Inc., 1964.

Manovich, Lev. "Assembling Reality: Myths of Computer Graphics." Afterimage 20, no. 2
(September 1992): 12-14.

Manovich, Lev. "'Real' Wars: Esthetics and Professionalism in Computer Animation." Design
Issues 6, no. 1 (Fall 1991): 18-25.

Marr, David. Vision. New York: W.H. Freeman and Company, 1982.

Matsa, I., ed. Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (Fifteen years of Soviet
art: materials and documentation). Moscow, IZOGIZ: 1933.

McArthur, David. "Computer Vision and Perceptual Psychology." Psychological Bulletin 92, no.
2 (1982): 283-309.

McFarlane, M. D. "Digital Pictures Fifty Years Ago." Proc. IEEE 60, no. 7 (1972).

McGraw-Hill Encyclopedia of Science & Technology: an International Reference Work in
Twenty Volumes Including Index. New York: McGraw-Hill, 1992.

Michelson, Annette. "Reading Eisenstein Reading 'Capital'." October 2 (1976): 27-38; October 3
(1977): 82-88.

Miller, G.A. Language and Communication. New York: McGraw-Hill Book Co., Inc., 1951.

Mitchell, J. William. The Reconfigured Eye: Visual Truth in the Post-Photographic Era.
Cambridge, The MIT Press, 1992.

Mitchell, W.J.T. Iconology. Image, Text, Ideology. Chicago: The University of Chicago Press,
1985.

Moholy-Nagy, L‡szl—. "Photography in Advertising." In Photography in the Modern Era,
edited by Christopher Phillips. New York: Aperture, 1989.

Moholy-Nagy, L‡szl—. The New Vision. New York: Wittenborn, 1947.

L‡szl— Moholy-Nagy. Painting, Photography, Film. 1925. Reprint. Cambridge: The MIT Press,
1973.

Molok, Yu. A. "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel Florensky's
'dictionary of symbols.' A few margins). Sovetskoe Iskusstvoznanie 26 (1990): 322-343.

MŸnsterberg, Hugo. The Photoplay: A Psychological Study. New York: D. Aplleton & Co.,
1916.

Newhall, Beaumont. Airborne Camera. New York: Hastings House, Publishers, 1969.

Nietzsche, Friedrich. "On Truth and Falsity in their Ultramoral Sense." In Early Greek
Philosophy and Other Essays. New York: The Macmillan Company, 1924.

Noble, Douglas. "Mental Materiel: The Militarization of Learning and Intelligence in U.S.
Education." In Cyborg Worlds: the Military Information Society, edited by Les Levidov and
Kevin Robins. London: Free Association Books, 1989.

Nšth, Winfried. Handbook of Semiotics. Bloomington and Indianapolis: Indiana University
Press, 1990.

Panofsky, Erwin. Perspective as Symbolic Form. New York: Zone Books, 1991.

Peat, David. Artificial Intelligence: How Machines Think. New York: Baen Enterprises, 1985.

Pinker, Steven. "Visual Cognition: An Introduction." Cognition 18 (1984): 1-63.

Posner, Michael I. and Gordon L. Shulman. "Cognitive Science." In The First Century of
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979.

Posner, Michael I. "Section V: Information Processing. Overview." In vol. 2, Handbook of
Perception and Human Performance, edited by Kenneth Boff, Lloyd Kaufman, and James P.
Thomas. New York: John Wiley and Sons, 1986.

Quastler, H., ed. Information Theory in Psychology: Problems and Methods. Glencoe, IL.: Free
Press, 1955.

Rabinbach, Anson. The Human Motor: Energy, Fatigue, and the Origins of Modernity. Basic
Books, Inc., 1990.

Rappoport, A.G. "El Lissitsky i ego 'Pun-geometiya'" (El Lissitsky and his 'Pan-geometry').
Sovetskoe Iskusstvoznanie 25 (1989): 113-130.

Reichardt, Jasia. The Computer in Art. London and New York: Studio Vista and Van Nostrand
Reinhold Company, 1971.

Resnikov, Howard. The Illusion of Reality. New York: Springer-Verlag New York Inc., 1989.

Reveaux, Tony. "Virtual Reality Gets Real." New Media (January 1993): 36-41.

Rheingold, Howard. Virtual Reality. New York: Simon & Schuster, Inc., 1991.

Riggs, L.A. et al. "The Disappearance of Steadily Fixated Visual Tests Objects." J. Optical Soc.
America 43 (1953): 495.

Roberts, L.G. "Machine perception of three-dimensional solids." In Optical and Electo Optical
Information Processing, edited by J.T. Tippett. Cambridge: The MIT Press, 1965.

Roberts, L.G. Homogeneous Matrix Representations and Manipulation of N-Dimensional
Constructs. MIT Lincoln Laboratory MS 1405, 1965.

Roberts, L.G. Machine Perception of Three-Dimensional Solids. MIT Lincoln Laboratory TR
315, 1963.

Rose, Jacqueline. Sexuality in the Field of Vision. London: Verso, 1986.

Saint-Martin, Fernande. Semiotics of Visual Language. Bloomington and Indianapolis: 1990.

Sampson, Geoffrey. Schools of Linguistics. Stanford, CA: Stanford University Press, 1980.

Sardarov, A.S. "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic
sign). Tekhnicheskaya Estetika 9 (1984): 14-19.

Scha, Remko. "Virtual Voices." MediaMatic 7, no. 1 (1992): 27-43.

Schramm, Wilbur. "How Communication Works." In The Process and Effects of Mass
Communication. Urbana: University of Illinois Press: 1954.

Sekula, Allan. "The Instrumental Image: Steichen at War." In Photography against the Grain:
Essays and Photo Works, 1973-1983. Halifax: The Press of the Nova Scotia College of Art and
Design, 1984.

Sekula, Allan. "The Body and the Archive." October 39 (1987): 3-64.

Shannon, Claude E. and Warren Weaver. The Mathematical Theory of Communication. Urbana:
The University of Illinois Press, 1949.

Shepard, S. and J. Metzler. "Mental Rotations of Three-dimensional Objects." Science 171
(1971): 701-703.

SIGGRAPH '89 Panel Proceedings. New York: The Association for Computing Machinery,
1989.

SIGGRAPH '92 Final Program. New York: The Association for Computing Machinery, 1992.

Sonesson, Gšran. Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance for
the Analysis of the Visual World. Lund, Sweden: Lund University Press, 1989.

Spriegel, William R. and Clark E. Myers, eds. The Writings of the Gilbreths. Homewood, IL.,
1953.

Stelzer, Otto. "Moholy-Nagy and His Vision." In L‡szl— Moholy-Nagy, Painting, Photography,
Film. 1925. Reprint, Cambridge: The MIT Press, 1973.

Taylor, Brandon. Art and Literature under the Bolsheviks. London: Pluto Press, 1991.

Taylor, Franklin. "Psychology at the Naval Research Laboratory." American Psychologist 2, no.
3 (1947): 87-92.

Taylor, Frederick Winslow. The Principles of Scientific Managment. New York, 1967.

Titchener, Edward Bradford. A Beginner's Psychology. New York: The Macmillan Company,
1915.

Todorov, Tzvetan. Theories of the Symbol. Ithaca: Cornell University Press, 1984.

Tretyakov, Sergei. "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to?
(Perspectives of futurism)]. LEF 1 (1923): 192-203.

Vasilieva, Yu. A. "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the concept of
'aggressive art'). Kinovedcheskije Zapiski 3 (1989), 205-209.

Vertov, Dziga. "Kinoki. Perevorot" (Kinoki. A revolution). LEF 3 (1923): 135-143

Virilio, Paul. Lost Dimension. New York: Semiotext(e), 1991.

Virilio, Paul. War and Cinema: the Logistics of Perception. London: Verso, 1989.

Viteles, Morris. Industrial Psychology. New York: W.W. Norton & Company, Inc., 1932.

Vitz, P.C. "Preference for Different Amounts of Visual Complexity." Behavioral Science 2
(1966): 105-14.

Vitz, Paul C. and Arnold B. Glimcher. Modern Art and Modern Science. The Parallel Analysis
of Vision. New York: Praeger, 1984.

Vygotsky, Lev. Psikhologija Iskusstva (Psychology of art). Moscow, 1968.

Weber, Max. From Max Weber: Essays in Sociology. New York: Oxford University Press, 1958.

Weber, Samuel. The Legend of Freud. Minneapolis: University of Minnesota Press, 1982.

Wiener, Norbert. The Human Use of Human Beings. Cybernetics and Society. New York: Avon
Books, 1967.

Wiener, Norbert. Cybernetics. Of Control and Communication in the Animal and the Machine.
New York: John Wiley & Sons, Inc., 1948.

FIGURES

Figure 1. Humbert de Duperville's "Synoptic Table," 1827-32.
Figure 2. Charles Henry's aesthetic protractor, 1988.
Figure 3. Diagram illustrating Georges Seurat's theory of expression.
Figure 4. The exercises developed by Rodchenko for his course on graphics at VKhUTEMAS,
1921-22. Top: examples of basic forms given by Rodchenko. Bottom: example of a student
work.
Figure 5. Example of computer generated visualization.
Figure 6. Representations of propositions as Euler's diagrams, an earlier form of Venn's
diagrams. Max MŸller, The Science of Thought (London: Longmans, Green, and Co., 1887),
543.
Figure 7. Composite photography. Frontispiece from Francis Galton, Inquiries into Human
Faculty (London: Macmillan, 1883).
Figure 8. Freud's diagram illustrating the relation between two triads of concepts: id, ego, and
superego, and unconscious, preconscious, and conscious.
Figure 9. Top: Sigmund Freud's schematic picture of sexuality. Bottom: example of a modern
diagram.
Figure 10. Example of diagrammatic graphic language.
Figure 11. Figure from Monge's GŽomŽtrie descriptive, 1799.
Figure 12. Two draughtsmen plotting points for the drawing of a lute in foreshortening, from
Durer's Underweysung, 1525.
Figure 13. Wireframe computer graphics by Boeing, early 1960s.
Figure 14. ACRONYM computer vision system processes an aerial photograph, late 1970s.
Figure 15. Laser range finder, early 1980s.
Figure 16. El Lissitsky. Tatlin at Work. Photomontage. Illustration for Ilia Erenburg's Six Tales
with Easy Endings. 1922.
Figure 17. Wireless telegraphed photography.
Figure 18. Schematic diagram of a general communication system.
Figure 19. Example of a stimulus used in experiments on preference as a function of the amount
of information.
Figure 20. One of the earliest models of a human as information processing system.
Figure 21. The model of a human as information processor which summarizes the available data
(by 1986) about human sensory, cognitive and motor performance.
Figure 22. NASA Ames Virtual Reality Environment Workstation, late 1980s.
Figure 23. SAGE (the "Semi-Automatic Ground Environment") -- the first human-machine
interactive display system, mid 1950s.

Wyszukiwarka

Podobne podstrony:
Nicholls; The Secularization of Revelation from Plato to Freud
David Runciman Political Hypocrisy, The Mask of Power, from Hobbes to Orwell and Beyond (2008)
Morimoto, Iida, Sakagami The role of refections from behind the listener in spatial reflection
Idea of God from Prehistory to the Middle Ages
Morimoto, Iida, Sakagami The role of refections from behind the listener in spatial reflection
Kosky; Ethics as the End of Metaphysics from Levinas and the Philosophy of Religion
The Battle of Hattin from a Muslim source
Alfred North Whitehead The Aim of Philosophy (from Modes of Thought , Lecture IX) 1956
Mencej The Role of Legend in Constructing Annual Cycle
The Persistence of Vision John Varley
The Engines of Dawn Paul Cook
SPME for the identification of MVOCs from moldy
Russian and Chinese Methods of going from Communism to?moc
Kinesiotherapy is the application of scientifically?sed exercise principles?apted to enhance the str
On the Effectiveness of Applying English Poetry to Extensive Reading Teaching Fanmei Kong
0198752091 Oxford University Press USA The Character of Mind An Introduction to the Philosophy of Mi

więcej podobnych podstron