Eye of the Beholder

Visual art is quite simply made for the human eye. It synchronizes smoothly with how our visual system works, thanks in large part to the astute observations and skills of artists.

Over millennia, artists have learned to use color and other tools of their trade to stimulate the brain’s visual circuitry in ways that create illusions: depth on a flat surface, movement on a static plane, emotion in an empty space. One does not have to strain to experience these effects. They are simply there, produced by the inner workings of light-sensitive cells in the eye, the complex wiring of the retina, and the trunks and branches of the computational tree that forms the visual cortex.

Today, science can explain most, but not all, of how color vision works. Those explanations developed over years of reductionist thinking, meticulous experimentation, and systematic exploration. Yet artists, some working before Sir Isaac Newton revealed that colors were physical and quantifiable parts of what he described as light’s spectrum, have managed to feel their way into the visual system, the way a jazz musician might feel his way through an improvisation.

Let the Light In

When light shines on Launching the Curragh, a work by the Irish painter Paul Henry, the differently colored pigments in the paint absorb light of differing wavelengths in the visible spectrum. To a large extent, we see what’s reflected to our eyes: red light from the men’s trousers, yellow from the glistening sand and churning sea foam.

When light from an object or scene enters through the pupil, it is focused on the retina, a wisp of nervous tissue that is actually an extension of the brain. It is this layer of tissue that begins processing visual information in the eye. Its outermost layer is made up of more than a hundred million light-sensitive cells known as photoreceptors. Of these, six to seven million are cone cells that support color vision in humans. These millions of cells are concentrated at the center of the human retina in a region called the fovea.

There are three types of cone cells, each type sensitive to wavelengths of either blue, green, or red light. The tips of these cells are filled with photosensitive pigments known as opsins. When a photon strikes a photosensitive pigment, its chemical structure alters within milliseconds, triggering a cascade of events that results in the transmission of a signal to the brain. Once this series of actions is complete, the chemicals reset, allowing for the seemingly endless translation of light into electrical energy that is perceived by our brain as color.

The human visual system is capable of detecting wavelengths between 400 and 700 nanometers, a tiny sliver of the electromagnetic spectrum. Other wavelengths go unseen: the meters-long sound wavelengths that carry voice and song; the tinier-than-the-breadth-of-an-atom wavelengths of cosmic rays that bombard Earth from outer space; and the 100-nanometer wavelengths of ultraviolet light, imperceptible to the eye yet powerful enough to burn the skin.

Inky Depths

When John Dowling gives presentations with his wife about vision and art, their focus is on ancient Asian ink drawings. This focus makes sense to each of them: Judith Dowling is an expert in Asian art, and John, the Gordon and Llura Gund Research Professor of Neurosciences at Harvard University, has made it his life’s work to study how the retina works and how artwork, such as Asian ink drawings, takes advantage of the visual system’s complex wiring to give depth and richness to what is merely black ink on white paper.

The processing that performs this transformation begins inside the eye, inside the retina. The retina is a complex biological computer that, in some species, contains as many as eighty different subtypes of nerve cells and at least ten biologically distinct functional layers that process light signals. The photoreceptor cells that transduce light into electrical signals begin that process.

The photoreceptor cells wire to bipolar and horizontal cells in the outer retina, while the bipolar cells connect to ganglion and amacrine cells in the inner retina. These cells process the incoming signals by making contextual comparisons: How is a spot of light in the center of a field affected by the surrounding light? How is its color affected? Is the spot moving? The layers of wiring create what could be described as parallel streams of specialized movie tracks that represent spots, colors, and motion. The ganglion cells of the retina, which are the output neurons, send these tracks through the optic nerve to cells in the brain’s visual cortex for further processing.

“Your eyes are constantly comparing things, so what you see depends not only on what you’re looking at but also on the surrounding illumination,” says Dowling.

Filling In

Contrast often is at the heart of what makes eighth-century Asian ink drawings so stunning, says Dowling. In Viewing Plum Blossoms by Moonlight, a painting by the thirteenth-century Chinese artist Ma Yuan, mountains loom large and forbidding against pale skies even though the heart of the mountain is the same shade of gray as the sky. The eye fills in the mountain’s color as it should be, darker than the sky beyond.

“Visual perception is reconstructive and creative,” says Dowling. “These illusions work because of what you’re expecting to see when you see the outlines in the painting.”

The same holds for processing color. The retina goes beyond the detection of blue, green, and red by recognizing local areas of contrast: light and dark, red and green, blue and yellow.

“Contrast is the driver in the retina,” says Michael Marmor, ’66, a professor of ophthalmology at Stanford University School of Medicine and co-author of the 2009 book The Artist’s Eyes, which investigates vision and art history. “When we look at a scene, we’re looking at the complex relationships between forms and colors. We see an object as red not so much because it reflects a discrete wavelength, but because it is more red than the greenness of objects around it.”

The retina is so complex that even though Dowling has written two books describing it, neither he nor others have a complete picture of how it works. Recently, he helped launch an effort to use new electron microscopy techniques to construct a precise three-dimensional map of the human retina’s central region, which mediates high acuity vision. The work will require him and his colleagues to slice human retinas into thousands of tiny slivers, image them, and then reassemble the images into a complete wiring diagram.

All That Shimmers

Some artists seem to revel in making the visual system, and, by extension the brain, struggle. The French painter Claude Monet was one such artist.

When Monet and others began showing their work in the late 1800s, critics collectively labeled them Impressionists. At the time, the term was meant to be derogatory; now, it is a designation of membership in a pivotal art movement.

According to many, the painting that spurred the critics’ ire, and inspired their derisive label, was Monet’s Impression, Sunrise. In it, a fiery globe hovers over a gray-blue harbor.

And the globe shimmers.

Monet attained this effect through his choice of colors. Color has three components: wavelength, also called hue; saturation, which indicates a color’s purity or how free it is of admixture with white; and perceived brightness, which is called value by artists and luminance by scientists.

In Impression, Sunrise, it’s luminance that matters. Monet brushed in an orange sun that had the same relative brightness as the blue haze surrounding it. The equality of brightness in this painting has been demonstrated using photometry and gray-scale rendering. A gray-scale image, which shows brightness only, effectively erases the sun in the landscape. The sky seems to contain only mottled gray.

According to Margaret Livingstone, Takeda Professor of Neurobiology at HMS, by selecting two colors with the same relative brightness, Monet rattled the brain’s visual system. In the 1980s, Livingstone figured out how.

Livingstone and her mentor, the late David Hubel, Nobel Prize recipient and an HMS professor of neurobiology, were trying to map the visual cortex by recording the activity of individual cortical neurons as they applied differing inputs to stimulate an eye.

Around the same time, the lab received a set of slides of tissue from a visual cortex. The slides showed blobs of pale tissue against a dark stained background. To Livingstone and Hubel, the fixed tissue suggested that the visual cortex contained distinct pathways. Their recordings of individual cortical neurons suggested the same thing.

Livingstone used the slides to map her recordings and confirmed that the area of the visual cortex they were investigating contained bands of cells that route signals coming in from the eye to specialized branches of neurons downstream. One branch, dubbed by scientists the “where” pathway, deals with orientation and the location and movement of things in space. Another, the “what” pathway, handles color and shape.

“We have all these modules in our brains for seeing things,” Livingstone says. “They make us expert at seeing those things.”

Fooling the Experts

Monet’s sunrise, however, confuses these expert systems. The where pathway is color-blind: Different hues of the same brightness are identically processed just as black and white photography melds differently colored items of equal brightness. In other words, the where pathway doesn’t register the sun in Monet’s sky, but the what system finds clear boundaries between the orange sun and the blue sky.

According to Livingstone, when the edges in one pathway don’t match up with those in the other, the resulting effect is often some sort of motion—the shimmering of Monet’s sun is an example, as are the rotating snakes by Japanese psychologist Akiyoshi Kitaoka and the boogie-woogie cubes in the work of Dutch painter Piet Mondrian.

Over the years, Livingstone has presented her findings about how special and how specialized the visual processing pathways are. Despite her emphasis on the brain and vision in these talks, she’s learned that what people remember is the art.

“Artists are really good. They do experiments. They try different things and figure stuff out,” says Livingstone. “They are empirical. Artists are observers.”

Livingstone has traced one branch of visual processing specialization to the inferior temporal area, a region of the brain that processes faces. Dowling speculates that there is a specialized cortical area that processes color. He bases that speculation on case reports of brain lesions that destroy color vision, but nothing else. These patients see form and movement perfectly well but do not see color.

Ultimately, these branches must reunite so that we can perceive the visual world as a coherent, continuous space, the same ability that allows us to see the sun shimmering in Monet’s sky.

“This integration, is called the binding problem,” says Dowling. “Right now, no one has a clue as to how it all gets put back together.”

Life as Art

Above Livingstone’s desk hangs The Enigma, a piece of op art by French artist Isia Leviant.

“Someday I’m going to work on that,” she says.

So, in the interest of science, I became a test subject.

Standing before the print and scanning the image, I worked hard to find whatever it is that intrigues Livingstone. Photons bombarded my cone cells; the layers in my retina sent signals that revealed edges and purples and oranges to my cortex. Concentric circles and lines took form, but no illusions. I decided to hazard a guess.

“It’s moving in and out?”

No, but Livingstone encouraged me. “Surely you see it.”

My brain and eyes were working overtime, searching for “it.”

More encouragement. “Really? You don’t see the nasty motion? The rings have fuzzy stuff spinning in them.”

Not to me, they didn’t. I sat, defeated.

Then, I saw it.

In 2008, neurobiologists in Arizona and Spain figured out that the illusion in Leviant’s painting is related to microsaccades, tiny movements of the eye that occur automatically as we gaze at a fixed point, such as the central circle in The Enigma. But it’s what happens inside the cortex that interests Livingstone.

Even though Livingstone has studied the visual system for decades, she doesn’t understand it completely. It’s a puzzle. Perhaps the same puzzle as the great binding problem described by Dowling.

Either way, the artist has figured something out. “I wouldn’t look at it too long,” says Livingstone. She laughs. “It’s dreadful.”

“I do like it, though,” she adds.

Elizabeth Dougherty is a science writer based in Massachusetts.

Images: Album/Art Resource, NY (top); Digital Image copyright The Museum of Modern Art/Licensed by SCALA/Art Resource, NY (Mondrian); Edward Campbell (Marmor); The Metropolitan Museum of Art/Gift of John Crawford, Jr., in honor of Alfreda Murck, 1986 (Yuan); John Soares (Livingstone)