Discovering the World’s Depth Later in Life

Article written by Gordy Slack

The full tsunamic impact of Virtual Reality (VR) technology has been described by industry experts and futurists as “only five-to-ten years away” for at least three decades now. Several devices are available for less than $1,000 today: the Oculus Rift, Samsung Gear, Playstation VR, and the HTC Vive. But for now they are restricted mostly to high-end game playing; battling ogres, mock warfare, and intergalactic travel. Who knows when in fact the virtual wave will disrupt the actual shore? But even before it does, the lab of UC Berkeley professor of optometry and vision science, Dennis Levi, is exploring a way to use the new generation of immersive VR devices, such as the HTC Vive, to help correct a notoriously difficult and widespread vision problem for millions of people around the world that do not perceive depth. In a June 2016 study published in Philosophical Transactions, Levi’s team used an expensive and relatively elaborate virtual reality set-up (not the commercially available HTC device) in their lab to help boost the depth perception for six of their 11 adult test subjects who were stereo-blind or stereo-deficient. Given that depth perception was for decades considered untrainable in adults, these results have profound implications for the recovery of stereo vision late in life. “More broadly,” the study concludes, “our approach demonstrates the potential power offered by VR for perceptual training of all kinds.”

One such stereo-blind person was Eric Gillet, Berkeley financial analyst, circus performer, photographer, martial arts enthusiast, and romantic. Gillet always knew that the world was a deep place. So he was surprised to learn, at age 38, that there was a common kind of depth that he was incapable of perceiving. The experience of the world as an immersive three-dimensional space, called stereopsis, is produced by a part of the visual cortex that fuses the offset images from each eye into one deep world, filled not only with things but with the empty spaces between them. Gillet could not see those spaces.

Most people who have full depth perception take it for granted. They develop the ability as infants and it quickly becomes one of the fundamental characteristics of their world. But for at least ten percent of North Americans, vision abnormalities preclude stereopsis. Usually, the problem arises because one eye is stronger than the other (amblyopia) or because the eyes are misaligned (strabismus). In both cases, the two images sent to the brain’s visual cortex, where they usually are interpreted and formed into coherent mental pictures, cannot be fused into one, deep, holistic image. To avoid the tangle of conflicting signals, the brain often opts either to ignore most information coming from one eye (which then further weakens) or to alternate back and forth between the eyes. Both strategies avoid the confusion of equivocal inputs, but neither is compatible with strong stereo vision.

Until Gillet showed up for a routine eye exam at the UC Berkeley Optometry Clinic fifteen years ago, he had no idea he was stereo-deficient. “I thought I knew what 3D-vision was, and I thought I had it,” he says. Upon hearing the diagnosis, Gillet remembers both wondering what he was missing and asking the optometrist if there was anything he could do to get it.

“He told me there was no way. It was too late,” says Gillet.

Bad news for sure, a philosophical Gillet says, but not devastating. On a practical level, stereo deficiency can make some tasks (like threading a needle or pouring cream into coffee) more difficult, but it is hardly disabling. We rely on many other cues to judge depth—the occlusion of distant objects by nearer ones, parallax, adjusting focal fields, and the fact that more distant images appear smaller, to name a few key ones—and most people can get along just fine with those alone. Not seeing the point of mourning the loss of something he couldn’t have, Gillet focused elsewhere.

Until, that is, about ten years later when he was contacted by Professor Levi, who was exploring the use of video games as tools for jump-starting stereo vision in those, like Gillet, who did not have it. For the past two decades, results from the Levi Lab have suggested that the visual processing components of the brain are much more plastic than had previously been assumed. In 1981, neurophysiologists David Hubel and Torsten Wiesel shared the Nobel Prize in Medicine partly for work showing that there was an unforgiving deadline for developing certain visual processes, including stereopsis. It was thought that if you didn’t develop stereo vision by about the age of eight, for example, you simply never would.

“This idea of a hardwired critical period became a core belief in neuroscience,” says Levi. “But it turned out not to be true. Our studies and others showed that there is a lot more plasticity than everyone thought.”

In 2012, in a collaboration with University of Rochester professor Daphne Bavelier and Rochester Institute of Technology professor Jessica Bayliss, Levi devised a video game that would “specifically attack some of the main issues with amblyopia,” he says. They took an open-source video game and divided the screen into two, so separate images reaching each eye could be independently controlled. That way, investigators could both offset the images a little and present a strong, high-contrast image to a participant’s weak eye and turn down the gain on the image going to the strong eye. In order to progress in
the game, a player has to learn to use both eyes together. Improvements in stereo vision were impressive.

The key, explains Levi Lab post-doctoral researcher Adrian Chopin, is to give a participant with weak or no detectable stereo vision plenty of monocular cues, along with stereo ones, when they start out. Then, slowly withdraw more and more of the monocular cues (such as shadows, occlusion, and motion parallax) until—like eventually learning to balance on a bicycle without training wheels—the participant learns to rely on the stereo cues alone. In well more than half the cases, the participant’s mastery of stereo vision becomes strong enough that they begin to experience stereopsis.

Gillet first saw in stereo while he was participating in Levi’s study. It was a beautiful thing, he says, in some ways subtle and in others not at all. “Trees became so powerful. I felt like I was more in the world, surrounded by it, instead of just looking at it. When you see depth,” he says, “basically you discover a new world.”

About four years later, though, his experience leapt up to an even new level of intensity. He was watching Star Wars in 3D on a giant screen when a spaceship seemed to hurtle straight toward him off the screen. “After that, I began—especially if I pay attention—seeing 3D much more deeply and more often in real time.”

Chopin says that immersive 3D movies, which can exaggerate depth by as much as a factor of ten, are helpful at boosting the perception of three-dimensional space up to conscious awareness. “It crosses a kind of threshold and suddenly people can see it,” he says. And once they know what it looks and feels like, it becomes much easier to find again and again.

In the ongoing study begun last year, Levi’s research team, hoping to exploit the power of the artificial 3D experience, took the leap into virtual reality (VR). They first used a sophisticated, lab-based VR device to host a simple game that involved squashing a virtual bug with a cylinder on a shifting tilted plane. As with the video game, “we wanted to reduce inhibition of the weak eye by the stronger one and to create cooperation between the eyes,” says Levi. Again, they slowly withdrew monocular cues, forcing players to rely on binocular ones. It worked. Extremely well!

The lab is currently collaborating with Vivid Vision — a San Francisco based startup company developing a VR based solution for vision therapy—on another generation of VR training programs that will be played on commercially available, easier to use, and less expensive headsets such as the HTC Vive. These programs promise to be much more entertaining than their bug-squashing predecessor and that should make it even more enjoyable for users to persist with the training program until they see results. The portability and affordability of the devices will also permit their use at ordinary optometry clinics and even at home, says Levi. From home, users could log in to a secure web site that would allow monitoring—and customizing—the treatment. This could raise exponentially the number of people treated for their amblyopia, strabismus, and stereo deficits.

For Gillet, who was stereo-blind until his late 40s, the availability of 3D-vision training on portable VR headsets might have given him extra decades of stereopsis. Even today, access would allow him to further strengthen his perception of depth. “I’m much much better, but I don’t think I’m one-hundred percent yet,” he says. But he really doesn’t have any way to further train his eyes and brain.

“One of our aims is to have a number of different, highly engaging video games that are tailored specifically to the patient’s needs,” says Levi. “We would want to push someone like Eric to finer and finer stereo requirements. Because the games are fun and engaging, he would want to play them.”

Furthermore, because the felt experience of depth in the VR environment is so close to that of experience in the real world, says Levi, stereopsis achieved with VR may more likely “stick,” to survive the translation to reality once the device is removed. Like the 3D epiphany Gillet had watching Star Wars, VR may key people in to what it “feels” like to see depth, so their brains can more easily seek out that neurological sweet spot again.

For his part, Gillet is still relishing his gain of a new way of seeing. And he is also looking forward to one day soon when he might be able to use off-the-shelf VR devices to deepen further the stereo perception that he first gained in Levi’s lab and that he has come to value so much.