How Nonny de la Peña, the “Godmother of VR,” Is Changing the Mediascape

How Nonny de la Peña, the “Godmother of VR,” Is Changing the Mediascape
Photo: Gregory Harris for WSJ. Magazine

WHEN NONNY de la Peña was in junior high, in West Los Angeles, her math teacher wrote a note home to her parents. “It was about something I’d rebelliously done,” she says. It was something minor, like speaking up, or speaking too loudly. Her father read the note, then turned the paper over and looked at her. “ ‘This is what she says you do,’ he said to me,” de la Peña recalls. “ ‘Tell me about her. What does she do?’ ” De la Peña listed some of her own unflattering observations about the teacher, and her father wrote them down on the back of the note, so that now the piece of paper contained two notes, two different perspectives. Then her father signed it. “So what does that make you do? Of course, it makes you think about the structure of things…. Of how situations are multidimensional.”

How Nonny de la Peña, the “Godmother of VR,” Is Changing the Mediascape
Photo: Gregory Harris for WSJ. Magazine

It’s not simply that observations are two-sided, but as multisided as there are individual witnesses, as infinitely sided as the world in all its depths and dimensions. The problem with the world, with reality, is that each of us is trapped within our own version of it (never, perhaps, has this been more true than in our present moment). So how do we shake ourselves out of our extremely personalized versions of reality? For de la Peña, the answer begins not just by flipping the paper over, but by changing the entire room.

De la Peña, 55, is the founder and CEO of Emblematic, a virtual reality (VR) and augmented reality (AR) studio based in Santa Monica, California. What de la Peña and her 15-person team do is relatively simple in concept and not at all simple in execution. The idea is to put people in places they wouldn’t normally find themselves, experiencing situations they would not normally experience. Often, these are related to urgent issues, issues that are, in their sprawling complexity, seemingly difficult to grasp or even care about. But suddenly, there you are, standing in a cell in solitary confinement, or at the foot of a melting glacier, or before a protest line outside an abortion clinic. When you are in these places—hearing the anguish of a prisoner, the calving of a glacier, the vitriol of the protesters—the issues no longer seem like issues, with vague names attached, like prison reform or global warming or women’s health. Rather, they seem like lived experience, like people you’ve met and places you’ve been—like memory.

Though it is still early days, study after study has shown how our minds categorize VR far differently than other, flatter mediums. All of the VR experiences de la Peña creates are grounded in reality. They are reported documentaries, in a sense. And so they become all the more memorable. De la Peña calls VR an “empathy machine,” because by putting you in places with people you may never have otherwise interacted with, you come to understand them far better than you did before.

But VR is still quite a young form, so de la Peña and her team must invent new methods—lines of code or means of motion capture—in order to build these environments. They must digitally reconstruct the spaces and the experiences, brick by brick, or byte by byte, innovating as they go.

In 2012, for example, the World Economic Forum commissioned de la Peña and Emblematic to create a piece on the Syrian refugee crisis. Part of the resulting work, Project Syria, involved meticulously reconstructing streets in Aleppo. You walk down a road and encounter a singing girl and then a mortar shell explodes and chaos ensues. A huge element of the impact of this experience was to allow the viewer to walk the streets freely. But this element was an advancement of de la Peña’s work. Nearly all VR experiences outside a lab up to that point had been stationary, like putting a fishbowl over your head that you could look around in but couldn’t swim through.

THE FUTURE LOOKS BRIGHT Nonny de la Peña, CEO and founder of Emblematic, a VR/AR studio based in Santa Monica, California. Hair, Dennis Gots; makeup, Lisa Storey.
THE FUTURE LOOKS BRIGHT Nonny de la Peña, CEO and founder of Emblematic, a VR/AR studio based in Santa Monica, California. Hair, Dennis Gots; makeup, Lisa Storey. Photo: Gregory Harris for WSJ. Magazine

Just a few months before she was to present her work at the World Economic Forum in 2014, de la Peña realized she would have to invent a new headset, because the headset she was using didn’t allow for as much freedom of movement as she wanted. The headset is the essential piece of hardware in VR, the thing that allows the entire magic trick to work. It’s not just the movie screen, but also the projector, the seat, the popcorn, the ticket booth—the gateway to the entire experience.

The particular headset she had on hand, the one that wasn’t doing all that de la Peña wanted it to do, was rather famous, as VR headsets go. Before she moved Emblematic to its Santa Monica offices, de la Peña had worked at the Mixed Reality Lab at the University of Southern California Institute for Creative Technologies. While there, one of her interns had helped build her this headset. That intern, Palmer Luckey, eventually launched a Kickstarter project off of a similar design to help fund his company, Oculus, which Facebook later bought for $3 billion. But in the ever-evolving world of VR, the headset Luckey had worked on for de la Peña had become, after a few years, ancient.

So de la Peña went about building herself an entirely new one. First, she tracked down a company that constructed lenses for NASA and used those lenses for the stereoscopic, depth-producing screens placed mere inches from your eyes. Then she purchased a 3-D printer so she could print and test parts for the rig. She set it up in her mom’s garage, and her brother, Stephen Brown, and Thai Phan—another engineer—took over the prototyping. She found a motion-tracking system at a bankruptcy sale and bought it for $7,000. They spent hours, then days, then weeks, building and dismantling existing headsets to figure out how to construct their own. “It was such hubris to think that I could do this, that we could do this,” she says, but she did it anyway. “This is what we want to do. I want to walk the streets in Aleppo. How do we reverse-engineer things to make this possible?” Within a month, they had built two headsets you could walk around in.

The same year she was commissioned to make the Syrian refugee piece by the World Economic Forum, the director Alejandro G. Iñárritu met with de la Peña. He’d sought her out because she was a pioneer in VR, and he was just getting interested in the possibilities of the form and thinking about doing a work of his own. Later, this would turn into the U.S.-Mexico border crossing experience called Carne y Arena (Virtually Present, Physically Invisible), a piece that won him a special achievement Academy Award.

“The great challenge is to discover or to understand the visual grammar of this new medium,” Iñárritu says, and with her expertise, de la Peña acts as a kind of translator. He continues, “This is another art, and we haven’t yet discovered the grammatical language.” Others working in VR readily lean on the tricks—the language—of mediums they understand (movies and videogames, mostly), but according to Iñárritu, de la Peña understands that “at the heart of the form is still a reality, a reality lived by somebody else. What I discovered is the genius of Nonny is the way she knows how to balance fact against the limitations [of the form].” As an example, he brings up Hunger in Los Angeles, an experience that was a special selection at the Sundance Film Festival.

In it, you stand in line at a food bank, and while you wait, the man in front of you collapses and begins convulsing. An ambulance arrives and you overhear paramedics say that the man is experiencing seizures resulting from his diabetes. It’s an older piece, from 2012. The technology has changed considerably since then, and it shows. The renderings are simple and strange. Yet it is hugely affecting to be standing there looking down at a man—even a poorly rendered one—struggling to survive, right at your feet.

SECOND SIGHT This headset—Apollo—was designed for Emblematic by artist Greg Brotherton based on earlier headsets de la Peña produced with her brother, Stephen Brown.
SECOND SIGHT This headset—Apollo—was designed for Emblematic by artist Greg Brotherton based on earlier headsets de la Peña produced with her brother, Stephen Brown. Photo: Gregory Harris for WSJ. Magazine

The reason Hunger in Los Angeles works so well is simple: The audio is a recording of the lived experience as it happened in line at a food bank in downtown Los Angeles in 2010, captured by one of de la Peña’s interns as her team researched and reported on the topic of hunger and food access across the city. “You don’t care about how it looks so much,” Iñárritu says, “because you recognize that it is true.”

WHEN DE LA PEÑA left Los Angeles for Harvard in 1980, she thought she would go into international politics. But she quickly fell in with documentary filmmakers, both students and professors, and received a university grant that allowed her to photograph and report on Chicano gangs in L.A.’s Mar Vista Gardens housing projects. After she graduated, de la Peña returned to L.A. full time to work for the Associated Press during the 1984 Olympics. Soon after she relocated to Mexico to freelance as a journalist and then moved to New York City, joining Time as a stringer and, in 1987, Newsweek as a correspondent. (Meanwhile, determined to be a freelance photographer, she had converted a closet in her tiny apartment into a darkroom.)

Her favorite reporting was similar to what she’d done in Mar Vista Gardens: enough hanging out to impart a sense of being there, either through writing or photography. But she was increasingly attracted to documentary film and, beyond that, the possibilities of newer forms. She was an early evangelist of the internet and taught herself HTML and other computer languages. Also in the early 1990s, she read Virtual Reality, by Howard Rheingold, and somehow knew that this was what she’d been waiting for.

Virtual reality, as an idea in art, is ancient. A huge canvas depicting a battle, the foreground life-size—that’s early VR. So are mid-19th-century stereoscopic photos and viewers. So is an early-20th-century flight simulator. And so are the 1960s prototypes for what we now consider modern VR, chunky headsets attached to room-size computers that ran images that took up your entire field of vision. Virtual reality as a phrase didn’t take hold until the 1980s, and virtual reality as a marketing tool arrived in the early 1990s (remember The Lawnmower Man or Nintendo’s Virtual Boy?). But VR, until very recently, was extremely niche, usually expensive and often very bad. Only in the past five years has it become anything resembling mainstream—Samsung, for example, advertised its VR headsets during the 2017 Super Bowl. But even so, VR headsets are found in very few households (6 percent of U.S. internet users ages 16 to 64 own a VR device, according to market-research firm GlobalWebIndex). Of these, the vast majority are using their headsets to play videogames.

De la Peña is both an early innovator in the form and an outlier, more concerned with capturing and bringing people into areas of our world than allowing them to leave our world behind. To get inspired, she says, “I close my eyes and think about my body in the space. What’s the best way to understand it if I were there?”

Later, she mentions a persistent worry she’s had about this interview, that she would sound overly head-in-the-clouds, ungrounded, like “a silly, dreamy woman.” Part of the trouble is that because VR is a new form, the technical hurdles are worked out mostly on a computer screen. Talking about it, one can’t help but sound a little dreamy, because we don’t have the words yet.

But the bigger challenge is the fact of her gender in a male-dominated field. Once, when she was ticking through all the ways she scraped together the business and made payroll—the grants and the pitches and the client work (including a historic re-creation of the Cartier mansion and an AR data visualization of the U.S. stock market for The Wall Street Journal’s iPhone and Android apps) and the licensing fees for the technologies they had developed—she indicated that Emblematic hadn’t taken much outside investment. Was this a result of some principled stance? “Why haven’t I taken [more] investment?” She raises her eyebrows. “I think you mean: Why haven’t I gotten investment? Something like 98 percent of VC funding goes to guys. So. You see?”

The Emblematic offices are in a squat two-story stucco building on the quiet, inland side of Santa Monica, near enough to de la Peña’s home—which she shares with her husband and kids—that it takes her about 10 minutes on her electric bike to get to work. One hot, hazy August afternoon, de la Peña and her team—which now includes New York City–based co-founder Jamie Pallot—are readying the rollout of a new project, Reach, an entirely original way to bring VR to the masses (a prototype was released earlier this year).

Reach is both a platform for WebVR content and a distributor of that content. It’s also a set of tools for making WebVR experiences a bit more streamlined and accessible. The idea for Reach is many years old, and a lot of what Emblematic has been building for the past year is essentially a series of clever hacks to existing software and hardware. De la Peña’s team is well-suited to this sort of work—most of their inventions here have come out of their own needs, with limited means, resulting in software that makes a higher-end but not wildly expensive camera able to capture both spaces and people while adding depth or volume. This so-called volumetric capture isn’t perfect, but Reach works well enough to allow a huge portion of people to suddenly create within this previously unplayable form.

By building the sense of space, de la Peña is able to trick you into believing in your presence in that space. This becomes all the more powerful when you are in a space with a person in it. You listen more intently because that person appears more real, because you believe—if only for an instant—that you are there with them. In one early demo of the Reach platform, de la Peña and her team captured a landscape in Utah canyon country, a lot of sandstone and open sky. But standing there on the rock with you are holograms of different people—including one of musician Reggie Watts—addressing you.

De la Peña’s continuing goal with Reach is to capture more environments for people to tell their stories in. She’d like to create a virtual space that is a segment of the walled U.S. border with Mexico, as well as the courtroom in the Supreme Court. Although we are constantly hearing about them, we rarely grapple with the places themselves, much less stand in them and listen. All de la Peña needs, she says, is “access to a place for, maybe, six hours, and now I can have a model for anybody to use whenever they want to use it.” In this way, on the Reach platform, she and her team will build a database of environments.

De la Peña tries out a headset.
De la Peña tries out a headset. Photo: Gregory Harris for WSJ. Magazine

But the places themselves are only half the challenge. The rest is up to all of us. In creating these virtual spaces for others to use, and building a tool kit for capturing and rendering people in depth within these spaces, de la Peña and her team are throwing the empathy machine out into the wider world. Perhaps, with Reach, we’ve finally begun moving toward something post-internet, something less anonymous, that seems closer to the lived experience—something kinder. But de la Peña says she isn’t as concerned with kindness. There is a more important step first: listening. “If you and I were facing each other in the virtual world, would we begin to at least try to understand each other? Would we listen? Maybe. Probably. Yes.”

Previous ArticleNext Article

Send this to a friend