Designers Are Exploring Ways to Introduce Users to a “Mixed” Reality

A wave of artists, designers, and academics are blurring boundaries between virtual, augmented, and physical reality.

Surfaces of all kinds are top of mind these days, so we decided to look at all aspects of them, in these articles, from A to Z. Thinking of surfaces less as a product category and more as a framework, we use them as a lens for understanding the designed environment. Surfaces are sites of materials innovation, outlets for technology and science, and embodiments of standards around health and sustainability, as well as a medium for artists and researchers to explore political questions.

The Cloud+ Labs (2017)
Seated in a stylized empty hot tub and wearing a VR headset and motion-capture gloves, viewers of Leah Wulfman’s CLOUD+ Labs project navigated an animated 3D digital model by swimming through a cave while the software sifted through a user’s Google search history and fed it back at key moments in-game. The project was produced in SCI-Arc’s Fiction and Entertainment program, led by Metropolis 2020 Game Changer Liam Young. At the 2018 Venice Architecture Biennale, artist Simone C. Niquille critiqued ergonomics and digital representations of the body through the installation Safety Measures. Courtesy Leah Wulfman

“Reality, however utopian, is something from which people feel the need of taking pretty frequent holidays,” wrote Aldous Huxley in his 1932 dystopian novel Brave New World, and the sentiment is acute nearly 90 years later. The pandemic has intensified and accelerated a digital drift, as culture turns to the virtual— to Zoom, Animal Crossing, or TikTok—for ways to escape and normalize current conditions. Going on a reality holiday, however, risks setting up a needless opposition between activities in our daily lives and our online interactions. The truth is that we are all operating somewhere in between—and it is this middle ground where a number of emerging artists, architects, and designers are staking out territory, using this nonbinary space to address questions of subjectivity and identity.“Mixed reality” is the term often used to describe the space between the real and the virtual, especially in relationship to augmented reality (AR) and virtual reality (VR). “I like to find this boundary between virtual reality, augmented reality, and physical reality, and to try to make work that is ambiguous,” says Theo Triantafyllidis, who trained as an architect in Greece before moving to Los Angeles to study design and media arts at UCLA. “My core takeaway from architecture is thinking about the human body in relationship to space in a primitive and archetypal way. Virtual spaces from web-sites and virtual reality have an embodied interaction, even if it is a cursor or a touch screen.”

Anti Gone Pilotstills (1)
A vanguard of visionary creatives is using virtual and augmented reality environments to subvert the boundaries of physical space and bodies. For Laida Aguirre, the director of stock-a-studio, this investigation has taken the form of installations like [ a kit of these some parts ] x budget gym ], which was exhibited at Materials & Applications in Los Angeles but continues to live online. Meanwhile, the architect and artist Theo Triantafyllidis has produced mixed-reality performance pieces such as Anti-Gone. Courtesy Theo Triantafyllidis

His 2020 mixed-reality performance Anti-Gone, for example, incorporates VR headsets, motion-capture technology, and a game engine running in real time with actors, an audience, and stage sets and costumes designed by collaborator Polina Miliou. Equipped with VR goggles and motion-capture sensors, the actors perform onstage while their movements map onto avatars in a projection of a virtual realm. The result collapses the two theatrical environments while doubling the number of performers, since the audience watches actors and avatars simultaneously.

“Leaky” is how the artist and designer Leah Wulfman categorizes the interface between worlds, and the word seems to capture the slightly surreal aesthetic better than the more straightforward “mixed.” Galo Cañizares, author of Digital Fabrications: Designer Stories for a Software-Based Planet, suggests that its aesthetic hallmarks include glitchy visuals, participatory environments, and unrecognizable imagery. There are also heavy doses of meme culture and the bright green used for screen capture. “The medium lends itself to nonbinary themes and questions,” he says. “You can not only design a space, but you can also design the bodies—the avatars—that go into it. [It] enables us to think about empathy and subjectivity in representation.”

In 2019, Laida Aguirre, assistant professor at the University of Michigan’s Taubman College of Architecture and Urban Planning and director of stock-a-studio, installed [ a kit of these some parts ] x budget gym ] in the storefront of Materials & Applications (M&A), a gallery on Sunset Boulevard in L.A. The installation, which consisted of a temporary workout space made from a “deployable kit-of-parts,” didn’t use AR or VR rigs—but was nevertheless designed to live equally in the real and virtual worlds. Now, after its exhibition, an online portal continues to list parts that can be assembled into ad hoc workout equipment, like at M&A, or into other IRL constructs decorated with stickers printed with processing code or thin skins displaying images scraped from the internet. “It’s not about creating other worlds,” they explain. “More like a space of the uncanny and the slightly off.”

Aguirre uses this work to challenge assumptions embedded in architecture’s DNA. “Architecture has always rested on rhetorical abstractions,” they say. “First-person experiences and fly-through spaces are of interest to me because they foreground qualities that are scenographic versus focusing on expertise, on geometric prowess of form.” They use the analogy of a teenager’s room to illustrate how virtual and hybrid realities create meaning. In plan, the room looks like four walls and a door. In virtual space it is rich with decoration and symbols. “We get the clues of what kind of space we are in from the stuff.”

Ven Safetymeasures 1
Courtesy Simone C. Niquille

And our “stuff,” especially these days, is as much digital as real. Wulfman’s CLOUD+ Labs, created as a thesis project in 2018 for SCI-Arc’s Fiction and Entertainment program, blended a spa-like experience with invisible data residue left behind on the internet. They also teach at SCI-Arc and Art Center, leading classes and workshops that blend IRL with green screens, motion capture, and game engines like Unity, and see leaky, hybrid spaces offering up the possibility of embodiment to a discipline that rarely includes people, much less bodies that seek to defy conventional categories. “Gayming”—Wulfman’s term—is all about plural identities and narrative.

For example, Amsterdam-based designer Simone C. Niquille critiqued the heteronormative body measurements of Jack, one of the first digital human representations developed at the University of Pennsylvania, in her piece Safety Measures (shown at the Dutch Pavilion of the 2018 Venice Architecture Biennale). The virtual avatar and female counterpart, Jill, are meant to reflect an average body in order to test architectural spaces, floor plans, or vehicle interiors, yet they embody the biases of body measurement databases. “The politics of these databases is crucial in understanding ‘who’ Jack and Jill are representing,” says Niquille.

“Architecture can’t even match where gaming culture is right now in terms of experience, engagement, and inclusivity,” says Wulfman, underscoring the field’s history of exclusion and relative silence on issues from #MeToo to Black Lives Matter. “Our tools are open for critique.”

You may also enjoy “Welcome to the Agrihood

Would you like to comment on this article? Send your thoughts to: [email protected]

Register here for Metropolis Webinars
Connect with experts and design leaders on the most important conversations of the day.

Recent Viewpoints