Year 2017

Categories Mixed Reality Design Research UX Design

Awards Most Provocative Design - Microsoft Design Expo 2017

Inspired by existing technologies such as the VIVE controller and photogrammetry software, the project imagines an interaction in which users can scan in objects from RR to VR. This creates digital models of personal objects:

↑ Left to right: 1. User scans in a cup in RR, prototyped with HTC VIVE and VIVE tracker. 2. VR view of scanning. 3. User can move the cup (pivot point) in RR and see its VR counterpart respond as well.

As demonstrated above, a pivot point is a mixed reality object created from real reality. These diagrams illustrate the scanning and sharing of pivot points.

↑ The digital model of the cup (on right) is generated for MR, and mapped onto the real cup in the original owner's RR space, becoming a pivot point.

↑ User A, the owner, uploads their scanned model (pivot point). It can now be downloaded and shared by other users in their MR/VR spaces.

↑ When spaces share the same pivot point, the MR spaces are connected.

↑ Each user can now see other people's MR spaces overlaid onto their own, creating shared layers of space around a pivot point.

We created a video demo to show how this might play out as a working prototype.

In the video above (and stills below), Sean first scans in a coffee cup (which he holds in his hand) from his RR world. This cup is now live as an object in his VR space, shareable with others.

In another RR setting, Christine pulls the cup from a virtual commons and uses it in her VR environment. The cup now exists in her VR space, and she is connected to Sean's VR space, which she sees rendered as a wireframe.

↑ Christine's view of her VR space, after placing the pivot point. The wireframe is Sean's space—a ghost.

What are design alternatives to avatars as representations of individuals and social interactions in digital space?

A network of VR spaces is created, all connected by different users' pivot points. Given the spatial dimension of embodying VR spaces, we called this network a neighborhood.

↑ Representational view of network formed around various pivot points.

↑ Spatial view of network, or neighborhood, as "mixed reality"—connected virtual spaces. Higher density (darker) areas are around pivot points that are shared more often (more popular).

In addition, we also created a simulation in Unity of a birds-eye view of multiple virtual spaces. These virtual spaces move, overlap, collide, and collapse into one another to demonstrate an alternative vision of "mixed" reality that the project proposes:

One question we asked was: Would visiting your neighbor allow you to touch, use, and borrow their other models and assets? This raises issues of privacy and etiquette: how is your space being seen? Are all of these spaces by default accessible to each other, and what are the customs of visiting?

As one way to answer these questions around privacy and customs, we imagine that a virtual commons could be created for accessing shared interests and assets. The neighborhood is essentially a spatial mapping of this commons. User-creators can generate and share Pivot Point, creating a different model of sociality in VR.

This project was developed in two parts, first for a class on "Everyday Immersions" and later for the Microsoft Design Expo 2017: Intentional Design for Positive Cultural Impact in Mixed Reality.

"Everyday Immersions" (2017) is an immersive media class at ArtCenter College of Design. The class' projects explore how we might use the performances of everyday life (now in multiple resolutions, places, and bodies) to propose new opportunities within the field of VR/AR, immersive media, and design, while avoiding stereotypical AR/VR content.

Nan and my project was multifold: it explored the "transition zone" between real and virtual realities, how to orient oneself in a headset's VR space, and also how to continue RR activities in VR. The project is documented here.

We developed this initial project for the Microsoft Design Expo: full presentation can be watched here.