Adobe After Effects
GUI for VR/MR
This design project takes multiple experimental approaches to answering these initial queries:
An initial point of interest for me is the swimming pool as an analog of immersive technologies: designed as a literally "immersive" space filled with water, the pool is a deliberate space carved out for purposes of leisure and play (derived from the observation that most applications of VR/AR currently is for gaming).
Over the course of the year, I created a diagrammatic drawing/visual diary of various immersive technologies, notes, spatial layouts, and documentation and sketches of prototypes. The drawing documents different immersive set-ups, such as Microsoft Hololens’ spatial mapping, photogrammetry, and HTC VIVE’s play space:
This fantastical landscape of sensors, actions, and objects show that as untethered as we might imagine VR to be, it is still supported by a system of concrete things that map our surroundings, capturing and recreating our “real world.”
The first chapter in this video thinks about immersion as recreating the physical into digital, and is a reinterpretation of Apple's FaceID. We are already scanning ourselves everyday when we use our thumbprints or facial recognition to unlock our phones.
While Apple’s version uses infrared sensors to perform facial recognition from arm’s length, these prototypes ask what might result if we “scan” our faces in by directly pressing our flesh against the screen. How might scanning become less impersonal and less “objective” (as in, performed from a distance)?
Our bodies are at times at odds with scanning devices—for instance, the phone cannot recognize your thumbprint if it is sweaty. I considered how this gesture of scanning a face directly on the phone might produce residue that is usually considered "incompatible" with technological devices, such as sweat, grease, and face oils.
Scanning objects and people into virtual space is an increasingly common practice as we create more elaborate immersive experiences, and photogrammetry is one of the most common solutions for this.
Photogrammetry is a process involving multiple photos taken from all angles around an object or person, then processing these photos through software to generate a 3D model. For this project, I opted for more accessible modes of photogrammetry, i.e. using smart phones as the camera:
But what is each individual's process of capturing or recreating real objects? How can we "read" each person's scan?
Earlier in the project, I set up a series of user tests using a VIVE controller with a camera attached to it, as well as a corresponding scene in Unity, to see if I could physicalize the trace or action of photogrammetry. I asked each user to scan a bicycle and a coffee cup.
As shown in the video, I designed multiple systems for visualizing the user interaction, including realtime documentation, symbols to serialize their scanning patterns, as well as a script that leaves a trace behind the VIVE controller's movements:
SKINSPACE is a slow pan in Unity in, around, and through a 3D model of a face. This is a closer exploration of a 3D model and its texture as inhabitable space, because one of the affordances of immersion is the mismatch of embodied and experienced scales.
Much of the design research in Surface Tension centers on photogrammetry as a process, and on the materials generated by photogrammetry. In addition to exploring the interactions and gestures that scanning allows, my design studies also looked at texture maps—the skin that lies on the model mesh.
∞FEED looks at how immersion is a result of activity, rather than technology. If we think of immersion as a mindset or state created by the interactions we perform on our phones, what might this look like? This video prototype is a means of bringing immersive activity down to the everyday, to our smartphone gestures.
In this short vignette, the screen remains a very digital green—the screen is left open to imagination, and viewers can easily project an Instagram feed, text message stream, Tinder profiles, etc. onto that screen.