Nicci Yin is a designer
of interactions and critical media, engaging design to bring together art and emerging technologies. She has contributed to curatorial projects, most notably with Space Caviar and Creative Time Reports, and produced feminist media while a fellow at Barnard Center for Research on Women. Most recently, Nicci's work has been shown at Ars Electronica, the Post-Internet Cities Conference, and Microsoft Design Expo. She is currently based in Seattle.

INDEX
CV, Instagram, Email
hello [at] nicciyin [dot] com






MFA Thesis Project


Immersive technologies (especially VR) propose new means of displacing our bodies through technology, and photogrammetry recreates these flesh and object bodies for virtual space. This year-long research project uses diverse design approaches to explore how scanning, photogrammetry, and touchscreen interactions mediate our relationship between the physical and digital. The designed scenarios and materials advocate for a "gooey"¹ approach to immersion.
Surface Tension is a series of proposed scenarios and materials for a gooier¹ take on immersion.

Immersive technology (virtual, augmented, and mixed realities) necessitates a reexamination of the boundary it seeks to transcend: the (touch)screen, the surface where flesh presses up against the device. Scanning 3D objects—including people—is an emerging interaction for screenless realities, and reinscribes the everyday gesture of touching a screen.

What gooey surfaces and interactions are created as we move from screen to screenless? Surface Tension proposes gooeyness as a feeling, an aesthetic, and an act, destabilizing and blurring the binary of physical and virtual, flesh and device.


The video above is a cumulative work, loosely broken down into “chapters” of sketches from different stages of the project’s research:


SCANNING / FACE ID
Immersion describes emerging VR/AR/MR technologies, but is expanded as an idea in this project:

 How might we think of immersion not as physical/digital, virtual/real, but about screenless-ness?

 In what ways are we already immersed in the merging of digital/physical worlds?
Different systems and set-ups for immersive technologies were documented in this drawing, which serves as both an analytical diagram and a fantastical landscape of sensors, actions, and objects (full version):


Some examples of scanning in the drawing are: Microsoft Hololens’ spatial mapping, photogrammetry, and HTC VIVE’s play space. These show that as untethered as we might imagine VR to be, it is still supported by a system of sensors and cameras that map our surroundings, capturing and recreating our “real world.”

Another instance of how we are already scanned everyday is when we use our thumbprints or facial recognition to unlock our phones:


These prototypes reinterpret an everyday act of scanning, Face ID. While Apple’s version uses infrared sensors to perform facial recognition from arm’s length, these prototypes ask what might result if we “scan” our faces in by directly pressing our flesh against the screen. Immersion becomes almost an impossibility as the screen is a frustrating, solid threshold.

How might scanning become less impersonal and less “objective” (as in, performed from a distance)?


PHOTOGRAMMETRY
Photogrammetry is a specific example of scanning. While designers, artists, and engineers often use photogrammetry for simulation, reproduction, and reconstruction, Surface Tension focuses on the act of the scan and the aesthetics of the texture maps that the process creates.

Photogrammetry involves taking a series of images around a person, object, or place, then compiling these images through software (such as Agisoft, Autodesk ReMake, etc.) to generate a 3D model. Above is a demonstration of a low-tech version of photogrammetry, where photos are taken via an iPhone.

The faces in the image above are all scans of individuals’ faces, shown here as models in Unity 3D. The choice of faces in Surface Tension is deliberate, since faces—personal, intimate—question how scanning appropriates biometrics and potentially displaces identity.

This section in the video imagines what it might be like to be immersed in the scan, and to use the affordance of infinite scale to explore a model as an inhabitable space. The leaks, fragments, and imperfections of the model are exposed as well:



GOOEY INTERACTIONS
One of the emerging outcomes of Surface Tension is to reframe immersion for the everyday, rather than thinking of immersion only as VR/AR/MR (or only as laborious, inaccessible, and expensive technology).

The project refocuses attention on our everyday interactions with the smartphone: how does the surface—the screen—also produce a "screenless," immersive feeling?


The actions we perform on our screens creates the feeling of immersion, whether it's an infinite number of faces to swipe through on Tinder or feeds to scroll through on Instagram.

A user performed various touchscreen interactions (examples diagrammed above) on a green screen as a study to demonstrate the various acts of attempted immersions that occur on our screen surfaces.


GOOEY SURFACES
The observations around photogrammetry surfaces, screen surfaces, and immersive interactions were synthesized in the final two sketches:


One clip shows touchscreen interactions being performed on a photogrammetry texture map (the skin of the model), which is simulated as fabric using p5.js. The other clip is an analog cousin of this: seen from within the phone, touchscreen interactions are performed until the screen is broken through.

For the installation of this project, an experiential prototype of the video was created, using agar.



¹ A word borrowed from American Artist (“Black Gooey Universe,” 2016).

Presented at Wobbly Realities: A Happy Hour in the Post-Geographic City (2018)


For further research documentation:






All images and content
(c) Nicci Yin.