Modern artists, engineers and filmmakers merged traditional cinematic narratives with the technology of the future to create a wholly immersive and interactive experience featuring science, art and technology at the 2016 Filmatic Festival on May 7 at UCSD’s Qualcomm Institute. A total of 22 installations were exhibited, some nestled away in darkened labs and black-box theaters, and others projected outdoors directly onto existing landmarks. Almost all involved one or a combination of interactive headsets, sleek remote controllers, 3-D cinematic virtual reality, audio-visual immersion and, perhaps most importantly, an open mind and vivid imagination. Filmatic’s curator Rebecca Webb noted the irony of the pervasiveness of headsets and screens that monopolised our attention at a festival which is generally understood to be a social event rooted in human interaction. The following are descriptions of just a few installations from the multi-spectral array explored by the UCSD Guardian.
Margaret Dolinsky’s “Figuratively Speaking” is a virtual environment for exploration and contemplation. Wearing 3-D glasses, the audience is able to enter a fantasy world of little, friendly-looking animated faces and figures bobbing around on an island. The figures are drawn up using simple, flat-faced geometric shapes by Dolinsky and then later brought into the third dimension via the Unity game engine. The noses are made of right triangles, the eyes of blobs, the bodies of circles and cones — all very aesthetically abstract. We navigate through the world via a remote control, following brown bulb-like blobs up hills — a metaphor for crowd mentality — and into trees made up of noses, because the figure “nose” something. According to Dolinsky, navigating this fantasy world represents psychic dilemmas and refracting consciousness. The longer we stay in this world, the more the figures reveal themselves to us, through their actions and their constantly changing forms — this is allegorical of reality as well. As a whole, Dolinsky’s piece is nonlinear and non-narrative, produced and experienced as a stream of consciousness.
“Quantum Imagination,” created by the San Diego-based NanoVR team, is a virtual reality medium for nanoscale design and simulation. The NanoVR team aims to create virtual reality tools for scientific design, giving users the tools to intuitively design using simple 3-D organic materials comprised of carbon and hydrogen. The simulation itself is impressive — wearing an interactive headset (HTC Vive) and earphones, the user is able to use two hand-held remote controls to drag, drop and connect multiple atoms in a 3-D, 360-degree space; whatever the user sees through the headset is simultaneously displayed on another screen for observers. Sensors and cameras placed around the room are used to read and capture matter and motion. The experience starts with instructions to step forward onto a lit blue square; a video introducing the potential for innovation in nano engineering plays, then the user is invited to create their own molecules from different atoms spread out throughout the space, surrounding the user like a 3-D polygonal structure. At this point, the user can walk around within a confine of about 20 feet by 20 feet, reaching in any direction to grab, drag, and drop atoms — the result is limited only by the user’s creativity.
The state-of-the-art Audio Spatialisation Lab offered a preview of the new “Mass Effect 4D” theme park attraction is based on the critically acclaimed BioWare game series set to open soon at California’s Great America in Santa Clara, California. Real-time acoustic simulations accompanied the virtual screen experience through multi-directional speakers equipped with constructive and deconstructive facial recognition, creating a sound cloud constantly attached to the viewer. Sitting directly in front of the screen with the speakers in front of them, the audience is thrust into a live-action scenario: they are aboard a spaceship set for Teranova, a planet in another solar system introduced as the new Earth. Their journey through space, narrated by the captain of the ship, sees them flying alarmingly close to the energy field of a another space ship, hurtle at light speed through a hole of purple waves, and arrive at Teranova only to be greeted by enemy hostiles. The second stage of the journey takes place on the new planet, flying around, shooting furiously at their enemies while avoiding opposition fire — this is reminiscent of any galactic battle scene from “Star Wars.” The main feature of this attraction is the sharp and crisp sound of missile fire, explosions, beeping spacecraft dials and engine reverberations projected by the high-tech acoustic speakers, making this a wholly immersive experience. The complete attraction, opening soon, is set to take place in a newly design theater with a holographic screen, motion seats and atmospheric surround-sound arrays.
The media presentation “Hearing Landscapes” showcased the latest work of Pulitzer nominee, UCSD music professor and Qualcomm Institute’s composer-in-residence Lei Liang. Liang’s goal was to reconstruct the lost images of Chinese artist Huang Binhong (1865–1955). Along with a team of engineers at QI, Liang translated a series of images which were scanned and digitally enlarged using multispectral imaging into a sound composition. Liang in particular worked on developing new concepts of sonic particles to create a new language of sound design and spatialisation, but his score was largely influenced by both his interpretation of Binhong’s original images and the newly scanned high-resolution, enlarged projections. What resulted was an electro-acoustic composition reflective of the landscape in all its glorious layers and intricacies, going a long way towards answering Liang’s exploration of decoding the simplicity of complexity. The work was a melding of engineers and artists: It looked through the eye of engineering at a piece of art in order to tell a more complete story, through auditory and visual experience, known as “Deriving Worlds.”
In “Making the Past Present,” UCSD professor and director of the Calit2 Centre of Interdisciplinary Science for Art, Architecture, and Archeology Falko Kuester explores the power of science, technology, engineering, arts and mathematics as an engine of disruptive thinking in pursuit of artful engineering. Kuester and his team are passionate about cultural heritage engineering as well as innovation and preserving world heritage digitally through the collection of big data. Kuester’s live narration of a short film details a recent project of digitally mapping the Baptistry di Giovanni, in Florence. Two UCSD structural engineering Ph.D candidates traveled last year to the site of the baptistry to conduct a health assessment of sorts on the centuries-old building and to understand the structural and historical interactions of the site, including its flaws. They used LiDAR (light detection and ranging) laser scanning, high-resolution photography, thermal imaging and photogrammetry to construct a 3-D, 360-degree live model of the entire baptistry. The fully interactive rendered model, consisting of billions of data points and 2.5 billion sensors, is spatially accurate and allows for high-resolution visualisation. Ground-penetrating radar technology combined with a custom-built mobile robotic thermal camera platform allows the team to see through walls and beneath floors. Everything from mosaic tiles in the ceiling to cracks in the wall can be explored at the molecular level. According to Kuester, the ability to bring the data back home to analyze not only allows us to step into areas of scientific discovery through tools of virtual reality and data interaction, but just as importantly, presents an opportunity for a story spun with compassion and human engagement.
Adding the third dimension and interaction to a medium previously only manipulated on a 2-D screen introduces new perspectives towards the material properties of these elements. In a panel moderated by Benjamin Bratton, UCSD visual arts professor and director of the Centre for Design and Geopolitics, the future of virtual and augmented reality was vigorously discussed. Bratton noted the need for a new vocabulary that doesn’t borrow from the jargon of past media. Sundance Institute’s New Frontier Labs director Kamal Sinclair spoke, similarly, on how language triggers higher thinking and function beyond pure instinct, and voiced curiosity over how new technologies will allow us to experience something neither physical nor normative. Artist Margaret Dolinsky addressed the power of virtual reality: its ability to bring together different disciplines across engineering and visual arts to break down long-standing semi-isolated systems within those fields. When asked by Bratton about VR’s potential for innovation in relation to its dangers, Ingress game designer Flint Dille voiced his concerns about uses and manipulations of virtual reality games and systems that diverge from the creator’s intentions. What is clear from the myriad virtual reality experiences on display at Calit2 is its monstrous potential for growth. Though virtual reality can be dangerous in its applications, it offers some people a sense of liberation in their experience of a reconstructed world.
Filmatic Festival impressively demonstrated the current state of constantly evolving technology and its application to different art forms, particularly in the field of virtual and augmented reality. Aside from its dazzling presentation of cinematic reality, and a complete hyperstimulation of the senses, it communicated the community’s extraordinary progress in the field and galvanised today’s generation of artists, engineers, and scientists to very real possibilities of tomorrow.
