A new analysis of 15 entries in the 2020 TriBeCa VR film festival Cinema360, undertaken to determine if the filmmakers fully utilized Virtual Reality, found “there is still a considerable distance for VR storytellers to travel to fully utilize the nature and potential of the developing medium of virtual reality,” study author Professor of Journalism and Media Studies John V. Pavlik said.

“Unless storytellers using the VR platform fully utilize its affordances, the impact of cinematic VR will fall short of its potential."

His paper, “Experiencing Cinematic VR: Where Theory and Practice Converge in the Tribeca Film Festival Cinema360,” was published in the Proceedings of the New York State Communication Association in November 2021.

Pavlik said while the field of cinematic VR has advanced substantially in recent years in terms of narrative design and user experience, its use of VR has not. “Unless storytellers using the VR platform fully utilize its affordances, the impact of cinematic VR will fall short of its potential."

Pavlik, who created an early form of augmented-reality-based storytelling called “Situated Documentary,” which is still in use, said his motivation for conducting research on Cinematic VR is “to understand the nature of virtual reality as a medium for narrative, or storytelling, and the extent to which professional storytellers are utilizing the qualities available in this new medium of communication.” 

Pavlik said he chose to examine the Tribeca Film Festival Cinema360 because he wanted to conduct his study “in the vein of critical events research dating to at least the broadcast of the 1938 ‘War of the Worlds’ radio play. The late Princeton University Professor Hadley Cantril and colleagues conducted research immediately after this broadcast. Reports at the time suggested many listeners panicked as a result of listening to the storytelling in this then-new medium.”

Employing a form of qualitative content analysis to conduct this study, Pavlik said he used the Oculus Quest headword display along with a six-part conceptual framework he developed previously for his analysis of experiential media, which includes virtual reality.

Pavlik viewed all 15 entries in the Cinema360 competition twice to determine, he said, if each entry employs the six dimensions of experiential media storytelling. “These are, Pavlik said, whether “a video employs interactivity and in what form; immersion includes three (or potentially six DOF) visual, three-dimensional imagery, and audio envelopment; multisensory engagement (sight, sound, haptics) are used; the role of AI and/or data is used to provide a zero-latency adaptable user experience; first or third-person perspective is used and which is dominant temporally; and whether natural user interface (voice, touch, gaze or other) is used.

Summarizing his key findings, Pavlik said this investigation revealed that the cinematic VR examined in this study featured a limited range of the narrative features available in VR. These features include visual and audio envelopment that can enable the user to look about the cinematic experiences, but with limited interactivity or haptic engagement.

“Future researchers can build on the findings of this investigation,” Pavlik said, “to develop and test a model of how VR narratives (e.g., cinematic VR) can generate a sense of presence for the user (i.e., a psychological sense of being within the story as a participant) and thereby generate other outcomes, including building empathy and understanding.”

Of the 15 cinematic VR productions studied here, Pavlik said he found that 100% of the productions employed 360-degree video (a test of immersion); 100% utilized a linear narrative and five (33%) featured 3D content, both visually and aurally; the evaluation of their use of multisensory qualities showed none (0%) utilized haptics and interactivity within the narrative and 100% utilized sight, sound and motion (both live-action and animation); six (40%) employed primarily a first-person perspective or in combination with a third-person perspective.

Pavlik viewed all 15 entries in the Cinema360 competition twice to determine, he said, if each entry employs the six dimensions of experiential media storytelling.

How can filmmakers improve their use of VR as a storytelling method? Pavlik said based on his research he can offer at least four recommendations: further develop the three-dimensional, or depth, available in virtual environments; better utilize the multisensory nature of VR (e.g., haptics or touch) in the storytelling; increase the level of interactivity in the narrative; and better utilize the natural user interface available in VR systems such as the Oculus Quest (e.g., user voice command and interaction, and gaze control). 

 “It is hoped these findings will be suggestive of the value and relevance of the key concepts of experiential media as they are manifested in this current generation of cinematic VR,” Pavlik said.

Learn more about the Journalism and Media Studies Department at the Rutgers School of Communication and Information on the website.