View the demo (Oculus Rift not required) »
View the source code »

Last week’s Oculus Rift demo demonstrated that it’s possible to use the web to deliver virtual reality “films.” But an interactive documentary is not really interactive if all you can do is look around. We also can’t assume the same forms of interaction that worked for flat content will work in virtual reality, and because virtual reality games, let alone films, haven’t yet established interaction conventions, there’s a lot for us to explore.

It’s always important to consider the context of an experience when determining the interaction model. If you’re presenting a virtual reality documentary at a film festival under controlled circumstances, there are a number of novel devices you could bring along. The Facebook-owned Oculus Rift is expected to become available to the public in 2015, so we’re targeting what we believe will be the best way to distribute virtual reality work to a large audience, the web browser. Because the audience is unlikely to own any of one these specialized control devices, the interaction model should include the devices that are most likely to be available: the keyboard and the mouse.

On the left, the mouse point is snapped to a surface. On the right, the viewer has set the pointer at an arbitrary distance.

On the left, the mouse pointer as it appears when it’s snapped to a surface. On the right, the viewer has set the pointer at an arbitrary, closer distance.

First, let’s see what we can accomplish with a good old-fashioned mouse. The traditional mouse paradigm, where the mouse pointer moves relative to the screen, doesn’t work well in virtual reality. When your screen is a fixed 2D monitor, the “world space” of your interaction is the same as the “screen space.” If you move your head, the mouse pointer stays right where you left it on the screen. This is true for flat interfaces, such as most web sites or documents, but it’s also true for 3D video games where your virtual head is fixed to your virtual body — when you look around, whatever you’re carrying (for example, the gun in a first-person shooter video game) comes with you. But in virtual reality, your viewpoint moves independently from your virtual body and the world around it, so attaching a mouse pointer to the screen is like taping a pencil to your face instead of holding it in your hand. In this demo, the mouse pointer (the white circle) will stick to objects — it’s de-coupled from the view and attached instead to the 3D world. You can scroll to move the pointer closer or farther away to potentially probe the environment or to indicate interest in something within it. If you scroll all the way back to you, the pointer will reset and stick to the first object directly in front of you. Moving the mouse forward and back moves the pointer vertically, just as it would in a 2D environment, but moving left and right moves the pointer radially, centered around your point of view. It seems to be a good compromise between fixing the pointer to the world-space and maintaining a connection with the user. With these interactions defined, the pointer can be placed anywhere you can see.

For this demo I also wanted to allow viewers to walk around as naturally as possible, and I implemented this using the keyboard. For virtual reality experiences others have created so far, including the famous “Tuscany” demo, you move in the direction you’re looking. It feels like your head is attached to your feet, which is not how people walk. So in this demo, the forward direction is set to whatever direction you’re facing when you first hit the arrow key on your keyboard, and you will continue to move in that same direction as long as you’re holding the key down, no matter where you look. Also, because research indicates that moving in any direction other than forward is likely to cause motion sickness, movement in any other direction is reduced to half speed.

The demo is a simple scene of cubes and a sphere built with three.js, and the code for the mouse pointer is a separate module that can be re-used in any three.js project. If you have an Oculus Rift, you may view the demo with a WebVR build of either Firefox or Chrome. Otherwise, it’s viewable in any modern browser as long as you have a mouse, and you can use the W, A, S and D keys to look around.

Let us know if you end up using the virtual reality mouse code or can think of how to improve the experience. Share a link. You can comment below, use the hashtag #povtech or email us at filmmakers@pov.org.

Get more documentary film news and features: Subscribe to POV’s documentary blog, like POV on Facebook or follow us on Twitter @povdocs!

Published by

Brian Chirls is the Digital Technology Fellow at POV, developing digital tools for documentary filmmakers, journalists and other nonfiction media-makers. The position is a first for POV and is funded as part of a $250,000 grant from John S. and James L. Knight Foundation. Follow him on Twitter @bchirls and GitHub.