This is the fourth post in this week’s series on the state of web video. Read parts one, two and three.

Having covered in previous posts a long list of features, bugs and other dangers to look out for, let’s discuss the different ways that browsers fail. Delivering, playing and remixing video in real time in a browser is an ambitious project, so the occasional failure is inevitable. Understanding the kinds of failure will help us figure out what to do about it when it happens.

Feature Detection

When your web experience requires a certain feature that may not be available everywhere, you’ll want to detect whether it’s supported. If you don’t account for unsupported features, users may see an obviously broken site or possibly nothing at all. Proper feature detection allows you to fall back to an acceptable, if not optimal, alternative experience. At the very least, if there’s no way to do without a missing feature, you can show a friendly error message and abort. Feature detection is widely considered a best practice, and there are JavaScript libraries like Modernizr for this purpose. But there was a time when common practice was to use a hard-coded white list of specific browser versions known to work for one’s web site. This is very bad, because it uses the user-agent string to identify the browser, which is difficult to parse and can be faked. Feature detection works much better because it doesn’t matter which browser you’re using, as long as it supports the feature you need. So, months or years after you’ve launched your site, when a future version of a browser starts supporting a feature you need, it will just work without any extra effort on your part.

Let’s examine how this works in practice. The cleanest and preferred way that a browser can fail with a given feature is to simply omit support for it. Most major APIs or features come with a globally accessible JavaScript object or method, and all you have to do is check if that object exists. For example, Internet Explorer doesn’t support Web Audio API, but presumably it will in the future. Here’s how we can detect it:

if (window.AudioContext || window.webkitAudioContext) {// Web Audio API supported!// todo: proceed with your awesome multimedia experience} else {// Web Audio API is missing// todo: implement an acceptable alternative or display an error message}

In other cases, we might need to detect a subset of a larger feature. Let’s try detecting whether the browser can play MP4 video files. This is impossible to do with a browser white list, because on some platforms Firefox will play MP4 only if external decoding software is installed on the device.

var video = document.createElement('video');if (video.canPlayType('video/mp4')) {// MP4 support detected!} else {// better try WebM}

This practice also comes in handy with WebGL, where capabilities depend not just on the browser but on the graphics hardware and drivers installed.

Bugs

Often, even if a browser supports the technology you need, it will have bugs that may diminish or completely break your experience. It happens particularly often with technology that hasn’t been around long enough to be heavily tested. Simple feature detection doesn’t work, because as far as the browser knows, it supports the feature just fine. But we can sometimes still find a way to detect the problem and write a workaround.

For example, Internet Explorer 11 has WebGL, but it doesn’t work with videos, just for still images and canvases. This is against the specification, so there isn’t a built-in way to ask the browser if it can handle a video. But we can formulate a test to try it out and see if it works. When you try to process a video with WebGL, you can query the WebGL context for an error code. If the error code is 0, then it probably worked, but when it fails it consistently returns an error code for “invalid value.” Then we know we need to try a workaround, which in this case involves first drawing the video to a 2D canvas and then draw that canvas with WebGL. It’s slower, but at least it works. The code is a bit complex for this blog post, but you can see it in the code for Seriously.js.

Fortunately, all four major browser vendors have web sites where you can report bugs so they can be fixed, though some are more useful than others. Firefox and Chrome (well, technically Chromium) are open-source projects so the public can track progress on all bugs and feature requests, and they have large communities of developers, many of whom work as volunteers, so progress is sometimes fast, as in the case of this bug that was fixed in two days. Other bugs can linger for years without a response. When bug reports are public, they also serve as documentation, so you can see if other people have dealt with the same problem and whether anyone has found a workaround. Microsoft also has a public bug tracker, but they’re a little less responsive and, since the code is private, you can’t track progress as easily. Apple will allow you to submit a bug report for Safari, but all of its bug reports are private so you can’t even share a link to one you’ve submitted. Apple is notoriously secretive about its software development, so you won’t know for certain if a missing feature or bug will be addressed until the software is released.

Broken on Purpose

The most difficult kind of failure to handle is when a browser’s behavior is intentionally contrary to what the standard dictates. This problem is more or less exclusive to mobile platforms, and Apple is notorious for it. One example (I mentioned several in part one) is the way that Mobile Safari completely ignores the volume property on video and audio elements, making it impossible to adjust volume programmatically. The kind of feature detection discussed above becomes difficult or impossible, forcing us to resort to the kind of browser sniffing we sought to avoid. These problems are usually a dead end because the browser developer actively goes out of his or her way to prevent a workaround, and they are applied broadly. In contrast, a bug usually applies to a narrow set of circumstances that you can avoid — once you know what those circumstances are, such as a specific codec or server configuration.

Supposedly, the justification for these variances is to protect the user, but I find it to be ineffective at best and potentially counterproductive. For example, on iOS a video will not start loading or playing until it’s triggered by a touch event. Apple explains that this is to prevent the user from incurring excess mobile bandwidth costs. But there is nothing to prevent a site from downloading a very large JPEG image, and there are no similar restrictions on native apps. Why couldn’t they allow the video to load when the device is connected over Wi-fi? Web standards give the browser a lot of leeway to decide how to allocate resources like network access, and it should be possible to work within the standard. By thwarting developers, they raise the cost of development and block the efforts of those of us making an honest attempt to give the user the best experience.

Breaking the standard in this way might seem justifiable only if you assume that all anybody will ever want to do is to play a single video like you do with a VCR. As soon as you want to do something interesting, such as splice together multiple clips or play a video as a page background, the whole thing breaks. A large part of the success of the web, like the Internet in general, is that projects are generative. This is, they are programmable and modular enough that people can build things with them that they weren’t originally designed for. This is why the open, standard web succeeded and left CompuServe and AOL behind.

The tools for creating and distributing rich, interactive media on the web are amazing and are beginning to outpace anything that came before, but they are still rough in places. As developers and artists, we can do our part to protect ourselves and to make the tools better. Of course, test your work on as many platforms as possible. When you hit a snag, search for existing bug reports and, if you don’t find any, submit your own. Continue to make great work that shows off what web video can do, and complain frequently and loudly when things don’t work.

Here are some resources that you can regularly check to help you along the way:

Get more documentary film news and features: Subscribe to POV’s documentary blog, like POV on Facebook or follow us on Twitter @povdocs.

Published by

Brian Chirls is the Digital Technology Fellow at POV, developing digital tools for documentary filmmakers, journalists and other nonfiction media-makers. The position is a first for POV and is funded as part of a $250,000 grant from John S. and James L. Knight Foundation. Follow him on Twitter @bchirls and GitHub.