RedBlue

Video hosting is expensive, so I wanted to allow cash-strapped indie filmmakers to use RedBlue in conjunction with the free video hosts they already use, like YouTube and Vimeo. While the YouTube Iframe API does not emit the timeupdate event I was relying on in the original HTML5 version, I was able to work around it using requestAnimationFrame.

Additionally, YouTube removed native annotations from their platform, seeing them as an underused legacy feature with high maintenance costs. The replacement, End Cards, are more limited in what they allow users to do. In the process the company ended up retroactively preventing the ability to host branching narratives on YouTube—as well as offer supplemental/corrective text to a video after uploading. This RedBlue feature provides a partial solution to that problem: the units of media behind hypervideos can be hosted on YouTube, but the interactive experience moves to your own website. Not be the worst idea from a branding perspective!

Firefox did not fully support the MSE API that the RedBlue prototype relied on. So I needed to develop fallback functionality; otherwise I would be limiting my framework’s utility and potential user base. I theorized that I could dynamically swap out <video> sources rather than appending to a buffer. While I had started the relevant Git branch in 2015, I hit a wall and put the project on the back burner. In 2017 I tried again and landed on a working solution!

Inspired by Gimlet’s “Startup” podcast and Casey Neistat, I embarked on a daily vlog series on YouTube and Facebook to document the ins and outs of starting a startup. I found that most business success stories were told after the success had already been achieved, which led to glossed-over details and mythologized half-truths. As a founder looking for inspiration and guidance, I wanted those nitty-gritty details that few seemed willing to share. So I endeavored to be as transparent as possible with my series, so others could learn from my unfiltered mistakes as well as my victories. I also wanted to keep myself accountable by working in public.

After releasing a few episodes and realizing that progress on the startup was too slow to have a substantive update every single day, but wanting to keep up content frequency, I broadened the scope of the show to my daily life in general. The name became “Hugh’s [Startup] Vlog” and then just “Hugh’s Vlog”, but I kept a separate playlist of startup-only episodes for those that might not care about what I do for fun.

Unfortunately I could not keep up with the output demands of a daily vlog, and have since fallen behind on the series, with many episodes shot but not edited. But I have received positive feedback on it from multiple people so I have every intention of resurrecting it.

Dropping out of college after only two semesters, I started my adult life as an aspiring actor/director. But after working on a handful of no-budget projects and realizing pretty quickly that it would be difficult to support myself that way, I pivoted to web development. I had taught myself HTML and CSS and had some experience working for SMBs via a part-time gig in high school. I landed a design/development internship at a nonprofit and leveraged that into more work.

By 2014 I had worked at or with a handful of organizations, from startups to established companies, and while the pay was unbelievable for someone without a degree, I wasn’t feeling fulfilled at work. So I thought about how I could combine my dual passions of filmmaking and programming. I came up with choose-your-own-story films—dynamic narratives that would prompt the viewer to choose between two options at predefined times, just like the red pill/blue pill choice Neo makes in “The Matrix”. Each choice would influence how the rest of the story unfolded.

This idea dovetailed nicely with OVML, my existing idea for a general-purpose metadata syntax for films. So I decided to add nonlinear playback instructions to the specification.

Using the developing MSE (Media Source Extensions) API for HTML5, I built a proof-of-concept that fetched OVML data via AJAX, then crawled it for media files that would be appended to a <video> source after the user clicked a choice.

The precursor to RedBlue was OVML (Open Video Markup Language; later renamed to Hypervideo Markup Language), a general-purpose XML format to codify metadata pertaining to audiovisual media.

I came up with OVML after noticing that video containers like MOV, AVI, and MKV all had competing ways of storing a very limited set of metadata, which would oftentimes be incomplete or incorrect due to bad encoding practices. This forced tools like ffmpeg to rely on heuristics to analyze video in many cases. I wanted a standardized way to associate a comprehensive set of technical data with a video file, as well as artistic/historical data about the film itself such as release date, running time, director, actors, etc. without relying on walled-garden or user-entered info from the likes of IMDb and Wikipedia.

The idea was that the filmmaker or post-production house could enter information using a to-be-developed GUI app, which would generate an OVML sidecar file that could then be read by compliant tools to facilitate better ingestion, encoding, remixing, and discovery of video content. The aim was to make it as granular as possible, so that individual scenes or even frames could be tagged with what appears in them: actors, props, locations, etc. This would open up the possibility of building a search engine for video. While copyright-heavy Hollywood might be resistant to it, I thought this would be a boon to a future wave of open-source indie filmmakers.