The 2019 Oscar winner for Best Editing, John Ottman, spoke exclusively with Assemble in a candid interview about the status of AI and post-production in 2019.
The state of artificial intelligence and its effects on post-production is a burgeoning topic for film crews around the world. While short films like Sunspring – written entirely by AI – are changing the scriptwriting process, products such as IBM Watson are changing the way we approach the post-production phase of film.
With AI’s massively impressive object recognition, data wrangling, cut culling and object rendering functionalities, AI is slowly but surely poised to eliminate the typical early-phase, mind-numbing hours of post-production. But will artificial intelligence in post-production, with new functions like object placement in video and cutting “bad takes” become so powerful that the human creative process will be rendered obsolete?
The Pioneers of AI Post-Production
IBM’s Watson began editing movie trailers, such as Morgan, in 2017.
Their shiny new toy, however, is image recognition. According to its website, IBM Watson’s image recognition, “quickly and accurately tags, classifies and trains visual content using machine learning.”
Industries at the forefront of AI in post-production include everything from adventure content creation labs to a full-scale AI marketing video suites that claim they can create compelling sales films with user generated content submissions from start to finish.
Technology developments in LiDar, GPS, smart camera technology, and proprietary algorithms are now able to automatically “capture, edit, and share cinematic videos & photos.”
ShredVideo.com, one company that’s found success in this area, says to “think of it like the technology of self-driving cars, but applied to adventure activity videos, with the cinematic flair you’d expect from Red Bull Media House.”
Post-Production Pain Points: Solved by AI
With the emergence of AI in post-production for film, editors can now search their entire video library based on words said by actors, animated characters or objects within digital frames from the actual film.
According to the team over at Axle AI, a film industry player in Europe who has developed these types of “radical search” AI practices, “the search uses multiple video analysis engines to analyze media, compare it to a massive image-recognition database and automatically tag and annotate video files based on specific visual information.”
Sounds like an expensive intern (but not when you consider time saved).
Other pain points AI’s already solving in 2019 include object recognition, data wrangling, cut culling and object rendering functionalities, as well as auto transcribing and quality control. Essentially, AI is stepping up and eliminating the need for humans to perform mundane, tedious tasks.
With the current ability to recognize faces, speech, movement and other factors – and place said objects in moving images – are the days of AI telling stories from an emotional, human perspective (such as using choice takes of an actor’s line of sight) on the horizon?
Would you be comfortable letting AI help edit your next magnum opus?
2019 Oscar Winner John Ottman Says Yes
When it comes to working with AI in post-production, 2019 Academy Award winner for Best Editing, John Ottman, says bring it on.
For his work on Bohemian Rhapsody, Ottman spent a year tucked away toiling over take after take – augmenting shots or creating almost-completely new shots and angles to tell his (or, Freddie Mercury’s) incredible, award-winning story.
Ottman is a pragmatist and welcomes any new form of AI into his post-production. However, having the final word on creative decisions and mirroring human emotions, such as fears and hopes, is something John can’t see AI accomplishing anytime soon.
Assemble: You just won the Oscar for Best Editing on Bohemian Rhapsody. Your first Oscar. Congrats.
Ottman: Ha, thanks. Yeah, it’s like I’ve graduated.
Assemble: What’s your take on AI’s role in post-production?
Ottman: It’s like asking a painter how AI can help. It’s sort of a baffling question because so many crafts depend on creative and instinctive decision making, many times personal to the artist’s point of view or way of looking at things, and so it’s difficult to imagine how an AI would help in that regard. However, if indeed it ever got to the point where an AI could literally have creative input then I think many artists, including myself, wouldn’t mind that. Having said that, we also don’t like back seat drivers. But we love other opinions and like to bounce ideas off of somebody else, which can be invaluable. If an AI were that person, as it were, in the room I think we’d all value that as long as we had the last word.
Assemble: Could you see AI helping you in the creative process?
Ottman: I had such a small crew on Bohemian Rhapsody that I didn’t really get to bounce ideas much. I felt rather isolated so, hell, again, if an AI were some entity I could debate decisions with, I mean, why not? However, at the end of the day you’ve got to go with your gut, and so AI might be a great partner to debate with, but at the end of the day you have to go with what you really feel and not get too influenced. I’ll just say we’re pretty far away from an AI making creative decisions like dictating paint strokes for a painter or sifting out the best moments of an actor’s performance. It’s very difficult to imagine. It feels like AI would have to be extremely advanced.
Assemble: What are some pain points you wish AI could help solve in 2019?
Ottman: It’s a laundry list, frankly, starting with the lifestyle. You’re kind of isolated from your friends and family for a year and a half or a year or so. I guess building consensus is a real issue down the pike when you’re making a film. Like I tell young filmmakers, you’ll find that the larger the projects get, it becomes … how do I say this … you realize that it’s not all about editing a film, for instance, as it is the added burden of politics and building consensus.
Assemble: Pleasing the stakeholders.
Ottman: Yes, diplomacy, basically. Your job becomes one of a diplomat and that also really depends on the dynamics between you and the director. Sometimes the director will give more power to the editor, to become that diplomat; and sometimes a more hands-on director will really shield the editor from being that person and will literally be a private partner. So really it depends on the editor and the editor’s relationship with the director and what kind of person that director is.
Assemble: So, you want artificial directors?
Ottman: Well, you know, it’s a blessing and a curse when you have a director who leaves you alone or isn’t around a lot. I am the kind of guy who likes to do my thing and not be influenced by any outside factors and be a filmmaker in my own little world with minimal interruption. So it works for me; although, then you have this resentment, which is this… damn, my director’s able to go have a great life while I’m sitting here in this dark room holding this film together. So, I think every editor can relate to that; but I think most of us like to be left alone.
Assemble: Can you see AI as a companion in the editing suite?
Ottman: Sure, the editing room can be lonely, so if all the AI did was keep me company, that’d be good enough for me! This is off the subject a little bit, but a lot of people have this misconception that an editor is just this person cutting these individual scenes. What they don’t realize is that the editor’s job is also to have the macro view of the film. The task is to shape the film as a whole and step back from it and tell a story to see how it’s working. That’s one of the major contributions of a good editor because that’s the storytelling. The individual scenes can be crafted beautifully but they’re only as strong as the entire story they’re involved in. Again, I don’t know how an AI would judge how well or involving or boring or too fast-paced feeling a story is, but that would be the other task, basically, of looking at the film as a whole and feeling it to see if it works, if it’s emotionally and moving, if it’s boring in certain areas and so forth.
Assemble: It sounds like you’re posing a challenge to the artificial intelligence industry. Show us some emotion. Show us AI can be artistic.
Ottman: That’s the classic threshold that we always talk about with AI’s. Once we cross that threshold where there’s feeling and intuition, then the whole world opens up; but until they really have a seemingly sentient personality or ability, none of this stuff is going to be that practical until that point. Because frankly, if that point is ever reached, then who knows, an AI could literally be a fascinating director, editor, composer or writer. Certainly a film executive!
Assemble: What’s your take on Sunspring, starring Thomas Middleditch?
Ottman: I haven’t seen it yet.
Assemble: An AI known as Jetson, fueled with hundreds of sci-fi TV and movie scripts, built by filmmakers Oscar Sharp and Ross Goodwin, wrote a short screenplay which was turned into a hilariously out-of-control, yet well-received short film.
Ottman: I’m sure the AI didn’t really know what it was doing, right? It’s taking random things and organizing it. I mean is it sort of similar to when the elephant makes those paintings. You know, when they put a paint brush in the elephant’s trunk and it paints these things that sell for thousands of dollars. I don’t know if the elephant really knows what the hell it’s actually doing, but we’d like to think it does.
Assemble: Do you think that AI will have a positive or negative effect on the industry?
Ottman: I don’t see why it would be negative as long as humans with personal life experiences are in control.
Assemble: Did you use AI during your work for Bohemian?
Ottman: If I was using any form of AI in my post-production efforts during Bohemian, you could have fooled me. But, things like creating new angles in a scene without having the camera angles to begin with, I guess you can say I did that from time to time in Bohemian Rhapsody. For the Ray Foster scenes, for example, I could turn a master shot of five characters into a three-shot, and in some cases turn a medium shot into a single or close up on one character. It’s hardly AI, but the new digital age of cameras enables us to do a more nuanced style of storytelling, and we can often avoid having to go shoot a pick-up shot. There’s a smorgasbord of digital things you can do in post, from changing backgrounds to even altering the direction of an actor’s eyeballs so they look where you need them to. I used this technique for a scene with Michael Fassbender in X-Men Apocalypse, where I loved the emotion of the take and angle, but his eye line was slightly off. Yet the ability to pick and choose the exact inflection of an actor’s eyes, choosing various takes and nuances, that’s not something I can’t see an AI having the instinct to do. But, hey, if there’s an AI out there that wants to join my team, I’m game.”
Assemble: In your opinion, will individual’s creative capacities be more valued or devalued at the expense of AI?
Ottman: Well, again, you’re talking about if we really want to go to the extreme with science fiction and go towards the Terminator future; then sure, we definitely could become devalued if AI basically took over. But until that point, I don’t think humans will be overvalued because I still think the AI will need us more than the humans need the AI.
The End Game
As John Ottman says, if there’s space in the room for AI to make his job easier (or less lonely) he’s all for it.
Like Uber’s automation putting old school cab drivers out of business, the best editors will continue to figure out ways to maximize AI’s current capabilities in 2019.
“The amount of money and time saved with AI in post-production could be impressive,” Ottman went on to say, “but the ability to see the film from the macro perspective, and edit the film to tell a truly human, emotional and compelling story – well, that onus is still on us humans.”
Like Ottman in 2019, will artificial intelligence, such as IBM Watson, one day give an Oscar speech for Best Editing in film?
Maybe Watson should start an IMDB first.
