HPE Tech Talk Podcast - The Walt Disney Studios StudioLAB on the Role of AI in Animation, Ep.15

[HPE-Editor]

At The Walt Disney Studios StudioLAB, the world of filmmaking meets the leading edge of AI innovation. On this episode, Erika Varis Doggett, Research Scientist at StudioLAB, discusses how designers, technologists and animators are using machine learning to advance the art of digital storytelling.

Also available on:   Spotify   /   Apple Podcasts   /   Other podcast apps

Transcript

Robert Christiansen:	Welcomes to HPE Tech Talk. I’m your host, Robert Christiansen. Animation has progressed significantly in the last few decades, especially the integration of AI and machine learning technologies. On today’s episode, we’re so excited to have Erika Varis Doggett, Research Scientist at the Walt Disney Studios StudioLAB to discuss how artificial intelligence is influencing modern entertainment. Erika, welcome to the show.
Erika Varis Doggett:	Great to be here. Thank you for having me.
Robert:	Thank you for joining us. I'm super excited about having you here. HPE has been pretty proud to be a founding partner at StudioLAB and it'd be great if you could tell us and our listeners about your role and what the overall mission of StudioLAB.
Erika:	Yeah. So, the StudioLAB is both a place and actual physical lab on the studio lot in the old animation building. Obviously, we're not using it a whole lot during COVID. But it does exist as a place for creatives and technologists and executives from around the studios to come and play around immediately and hands-on see the latest and greatest technology that is available for filmmakers, and for everyone around the company to start really looking at pushing the boundaries of what is available and what we can do with storytelling. So, it's both this, this physical place for that to happen and to have those conversations but it's also a program. We have a number of external collaborations with different technology partners like HPE, like Accenture, and a few others where we do an innovation project together. So, the external company will have their teams and resources and expertise, and they will contribute towards a project, and we contribute our expertise and resources towards the project. And we get to build really cool stuff together all in the name of really seeing what can we do with innovative storytelling today.
Robert:	It sounds like the perfect role for somebody. Tell us about what you do there specifically and how and why, what brought you there? What was the attraction?
Erika:	My role on the team is to be looking at AI and machine learning technologies. What is out there both in the academic space, as well as the startup space and the technology industry and what could be useful for studios. I'm kind of keeping an eye out on what is out there and also doing adaptations of that research adaptations of different technologies for our use cases and building prototypes. We're in the process of building and trying to  do a tech transfer of some of our R&D work into production right now which is very exciting. We have a small team of engineers, AI engineers, and we, you know, we build stuff both in collaboration with partners, as well as internally to make cool stuff and see what we can do that would make the lives and jobs of our studio stakeholders much, much easier. Whether those are folks in studio operations that are doing something like manual QC quality control on our video files and, and video formats and things, all the way through to different animation artists among the different studios and, you know, talking to what are some of the maybe VFX use cases that could make use of some of the latest-
Robert:	Mm-hmm (affirmative).
Erika:	... computer imaging, computer vision technology. And it just, there's so much (laughs)-
Robert:	There is a lot-
Erika:	... there's so many-
Robert:	... there, you could go endlessly.
Erika:	I could go on and on. Yeah. I'm looking at how can we utilize artificial intelligence and machine learning techniques at every stage of the movie making pipeline?
Robert:	Yeah, that's really interesting because I, I think about all the ways that a team like yours at the StudioLAB could be modernizing the filmmaking process. … How is that showing up in one of the things that we've worked in together want- unwanted anomalies and films? Can you just describe how that works and why, why AI, specifically machine learning models are, are useful for determining anomalies specifically ones that you want to take out of the films?
Erika:	The QC's case is pretty fascinating. When I learned about it, it was a bit mind blowing because what's happening right now is you have human QC operators who are very skilled. They're very good at what they do. What they do is watch the films and watch every version of every film before it goes out for distribution to make sure that there aren't any errors in the image. Those errors in this case, one of the things that we're looking at, whether we're looking at detecting are these pixel level anomalies which means a pixel on the image, that's the wrong color. As you can imagine, there's a lot of different pixels on an image, particularly when you're talking just a standard HD resolution which is about 2K, but then like expanding up into some of the latest 4K formats. There's a lot of spots that could potentially be wrong. We have a very high bar of quality at the Walt Disney Company. I think our, our fans also have a pretty high bar of quality that they're expecting to see from our images. So, we really do want to be careful that we're not putting something out subpar — that t is matching the creative vision. We want to enable our QC experts to be able to do more with less time so that we can use their expertise… not necessarily on seeing a pixel fly by but on saying, "Okay, once we've identified this pixel, is this something that needs to get fixed or not?" … That's the next stage of making a decision of, are you going to do something, what kind of an error, how egregious it is, you know, all of that.
Robert:	Right.
Erika:	So by providing automated tools to do parts of that, that QC workflow, we get to, you know, give them superpowers. (laughs)
Robert:	Yeah.
Erika:	... and just let them do their jobs so much faster.
Robert:	Mm-hmm (affirmative).
Erika:	... a model is and why do models change and why you are important to a model in like specifically like the film, and do they differ? Can you just take a little minute about what a model is and why-
Robert:	... that's useful here?
Erika:	… A machine learning model is essentially a trained statistics function or model. So, in statistics, you'll train an algorithm according to a sampling of data so that you can say something insightful about the entire population, right? But the algorithm itself-
Robert:	You can, you can craw a conclusion-
Erika:	Right.
Robert:	... so it can say, "Hey, if this many people did this way, I conclude with some degree of certainty that this will happen-
Erika:	Right.
Robert:	... or this is true." Okay.
Erika:	The way in which you can use that same algorithm for a machine learning purpose, where instead of saying, "Okay, I want to use this to give me insight about the entire population." You're saying, "I just want to get the insight or prediction on a new piece of data that's coming in." So, say I trained on this one sample and now I have a trained function. And now I want to use that to make predictions about a new sample that is coming in. Now you're doing machine learning. That is-
Robert:	Mm-hmm (affirmative).
Erika:	... 100% how machine learning is functioning. Now these functions can get very complex. The latest technique that is used in a lot of different areas now, uh, is called deep learning which are based on a neural net-artificial neural networks which are more or less modeled our understanding of how the human brain works. So the way we have this particular one set up, we're using what's called an inpainting approach. So-
Robert:	Mm-hmm (affirmative).
Erika:	... the model that we have trained has been trained on like clean data, clean images that don't have anomalies in them. And it's trained to say, okay, given the surrounding pixels, can I generate what is supposed to be on the inside section?
Robert:	Yeah.
Erika:	So if I take out like a little box of pixels from like some quadrant I can look at all of the pixels around and then I can infill that I can inpaint this little section.
Robert:	Mm-hmm (affirmative).
Erika:	So that's the kind of model that we're using- ... and what we're doing is we're actually having it go across the entire image so that it can inpaint every single pixel, we have to do some, some clever masking-
Robert:	Wow. That's some serious compute work going on there too. I know we're just on the tip of the iceberg, right?
Erika:	Yeah.
Robert:	How, how- ... is this leading, do you think to the future of film production, what do you think the future applications are?
Erika:	I mean, I think there's a lot of different ways that these techniques and tools can assist our filmmakers. But one of the things that I always say when I'm speaking to anybody more on the creative side or outside when everybody says. "You know, do you think we're going to be making our movies entirely with AI, or we're going to be- ... writing our new scripts with AI?" And I say, "No. No." That is, that's not what we want to do, you know? I want to come back to that notion of like, we want to give our humans superpowers. We want to let our people do the things that they are so good at because you know, the creative storytellers that we have at the Walt Disney Company are incredible. They're incredible at what they do. What we want to do is instead give them new tools to explore and see what is possible. Can we do different kinds of stories with technology? Are there different mediums in which we can tell stories or are there ways that I can do this other tedious part of the filmmaking process faster? But there were so many, so many little ways in which these machine learning techniques can help kind of behind the scenes. That's really what I'm interested in doing.
Robert:	Well, I think about also the speed of production that if, if a- ... director, you know, the production managers, set managers, assistant directors, et cetera, were educated in the capacities or the capabilities of AI and machine learning on the back end, specifically in the ability to create scenes automatically or to-
Erika:	There's, yeah, there's-
Robert:	... you know, build out that stuff.
Erika:	... there's a lot of folks working in-
Robert:	Changes the economics.
Erika:	... yeah. There's a lot of folks working in like the virtual production space right now.
Robert:	Mm-hmm (affirmative). Mm-hmm (affirmative).
Erika:	So, a number of filmmakers are exploring that both, you know, who have worked with, with Disney before. So, if you think about something like Jungle Book or the live action “Lion King."
Robert:	Yeah.
Erika:	You know, the Lion King was built in like that was a virtual production film that was-
Robert:	Was it really?
Erika:	Yeah. So, they all have the planning for what the shots were going to be that was all done in a virtual production system. Those kinds of tools, those kinds of tools are going to get faster and better because of some of the machine learning capabilities that we have now. I know there are there, there's just so many folks exploring and researching how to do things like, I don't know, automatic green screening- speeding up some of the processing for- ... heavy, heavy amounts of data. There's, there's so much possibility. So much is being done.
Robert:	Well then, and then I keep, so Erika, you just got, I'm going to geek out totally, right? Because my-
Erika:	(laughs)
Robert:	... my family is over-rotated and films, right? We had a movie night at our house every Friday night for almost two decades (laughs) with my family. So, we would schedule movie, we'd have a dinner all laid out on a table with this very specific, you know, food that related to the movie, that kind of stuff like that.
Erika:	Right.
Robert:	So our whole families relate to that. And we learned everything about film production and we made animations like that. And I think about the opportunities for folks that are actually doing production to maybe see a more end result, pre editing, you know-
Erika:	Mm-hmm (affirmative).
Robert:	... to see, Hey, how this would look and the machine, just all that technology that will overlay on top of that. So if you think about like the dream project that you could start, what would it be?
Erika:	I don't know. I feel like-
Robert:	(laughs) So many.
Erika:	Yes. That's really the problem is that A, there are so many and B I'm already working on several (laughs) of them.
Robert:	Right. The ones that you already would have dreamed of. Yes, I get it.
Erika:	Yeah.
Robert:	I totally get it. You know, I think of myself, I have to say that in my own personal career, I've kind of made an act three of my own personal career, right? So, I think about just the opportunity. I'd never want to retire because I think about these opportunities in technology that are forces for good, how we do good things with this.
Erika:	Yeah.
Robert:	And you know, in the, in this whole area that you're talking about, I think this is a really great place where this force for good and what the Walt Disney Studios and StudioLAB is doing is just, it makes people feel good.
Erika:	Yeah.
Robert:	I mean, there's like nothing bad (laughs) anywhere on this. And so-
Erika:	Well, that's the idea, that's the idea.
Robert:	It is. Isn't it? Have fun, have fun. Well, Erika, thank you so much for joining us on the HPE tech talk. I really appreciate it.
Erika:	Yeah, absolutely. It's great. Anytime you want to geek out. I am here for it.
Robert:	For us that HPE, it's been a pleasure to partner with StudioLAB and to create the solutions that are driving the future of film industry today, to our listeners, I really hope you enjoyed this episode. … If you, if you don't already subscribe, leave us a review and tell us what you think. Thank you for tuning in and we'll catch you next time. Bye-bye.