Film making is war, and if you want to win, you better have some damn talented generals to whip your army into shape. As of this writing, it is Sunday the 24th, and tonight is Oscar night. If you are watching it, pay close attention to the nominees for best visual effects. Almost all of them are from ILM, Industrial Light and Magic. These guys are the ultimate generals, the Dynasty Warriors of visual effects.
The process of visual effects is one of the most complicated, if not downright painful, in all of film making. There's a reason why, even to this day, so many Hollywood directors try not to use them, or why George Lucas always overuses them. On the one hand, sometimes it's necessary to have an alien spaceship shooting laser beams at cowboys, and there's no way to convincingly do that in camera. On the other hand, the required hardware, extra staff, extra steps to take, etc can all become very overbearing, and very expensive, very fast.
On the Painted Lady, it was obvious from day one that this piece would require a lot of effects work. I think there's only two or three shots that won't have some sort of visual effect added, and even those will have color correction, grading and camera stabilization. So post production is often the longest process of the three, and with the Painted Lady, it's no different.
The first trick is to make sure you are using visual effects properly. Meaning, to use them to tell the story, not be the story. You hear people saying this all the time in behind the scenes documentaries, but it doesn't hurt to say it again. Maybe George Lucas is listening. Oh wait, he sold Star Wars to Disney? Never mind, he can't hurt us anymore.
With the Painted Lady, story is key and a lot of time was spent wrapping our heads around how to tell the story. There are several time periods and shots that we wanted, that we had no way of shooting, in reality. When you don't have the budget to build a European town bombed out in World War II, you don't have much of a choice but to go with matte paintings. But we always knew, when it came down to it, we had to make a character in a painting, leave the painting. Now how the hell were we going to pull this off?
A great deal of time was spent in preproduction researching how to do this effect. Most of the effects work we had used in the past was basic compositing ABCs. Making someone disappear? Easy. Need a J. J. Abrams flare? Videocopilot has a plug in called optical flares.(By the way, if you are an after effects user and you haven't checked out Video Copilot yet, then do so NOW. Andrew Kramer, the man who runs it, is pretty much the Lu Bu-esque god of After Effects.) But doing something that's never been done before? That's a fun challenge.
Now, a lot of jokes can be made about those involved in the post process being lazy computer nerds, and this is true to a point. We the people of post are the nerdiest buggers on the set. Visual Effects Supervisor Brandon Dostie and I have been known to lull a production meeting into a comatose state, talking about depth passes and render qualities. But hey, someones gota do it. In defense of any post production team, however, a lot of the work is just figuring out how the hell you're going to create XYZ effect. It requires a lot of problem solving, a bit of calculus and an open mind. Sometimes the solution is so dumb, so ridiculous, that people think you've lost your mind. But it works.
So lets give an example. In the previous blog in this series, we talked about shooting actress/model Neyney Light on a green screen. This specific shot is of her waking up, so to speak, in the painting world. First, we had to figure out how to portray this cinematically, then we had to figure out how to show this, physically. She starts off in the painting, holding an apple:
So what happens when she comes to life? Does the apple come to life too? No, it's not part of her. So what happens to the apple? Does it fall? Well, again, no, because there is no gravity in a painting. So it just stays where it is, where it was painted. But then you have the problem of the nature of the image: it is a painting, made out of individual paint strokes. So what is behind her, in the environment, doesn't exist, because it was never painted. Which, for a post production guy, is kind of a god send, as it reduces the need for a back plate, or creating the illusion that a character is in a three dimensional environment that they aren't really occupying. However, this does provide another interesting challenge: How does a moving painting effect itself?
As previously stated, the physics of the painted world were an interesting hurdle all by itself. The character has to appear to be there, but not be effected by its environment. Instead, it effects the entire composition. Since both the figure and the painting it occupies are made of paint strokes, it follows that the entire piece would distort and smear if some part of it shifted.
What we decided upon was a sort of echo effect (not to be confused with the actual echo plug in, for those after effects compositors out there) whereby as she moves inside the painting, she would leave 'echos' of herself that would bleed into the rest of the painting around where she had moved, giving the illusion that she was smearing the paint around her and causing it to displace. Still following me?
If this is a hard idea to conceive of without seeing it in action, it's a terribly difficult effect to create. So here is how we did it. First, we filmed Neyney on the green screen, doing her best to mimic the pose she had used two weeks prior during the still shoot. The lighting also had to be matched with that from the set. She holds the pose for a few moments until she subtly starts to move as if coming out of a trance, and looks around her environment before exiting, stage canvas.
The first step after we enter the post process, is to take this raw footage and import it into After Effects. The process for removing her from her background is called Chroma keying, whereby we take the color green and spend approximately forever carefully removing the green, but keeping the edge detail. For a decent key, this takes several hours, breaking the clip down into separate pieces, keying them all individually, then reassembling the pieces. Sometimes a perfect key isn't always possible, especially when thin stringy hair is involved. Luckily, there are many tools available later to help sell this and cover up any loose edges.
Now that we have our keyed out footage, we are tasked with making it look like the painted Neyney. To do this, we render out our footage as a .png sequence. A .png is an image, like a .jpg, except that it supports an Alpha channel, or transparency. That way any effects we apply to it will only effect Neyney. Depending on your hardware, this takes 20 min, up to several hours.
Prior to this, an action, or series of effects, was created in Photoshop CS6, including several painting effects, an enlargement and noise reduction. All of the .pngs are then batch actioned, with all the effects applied to each of the .pngs, one at a time. It's an automated process, but it can take a couple hours. By the time this is done, our finished footage has been upgraded to almost 4k in size, and is starting to look more like the original painting, in terms of texture.
After this, the next step is to import the sequence back into after effects. Even with the same costume and as close to the same lighting scenario as the original, the are still some discrepancies. Namely, the color. Because the original was shot in RAW, we had a lot of options to get the color as close to Vermeers' palette as possible. Unfortunately, we don't have that option with the shot footage, so we have to take another step to make sure the colors are accurate to the painting.
Luckily, the brilliant bastards of Red Giant were nice enough to invent a tool called Colorista. While not a total replacement for such high end color treatments as Da Vinci Resolve, it works for our purposes. Because of the discrepancies between the RAW photo and 8 bit AVCHD footage that the Sony camera gave us, we need to isolate all the major color bodies for individual grading and Colorista is just the tool for the job.
The skirt, the lips, the blouse, the hair, the skin and the corset all require separate instances of Colorista for this, as well as to switch to 32 bit floating color, and a touch of Magic Bullet Looks, another color grading product from Red Giant. The final product is then rendered out into yet another image sequence, this time a .TIFF... because I'm a masochist.
Now we have our .TIFF sequence, rendered out in painfully high res 4K. Since we started working on the shot, assuming we didn't sleep, it's been around 3 days, on our average workstation. And that's only one shot. And we're barely 1/3 of the way through making this one shot. As I write this, this specific shot is rendering out again, with new effects, in another process, so I can't show you anymore making of pics at the moment. But I'll give you the basic breakdown.
So as you can see, we are nowhere near done with this shot. Also, somewhere in there you can see just where I lost my mind and why none of us have social lives. But that's what is necessary for the level of quality that we are looking for. But don't cry yet. Later this week, we are meeting with some potential new blood, animators and compositors that will help with the workload and bring this baby to a speedy and beautiful conclusion. Next week, after we have some of these crazier shots done, I will post some more making of pics, so you fans at home will know that we are doing more than just sipping martinis.
No comments:
Post a Comment