Arowin writes: I was recently privileged enough to view a lecture by Milton Ngan. As far as IT stuff goes, Milton has a pretty good job. You see, he is the Digital Operations Manager at Weta Digital. He is basically the architect for all the technical side of things at Weta.
Last night he came and gave a 1 hour lecture at Victoria University outlining the hurdles and obstacles that needed to be overcome to produce the stunning 3D graphics lying in each of the Lord of the Rings movies. The lecture itself was full of lots of facts about Weta, the IT side of things and it also included some very cool behind the scenes shots of The Two Towers.
Weta
Well as you probably already know Weta is the Wellington based company in charge of bringing the Lord of the Rings special effects to life. It is composed of two companies – Weta Workshop and Weta Digital. Weta Workshop deal specifically with the models, props and miniatures – all the physical things. Weta Digital on the other hand, work with the virtual – all the 3D characters, Motion Capture and Special effects.
A lot of information about Weta Digitals IT structure was thrown at us some of which includes:
All the film first comes to WD and is scanned into the computer system through the use of one of two Imagica Film Scanners. These scanners can scan at 3.2 mega pixels at an amazing 64 billion colours. The speed of this is about 4.5 frames/second
The data is then fed to the storage facility along a 4 gigabit Ethernet.
From the reels alone Weta stores about ½ a petabyte of data. By the end of film three this amount is expected to reach 1 petabyte.
The workstation structure at Weta is astronomical. They have:
-125 SGI Octane systems
-220 Linux systems
-35 NT systems
-15 Mac systems
Then there’s the rendering system…. The renderer alone is run 24 hours a day rendering out the 1000 odd shots that were required for TTT. It consists of 192 Dual Pentium 1 GHz and 448 Dual 2.2 GHz processors. A total of 1280 processors running at approximately 2,355 GHz…. Mmmmm…..
Creatures
The creatures are first sketched out on paper and then modelled with modelling clays. After Peter Jackson’s approval, these models are then scanned with a mobile hand held 3D scanner (developed in Christchurch, NZ) which scan into the computers every single bump and feature of the model. In fact so much data is scanned in that it would be next to impossible to use it all so the mesh is then converted to a NURBS model, which smoothes most of it out. The detail that was lost is still kept and the difference between the NURBS model and the original scan is uses as a displacement map (A texture that applies bumps based on a greyscale image).
Once the models have been put into the system and textured, it is up to the animators to animate the creatures. A great deal of the animation was done automatically. For example, when Gollum walks, the system automatically splays his toes as he steps down, this allows the animators to focus more on the artistic side rather than the mechanical. A good example of this was shown with a simple mock up of Gandalf standing still with Gollum jumping up onto his back and ripping off his head. Most of the actions were done by the computer.
Each animation must go through a rigorous development process. Most of the actions where Gollum is by himself can be taken from Motion Capture with Andy Serkis, but for other scenes (In particular the first scene of Gollum when he jumps on Sam and bites him) Andy wasn’t right for the situation (“He looked too much like WWF). So the animators made a few animations including Gollum ferociously attacking Sam to Gollum coming up and licking him.
The motion capture uses about 20 different cameras and can support multiple actors at the same time. Andy was used for almost all of the actions and most of the animations were done using him as the actor. A blooper that was shown was Gollum playing an electric guitar. Also a few shots were shown with Kermit the frog instead of Gollum. Even after the motion capture is done. The animators will still need to go in and clean up a few details. A few shots were shown with Gollum with his hair dyed pink and standing on end, another with his eyes floating out away from his eyes.
Andy Serkis as we know was also the basis for the facial animation of Gollum. The animators based Gollum’s actions on Andy. Side by side shots were shown with Andy and Gollum (In the split personality scene) showing exactly the same facial expressions and speech expressions.
To composite Gollum into the scene is quite a rigorous process.
1) Firstly, Andy acts out the scenes with the other actors.
2) The scene is then shot without the actors using exactly the same camera movements.
3) The Computer camera is then matched to make sure the angles will still be right
4) The lighting in the Computer environment is then matched to the camera footage
5) The compositing is done. Andy is literally painted out of the scene and Gollum placed in.
6) More painters come in and now have the arduous task of painting all the areas where Gollum is behind something or in front of.
Massive
Massive was described to us as “AI on steroids” and by all accounts it is. I’m talking about the computer software that was developed on site to do all that large battle scenes that rage in both FotR and TTT. It is a system that creates hundreds and thousands of ‘agents’ – individual 3D creatures that think for themselves and battle it out on the field. They can react, fight and make logical decisions based on inputted given data. The program is so details that agents can get dirtier as the battle progresses.
For example, in the battle of helms deep, thousands of computer generated orcs fought against the stronghold of men and elves. Each orc would react and think as an individual. A top down render was shown when the wall exploded and ripples were sent through the agents. Now, traditionally this would have been done with particle effects and a wave effect but this was made simply by the reactions of each agent.
Each agent is given a complex tree ‘brain structure’ whereby data is inputted to the agent and based on the tree an output is given. If two outputs come through then the result is mixed together. An example of this was an agent that was set so it doesn’t walk off a cliff. It walked around by staying on the flattest surface. As soon as it was ‘allowed’ to walk off the edge, its arms flew around in a very lifelike manner. Each hit against the cliff gave different results and those results were added together to form the movement of the fall.
Each agent is also given certain characteristics according to its race. Each race has its own unique fighting style (which was motion captured originally). Each individual agent is given random variables to make them shorter, taller, a larger walk almost everything. Massive then calculated every single movement based on the agent – A shorter agent will have to walk faster to keep up with all the others.
When Massive was first tested two armies were pitted against each other to fight it out. Once the scene was rendered, a bug in the program was found. Agents were actually seen running away from the battle field! This simple bug was resolved by adding the rule “If you can’t see an enemy, turn around”.
With The Return of the King, Massive is apparently being stretched to its limits. Peter Jackson is saying that the great battle must be several times larger than that of Helm’s Deep. This is not only stretching Massive to it’s limits but also the Intel 32bit processor architecture as well and Weta is looking at replacing the processors with 64bit ones. Whatever they do, RoTK is set to be pretty spectacular.
Other Random Stuff
– When Peter was working in London and the animators needed his approval on certain shots a conference was set up from New Zealand to London through a private 10 Megabit connection. Each day for 3 months, these conferences would go over about 10GB of data that was rendered during the night.
– Gollum alone has about 20GB of textures
– Gollum was completely redesigned from the ground up for TTT
– The room at Weta that stores the servers started out at about 40m2. For TTT this was made larger by about 60%. Now, it is about 4 times as big as it was at the start and work is under way to increase the size again
– Weta Digital has only now finished the extra footage for TTT DVD that will be released later this year and is now starting work for RoTK
– In TTT there were approximately 800 Computer Graphics shots. They are expecting in excess of 1000 for the third film.
– Thousands of different programs were used in the development of the special effects. Most prominently Maya was used for the modelling and animation, Pixars Renderman was used for the rendering and finally Apple’s Shake was used for compositing the film.
– The developer of Massive is under talks at the moment to release Massive commercially.
