CGI in Films

Recently I have had some communication via email with an Australian artist called David Rutherford. He interests me simply because he started out as a miniature model builder in the film industry, but then made the transition in the 90’s to working as a CG artist. I asked him a few questions about this, such as why did he make this decision, the differences in the two art forms, and where he sees the use of miniatures in film in the future.

This is a quick summary if his skills and experience taken from Mr Rutherfords’ Linkedin page:

“I have experience in every facet of visual and special effects.
From practical rigs, props, miniatures, and pyrotechnics , right through to digital effects, creature animation lead, FX animation, compositing lead, on set supervision, previz and cg effects supervision. I’ve directed animated commercials and been nominated for an AFI visual Effects award for a documentary series.
I have strived to learn as much as I can about all facets of the visual effects field.”

Below are some of the responses to my questions. I have highlighted the most important and enlightening points in relation to my research question, Is there still a place for miniatures in contemporary film?

On his background:

“I was always making things as a kid, robots from old batteries and transistor radio parts. I always pulled things apart to see how they worked.”

“I was going to be a film director and I was planning to go to the Australian Film Television and Radio School then at age 17 Star Wars came out, along with a massive amount of information about special effects that was very hard to find previously. I realised it was the special effects I was interested in rather than the live action parts of the movies and that it was a job that you could actually try to do.”

From here, Mr Rutherford went on to study media design, mostly focusing on animation. Outside of University:

“I thought maybe a way into effects was through model making, so I sent off photos to all the industrial and architectural model making companies in Perth. Got lucky and was employed at the largest of these companies in Western Australia and learnt the trade for two years. I did a lot of plant and piping models, ship models, and Petroleum, gas, and mining models and some small architectural stuff, still making the odd spaceship at home.

I moved to Sydney at the beginning of 1987 and worked at Mirage Effects for a year and got a special effects education. We did everything from models and miniatures, props, rigs, floor effects like wind, dry ice, Liquid Co2, set detailing (sci-fi) whatever.

One of the last things I did was a documentary series about early aircraft and I really enjoyed it. I was utterly sick of advertising at this point and so moved once again across the country to work on prehistoric furry creature TV documentaries among others. Did that for 4 years learnt a pile of new stuff and could specialise in animation and compositing . After that all wound up I moved back again to Western Australia and have been working for the Producer of the Aircraft documentary series on developing projects.”
On steps for producing CGI and miniatures:
“In a nutshell the same. Build the shape, paint or texture the shape.

Its in the controlling of the action and getting the image onto the screen that is different.
The miniature is lit, controlled and shot all in one session ( day, days or weeks). Depending on the subject many variations or takes can be tried at the same time.
The Cg models motion is animated or simulated, passed on to the lighting department to be lit and rendered.
A CG shot is usually rendered in discrete passes where each contributing element, such as diffuse colour, specular, reflection, shadow, global illumination, etc etc is rendered as a separate file and recombined in the composite. There is an alternate more recent view that this is overcomplicating in time and storage space and less efficient than just rendering a near final image in one beauty pass that can be subtly corrected in the comp. This has come about because of the growing push to unbiased renderers in which the light interacts and bounces about based on physical principles. Traditionally there have been many fixes and workarounds and tricks to get around the slowness of the rendering process but with improvements in hardware and software it is more efficient to discard all the tricks and just do it like it really happens in the real world. It is easier and takes less time to set up than all the workarounds take to save in rendering time.
In both scenarios there could be a compositing step before the image is finished. The model shoot could capture the final shot in camera but in this day and age less likely, it is more usually going to be an element of a shot to be composited with other elements.
It takes a good deal less people to do a model shoot than it does to have all the modelers, texture artists, simulation TDs, lighters etc you need to do a CG shot.
There is a myth that miniatures are more expensive, I don’t see how, the crew lists on cg films are huge.”
On CGI vs Miniatures:
“In terms of CG I have never worked in a big facility. I have always had to do pretty much everything in smaller places with very little money. It was such a luxury to be able to concentrate on a couple of things like animation and compositing and let other people worry about the other aspects. My skills in those two areas improved greatly because of it. Now I am back to being by myself and having to do everything again. I find I am getting increasingly frustrated by CG and am pining for the old days of miniatures, when I knew or could work out how to solve the problems. Actually miniatures are just more fun. Sitting in front of a computer all day is frankly dull. There is a level of excitement on a miniatures shoot that is simply not matched by any CG process. For many people in VFX these days their work is pretty tedious, rotoscoping, data wranglers etc very few actually see a whole shot through.
CG has got more complex and it is pretty much impossible for one person to be able to do everything at a level that is acceptable to audiences used to what a facility of 200 plus crew can achieve on 200 million dollar movie.
I am also finding that I am so unmoved by CG these days, the Hobbit being a case in point. Everything in the frame is synthetic, the camera moves in an impossible way, there is spectacle but not any excitement.
It is what I term low stakes,  there is no risk to limb or property in CG.  In miniatures there is risk, the model gets destroyed, somebody worked on it for weeks. The shot might go wrong so there has to be a backup.
The other important aspect is that visual effects no longer have a hand made quality to them. They are like the most refined processed packaged food, tasty at first but ultimately lacking nutrition.
There are very few happy accidents in CG. Usually everything you see has been deliberately put there. There are very few surprises.”
On the future of miniatures:
“Ironically miniatures now more than ever could be a solution to many visual effects shots. With digital compositing it is so easy to remove control rods wires etc.

Radio Control has become ever so much more reliable and glitch free than in the past with 2.4 gigahertz technology.
It takes about the same amount of time to build a miniature as it does a CG model. The cost of the hardware and software would easily equate to or exceed the cost of workshop equipment for a model shop. It costs about $10,000 dollars to equip each CG person with hardware, software, chair and desk.
The interchange between digital modeling and a physical model is now a simple task for the 3d Printer 3D scanner.
You can print water slide decals on your home printer.
The same can be said for animatronics. The technology for doing really good lip sync and computer control started appearing as the industry jumped to CG creatures.
Now more than ever it would be possible to do really amazing animatronic creatures where the facial performance can be done ahead of the shoot like an animator would animate a CG creature. That performance can then be played back on set in sync with the prerecorded character voice over and over perfect every time. Variations could be called up for alternate takes. A layer of improvised control could be mixed in live over the top.
The problem is that all the VFX supervisors come from the computing world now. They have no experience of miniatures or animatronics. They are used to their world and are unlikely to change.
Directors are followers of fashion. They think that they have to go CG because everybody else is doing it.
People only remember the dodgy model shot, the good ones go unnoticed whereas the dodgy CG shot is largely forgotten and dissmissed.
It will take a maverick director to say no I want to use miniatures here, in the face of studio pressure.
It will take a maverick VFX supervisor to suggest miniatures in the face of that same pressure.
Having said that I detect a resurgence of interest in the old school. I think there is a growing backlash against the CG or nothing approach.
I think it will come from the indy film sector, not from the majors who only make comic book films these days , though the economics of making those films are at the cost of bankrupting the VFX facilities.”

I have written previously about some of the master classes you can find on the BAFTA Guru website, and have been watching and doing further research through this website on CGI and what digital artists feel are the advantages and disadvantages. This basic information is important as I need to have a broader understanding of procedures involved with CGI, and what it can actually achieve today in order to contrast and compare with my deeper understanding of miniatures. On the BAFTA Guru website were recordings of 3 talks that I would like to summarise here in relation to the capabilities of CGI today.

Firstly I’d like to write about the masterclass by Jonathon Faulkner of Framestore, in which he spoke about the role of VFX as a whole, and as a solution to practical problems. I found as someone who has a very limited understanding of the potential of CGI this was a useful masterclass in explaining the basics.

There are five types of shots where visual effects are used when it is broken down. Cleaning up – wires and rigging that are left on camera at the time of filming. Environment enhancement – changing from day to night, set extensions when filming everything on location has cost implications, entire digital locations. Atmospheric effects – smoke and water etc. Characters – face replacement between hero and stunt men, to a certain extent crowd replication. Props and stunts – models and miniatures, bluescreen.

It was also interesting to hear about the on-set roles of VFX technicians. One such role is to be there to explain to actors, directors, lighting and cameramen what will eventually be there once the shot has been handed over to post production so that everyone can allow for this. Someone also needs to record info on atmospheric conditions at the time of shooting, take measurements of the actual set and take continuity photographs so that the digital work can match in as closely as possible. Finally, someone needs to be on set to advise others of the cost implications of their decisions and actions at the time of filming, so that changes can be made to ease the production of the digital work later.

He talked about how everything from locations to hair, makeup and costume, could be achieved digitally if the relative cost of doing so was irrelevant, and of course the actual thing will always be more realistic on camera:

“If I am asked about how to achieve a shot, and I’m asked about the visual effects solution to it, the best solution is always to shoot it for real if you can”

In fact, I found it really interesting how Mr Faulkner made the point that the digital artist does often have to fulfill the job of the costume designer, and the hair and makeup technicians, and so the skill of working in digital VFX does involve a high level of artistry and understanding alongside the technical skills of working with the software. I have during my research questioned how many digital artists do have this level of artistry and imagination, or is the digital artist someone who understands the capabilities of the computer and the software available, so it was interesting to hear this from a VFX artist.

The second talk was led by representatives of the company Jellyfish, who are a BAFTA award winning VFX company based in Soho, London. Their masterclass was based on the idea of saving money on VFX and what should and shouldn’t be done in this respect, in their opinion. They emphasised how important the planning stage should be, and how at this stage cutting corners will eventually result in further cost. So, make sure you do hire a VFX supervisor, and use storyboarding and pre-viz to plan out your film. It is also worth getting in touch with VFX companies even at the script writing stage to get their input right from the start. This ties in with what José Granell from the Magic Camera Company emphasised when I interviewed him earlier in the year. Mr Granell believes that a lot of poor CGI shots are instigated by late or no planning on behalf of the director, and a lack of understanding of the complexities digital artists are dealing with. These artists also need to be guided and informed throughout the process in order to produce the best work.

Using miniatures requires forward planning which includes a set deadline and procedure for building and filming that ultimately results in a more convincing outcome.

The third master class was taken by the makers of the low budget film “Monsters”, Gareth Edwards and Colin Goudie. With 250 VFX shots to produce over a short period of five months, it often involved judging when they could afford to take short cuts and asking themselves how lazy could they be with each shot as opposed to how to get the best most technical shots. Their basic test was to watch each shot once instead of on a continuous reel: did it convince you? If so, move on. What is the main focus on camera? If it is not the VFX element, the audience won’t notice how poorly the effect is achieved. To save on money you have think how to make a shot easy to achieve. In “Monsters” no creature is ever closer to the camera than 10 meters, so that the camera doesn’t have to capture close up interaction with the environment which would make it much harder for the digital artist. All the digital work was done using basic programs like Adobe Photoshop and Aftereffects with a small amount of 3D work in 3D Studio Max. The main point that I took from this talk was to remember that all film making is an optical illusion, nothing is reality: the actors aren’t real characters, the illusion of movement is actually 24 still frames per second etc. Although it was a deliberate poor use of CGI elements, it’s interesting to hear about what these short cuts might be. They in turn highlight some of the disadvantages and the difficulties CGI face, which I can take forward to make comparisons with the technique of building miniatures.

On the BAFTA website I found a recording of a seminar with various visual effects artists working on Harry Potter and The Deathly Hallows Part 1 and 2 ( The VFX Advantage : Harry Potter Case Study, BAFTA  Guru 2011). It involved describing why, in their opinion, their use of such extensive digital environments and creatures was an advantage over using real locations for the films.

Emma Norton (Visual Effects Producer for The Deathly Hallows parts 1 and 2) , Tim Burke (Visual Effects Supervisor for The Deathly Hallows parts 1 and 2) and David Vickery ( Double Negative Visual Effects Supervisor for The Deathly Hallows parts 1 and 2) were the guest speakers at the event.

This is directly relevant to the themes in my project as the main competitor for the use of miniatures in film is CGI, and therefore it is important for me to have an understanding of the advantages of digital environments in order to contrast this to the advantages of using miniatures.

The most simplistic way of observing the growth and development of CGI techniques is to physically count how many CG shots were involved in each film. Emma Norton does just this, telling us that although the films are a similar length, in the first film out of 2294 shots, 669 were visual effects shots (these will also include those involving José Granell’s miniature shots). In the fifth film, The Half Blood Prince, out of a total of 2090 shots, 1400 involved visual effects. From these figures we can see the increase in demand and volume of CG shots chosen for the franchise. The question is, why are more and more producers and directors leaning towards CGI?

One of the main arguments against “excessive” use of CGI is that there is no interaction for the actors and director, and therefore a less enthralling performance overall. It was interesting to hear how the speakers did defend against this, commenting that the actors were shown images of the real locations before being filmed in front of a green screen. The idea is to at least give them a suggestion of atmosphere and the potential of the space once the environment is created digitally. To a certain extent the same can be said about using miniatures, that the actor has no tangible set to interact with. However, often a section is recreated at full scale for the actors to work in, since the detail of a scaled miniature would not stand up to the camera. Does this balance of visual effects and real time sets produce a better overall performance from the actors?

The actor Ian McKellen admitted in an interview with Contact Music to be being reduced to tears of frustration at acting alone in a green screen set on The Hobbit, saying “it stretches your technical ability to the absolute limits”. ( Ian McKellen : “Filming “The Hobbit” made me cry with frustration”, NME, 2012) Clearly, Ian McKellen believes he performs best when there is interaction between actors and tangible environments.

It is also true that having total control of elements like light is, on it’s own, an advantage that CG environments have over shooting on location. If it is needed, the illusion of a mid day sun can be kept for hours of footage, keeping continuity for the narrative. On location, the time slot for filming is dictated by levels of light and weather conditions, but using a cg environment, or even miniatures, in combination with green screen sets alleviates this problem. It gives more time for the director, lighting and sound technicians to perfect shots, which is of course a positive thing. With the technology available now, a real environment can be visited and captured by a digital team using engineering equipment etc, and kept in a library for future use.

The choice to build a series of green screen sets on the backlot at Leavesden came down to mainly to money and schedule. It allows the team to jump from filming scenes in Scotland back to Surrey quickly and efficiently, since the scenes were not being shot on location. It is important to remember that not only fantastical locations are built in a digital world. All the CG environments in the Harry Potter films were taken from real locations and meticulously reconstructed in the computer.

A final note on the development of CGI in films during the period in which the Harry Potter films were produced: at the beginning of the project, 90% of the visual effects shots were sent overseas. By the time of the final film 90% of the visual effects shots were produced by vendors in the UK, companies such as Double Negative, who are now the biggest producer of visual effects shots in Europe.

I’ve been reading up on The Hobbit since I saw it over the Christmas holidays because I couldn’t understand why the look of the film felt different even though it was directed by Peter Jackson, the same director as the Lord of the Rings trilogy. The answer – all CGI and you can tell. I’m hopefully not biased because of my project, but even though I enjoyed the film it was niggling at me the whole time. Admittedly the most obvious CGI was in the creatures rather than the environments, apart from Gollum who I thought they had improved. I can’t help making a comparison between the character of Gothmog in The Return of the King, achieved through physical effects by using an actor in prosthetics who still looks more convincing than the white orc in The Hobbit. Peter Jackson has reportedly said that it was the flexibility of the shots you can get in a CGI environment, swooping over and through doors and closer up than you can achieve with a miniature that affected his choice. However that might also be the problem; in terms of achieving realism, if a camera can’t achieve it in real time then it’s going to look fake done in CGI because our eyes tell us that it can’t be done.

When producing The Lord of the Rings Peter Jackson said that he wanted it to feel real, like it’s an alternative history for our world that really happened and I love this idea. I think if you want to escape into another world it needs to relate to what you know not bring you out of that, and I personally felt I couldn’t relate to it; the world of the Hobbit, although the same location, felt like disney fantasy to me. I think this may be down to the higher frame rate that it was filmed with partly, which is a new tool that allows for 50fps, similar to how our eyes process what we see, which added to the surreal childishly bright and sharp look of the film.

Originally Guillermo Del Toro was set to direct The Hobbit before having to turn it down due to money issues. He is famously known for opting for physical effects over CGI, and had planned to use less digital work that Peter Jackson used for The Lord of the Rings, opting for animatronics and miniatures instead. Before he handed the reins over to Peter Jackson he did an interview with about his plans for the films, including his opinions on the use of CGI (, 2008) :

“I think green screen photography is exactly like CGI, it is a tool, I don’t think it should be overused. Things like ‘Pan’s Labyrinth’ and ‘The Devil’s Backbone’ are incredibly dependent on location, we shot on location for more than half the time. Those locations can be enhanced by technology however, both digital and physical. What I would like to avoid is the recreation of the natural environments in CG, I don’t like doing that. The movie is essentially a journey movie, I think you need to use locations as much as possible.”

What I’m trying to do is keep the elements in place but allow you to feel a progression from ‘The Hobbit’ until ‘The Return of the King’. I believe ‘The Hobbit’ is a very crucial volume in The Lord of the Rings, it is a narrative that starts out very much in an innocent and golden way. It is permeated from England going through World War One, so there is a loss of innocence and a darker tone as the book and the film progresses. We’ll be doing that in the first film, taking you from a time of more purity to a darker reality throughout the film, but I think that is in the spirit of the book.

“What will differ from your films versus Peter’s?

The only thing I will be pushing for more in these films that the other three are full animatronics and animatronic creatures enhanced with CGI, as opposed to CGI creatures themselves. We really want to take the state-of-the-art animatronics and take a leap ten years into the future with the technology we will develop for the creatures in the movie. We have every intention to do for animatronics and special effects what the other films did for virtual reality.

Another thing people will notice, at the beginning of the film will be the palette, that will be slightly different, the world will be the same but it will be a more ‘golden’ world, a more wide-eyed world. But by no means will we depart from the canon, we will take the three previous films as canon. When I become part of a world that I love, such as this, I really come with a lot of enthusiasm and hard work, and we know we are recreating and creating a world that is part of the mythos of millions of people and we will approach it as passionately and respectfully as it needs to be taken.”

I notice that the extensive use of CGI in Jackson’s interpretation for me did not keep continuity with LOTR as Del Tor had planned. Neither did it have any sense of darkness or danger as it progressed, perhaps to do with the sharpness of the colours in the higher frame rate, but also because I felt it had lost the authenticity that LOTR gets from the extensive and beautiful location work and the amazingly detailed miniatures.

Just as an after thought, I was actually listening to someone discussing The Hobbit yesterday after I had finished this post. She mentioned how she loves The Lord of the Rings but wasn’t sure if she had liked The Hobbit because it had a weird surreal fake quality about it and was missing the gritty realism of The Lord of the Rings trilogy. I thought it was interesting that someone not studying film had picked up on all the points I had made myself, without understanding why.

I’d love to hear anyone else’s opinion on the film, since I seem to have such strong views on it myself!

When reading about the making of the Harry Potter films, I found a chapter on the miniatures they built for the film. I want to talk more in depth about the models later, but first I thought I’d do a quick background of the company that built them, Cinesite. The really sad thing is that this British company has now closed it’s model division, Cinesite Production Services, in favour of moving into the new technologies involved in CGI, like conversion of 2D to 3D. For the last two Harry Potter films they produced some of the digital work, such as the dust Dumbledore and the patronuses, which I do think look amazing and obviously couldn’t have been achieved any other way, but I do think it’s really sad that they have left behind such a tried and tested traditional craft, which may even come back into favour in the future. Who knows? I guess that’s what I am trying to find out this year. Before the model shop closed at the end of 2011, they had build models of the parliament buildings and consequently blew them up for the final sequence in V for Vendetta, built miniatures for the film adaption of Andrew Lloyd Webers’ Phantom of the Opera and created Wonkaville for Charlie and the Chocolate Factory (2005) alongside those built for the Harry Potter films.

Cinesite is a London based visual effects and post-production company recognised for their photorealistic work. The studio was created in 1994 in London, with a Hollywood counterpart also opening in 1991. They worked on Titanic and Space Jam in the 90’s, and various TV films and series including a remake of The Prisoner (2009) which I remember watching at the time.

In 1982, a barely acknowledge film called Tron was released, the first film to use computer generated imagery extensively. At the time the Academy of Motion Picture Arts and Sciences held the view that the use of computers in film was cheating, and yet now Tron is seen as a turning point in the use of CGI.
John Knoll, a visual effects supervisor at ILM has lived through changes in the techniques used to create visual effects in film for the past 30 years. He believes that the plot of the film, and being set inside a computer, in some ways helped to cover up the poor quality of the computer generated imagery. It therefore didn’t have to pretend to be live action, unlike  CGI shots in modern film. It did however show a lot of potential, which Knoll picked up on, for the future of film making.  At ILM, they are now at the forefront of technology, not only using off-the-shelf software, but also developing their own to solve problems as and when they come across problems that cannot be solved using what already exists.
On the subject of CGI versus miniatures and other traditional in-camera techniques, Knoll says (Gagne, 2007) :

Twenty-five years ago, the collaboration between live action and computer animation was innovative. Today, Pixar is making films that are wholly computer-generated; on the other end of the spectrum,Indiana Jones 4 is going back to its roots with stunt doubles and traditional special effects. Are live action and computer graphics merging or splitting? Will one replace the other? Oh, no — there are good complementary tools. Any new technology goes through a fad phase; gradually, as the technology matures and people become facile and used to the process, it becomes an accepted creative tool when used appropriately. We’re starting to get to that — it’s a gradual process, but we’re settling on less gimmicky stuff. One of the analogies I like to use is, when the Macintosh first came out, suddenly you could put twelve different fonts in a letter — and people would! Just because you had this new freedom, it was horribly abused. The novelty started wearing off, and it started being used tastefully. Over the last few years, I’ve seen a number of cases where people have tried really interesting, different photographic looks for movies. Some of these have gone overboard, about 50% too far. It’s the same thing where it’s kind of a fad: “Look at all this power! I can dothis with the movie!” As people get more used to having this kind of control, it’s going to settle down into a more tasteful use and will just be a tool like anything else, and you won’t see the extreme fringes quite as much.

One of the arguments for the use of CGI over models is the fact that once the software is set up, it takes less time and therefore less money to produce shots for film and television in comparison to models. However, I believe this is dependent on the quality of the work. Of course a poorly produced shot, with little attention to the details that give believability, will take less time and money to produce. But the point of both CGI and models is to trick the audience into believing what they are seeing is real, and in order to achieve this successfully time needs to be spent on the details.

I have found an example of a very popular television show aired on the BBC on a Saturday night that slots in some awful digitally produced shots that stand out as poorly, and probably quickly produced shots. Merlin is based on the legend of King Arthur, and is partly filmed on location at Chateau de Pierrefonds in France, which stands in as the castle. The screen capture is to a certain extent a true depiction of the castle, which was restored in the 19th century and is therefore not falling apart as many English castle are (this saved time by not having to build up a ruin using CGI) but it’s flat and has no depth or texture, which marks it out as CGI. The second screen capture shows a fictional tower with the same problems. The lighting doesn’t give it any weight.