Green Screen & It’s Future in The Visual Effects Industry

A before & after still from Godzilla (2014)

Green screen or chroma keying has become an integral part of visual effects in the film post-production and VFX industry. It is also called colour keying since it involves the removal of a specific colour from a media file, in order to replace it with another ‘plate’ (video/image or CGI elements). Generally green or dull blue shades of blue are used as on a background screen during the production process since these colours are rarely used in the foreground, especially for aesthetical purposes.

The above image shows the before and after shots of Godzilla (2014), a popular Hollywood film. Green screens greatly reduce manual efforts since it would be a tedious task to rotoscope and separate out the foreground from background in a motion picture. Notice how the minute details like the gaps between the railings and the actors are replaced with a new post-processed background in the above picture.

The mockup plate can either be an image or a video

Color keying is also used in product mockups. In the above example, the screen of the laptop can be replaced with another video or an image, demonstrating a new software product. This technique is even used in multiple news studios for replacing the background during a live stream in real time.

Green screens reflect their colour and tend to ruin the intended colours. (Source: Justice League)

Hence, colour correction is necessary after green screening. It involves correcting or altering the colour of light in a media, to get the intended colours. Since this is a major hurdle in the post-production process, high budget franchises like The Mandalorian are moving to ‘virtual sets’ which consist of a screen-like high resolution LED wall and the appropriate cameras to film them. This also helps the creative team to change the look of the background at the time of filming, hence cutting revision costs during the compositing process. The LED screens produce a real-time parallax effect too based on the movement of cameras! And the colours and reflections on the foreground object are already as intended! This is possible due to the use of Unreal Engine’s real time features.

The virtual sets of The Mandalorian

How Green Screen Works?

There are several methods. Here’s one involving masking and color thresholds:

1. The media plate with the green/blue screen is selected
2. The colour threshold/range for removal is selected, depending on the lighting conditions.
3. With the above settings, the keying software removes the green/blue shades and a black-and white mask for the image is created.
4. The masked plate then shows the foreground objects with the background removed.
5. The masked image/plate is added and placed onto the required background image.

This object has been masked out from the original image. (Courtesy: Wikimedia)

Let’s try another easier method in Python. This method is more suitable for conditions where the green color is uniform and thresholding isn’t required (like an individual image or animated videos or vector graphics with a uniform green color background). We will be using the following images:

The Planet of Pandora from Avatar (Credits: Image from wallpaperaccess.com)

Thanos Green Screen Image by: Roughened Edges YouTube Channel

Before putting Thanos on Pandora, it is important to note that both images should be of the same size! In this case, both images are 1920p x 1080p.

Now, let’s find the value of the dominant green color in this image using imagecolorpicker.com

imagecolorpicker.com

Note the RGB value which is (1, 128, 1) in our case.

Coding Time:

Import the following libraries in your favorite Python IDE:

import cv2
import numpy

Load the necessary images in the following variables:

fore=cv2.imread(‘thanos_foreground.png’) #the image to be keyed
back=cv2.imread(‘background.png’) #background

Store the RGB values from above in variables:

height, width = 1080, 1920 #image resolution
r, g, b = 1, 128, 1 #the RGB value of dominant green region

We now iterate, pixel by pixel, through the foreground image, looking for the mentioned color values and replace them with the pixel at same position from the background:

for i in range (height):
    for j in range (width):
        if fore[i][j][0]<=r and fore[i][j][1]>=g and fore[i][j][2]<=b:
            fore[i][j][0] = back[i][j][0]
            fore[i][j][1] = back[i][j][1]
            fore[i][j][2] = back[i][j][2]

The above code will successfully key the image! Save it with the following code:

cv2.imwrite(“keyed_image.png”, fore)

Run the code and let it process. This may take a minute. Let’s check the file for our keyed image:

Keyed Image!

Voila! Thanos is now on Pandora! But wait, see those green outlines around him? It’s because those color values weren’t the part of our range. Since it’s rudimentary and takes a while, it definitely isn’t a practical method (except when you want to chroma key a single image). But it’s a good way to develop understanding of the technical aspects of Green Screening!