Chromakeying in Matlab

Chromakeying in Matlab

The aim of this code example is to extract the green background, also called chroma key, and replace it with the background from a different image. In fact what we’re doing is keying out a black and white mask from the green component of the foreground image and using it to mask any other background image.

% Read any two images for background (bg) and foreground (fg)
bg=double(imread('mapa2.tif'));
fg=double(imread('top.jpg'));

% extract 3 different color channels
% and luminance matrix
fgR = fg(:,:,1);
fgG = fg(:,:,2);
fgB = fg(:,:,3);
fgY = 0.3*fgR+0.59*fgG+0.11*fgB;

% subtract luminance from green
fgG_Y=mat2gray(fgG-fgY);

% Calculate and plot histogram
histo=hist(fgG_Y(:),256);
plot([0:255],histo);

% Define and normalise a reasonable threshold to try to avoid the spills
% too high will trim the image
% too low will let some green through the borders
thres = 80/255;

% set to 1 in your mask all those values where 
% fg(G-Y) is lower than the threshold
mask = fgG_Y < thres;

% finally, save channel by channel the foreground where mask = 1
% and the background where mask = 0 (1-mask = 1)
final(:,:,1)=fg(:,:,1).*mask + bg(:,:,1).*(1-mask);
final(:,:,2)=fg(:,:,2).*mask + bg(:,:,2).*(1-mask);
final(:,:,3)=fg(:,:,3).*mask + bg(:,:,3).*(1-mask);
imshow(mat2gray(final));

The result is something like this:

Screen Shot 2014-11-13 at 11.08.36

But what about casting a shadow?

Let’s observe the histogram:

histogram

We can clearly distinguish 3 entities from it: the person, the shadow cast and the background.

By placing a threshold at a value of 150 we would be extracting the person only. If we want to keep the shadow we should plan a threshold at about 200. So the answer is to create two different masks this time to be able to distinguish between 3 different areas: foreground, shadow, background.

Foreground

– Mask 1 is active (= 1)
This means it’s getting it’s source from the foreground image.

Shadow

– Mask 1 is not active (= 0)
This means it’s getting it’s source from the background image.
– Mask 2 is active at 0.5
This means it will only be half as bright as the original image.

Background

– Mask 1 is not active (= 0)
This means it’s getting it’s source from the background image.
– Mask 2 is not active (= 0=
So that it doesn’t interfere with the shadow.

bg=double(imread('mapa2.tif'));
fg=double(imread('top.jpg'));
fgR = fg(:,:,1);
fgG = fg(:,:,2);
fgB = fg(:,:,3);
fgY = 0.3*fgR+0.59*fgG+0.11*fgB;
fgG_Y=mat2gray(fgG-fgY);
histo=hist(fgG_Y(:),256);
plot([0:255],histo);
thres1 = 150/255;
thres2 = 200/255;
mask1 = fgG_Y < thres1;
mask2 = fgG_Y < thres2;
final(:,:,1)=fg(:,:,1).*mask1 + bg(:,:,1).*(1-mask1).*(mask2)*0.5 + bg(:,:,1).*(1-mask1).*(1-mask2);
final(:,:,2)=fg(:,:,2).*mask1 + bg(:,:,2).*(1-mask1).*(mask2)*0.5 + bg(:,:,2).*(1-mask1).*(1-mask2);
final(:,:,3)=fg(:,:,3).*mask1 + bg(:,:,3).*(1-mask1).*(mask2)*0.5 + bg(:,:,3).*(1-mask1).*(1-mask2);
imshow(mat2gray(final));

Screen Shot 2014-11-13 at 11.07.03

Softening the shadow

For a more realistic effect we would want to soften out the shadow a little bit. This is simply done by averaging the shadow mask.

H=fspecial(‘average’, [20,20]);
mask_av=imfilter(double(mask2),H);

and using mask_av instead of mask2.

This is the result:

Screen Shot 2014-11-13 at 11.08.04

One Comment

Cancel

  1. when i use the code program in here , notifed : Undefined function ‘mat2gray’ for input arguments of type ‘double’.
    how solve it? please help me :(