GB2168564A - Improvements in or relating to keying of video signals - Google Patents

Abstract

A keying apparatus for providing colour separation overlay of foreground and background video signals derives a keying signal by non-additive mixing from the foreground signal (4, 7), Keying means (9, 10, 11, 18, 19, 20) are controlled by the keying signal Bv and select between the foreground signal Rf, Gf, Bf and a combination of the background signal Rb, Gb, Bb and the luminance signal Y of the foreground signal. By making use of the foreground luminance signal to modulate the luminance of the background signal when the background colour portions of the video signal is being received, transparency and, optionally, shadow effects are obtained in a relatively simple circuit arrangement requiring little user adjustment for setting up video effects. Also, by making use of the foreground luminance signal to modulate the luminance of the background signal, relatively little noise is introduced into the background signal. <IMAGE>

Description

SPECIFICATION Improvements in or relating to keying of video signals The present invention relates to keying of video signals and is particularly useful in the techniques which are variously known as colour separation overlay, chromakey, and travelling matte. For the sake of convenience, the term "colour separation overlay" will be used hereinafter to refer to all such techniques of keying between video signals on the basis of background colour of a foreground signal.

Colour separation overlay is a well known technique in the field of television. Essentially, this technique permits distinct background and foreground video signals to be processed into a composite signal in which foreground images from a foreground signal are superimposed over the background images. In order to achieve this, the foreground signals from a camera or the like are obtained by imaging the foreground objects against a background of a specific colour, usually a highly saturated blue.

Signal processing apparatus can identify the presence of the blue background from the video signal and can therefore produce a two level signal having a first level when the video signal corresponds to the foreground subject and a second level when the video signal corresponds to the background.

A separate background signal, which may be obtained from a video camera, video recorder, or any other suitable means, is then supplied to one input of an electronic video switch whose other input receives the foreground signal. The keying signal controls the video switch so that, when the foreground subject is being scanned, the switch passes the foreground video signal and, when the background colour of the foreground signal is being scanned, the video switch passes the background signal.

Such techniques are in daily use in television studios throughout the world and permit picture effects such as actors in a studio placed against outdoor backgrounds, news material projected behind or along side a newscaster, and the like.

In known apparatuses, transparency and shadow effects have been provided by the colour separation overlay technique. Essentially, windows, smoke, and shadows which modulate the blue background of the foreground signal have been used to create video effects by appropriate processing of the background signal. This has been achieved by complicated linear circuitry for mixing the background and foreground signals, which have generally been in the form of RGB signals. Apparatuses for performing such techniques have, however, been expensive and have required considerable adjustment of signal levels in order to obtain a sufficiently convincing effect. Such apparatuses have therefore been complicated to use, requiring a high degree of training and substantial amounts of time in order to set up a video effect.

Examples of known apparatus are disclosed in British Patents Nos 1286051 and 1506836.

According to the invention, there is provided a keying apparatus as defined in the appended claim 1.

Preferred embodiments of the invention are defined in the other appended claims.

The invention will be further described, by way of example, with reference to the accompanying drawings, in which: Figure 1 is a block circuit diagram of a keying apparatus constituting a first preferred embodiment of the invention; Figure 2 is a block circuit diagram of a keying apparatus constituting a second preferred embodiment of the invention; Figure 3 is a block circuit diagram of a keying apparatus constituting a third preferred embodiment of the invention; and Figure 4 is a block circuit diagram of a keying apparatus constituting a fourth preferred embodiment of the invention.

The keying apparatus shown in Figure 1 receives RGB foreground signals Rf, Gf, and Bf and RGB background signals Rb, Gb, and Bb.

The foreground signals are supplied to input clamps and buffers 1, 2, and 3 where the signals are buffered and clamped and, optionally, amplified. The outputs of the clamps and buffers 1, 2, and 3 are supplied to respective inputs of a non-additive mixer 4 which performs a non-additive mixing function so as to provide at its output a signal whose level indicates the presence of a particular colour in the foreground video signal. This is usually a highly saturated blue colour, but other colours may also be used. For instance, by exchanging the connections indicated at 5 and 6, a highly saturated green colour may be chosen for keying.It is also possible to use nonprimary colours by performing further mixing of the RGB signals supplied to the mixer The output of the mixer 4 is supplied to a clipper 7 which clips the waveform of the output signal of the mixer 4 and supplies at its output a keying signal Bv. The clipper 7 has a control input 8 for receiving a key clip level signal to allow the clipping level of the clipper 7 to be adjusted.

The apparatus of Figure 1 has three first keying mixers comprising video switches 9, 10, and 11. Each video switch has first and second inputs a and b, a control input which receives the keying signal B from the clipper 7, and an output to which the input signals are selectively switched by means of the keying signal. The first inputs a of the switches 9, 10, and 11 are connected to the outputs of the clamps and buffers 1, 2, and 3, respectively. The second inputs b of the switches 9, 10, and 11 are connected together and via matrixing or weighting resistors 12, 13, and 14 to the outputs of the clamps and buffers 1, 2, and 3, respectively. The second inputs b are further connected via a resistor 15 to a black level direct voltage source and via a resistor 16 and a switch 17 to the output of the non-additive mixer 4.

The apparatus further comprises second keying mixers comprising video switches 18, 19, and 20 which are of the same type as the video switches 9, 10, and 11. The first inputs a of the switches 18, 19, and 20 are connected via isolating resistors 21, 22, and 23 to the outputs of the switches 9, 10, and 11, respectively, which provide processed foreground signals R', G', and B', respectively, and which are provided with black clippers 24, 25, and 26, respectively. The second inputs b of the switches 18, 19, and 20 are connected via isolating resistors 27, 28, and 29 to the outputs of the switches 9, 10, and 11, respectively, and via isolating resistors 30, 31, and 32 to receive the background signals Rb, Gb, and Bb, respectively.The control inputs of the switches 18, 19, and 20 receive the keying signal B and their outputs provide the composite output signals Rc, Gc, and Bc.

In use, the apparatus receives the foreground signals Rf, Gf, Bf from a video camera or other video source of a foreground subject against a coloured background which is a highly saturated blue colour in the embodiment shown. The foreground subjects cast shadows on the blue background and smoke or transparency effects similarly alter the luminance level of the blue background. The clamped and buffered RGB signals are supplied to the non-additive mixer 4 which produces a signal of the form (B-(R non-additively mixed with G)). The level of this signal is such as to indicate when the video camera or source is scanning or reproducing the blue background, and the clipper 7 removes undesirable modulation of this signal so as to produce the keying signal Bv.Th clipping level can be adjusted at the input 8 and represents one of only two user-adjustable controls for setting up a video effect.

The matrixing or weighting resistors 12, 13, and 14 have values which are weighted so as to produce a luminance signal Y of the foreground signals. The black level of this luminance signal can be adjusted by means of the black level direct voltage supplied via the resistor 15, which provides the second useradjustable control. This operates in conjunction with the black clippers 24, 25, and 26 to set the actual black level of the processed foreground signals R', G', and B'.

The video switches 9, 10, and 11 are controlled by the keying signal B so that the foreground signals Rf, Gf, and Bf are keyed through the switches when the foreground subject video signals are present whereas the luminance signal Y is keyed through all three switches when the input video signal corresponds to the blue background.The video switches 18, 19, and 20 are controlled by the keying signal B so that, when the foreground subject video signals are present, the signals R', G', and B' from the switches 9, 10, and 11, respectively are keyed through the switches 18, 19, and 20 whereas, when video signals corresponding to the blue background are present, the switches 18, 19, and 20 key through a combination of each of the background signals Rb, Gb, and Bb and the luminance signal Y attenuated if necessary and with its black level clipped by the clippers 24, 25, and 26. In particular, each of the background signals Rb, Gb, and Bb is added to the processed luminance signal so that the level of each of the signals, and hence the level of intensity of the composite output signal Rc, Gc, and Bc, is modulated by the luminance signal corresponding to the background of the foreground signals.Thus, shadow and transparency effects such as windows, smoke, and the like modulate the background signals to produce a pleasing and, in many cases, more natural composite image.

If shadow effects are not required, then the switch 17 is set so as to subtract a proportion of the output signal of the non-additive mixer 4 from the luminance signal Y supplied to the switches 9, 10, and 11. By selecting the value of the resistor 16 appropriately, this removes the part of the luminance signal corresponding to shadows on the blue background, thus preventing shadow modulation of the background signals.

The keying apparatus shown in Figure 1 is relatively simple and cheap to manufacture while providing very effective transparency and, selectively, shadow effects. There are only two user-adjustable controls, namely for key clip level at the input 8 and black level via the resistor 15, so that an effect can be set up rapidly and with minimal technical training.

The use of the luminance signal Y to "modulate" the background signals is advantageous over the use merely of the blue foreground signal Bf, as has been used previously, since the noise of the signal which modulates the background signals is far less. In particular, the blue channel is the noisiest of the three RGB channels in video equipment, and use of the blue signal alone has the disadvantage that noise is added to the background signals and particularly affects the switching edges between foreground and background signals.

By using the luminance signal suitably matrixed from the RGB signals, the effective noise level is greatly reduced and the switching edge due to such noise is substantially reduced or masked.

The keying apparatus shown in Figure 2 has many parts which correspond to those of the apparatus of Figure 1 and which are referred to by the same reference numerals. Conse quently, description of those parts of the apparatus will not be repeated. The apparatus of Figure 2 differs from that of Figure 1 essentially in that it provides "parallel" switching whereas that of Figure 1 provides "series" switching.

The outputs of the switches 9, 10, and 11 are supplied via respective isolating and mixing resistors 40, 41, and 42 to the inputs of buffers 43, 44, and 45, respectively, whose outputs provide the composite output signals Rc, Gc, and Bc, respectively. The inputs of the buffers 43, 44, and 45 are also connected via respective isolating and mixing resistors 46, 47, and 48, respectively, to the outputs of video switches 49, 50, and 51, respectively, which are of the same type as the switches 9, 10, and 11 and which constitute second keying mixers of the apparatus. The video switches 49, 50, and 51 are controlled by the keying signal Bv and have first inputs a connected together and to an input 52 for receiving a black level voltage corresponding to the black level of the background signals, and second inputs b connected to clamps and buffers 53, 54, and 55, respectively.The inputs of the clamps and buffers 53, 54, and 55 receive the background signals Rb, Gb, and Bb, respectively.

In use, the keying signal Bv controls the video switches 9, 10, and 11 as described with reference to Figure 1. The keying signal controls the video switches 49, 50, and 51 so that, when foreground signals corresponding to the foreground subject are being keyed through the switches 9, 10, and 11, the switches 49, 50, and 51 switch the black level voltage to their outputs whereas, when the background colour signal of the foreground signals is being received, the switches 49, 50, and 51 pass the background signals Rb, Gb, and Bb to the resistors 46, 47, and 48, respectively. This background signal is thus mixed with the foreground luminance signal to provide transparency and, optionally, shadow effects in the composite output signals. The remainder of the operation of the apparatus of Figure 2 and its advantages are as described with respect to Figure 1.

The keying apparatus shown in Figure 3 differs from the apparatuses of Figures 1 and 2 in that it makes use of already-encoded foreground and background composite signals. It is common for video sources, such as cameras, video recorders, and the like to provide colour video signals which have already been encoded in one of the standard formats, in particular PAL, NTSC, and SECAN. Thus, such already-encoded foreground and background signals are supplied to the first and second inputs a and b of a video switch 60 of the same type as the video switches shown in Figures 1 and 2 and constituting a second keying mixer corresponding to the second keying mixers of Figure 2. In addition, foreground RGB signals, Rf, Gf, and Bf which are obtained from the foreground signal prior to its encoding are also received by the apparatus in order to provide the foreground luminance signal.The foreground signals Rf, Gf, and Bf are supplied via video delay lines 61, 62, and 63, respectively, to the inputs of clamps and buffers 64, 65, and 66, respectively, which correspond to the clamps and buffers 1, 2, and 3 of Figures 1 and 2. The delay lines 61, 62, and 63 are provided so as to compensate for the delay introduced into the encoded composite signals during the encoding process.

The outputs of the clamps and buffers 64, 65, and 66 are supplied to a non-additive mixer 67, whose function, together with links 68 and 69, a clipper 70, and a switch 71, perform the same functions as the corresponding parts in Figures 1 and 2. Also, the outputs of the clamps and buffers 64, 65, and 66 are supplied via matrixing resistors 72, 73, and 74 which, together with the resistor 75, perform the same functions as the corresponding resistor 12 to 15 in Figures 1 and 2.

The apparatus further comprises a first keying mixer comprising a video switch 76 of the same type as the switch 60. The switch 76 has a first input connected to a direct voltage source corresponding to black level and a second input which receives the foreground luminance signal, the black level voltage, and optionally the shadow-removal signal in the same way as the second inputs of the video switches 9, 10, and 11 of Figues 1 and 2.

The output of the switch 76 is provided with a black clipper 77 which corresponds to the black clippers 24, 25, and 26 of Figures 1 and 2. The outputs of the switches 76 and 60 are connected via isolating and mixing resistors 78 and 79, respectively, to the input of a buffer 80, whose output provides the combined encoded composite video signal.

The switches 76 and 60 are controlled by the keying signal Bv from the clipper 70. In addition, a composite blanking input signal is supplied to an input 81 and is mixed with the keying signal for controlling the switches 76 and 60.

Operation of the apparatus of Figure 3 is similar to that of Figure 2. Thus, when a video signal corresponding to the foreground subject is received, the switch 60 passes the foreground encoded composite signal whereas the switch 76 supplies the reference black level voltage on its first input a. The signals are mixed via the resistors 78 and 79 and supplied as output via the buffer 80. When the video signal corresponding to the blue background of the foreground signal is received, the switch 60 passes the background encoded composite signal and the switch 76 passes the foreground luminance signal. These signals are mixed via the resistors 78 and 79 and supplied to the output by the buffer 80.

The composite blanking signal supplied to the input 81 is used to select the black level voltage via the switch 76 and the blanked portion of the foreground encoded composite signal via the switch 60 so as to provide a blanked output video signal which also contains the foreground synchronising and burst signals as is normal in encoded signals.

The remainder of the operation and the advantages of the apparatus of Figure 3 are as described with reference to Figure 1.

The keying apparatus shown in Figure 4 has inputs 101, 102, and 103 for receiving RGB foreground signals Rf, Gf, and Bf which represent a picture having a foreground part which is desired to be present in the combined picture and a background part which is to be replaced in the combined picture. The background part has a substantially uniform saturated or nearly-saturated blue colour, although other colours may be chosen if desired. The foreground signals may, for instance, be supplied from a television camera directed at a scene in which the unwanted background part is provided by opaque screens painted in the appropriate colour. Alternatively, suitable foreground signals may be provided from other sources, such as video tape recorders, telecines, etc.

The foreground RGB signals are amplified and have their black levels clamped in respective buffer amplifiers and clamps 104, 105, and 106. The outputs of the amplifiers and clamps 104, 105, and 106 are connected to the inputs of controllers 107, 108, and 109, respectively, and to inputs of a non-additive mixer 110. The mixer 110 produces a colour vector output signal Bv which represents the presence of the undesired background colour part of the input foreground signals. In the embodiment shown, the non-additive mixer 110 processes the RGB signals so as to provide the signal Bv whenever the input signals represent a colour whose hue falls within a predetermined range of hues representing a predetermined range of blue colours. The signal Bv may, for instance, be equal to the level of the blue signal B when a colour within the predetermined range of hues is detected.Alternatively, the signal Bv may be made equal to the amount by which the blue signal B exceeds the red signal R or the green signal G for colours within the predetermined range of hues. Alternatively, any conventional or suitable form of processing to provide a colour vector signal may be adopted in place of the non-additive mixing of the mixer 110.

The controllers 107, 108, and 109, receive the colour vector signal Bv and the respective foreground colour signals Rf, Gf, and Bf and process these signals according to the following equations: R' = (R; K1) + (Bv; K2) G' = (B; K3) + (Bv; K4) B' = (G; Us) + (Bv; K6) wherein each sign ";" represents multiplication or addition, each of K1-K6 represents a positive or negative constant, and R', G', and B' represent the output signals of the controllers 107, 108, and 109, respectively. The constants K1, K3, and K5 are provided by presettable adjusting means which allow compensation for contamination of the blue colour (or other colour as appropriate) used on the unwanted background part of the scene represented by the foreground signals.The constants K2, K4, and K6 have values which are controlled so as to remove any residual contamination and to add or eliminate shadows and transparency effects. The range of values of the constants K2 and K4 may, for instance, be from 0-1 and the range of values of the constant K6 may be from 0-2. In particular, when the blue background signal is present, Bv represents the luminance of the background signal and the controllers allow this to be used to modulate the background signals for transparency and shadow effects.

The signal Bv is used to take the signals Rf, Gf, and Bf below their respective black levels and then to clip these signals, thus effectively keying out the unwanted parts of the foreground signal. Then proportions (set by K2, K4, and K6) of the positive and negative values of Bv relative to black levels can be added for shadow effects, transparencies, etc as required. When K2 = K4 = K6, then the added proportions produce a luminance or monochrome brightness effect which has no colour hue, but reflected light causing colour hue contamination can be corrected by adjusting K2, K4, and K6.

The signals R , G', B' are supplied to first inputs of control mixers 111, 112, and 113 respectively. The control mixers 111, 112, and 113 have second inputs which receive background colour signals Rb, Gb, and Bb, respectively. The control mixers have first outputs 114, 115, and 116 which supply the signals R' G', and B', respectively, without modification. The control mixers also have second outputs 117, 118, and 119 which supply signals in accordance with the following respective equations: = = (Ka; Rb) + (Kb; R') G" = (Kc; Gb) + (Kd; G') B" = (Ke; Bb) + (Kf; B') wherein each sign ";" represents addition or multiplication, each of Ka-Kf is a positive or negative constant, and R", G", B" are the out put signals provided at the outputs 117, 118, and 119, respectively.

For instance, Ka-Kf may all be made unity and ";" may represent multiplication, so that the second outputs supply the following sig nals: R" = R' + Rb G" = G' + Gb B" = B' + Bb If desired, variations in level can be cor rected by adjustable Ka to Kf factors.

The first and second outputs of the control mixers 111, 112, and 113 are connected to first and second inputs of keying mixers 120, 121, and 122, respectively. The keying mixers 120, 121, and 122 have third inputs connected to the outputs of key control circuits 123, 124, and 125 respectively. The inputs of the key control circuits 123, 124, and 125 receive the colour vector output signal Bv from the non-additive mixer 110. The key control circuits provide output signals to the respective keying mixers in accordance with the following equations: Bv; KR Bv ; K6 Bv ; K8 wherein each sign ";" represents multiplication or addition and each of KR, K6, and KR is a constant.The keying mixers are controlled by the key control circuits so as to determine which of the signals from the first and second outputs of the control mixers is supplied to the output of the apparatus as the combined output signals Rc, Gc, and Bc. In particular, each of the keying mixers comprises means for fading the signals supplied to the first and second inputs in accordance with the level of the control signal provided by the respective key control circuit to the third input so that the input signals from the control mixers may be cross-faded. Such cross-fading may be controlled so as to provide an abrupt change between the foreground and background signals, resulting in sharp edges to the foreground parts of the picture which therefore stand out against the background.Alternatively, the cross-fading may be performed more gently so that the foreground parts merge more naturally into the background parts of the composite picture.

Thus, the signal Bv may be used to key between the signals R', G', B' and R", G", B".

If KR = K6 = KR, the modulation effects produced are monochromatic in nature and cause no change of colour hue. However, reflected light causing colour contamination can be eliminated or reduced by adjustable KR, KG, and Ka. The exact DC level of the keying signal Bv can be controlled so as to eliminate unwanted "holes" appearing in the foreground part ie so-called "false keying".

The constants K2, K4, and K6 may be adjusted so that the colour vector signal Bv proportionally controls the black level of the foreground RGB signals in the background part.

When added to the background RGB signals in the control mixers, this effectively modulates the background signals so as to produce shadows in the composite picture and so as to provide transparency effects. Alternatively, these constants may be adjusted so that such effects are eliminated from the composite signal. The colour vector signal supplied to the controllers 107, 108, and 109 may be in digital form so as to provide digital control of the black level. Similarly, the signals from the key control circuits 123, 124, and 125 may also be in digital form to provide digital control of the keying mixers 120, 121, and 122. Six bit digital code has been found to be adequate for this purpose.

It has occasionally been found that black or grey objects in the foreground part of the foreground signals reflect sufficient of the background colour to cause false keying of the background signals. This may be compensated by adjusting the constants K2, K4, and K6 appropriately so that this effect is eliminated while permitting proper overlaying at the background part of the foreground signals.

Claims (11)

1. A keying apparatus for keying between first and second video signals, comprising means for deriving a colour-based keying signal from the first signal, and means responsive to the keying signal for keying between the first signal and a combination of the second signal and a signal representing the luminance of the first signal.

2. An apparatus as claimed in claim 1, in which the keying means comprises first keying mixer means responsive to the keying signal for keying between the first signal and the luminance signal of the first signal, and second keying mixer means responsive to the keying signal for keying between the output signal of the first keying mixer means and a signal comprising the sum of the output signal of the first keying mixer means and the second signal.

3. An apparatus as claimed in claim 1, in which the keying means comprises first keying mixer means responsive to the keying signal for keying between the first signal and the luminance signal of the first signal, second keying mixer means responsive to the keying signal for keying the second signal, and means for mixing the outputs of the first and second keying mixer means.

4. An apparatus as claimed in claim 1, in which the keying means comprises first keying mixer means responsive to the keying signal for keying the luminance signal of the first signal, second keying mixer means responsive to the keying signal for keying between the first and second signals, and means for mixing the outputs of the first and second keying mixer means.

5. An apparatus as claimed in anyone of claims 2 to 4 in which each of the first and second signals comprises RGB signals and each of the first and second keying mixer means comprises three electronic video switches.

6. An apparatus as claimed in claim 5, including means for providing the luminance signal of the first signal by matrixing the RGB signals of the first signal.

7. An apparatus as claimed in anyone of claims 2 to 4, in which each of the first and second signals comprises an encoded video signal and each of the first and second keying mixer means comprises an electronic video switch.

8. An apparatus as claimed in claim 7, including means for delaying pre-encoded RGB signals of the first signal and means for providing the luminance signal of the first signal by matrixing the delayed RGB signals.

9. An apparatus as claimed in anyone of the preceding claims, in which the means for deriving the colour-based keying signal comprises means for non-additively mixing RGB signals of the first signal and means for clipping the output signal of the non-additive mixing means.

10. An apparatus as claimed in claim 9, including means for selectively subtracting a portion of the output signal of the non-additive mixing means from the luminance signal of the first signal.

11. A keying apparatus substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.