CA1179051A - Encoded signal color image compositing - Google Patents
Encoded signal color image compositingInfo
- Publication number
- CA1179051A CA1179051A CA000429718A CA429718A CA1179051A CA 1179051 A CA1179051 A CA 1179051A CA 000429718 A CA000429718 A CA 000429718A CA 429718 A CA429718 A CA 429718A CA 1179051 A CA1179051 A CA 1179051A
- Authority
- CA
- Canada
- Prior art keywords
- foreground
- backing
- blue
- control signal
- colored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired
Links
Abstract
ABSTRACT
A linear color television compositing system of the "blue backing"
type, in which the foreground signal channel is always open. The foreground and background channels are not switched by the edges of the pictorial subject matter. The hue of the colored (typically blue) background is removed by vector subtraction and the luminance thereof by arithmetic subtraction. Flare illumina-tion of the foreground subject(s) from the backing is removed; improving color fidelity. The background pictorial scene that takes the place of the removed colored backing area is linearly turned up or down, allowing shadows of fore-ground subjects to appear in the background.
A linear color television compositing system of the "blue backing"
type, in which the foreground signal channel is always open. The foreground and background channels are not switched by the edges of the pictorial subject matter. The hue of the colored (typically blue) background is removed by vector subtraction and the luminance thereof by arithmetic subtraction. Flare illumina-tion of the foreground subject(s) from the backing is removed; improving color fidelity. The background pictorial scene that takes the place of the removed colored backing area is linearly turned up or down, allowing shadows of fore-ground subjects to appear in the background.
Description
This invention per-tains to separate foreground and background com-positing, employing a colored ~i.e., blue) backing for the foreground scene.
The so-called chroma-key process for compositing was developed decades ago by engineers of the National Broadcasting Co. in New York City. This was a foreground-background switching system.
Recently, Nakamura et al modified this system to a "soft edge chroma-key", in which edges between foreground and background are purposely blurred.
This tends to hide the effects of switching from foreground to background and vice versa. However, detail is lost in this boundary area.
Additionally, Nakamura added a subtraction circuit to eliminate the blue tint at the soft edge.
This system is disclosed in the SMPTE Journal, Vol 90, No. 2, February 1981, page 107.
All chroma-key ~i.e., switching devices), including the Nakamura soft edge device, suffer from two defects; i.e., loss of sharp edges on objects that have such edges, and loss of fine detail at the edges. The soft edge frequency response must be about half that of the capability of the foreground camera.
Vlahos United States Patent No. 3,595,987 introduces the concept of developing a control signal that is proportional to the brightness and visi-bility of the colored backing, also of controlling the level of the background scene as a linear f~mction of the amplitude of the control signal, and also of eliminating the blue backing by limiting that video signal amplitude to a maxi-mum that is represented by the amplitude of one of the other primary colors.
The above mentioned patent does not disclose removal of residual contaminating colors by subtraction, the reJection of foreground luminance be-cause of secondary limitation from the backing, of encoded video, nor defining ~75~
mixing as additive or non-additive.
; Vlahos United States Patent No. 4,007,487 introduces a ~B-G) + (G-R) control signal; the ~G-R) term permits the reproduction of blue eyes. A color ratio is established to distinguish the colored backing from the subject even with a backing of non-uniform brightness.
United States Patent 4,007,487 does not disclose removal of contaminat-ing colors by subtraction, nor of removing or retaining secondary luminance on the subject. Nor does it disclose encoded video, or specify additive or non-additive mixing.
Vlahos United States Patent No. 4,100,569 introduces full rejection of the colored backing by subtracting its red, gree ~ blue components in accordance with a control signal that varies as a function of the intensity of these colors as they appear in the backing area.
A control for the color magenta is provided. Control Ec is described, but without the -K~l-B) term.
An encoded color signal is not disclosed, nor is mixing of foreground and background video signal specified as additive or non-additive.
The present invention is a linear non-switching compositing system that does not switch between foreground and background scenes. This is opposed ~0 to the foreground/background chroma-key switching systems.
The foreground channel is always open. Except for the specific hue of the colored (blue) backing, all information seen by the foreground camera is retained in the final composite image up to the limiting resolution of the camera. The normal transparency of smoke, dust, glassware, etc. is retained.
In the operation the specific hue of the colored backing is removed by a vector subtraction process.
The colored backing also reflects this hue onto foreground subjects, , s~
causing color contamination of these subjects. Also, the field of the camera lens is essentially fi.lled with the color of the backing, which induces a lens flare of that color over the entire field.
The subtraction process of this invention removes the hue of the backing color from the backing area and also the hue conta.mination of the foreground subjects, either by reflection thereonto or as added by the lens flare.
The subt~action process has previously been applied to the individual red, green and blue (RGB) components of the backing color, as in the Vlahos United States Patent 4,100,569.
In this invention the subtraction is applied to the 3.58 megahertz (mHz) vector after the encoding process.
The luminance of the colored backing is removed by arith~
metic subtraction. With both the chroma and the luminescence of the colored backing subtracted from the foreground scene sig-nal, the backing area becomes black. Thus, there is no need to switch it off.
The background scene that takes the place of the new ~0 removed colored bac~ing area, is not switched as such; rather, it is turned up or down in intensity over a continuous range from zero (off) to unity (on). The turn-on of the background scene depends upon the brightness of the colored backing and the extent to which the colored backing is obscured by opaque and/or semi-transparent foreground subjects, or as reduced in intensity by shadows falling UpOIl the backing.
. .
~7~51 In this way, shadows cast upon the colored backing are retained and transferred onto the background scene.
Because only the specific hue of the backing is subtrac-ted other colors are not affected. If the backing is an ultra-marine blue, for example, all evidence of this blue flare on faces, wardrobe and the over-all discoloration from the lens flare is eliminated. However, importantly, the process of this invention does not alter the natural reproduction of blue eyes and other shapes of blue.
The present invention may be summarized as a method of compositing color image video signa]s involving a colored backing, which comprises the method steps of: (a) forming a first control signal as a function of the brightness and visibility of the colored backing and also as a function of the colored secondary illumination on foreground subjects being received from the color-ed backing, (b) removing the chroma of the colored backing in the backing area and removing discoloration chroma from the fore-ground subjects resulting from secondary illumination received from the colored backing by vector subtraction as determined by the first control signal, (c) forming a second control signal as a function of the brightness and visibility of the colored backing but not as a function of the backing illumination being reflected from foreground subjec-ts, (d) removing the luminance of the colored backing from the foreground scene video signal in the backing area by subtraction as determined by the first or second control signal, and (e) combining the foreground scene video signal as controlled by the first control signal or as selectively controlled by the first and second control signals with the background seene video signal, -the level of which has been eontrolled by the seeond eontrol signal, to form a eomposite image.
According to another aspect, the invention is an elec-` tronic image compositing system of the colored backing type, . eomprising: (a) a souree of foreground eomposite video signals containing the red, green and blue components of the foreground scene, (b) a source of background composite video signals, - 10 (e) means for generating a first eontrol signal utilizing the red, green and blue eomponents of the foreground video signal, said first eontrol signal being a function of the intensity of .` these said eomponents emanating directly from the colored backing - and also indirectly as a reflection resulting from backing illum-ination of foreground subjeets, (d) means for utilizing said first eontrol signal to remove the chroma of the colored baeking and to remove the chroma discoloration cast upon foreground sub-jeets by the eolored baeking, by subtraetion, (e) means for gen-erating a seeond eontrol signal utilizing the red, green and blue eomponents of the foreground video signal~ said second con-trol signal being a function of the intensity of these said com-ponents emanating directly from the backing, but being unaffected by these sai.d eomponents refleeted from foreground subjeets, (f) means for selectively subtracting the first and second control signals from the composite foreground video signal to remove the luminance component of the colored baeking therefrom, (g) means for utilizing the second eontrol signal to control the level of -~a-5~
the background video signal, and (h) means for combining the foreground video signal and the background video signal after said signals are subjected to control by said first and second control signals.
The invention will now be described in greater detail with reference to the accompanying drawings, in which:
Figure 1 is a schematic diagram of a specific simplified form of the invention; and Figure 2 is a schematic diagram of the preferred form of the invention.
In Figure 2, the composite (encoded) video signal, as well as the individual red, green and blue video outputs of tele-vision color camera 1 are all connected to the encoded signal color image compositing system of this invention.
The composite video signal via conductor 5 is connected to the non-inverting (+) input of a differential amplifier 6, which may be a Harris HA2625. The colored backing chroma is removed in this amplifier by an input to the inverting (-) ter-minal thereof from apparatus to be later described.
The output from amplifier 6, via conductor 8, enters the + x input of a multiplier 10. This may be a Motorola four-quadrant multiplier MC1595L. Operational amplifier 24 follows the multiplier to remove the inherent offset of the multiplier.
Output amplifier 25 sums the foreground and the back-ground signals delivered to its inverting (-) input; i.e., the output from amplifiers 24 and 27. Amplifier 25 may be a Harris HA5195 operational amplifier.
-4b-75~5~
The R G s video signal outpu-ts from camera 1 are connec-ted to both chroma control circuit entity 15 to produce a voltage Eb and to background scene control circuit entity 26 to produce a voltage Ec.
In entity 15, differential amplifier 16 provides a G - R
term (green minus red) to potentiometer 21, which may also be termed a color gate. Differ-5~
ential amplifier 17 provides a R - G term (red minus green) to potentiometer 20, another color gate.
The outputs of potentiometers or color gates 20 and 21 enter the two ` input terminals of linear OR gate 18. This may consist of four transistors, as supplied by a Motorola MPQ6002 PNP/NPN transistor integrated circuit (IC).
The output of OR gate 18 is connected to the inverting (-) input terminal of differential amplifier 19, as is the green video (G). The blue video (B) is connected to the non-inverting input terminal of differential amplifier 19.
The output of amplifier 19 is connected to zero clipper 22, which removes any negative signals. This output is an electronic evaluation of equation, Eb, to be presented later. Zero clip 22 may be an OR gate with one input connected to ground.
The logic to evaluate equation E of entity 26 includes linear OR gate 34, linear AND gate 35 and operational amplifier 39, along with potentiometer 40 and zero clip 41. OR gate 34 is identical to OR gate 18. The AND gate 35 is an inverted OR gate. Four transistors may be used to construct both the OR and AND gates.
The green (G) and red (R) video signals from camera 1 are independent-ly controlled in amplitude by potentiometers 36 and 37, respectively. These are both connected to linear OR gate 34 and linear AND gate 35. The output of gate 34 is connected to one extremity of potentiometer 38 and the output of gate 35 is connected to the other extremity. The wiper comlection of this potentiometer allows an exclusive output of OR gate 34 at its upper extremity, or the exclu-sive output of AND gate 35 at its lower extremity. At intermediate positions a selected proportion of the two gates outputs are provided.
The wiper of 38 is connected to the inverting input of differential amplifier 39, and is subtracted from the blue (B) video signal from camera 1, 5~
which is connected to the non-inverting input of amplifier 39.
Also connected to the non-inverting input of amplifier 39 is potentio-meter 40. The output of amplifier 39 is an electronic evaluation of the Ec equation, of which potentiometer 40 implements the last term.
The background pictorial subject matter is supplied by camera 2. This may alternatively be an equivalent source, such as a pre-recorded video tape, a film scanner, or the like. The signal therefrom is in composite (encoded) form. It enters the +x input to multiplier 27. Control signal E enters the +y input of the same multiplier. The thus controlled background signal passes throug}l an operational amplifier 45 to remove the inherent offset of the multi-plier, as was discussed in connection with elements 10 and 24.
The background output signal is conveyed to the inverting input of amplifier 25, where it is added to the foreground video signal.
Non-additive mixer 29 may be comprised of two multipliers, such as the Motorola MC1595 multipliers. The two x inputs of these multipliers are sup-plied by the outputs of entities 15 and 26, Eb and E , respectively. The y inputs are supplied by a signal from filter 30. The input to the filter is the composite video signal from background camera 2. The filter is typically of the resistor-capacitor type with a time constant of approximately 1/30 second.
The y inputs are reversed, so that the same rising signal amplitude from filter 30 causes Ec to increase at the mixer output while causing Eb to decrease by an equal amount, thus maintaining a constant output.
In the apparatus, unless otherwise specified, the multipliers, as 10, 13, 27, may be the Motorola type MC1595L. The differential amplifiers may be Harris HA2625. The OR gates and AND gates are conveniently constructed by utilizing four transistors in a common package, such as the Motorola type ~IPQ6002.
5~
A resistance value of 1,000 ohms is suitable for all potentiometers.
A resistance value of 1,000 ohms is also suitable for all input, summing, and feedback resistors associated with the operational amplifiers.
Considering now the functioning of the apparatus, the positive input 5 of differential amplifier 6 is provided with the usual functions of isolation, and dc restoration ~clamping), but these are not shown in Figure Z.
The video signal at 5 includes a range of frequencies representing the detail in the foreground image. Also present in this video signal is the 3.58 ~z color subcarrier (for NTSC). The phase angle ~vector) represents the hue and the amplitude represents the saturation of a given foroground color. At the points in the scene containing a bright blue of the blue backing, a specific vector angle and amplitude will be generated in the encoder to represent the specific blue paint or fabric used for the backing. This is the normal function of the NTSC encoding system.
A second vector is generated in this nvention that has exactly the same phase angle and amplitude as the vector representing the blue backing, the amplitude of which increases and decreases in exactly the same manner as the vector for the blue backing and thus exhibits all the same changes in illumina-tion level. The electrical representation of this second vector is fed into the - input 7 o differential amplifier 6. This completely cancels, by common mode rejection, the picture vector fed in at + input 5. Output 8 will therefore contain only luminance information of the blue backing. This is fed to multi-plier 10.
The second vector is generated as follows. The 3.58 Mllz color sub-carrier is normally routed to all color cameras, and is also rou-ted to input terminal 11 in Figure 2. It then passes to phase shifter 12, which is connected to the +x input of multiplier 13. Phase shifter 12 is capable of rotating the ~7~5~
phase angle of the subcarrier through 360. Phase shifter 12 is adjusted to match the phase angle from terminal 11 to the phase angle of the 3.58 MHz subcarrier that represents the colored backing as it appears in the video signal on conductor 5.
The amplitude of the 3.58 MHz subcarrier leaving multiplier 13 is the product of the x and y inputs thereto. The y input is therefore adjusted by control potentiometer 14 to produce a level at point 7 to equal the 3.58 vector amplitude representing the colored backing appearing on conductor 5.
Cancellation at output 8 is thus obtained. As with multipliers 10 and 27 an operational amplifier, referenced 46, follows multiplier 13 to remove the inherent offset of multiplier 13.
The voltage output from entity 15 is Eb, the amplitude of which is adjusted by control potentiometer 14. The logic equation defining the vector amplitude Eb is as follows:
Eb = K[(B-G) - Kl(G-R) - K2(R-G) ] (1) In the e~uation ~1), the ~ signs signify that all terms must have positive, not negative, values. This is ensured by zero clip 22 in Figure 2. The various values of the individual terms and of the coefficients K have values between 0 and 1.
The value of coefficient K is determined by the adjustment of potenticmeter 14, of Kl by potentiometer 21, and of K2 by potentiometer 20.
Assume that the idealized video values for the blue backing are Blue = 1, Green = 0 and Red = 0. The signal level to control potentiometer 14 is therefore 1.0 for that area of the foreground scene that consists of the blue backing. As a :~7~C~S~
practical matter, the blue video amplitude is not quite 1.0 and the green (or red) video is not quite 0, and the difference of (B-G) may vary.
The above condition, where ~he generated vector at 7 matches the amplitude and phase angle of the vector at 7, holds throughout the area covered by the blue backing regardless of the changes and variations in the light level falling on the blue backing. A deep ultra-marine blue painted backing, for example, will have a blue content (the reflectivity) of approximately 60%, a green content of approximately 20%, and a red content of approximately 20%~
Changes in the light level will change all three componen~ vectors by an equal percentage. The vector sum will change in amplitude but will not change in phase angle. The vector angle does not change because the hue (color) of the backing is independent of the light level falling upon it. That is, the shadow of a performer will result in a shorter vector, but the phase angle will remain the same.
If constants Kl and K2 in equation (1) are set to zero, the equation becomes Eb = K (B-G)+. When foreground subject matter other than the blue backing is considered, all colors will have a green content that is equal to or greater than the blue content, with the exception of the colors blue and magenta (purple). This relationship is inherent in the physics of color.
Table I lists several colors in column 1, some of which include color contamination by secondary illumination from the blue backing~ The B (blue), G (green), and R (red) color components 7~5~l :
of the listed colors are shown in columns 2, 3 and 4. These component values were obtained by measurement, but are also to be found in handbooks of color science.
The value of blue that should exist if the backing had no color and did not cause blue flare on the subjects is shown in column 5. Column 6 lists the amount of blue to be removed so as to eliminate the blue of the backing as well as its influence on foreground subjects.
In most chroma-key devices the formula B _ G2R is used as the control signal for switching. When used as the criterion for blue removal, the result in column 7 is obtained. Note that the blue removal is incorrect for blue eyes, magenta, cyan and flare on flesh tones. (Numeral in parentheses). For blue eyes no blue removal is desired (column 6) but chroma-key logic (column 7) shows a blue reduction of 3/8, or 37~. This results in blue eyes and bluish objects being reproduced as a pale green.
Note also that the purplish tint to flesh tones is not removed.
Columns 8 through 12 illustrate the blue control logic of the present invention, as based upon equation (1). Column 8 is the basic logic (B-G). Column 9, (G-R) is identified with color gate 21, and is required to correctly reproduce bluish objects.
Column lO, (R~G) is identified with color gate 20, and is required to correctly reproduce magenta.
Table I is reproduced on the following page.
~7~
+
t ~
_ p:; I ~ O ~1 0 N O O O O O O O
~`I I ~ _ ~1~-- I
+
~;
I _ tl o ~1 0 t`J O ~ O O O O r-l I
m +
O
~1 1 o o o o o o ~ o o o . .
+
P~
O~ I O O O O O~ O O~D ~9 0 0 ~ . .
+~
0~ 1 ~ O ~,O ~ ~ ~ O O O O ~
m . . . ..
~1~
,~ ~ O ~ O ~ ~ ~ O O ~ O O
m _ _ _ U~
~n r o ~ o ~ o o o o o o ~1 m m ~,~ .. ......
O l'C h C~ ~ O
~rP; ~ oo ~ o o ~r co oo ~ ~ I` I`
O o ~ ~ ~ ~ a ~ m w 0~ ~ o ~
.
C~
H
Z E~ m ~4 H
~ ;~; O E~ P: ~ ~ Z O Z
m ~ ~ H ~ ~ ) V ~
o m ~ æ ~ ~ z ~ ~ ~ z E~ c~ o m ~ m o m m o m ~ ~ m o ~7~6~S31 In the general run of video program productions, the color magenta (purple) is rarely used in wardrobe or in commercial products. For this reason K2, color gate 20, is set to zero, thus eliminating the term (R-G)+. Equation (1) is thus reduced to the first two terms; (B-G)~ - (G-R)~. Representative values for various colors are shown in column 11. Assuming that magenta is omitted from the foreground scene, column 11 shows that all colors are correctly reproduced.
In those rare cases where magenta is required in the foreground scene, color gate 20, K2(R-G)+, is opened just enough to permit the reproduction of magenta. When color gate 20 is fully open, the results of column 12 are obtained.
Note that the values in column 11 (or 12) ar~ applied to multiplier 13 of Figure 2. The numerical value in column 11 is the amount of the blue vector supplied to the negative terminal of differential amplifier 6. Thus, this is the amount of blue removed from the video signal. It is principally in the blue backing area, and in white and flesh colors flooded with blue flare light, that blue removal is required. All other colors are true colors and are unaffected. By adjusting control potentiometer 14, connected to multiplier 13, equation (1), Eb, generates the correct level of vector to be substracted. The blue backing is thus reproduced as a gray backing.
By definition, a white subject reflects R G B equally;
i.e., 0.8, 0.8, 0.8. Line 3 of Table I, however, shows the R G B values for white that is flooded with blue flare light to be 0.9, 0.8, 0.8. Blue is shown to be in excess of green , .
.
or red by 0.1. Such excess blue would have caused a blue tint to the white subject. However, column 11 shows a subtraction value of 0.1 for removing ~he blue flare light.
The blue backing, reflecting blue light onto a person's face, the bottom line in Table I, will increase the blue content ` of the face. This gives it a magenta look. Column 11 shows a ; subtraction value of 0.1. This exactly removes the excess blue.
The logic of equa-tion (1), Eb, not only reduces the color (chroma vector3 of the blue backing to zero; it also removes the blue flare light that would otherwise affect the colors of subjects in the foreground scene. This is the significance of the numbers in columns 11 and 12 of Table I.
Kl of equation (1), Eb, must be unity if blue eyes and other pastel blue colors are to be reproduced as blue. K2 must be unity to reproduce magenta colors. When K2 is at zero, magenta colors are reproduced as reddish colors. However, with K2 at zero, all blue flare in the camera lens and on wardrobe and flesh tones is entirely eliminated.
Equation (1), Eb, is implemented by entity 15 of Figure
The so-called chroma-key process for compositing was developed decades ago by engineers of the National Broadcasting Co. in New York City. This was a foreground-background switching system.
Recently, Nakamura et al modified this system to a "soft edge chroma-key", in which edges between foreground and background are purposely blurred.
This tends to hide the effects of switching from foreground to background and vice versa. However, detail is lost in this boundary area.
Additionally, Nakamura added a subtraction circuit to eliminate the blue tint at the soft edge.
This system is disclosed in the SMPTE Journal, Vol 90, No. 2, February 1981, page 107.
All chroma-key ~i.e., switching devices), including the Nakamura soft edge device, suffer from two defects; i.e., loss of sharp edges on objects that have such edges, and loss of fine detail at the edges. The soft edge frequency response must be about half that of the capability of the foreground camera.
Vlahos United States Patent No. 3,595,987 introduces the concept of developing a control signal that is proportional to the brightness and visi-bility of the colored backing, also of controlling the level of the background scene as a linear f~mction of the amplitude of the control signal, and also of eliminating the blue backing by limiting that video signal amplitude to a maxi-mum that is represented by the amplitude of one of the other primary colors.
The above mentioned patent does not disclose removal of residual contaminating colors by subtraction, the reJection of foreground luminance be-cause of secondary limitation from the backing, of encoded video, nor defining ~75~
mixing as additive or non-additive.
; Vlahos United States Patent No. 4,007,487 introduces a ~B-G) + (G-R) control signal; the ~G-R) term permits the reproduction of blue eyes. A color ratio is established to distinguish the colored backing from the subject even with a backing of non-uniform brightness.
United States Patent 4,007,487 does not disclose removal of contaminat-ing colors by subtraction, nor of removing or retaining secondary luminance on the subject. Nor does it disclose encoded video, or specify additive or non-additive mixing.
Vlahos United States Patent No. 4,100,569 introduces full rejection of the colored backing by subtracting its red, gree ~ blue components in accordance with a control signal that varies as a function of the intensity of these colors as they appear in the backing area.
A control for the color magenta is provided. Control Ec is described, but without the -K~l-B) term.
An encoded color signal is not disclosed, nor is mixing of foreground and background video signal specified as additive or non-additive.
The present invention is a linear non-switching compositing system that does not switch between foreground and background scenes. This is opposed ~0 to the foreground/background chroma-key switching systems.
The foreground channel is always open. Except for the specific hue of the colored (blue) backing, all information seen by the foreground camera is retained in the final composite image up to the limiting resolution of the camera. The normal transparency of smoke, dust, glassware, etc. is retained.
In the operation the specific hue of the colored backing is removed by a vector subtraction process.
The colored backing also reflects this hue onto foreground subjects, , s~
causing color contamination of these subjects. Also, the field of the camera lens is essentially fi.lled with the color of the backing, which induces a lens flare of that color over the entire field.
The subtraction process of this invention removes the hue of the backing color from the backing area and also the hue conta.mination of the foreground subjects, either by reflection thereonto or as added by the lens flare.
The subt~action process has previously been applied to the individual red, green and blue (RGB) components of the backing color, as in the Vlahos United States Patent 4,100,569.
In this invention the subtraction is applied to the 3.58 megahertz (mHz) vector after the encoding process.
The luminance of the colored backing is removed by arith~
metic subtraction. With both the chroma and the luminescence of the colored backing subtracted from the foreground scene sig-nal, the backing area becomes black. Thus, there is no need to switch it off.
The background scene that takes the place of the new ~0 removed colored bac~ing area, is not switched as such; rather, it is turned up or down in intensity over a continuous range from zero (off) to unity (on). The turn-on of the background scene depends upon the brightness of the colored backing and the extent to which the colored backing is obscured by opaque and/or semi-transparent foreground subjects, or as reduced in intensity by shadows falling UpOIl the backing.
. .
~7~51 In this way, shadows cast upon the colored backing are retained and transferred onto the background scene.
Because only the specific hue of the backing is subtrac-ted other colors are not affected. If the backing is an ultra-marine blue, for example, all evidence of this blue flare on faces, wardrobe and the over-all discoloration from the lens flare is eliminated. However, importantly, the process of this invention does not alter the natural reproduction of blue eyes and other shapes of blue.
The present invention may be summarized as a method of compositing color image video signa]s involving a colored backing, which comprises the method steps of: (a) forming a first control signal as a function of the brightness and visibility of the colored backing and also as a function of the colored secondary illumination on foreground subjects being received from the color-ed backing, (b) removing the chroma of the colored backing in the backing area and removing discoloration chroma from the fore-ground subjects resulting from secondary illumination received from the colored backing by vector subtraction as determined by the first control signal, (c) forming a second control signal as a function of the brightness and visibility of the colored backing but not as a function of the backing illumination being reflected from foreground subjec-ts, (d) removing the luminance of the colored backing from the foreground scene video signal in the backing area by subtraction as determined by the first or second control signal, and (e) combining the foreground scene video signal as controlled by the first control signal or as selectively controlled by the first and second control signals with the background seene video signal, -the level of which has been eontrolled by the seeond eontrol signal, to form a eomposite image.
According to another aspect, the invention is an elec-` tronic image compositing system of the colored backing type, . eomprising: (a) a souree of foreground eomposite video signals containing the red, green and blue components of the foreground scene, (b) a source of background composite video signals, - 10 (e) means for generating a first eontrol signal utilizing the red, green and blue eomponents of the foreground video signal, said first eontrol signal being a function of the intensity of .` these said eomponents emanating directly from the colored backing - and also indirectly as a reflection resulting from backing illum-ination of foreground subjeets, (d) means for utilizing said first eontrol signal to remove the chroma of the colored baeking and to remove the chroma discoloration cast upon foreground sub-jeets by the eolored baeking, by subtraetion, (e) means for gen-erating a seeond eontrol signal utilizing the red, green and blue eomponents of the foreground video signal~ said second con-trol signal being a function of the intensity of these said com-ponents emanating directly from the backing, but being unaffected by these sai.d eomponents refleeted from foreground subjeets, (f) means for selectively subtracting the first and second control signals from the composite foreground video signal to remove the luminance component of the colored baeking therefrom, (g) means for utilizing the second eontrol signal to control the level of -~a-5~
the background video signal, and (h) means for combining the foreground video signal and the background video signal after said signals are subjected to control by said first and second control signals.
The invention will now be described in greater detail with reference to the accompanying drawings, in which:
Figure 1 is a schematic diagram of a specific simplified form of the invention; and Figure 2 is a schematic diagram of the preferred form of the invention.
In Figure 2, the composite (encoded) video signal, as well as the individual red, green and blue video outputs of tele-vision color camera 1 are all connected to the encoded signal color image compositing system of this invention.
The composite video signal via conductor 5 is connected to the non-inverting (+) input of a differential amplifier 6, which may be a Harris HA2625. The colored backing chroma is removed in this amplifier by an input to the inverting (-) ter-minal thereof from apparatus to be later described.
The output from amplifier 6, via conductor 8, enters the + x input of a multiplier 10. This may be a Motorola four-quadrant multiplier MC1595L. Operational amplifier 24 follows the multiplier to remove the inherent offset of the multiplier.
Output amplifier 25 sums the foreground and the back-ground signals delivered to its inverting (-) input; i.e., the output from amplifiers 24 and 27. Amplifier 25 may be a Harris HA5195 operational amplifier.
-4b-75~5~
The R G s video signal outpu-ts from camera 1 are connec-ted to both chroma control circuit entity 15 to produce a voltage Eb and to background scene control circuit entity 26 to produce a voltage Ec.
In entity 15, differential amplifier 16 provides a G - R
term (green minus red) to potentiometer 21, which may also be termed a color gate. Differ-5~
ential amplifier 17 provides a R - G term (red minus green) to potentiometer 20, another color gate.
The outputs of potentiometers or color gates 20 and 21 enter the two ` input terminals of linear OR gate 18. This may consist of four transistors, as supplied by a Motorola MPQ6002 PNP/NPN transistor integrated circuit (IC).
The output of OR gate 18 is connected to the inverting (-) input terminal of differential amplifier 19, as is the green video (G). The blue video (B) is connected to the non-inverting input terminal of differential amplifier 19.
The output of amplifier 19 is connected to zero clipper 22, which removes any negative signals. This output is an electronic evaluation of equation, Eb, to be presented later. Zero clip 22 may be an OR gate with one input connected to ground.
The logic to evaluate equation E of entity 26 includes linear OR gate 34, linear AND gate 35 and operational amplifier 39, along with potentiometer 40 and zero clip 41. OR gate 34 is identical to OR gate 18. The AND gate 35 is an inverted OR gate. Four transistors may be used to construct both the OR and AND gates.
The green (G) and red (R) video signals from camera 1 are independent-ly controlled in amplitude by potentiometers 36 and 37, respectively. These are both connected to linear OR gate 34 and linear AND gate 35. The output of gate 34 is connected to one extremity of potentiometer 38 and the output of gate 35 is connected to the other extremity. The wiper comlection of this potentiometer allows an exclusive output of OR gate 34 at its upper extremity, or the exclu-sive output of AND gate 35 at its lower extremity. At intermediate positions a selected proportion of the two gates outputs are provided.
The wiper of 38 is connected to the inverting input of differential amplifier 39, and is subtracted from the blue (B) video signal from camera 1, 5~
which is connected to the non-inverting input of amplifier 39.
Also connected to the non-inverting input of amplifier 39 is potentio-meter 40. The output of amplifier 39 is an electronic evaluation of the Ec equation, of which potentiometer 40 implements the last term.
The background pictorial subject matter is supplied by camera 2. This may alternatively be an equivalent source, such as a pre-recorded video tape, a film scanner, or the like. The signal therefrom is in composite (encoded) form. It enters the +x input to multiplier 27. Control signal E enters the +y input of the same multiplier. The thus controlled background signal passes throug}l an operational amplifier 45 to remove the inherent offset of the multi-plier, as was discussed in connection with elements 10 and 24.
The background output signal is conveyed to the inverting input of amplifier 25, where it is added to the foreground video signal.
Non-additive mixer 29 may be comprised of two multipliers, such as the Motorola MC1595 multipliers. The two x inputs of these multipliers are sup-plied by the outputs of entities 15 and 26, Eb and E , respectively. The y inputs are supplied by a signal from filter 30. The input to the filter is the composite video signal from background camera 2. The filter is typically of the resistor-capacitor type with a time constant of approximately 1/30 second.
The y inputs are reversed, so that the same rising signal amplitude from filter 30 causes Ec to increase at the mixer output while causing Eb to decrease by an equal amount, thus maintaining a constant output.
In the apparatus, unless otherwise specified, the multipliers, as 10, 13, 27, may be the Motorola type MC1595L. The differential amplifiers may be Harris HA2625. The OR gates and AND gates are conveniently constructed by utilizing four transistors in a common package, such as the Motorola type ~IPQ6002.
5~
A resistance value of 1,000 ohms is suitable for all potentiometers.
A resistance value of 1,000 ohms is also suitable for all input, summing, and feedback resistors associated with the operational amplifiers.
Considering now the functioning of the apparatus, the positive input 5 of differential amplifier 6 is provided with the usual functions of isolation, and dc restoration ~clamping), but these are not shown in Figure Z.
The video signal at 5 includes a range of frequencies representing the detail in the foreground image. Also present in this video signal is the 3.58 ~z color subcarrier (for NTSC). The phase angle ~vector) represents the hue and the amplitude represents the saturation of a given foroground color. At the points in the scene containing a bright blue of the blue backing, a specific vector angle and amplitude will be generated in the encoder to represent the specific blue paint or fabric used for the backing. This is the normal function of the NTSC encoding system.
A second vector is generated in this nvention that has exactly the same phase angle and amplitude as the vector representing the blue backing, the amplitude of which increases and decreases in exactly the same manner as the vector for the blue backing and thus exhibits all the same changes in illumina-tion level. The electrical representation of this second vector is fed into the - input 7 o differential amplifier 6. This completely cancels, by common mode rejection, the picture vector fed in at + input 5. Output 8 will therefore contain only luminance information of the blue backing. This is fed to multi-plier 10.
The second vector is generated as follows. The 3.58 Mllz color sub-carrier is normally routed to all color cameras, and is also rou-ted to input terminal 11 in Figure 2. It then passes to phase shifter 12, which is connected to the +x input of multiplier 13. Phase shifter 12 is capable of rotating the ~7~5~
phase angle of the subcarrier through 360. Phase shifter 12 is adjusted to match the phase angle from terminal 11 to the phase angle of the 3.58 MHz subcarrier that represents the colored backing as it appears in the video signal on conductor 5.
The amplitude of the 3.58 MHz subcarrier leaving multiplier 13 is the product of the x and y inputs thereto. The y input is therefore adjusted by control potentiometer 14 to produce a level at point 7 to equal the 3.58 vector amplitude representing the colored backing appearing on conductor 5.
Cancellation at output 8 is thus obtained. As with multipliers 10 and 27 an operational amplifier, referenced 46, follows multiplier 13 to remove the inherent offset of multiplier 13.
The voltage output from entity 15 is Eb, the amplitude of which is adjusted by control potentiometer 14. The logic equation defining the vector amplitude Eb is as follows:
Eb = K[(B-G) - Kl(G-R) - K2(R-G) ] (1) In the e~uation ~1), the ~ signs signify that all terms must have positive, not negative, values. This is ensured by zero clip 22 in Figure 2. The various values of the individual terms and of the coefficients K have values between 0 and 1.
The value of coefficient K is determined by the adjustment of potenticmeter 14, of Kl by potentiometer 21, and of K2 by potentiometer 20.
Assume that the idealized video values for the blue backing are Blue = 1, Green = 0 and Red = 0. The signal level to control potentiometer 14 is therefore 1.0 for that area of the foreground scene that consists of the blue backing. As a :~7~C~S~
practical matter, the blue video amplitude is not quite 1.0 and the green (or red) video is not quite 0, and the difference of (B-G) may vary.
The above condition, where ~he generated vector at 7 matches the amplitude and phase angle of the vector at 7, holds throughout the area covered by the blue backing regardless of the changes and variations in the light level falling on the blue backing. A deep ultra-marine blue painted backing, for example, will have a blue content (the reflectivity) of approximately 60%, a green content of approximately 20%, and a red content of approximately 20%~
Changes in the light level will change all three componen~ vectors by an equal percentage. The vector sum will change in amplitude but will not change in phase angle. The vector angle does not change because the hue (color) of the backing is independent of the light level falling upon it. That is, the shadow of a performer will result in a shorter vector, but the phase angle will remain the same.
If constants Kl and K2 in equation (1) are set to zero, the equation becomes Eb = K (B-G)+. When foreground subject matter other than the blue backing is considered, all colors will have a green content that is equal to or greater than the blue content, with the exception of the colors blue and magenta (purple). This relationship is inherent in the physics of color.
Table I lists several colors in column 1, some of which include color contamination by secondary illumination from the blue backing~ The B (blue), G (green), and R (red) color components 7~5~l :
of the listed colors are shown in columns 2, 3 and 4. These component values were obtained by measurement, but are also to be found in handbooks of color science.
The value of blue that should exist if the backing had no color and did not cause blue flare on the subjects is shown in column 5. Column 6 lists the amount of blue to be removed so as to eliminate the blue of the backing as well as its influence on foreground subjects.
In most chroma-key devices the formula B _ G2R is used as the control signal for switching. When used as the criterion for blue removal, the result in column 7 is obtained. Note that the blue removal is incorrect for blue eyes, magenta, cyan and flare on flesh tones. (Numeral in parentheses). For blue eyes no blue removal is desired (column 6) but chroma-key logic (column 7) shows a blue reduction of 3/8, or 37~. This results in blue eyes and bluish objects being reproduced as a pale green.
Note also that the purplish tint to flesh tones is not removed.
Columns 8 through 12 illustrate the blue control logic of the present invention, as based upon equation (1). Column 8 is the basic logic (B-G). Column 9, (G-R) is identified with color gate 21, and is required to correctly reproduce bluish objects.
Column lO, (R~G) is identified with color gate 20, and is required to correctly reproduce magenta.
Table I is reproduced on the following page.
~7~
+
t ~
_ p:; I ~ O ~1 0 N O O O O O O O
~`I I ~ _ ~1~-- I
+
~;
I _ tl o ~1 0 t`J O ~ O O O O r-l I
m +
O
~1 1 o o o o o o ~ o o o . .
+
P~
O~ I O O O O O~ O O~D ~9 0 0 ~ . .
+~
0~ 1 ~ O ~,O ~ ~ ~ O O O O ~
m . . . ..
~1~
,~ ~ O ~ O ~ ~ ~ O O ~ O O
m _ _ _ U~
~n r o ~ o ~ o o o o o o ~1 m m ~,~ .. ......
O l'C h C~ ~ O
~rP; ~ oo ~ o o ~r co oo ~ ~ I` I`
O o ~ ~ ~ ~ a ~ m w 0~ ~ o ~
.
C~
H
Z E~ m ~4 H
~ ;~; O E~ P: ~ ~ Z O Z
m ~ ~ H ~ ~ ) V ~
o m ~ æ ~ ~ z ~ ~ ~ z E~ c~ o m ~ m o m m o m ~ ~ m o ~7~6~S31 In the general run of video program productions, the color magenta (purple) is rarely used in wardrobe or in commercial products. For this reason K2, color gate 20, is set to zero, thus eliminating the term (R-G)+. Equation (1) is thus reduced to the first two terms; (B-G)~ - (G-R)~. Representative values for various colors are shown in column 11. Assuming that magenta is omitted from the foreground scene, column 11 shows that all colors are correctly reproduced.
In those rare cases where magenta is required in the foreground scene, color gate 20, K2(R-G)+, is opened just enough to permit the reproduction of magenta. When color gate 20 is fully open, the results of column 12 are obtained.
Note that the values in column 11 (or 12) ar~ applied to multiplier 13 of Figure 2. The numerical value in column 11 is the amount of the blue vector supplied to the negative terminal of differential amplifier 6. Thus, this is the amount of blue removed from the video signal. It is principally in the blue backing area, and in white and flesh colors flooded with blue flare light, that blue removal is required. All other colors are true colors and are unaffected. By adjusting control potentiometer 14, connected to multiplier 13, equation (1), Eb, generates the correct level of vector to be substracted. The blue backing is thus reproduced as a gray backing.
By definition, a white subject reflects R G B equally;
i.e., 0.8, 0.8, 0.8. Line 3 of Table I, however, shows the R G B values for white that is flooded with blue flare light to be 0.9, 0.8, 0.8. Blue is shown to be in excess of green , .
.
or red by 0.1. Such excess blue would have caused a blue tint to the white subject. However, column 11 shows a subtraction value of 0.1 for removing ~he blue flare light.
The blue backing, reflecting blue light onto a person's face, the bottom line in Table I, will increase the blue content ` of the face. This gives it a magenta look. Column 11 shows a ; subtraction value of 0.1. This exactly removes the excess blue.
The logic of equa-tion (1), Eb, not only reduces the color (chroma vector3 of the blue backing to zero; it also removes the blue flare light that would otherwise affect the colors of subjects in the foreground scene. This is the significance of the numbers in columns 11 and 12 of Table I.
Kl of equation (1), Eb, must be unity if blue eyes and other pastel blue colors are to be reproduced as blue. K2 must be unity to reproduce magenta colors. When K2 is at zero, magenta colors are reproduced as reddish colors. However, with K2 at zero, all blue flare in the camera lens and on wardrobe and flesh tones is entirely eliminated.
Equation (1), Eb, is implemented by entity 15 of Figure
2. Differential amplifier 16 provides the -term (G-R), through color gate 21. Differential amplifier 17 provides -the (R-G) term, through color gate 20. Linear OR gate 18 passes whichever term is positive.
The (B-G) term of equation (1) is formed at the + and -input terminals of differential amplifier 19, which also subtracts the (G-R) or the ~R-G) term. Potentiometers 14, 20 and 21 provide the K, Kl, and K2 of the equation.
.
5~
Having removed, by subtraction, the 3.58 vector that represents the hue of the blue backing, the signal leaving amplifier 6 via conductor 8 and entering multiplier 10 contains only luminance (black & white) information, which represents the "brightness" of the blue backing. It is necessary to remove this luminance so that it does not cause a gray veil over the entire background scene.
Chroma-key apparatus removes the luminance and the chroma of the backing by switching-off the foreground scene in the blue backing area.
In the present invention the luminance of the blue backing area is removed by arithmetic subtraction.
The luminance component of the blue backing and of the foreground objects resulting from secondary illumination from the backing are removed from the video signal by the connection of the Eb output into the negative terminal of the x input of multiplier 10. Luminance bias control potentiometer 23 allows the proper amplitude to be selected. Thus, the output of amplifier 24 has zero chroma and zero luminance in the blue backing area, while showing normal chroma and luminance for other foreground subject areas. Since the backing is thus in no way reproduced, the foreground video channel may be retained fully "on", and so reproduces all detail seen by the camera 1.
The foreground video signal, with chroma and luminance removed in the region of the blue backing is routed to combining amplifier 25, where the background scene is added to those areas from which the blue has been removed. Amplifier 25 sums the 05~
two signals by simple addition, as opposed to non-additive mixing required by soft chroma-key and other switching systems.
Background control voltage Ec is generated by the circuit of entity 26 in Figure 2. This circuit provides the following relationships:
c 1 2( r OR KgG~ + (l-K2)(KrR AND K G)-K (l-B) ]~ (2) OR indicates the larger of G or R, and AMD indicates the smaller of G or R. The + symbol indicates that negative values are clamped to zero.
Control voltage E will be large in the blue backing area, since blue is high while both red ana green are low. For any opaque object Ec falls to zero. At intermediate values of blue backing intensity E will have intermediate values. Ec is directly proportional to the luminance and visibility of the blue backing, and is zero for opa~ue objects whether or not illumination is received from the blue backing.
Ec is thus used to control the level of the background scene. This is accomplished by feeding the Ec signal to the y input of multiplier 27 of Figure 2. This control signal varies the level of the background scene from zero to an upper value determined by the adjustment of control potentiometer 28. This is normally adjusted for unity gain for the background image in those areas of the blue backing that receive full illumination.
From the general equation (2~, Ec, one can obtain the simpler e~uation Ec= B-G by setting Kl to unity, K2 to unity, K3 and Kr to zero, and Kg to unity. Similarly, by adjusting the values of the constants to other values, one can obtain 5~
Ec= B-R, and so on.
The num~er of terms employed for a suitable evaluation f Ec is a function of the color purity of the colored backing, of the range of colors present in the foreground subjects, and by the presence or absence of dark glossy objects therein. The evaluation of E is accomplished by manipulating the poten-tiometers representing the K's in the Ec e~uation (2).
Elements 29, 30, 31 and 32 of Figure 2 provide means for enhancing realism with various foreground - background subject matters.
When a subject is placed in front of a well-illuminated color backlng, it will receive substantial luminance from that backing, particularly at the sides and edges of the subject. If the backing is dark or black, no such side illumination occurs.
If the foreground subject is composited into a background day scene, that scene, if real, provides substantial back and side illumination to the subject. ~owever, if the foreground subject is composited into a background night scene, that scene pro~ides very little illumination to the subject.
The compositing system of this invention provides the option of day or night background simulation, or any degree between the two.
For manual adjustment of what this simulation shall be switch 31 is moved to the left contact. This connects to the wiper of potentiometer 32. In Figure 2 the top end of that potentiometer is connected to the Eb output from entity 15. The bottom end of that potentiometer is connected to the Ec output .~7~
from entity 26. Thus, any proportion of Eb vs Ec can be obtained by moving -the wiper. When the wiper is at the bottom of the potentiometer, E control signal is in control and background luminance is retained (day effect). When the wiper is at the top, Eb control signal is in control and background luminance is rejected (night effect). An intermediate position of the wiper provides an intermediate effect.
It is possible to obtain automatic selectlon (or mixing) of the Eb and Ec control by moving switch 31 to the right contact.
Both Eb and Ec control signals are inputted to non-additive mixer 29.
The background composite video signal from camera 2 is integrated by filter 30 to obtain a signal proportional to the average brightness of the background scene and is also inputted to mixer 29.
Depending upon the background brightness from filter 30 mixer 29 provides a luminance on the foreground subject as though the luminance of the background was illuminating the subject.
This automatic function is especially useful for train window scenes and the like. Should the train go through a tunnel the illumination of the subject from the background ceases, and vice versa when in daylight out of the tunnel.
Figure 1 is a schematic diagram of a simplified embodiment of the subject invention, in which the equation (2), Ec, carries the whole control function. The apparatus of entity 26 is retained; the apparatus of entity 15 (equation ~1), Eb,) is ~.~'7~a53~
.
deleted from the showing in Figure 2.
Additionally, switch 31, conkrol 32, mixer 29 and integrating filter 30 are deleted, since there is no longer two control functions to switch between.
In the place of these elements potentiometer 33 is connected to the E output of entity 26, the wiper of which is connected to the - x input (differential) of multiplier lO. Also, the output of entity 26 is connected to potentiometer control 14 in tha place of prior entity lS output.
The setting of potentiometer 33 determines the level of luminance to be subtracted at the differential x input of multiplier lO.
With the simplified embodiment of Figure l, the turn-on of the background is proper, as with the embodiment of Figure 2.
However, the equation for E goes to zero in the areas occupied by the foreground subject(s?, thus the blue illumination from the background and lens flare are not removed.
With the simplified embodiment, particular selection of foreground colors to the degree possible to minimize the effect of the blue illumination, is indicated. Also, the selection of a lens for camera 1 having a minimum flare is of assistance.
For reasons of simplification and clarity of explanation the color blue has been widely discussed herein. Typically, this is the color that is used in practice.
However, any color may be used for the backing in the practice of this invention. Whichever color is dominant is designated by the term B in the e~uations. The next strongest ~7~
color is designated by the term G, while the weakest color is designated by the term R.
Should a green colored backing be employed, the blue and green video signals from camera 1 are merely interchanged as connected to entities 15 and 26 of Figure 2.
Also, specific type numbers of named manufacturers have been given. Equivalent circuit elements that would perform the circuit function required may be su'r~stituted.
The (B-G) term of equation (1) is formed at the + and -input terminals of differential amplifier 19, which also subtracts the (G-R) or the ~R-G) term. Potentiometers 14, 20 and 21 provide the K, Kl, and K2 of the equation.
.
5~
Having removed, by subtraction, the 3.58 vector that represents the hue of the blue backing, the signal leaving amplifier 6 via conductor 8 and entering multiplier 10 contains only luminance (black & white) information, which represents the "brightness" of the blue backing. It is necessary to remove this luminance so that it does not cause a gray veil over the entire background scene.
Chroma-key apparatus removes the luminance and the chroma of the backing by switching-off the foreground scene in the blue backing area.
In the present invention the luminance of the blue backing area is removed by arithmetic subtraction.
The luminance component of the blue backing and of the foreground objects resulting from secondary illumination from the backing are removed from the video signal by the connection of the Eb output into the negative terminal of the x input of multiplier 10. Luminance bias control potentiometer 23 allows the proper amplitude to be selected. Thus, the output of amplifier 24 has zero chroma and zero luminance in the blue backing area, while showing normal chroma and luminance for other foreground subject areas. Since the backing is thus in no way reproduced, the foreground video channel may be retained fully "on", and so reproduces all detail seen by the camera 1.
The foreground video signal, with chroma and luminance removed in the region of the blue backing is routed to combining amplifier 25, where the background scene is added to those areas from which the blue has been removed. Amplifier 25 sums the 05~
two signals by simple addition, as opposed to non-additive mixing required by soft chroma-key and other switching systems.
Background control voltage Ec is generated by the circuit of entity 26 in Figure 2. This circuit provides the following relationships:
c 1 2( r OR KgG~ + (l-K2)(KrR AND K G)-K (l-B) ]~ (2) OR indicates the larger of G or R, and AMD indicates the smaller of G or R. The + symbol indicates that negative values are clamped to zero.
Control voltage E will be large in the blue backing area, since blue is high while both red ana green are low. For any opaque object Ec falls to zero. At intermediate values of blue backing intensity E will have intermediate values. Ec is directly proportional to the luminance and visibility of the blue backing, and is zero for opa~ue objects whether or not illumination is received from the blue backing.
Ec is thus used to control the level of the background scene. This is accomplished by feeding the Ec signal to the y input of multiplier 27 of Figure 2. This control signal varies the level of the background scene from zero to an upper value determined by the adjustment of control potentiometer 28. This is normally adjusted for unity gain for the background image in those areas of the blue backing that receive full illumination.
From the general equation (2~, Ec, one can obtain the simpler e~uation Ec= B-G by setting Kl to unity, K2 to unity, K3 and Kr to zero, and Kg to unity. Similarly, by adjusting the values of the constants to other values, one can obtain 5~
Ec= B-R, and so on.
The num~er of terms employed for a suitable evaluation f Ec is a function of the color purity of the colored backing, of the range of colors present in the foreground subjects, and by the presence or absence of dark glossy objects therein. The evaluation of E is accomplished by manipulating the poten-tiometers representing the K's in the Ec e~uation (2).
Elements 29, 30, 31 and 32 of Figure 2 provide means for enhancing realism with various foreground - background subject matters.
When a subject is placed in front of a well-illuminated color backlng, it will receive substantial luminance from that backing, particularly at the sides and edges of the subject. If the backing is dark or black, no such side illumination occurs.
If the foreground subject is composited into a background day scene, that scene, if real, provides substantial back and side illumination to the subject. ~owever, if the foreground subject is composited into a background night scene, that scene pro~ides very little illumination to the subject.
The compositing system of this invention provides the option of day or night background simulation, or any degree between the two.
For manual adjustment of what this simulation shall be switch 31 is moved to the left contact. This connects to the wiper of potentiometer 32. In Figure 2 the top end of that potentiometer is connected to the Eb output from entity 15. The bottom end of that potentiometer is connected to the Ec output .~7~
from entity 26. Thus, any proportion of Eb vs Ec can be obtained by moving -the wiper. When the wiper is at the bottom of the potentiometer, E control signal is in control and background luminance is retained (day effect). When the wiper is at the top, Eb control signal is in control and background luminance is rejected (night effect). An intermediate position of the wiper provides an intermediate effect.
It is possible to obtain automatic selectlon (or mixing) of the Eb and Ec control by moving switch 31 to the right contact.
Both Eb and Ec control signals are inputted to non-additive mixer 29.
The background composite video signal from camera 2 is integrated by filter 30 to obtain a signal proportional to the average brightness of the background scene and is also inputted to mixer 29.
Depending upon the background brightness from filter 30 mixer 29 provides a luminance on the foreground subject as though the luminance of the background was illuminating the subject.
This automatic function is especially useful for train window scenes and the like. Should the train go through a tunnel the illumination of the subject from the background ceases, and vice versa when in daylight out of the tunnel.
Figure 1 is a schematic diagram of a simplified embodiment of the subject invention, in which the equation (2), Ec, carries the whole control function. The apparatus of entity 26 is retained; the apparatus of entity 15 (equation ~1), Eb,) is ~.~'7~a53~
.
deleted from the showing in Figure 2.
Additionally, switch 31, conkrol 32, mixer 29 and integrating filter 30 are deleted, since there is no longer two control functions to switch between.
In the place of these elements potentiometer 33 is connected to the E output of entity 26, the wiper of which is connected to the - x input (differential) of multiplier lO. Also, the output of entity 26 is connected to potentiometer control 14 in tha place of prior entity lS output.
The setting of potentiometer 33 determines the level of luminance to be subtracted at the differential x input of multiplier lO.
With the simplified embodiment of Figure l, the turn-on of the background is proper, as with the embodiment of Figure 2.
However, the equation for E goes to zero in the areas occupied by the foreground subject(s?, thus the blue illumination from the background and lens flare are not removed.
With the simplified embodiment, particular selection of foreground colors to the degree possible to minimize the effect of the blue illumination, is indicated. Also, the selection of a lens for camera 1 having a minimum flare is of assistance.
For reasons of simplification and clarity of explanation the color blue has been widely discussed herein. Typically, this is the color that is used in practice.
However, any color may be used for the backing in the practice of this invention. Whichever color is dominant is designated by the term B in the e~uations. The next strongest ~7~
color is designated by the term G, while the weakest color is designated by the term R.
Should a green colored backing be employed, the blue and green video signals from camera 1 are merely interchanged as connected to entities 15 and 26 of Figure 2.
Also, specific type numbers of named manufacturers have been given. Equivalent circuit elements that would perform the circuit function required may be su'r~stituted.
Claims (19)
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:
1. The method of compositing color image video signals involving a colored backing, which comprises the method steps of;
(a) forming a control signal Eb as a linear function of the red, green and blue video components of the foreground scene in accordance with the equation;
Eb = K [(B-G) - K1(G-R)+ - K2(R-G)+]+, (b) generating a chroma vector and adjusting the phase angle of the generated vector to match the phase angle of the vector representing the colored backing in the foreground scene video signal, (c) removing the chroma of the colored backing by vector subtraction in which the amplitude of the generator vector is controlled by Eb and where K
is initially adjusted to match vector amplitudes, (d) removing the spurious chroma from the foreground subject caused by illumination of the subject by colored light from the backing by setting K
to unity and K2 to zero in the Eb equation, (e) forming a control signal Ec as a linear function of the red, green and blue video components of the foreground scene in accordance with the equation;
Ec = K1 [B - K2(KrR OR KgG) + (1-K2) (KrR AND KgG) - K3 (l-B)]+, (f) arithmetically subtracting Eb and Ec in a selected proportion from the foreground scene video signal at an amplitude that results in zero luminance over the area of the colored backing, (g) linearly controlling the level of the background image video signal in proportion to the amplitude of control signal Ec, (h) combining by arithmetic addition the foreground scene video signal from which has been removed the chroma and luminance of the colored back-ing, with the background scene video signal whose level has been controlled by Ec.
(a) forming a control signal Eb as a linear function of the red, green and blue video components of the foreground scene in accordance with the equation;
Eb = K [(B-G) - K1(G-R)+ - K2(R-G)+]+, (b) generating a chroma vector and adjusting the phase angle of the generated vector to match the phase angle of the vector representing the colored backing in the foreground scene video signal, (c) removing the chroma of the colored backing by vector subtraction in which the amplitude of the generator vector is controlled by Eb and where K
is initially adjusted to match vector amplitudes, (d) removing the spurious chroma from the foreground subject caused by illumination of the subject by colored light from the backing by setting K
to unity and K2 to zero in the Eb equation, (e) forming a control signal Ec as a linear function of the red, green and blue video components of the foreground scene in accordance with the equation;
Ec = K1 [B - K2(KrR OR KgG) + (1-K2) (KrR AND KgG) - K3 (l-B)]+, (f) arithmetically subtracting Eb and Ec in a selected proportion from the foreground scene video signal at an amplitude that results in zero luminance over the area of the colored backing, (g) linearly controlling the level of the background image video signal in proportion to the amplitude of control signal Ec, (h) combining by arithmetic addition the foreground scene video signal from which has been removed the chroma and luminance of the colored back-ing, with the background scene video signal whose level has been controlled by Ec.
2. The method of claim 1, in which the control of the luminance cast upon the subject by the backing comprises the method step of:
(a) arithmetically subtracting Eb and Ec in a proportion selected to retain a desired level of luminance on the subject derived from the backing.
(a) arithmetically subtracting Eb and Ec in a proportion selected to retain a desired level of luminance on the subject derived from the backing.
3. The method of claim 2, which comprises the method step of:
(a) the proportion between Eb and Ec is selected manually.
(a) the proportion between Eb and Ec is selected manually.
4. The method of claim 2, which comprises the additional method steps of;
(a) integrating the background scene video signal to obtain a control signal representing the average brightness of the scene, and (b) controlling the proportioning between Eb and Ec by the control signal.
(a) integrating the background scene video signal to obtain a control signal representing the average brightness of the scene, and (b) controlling the proportioning between Eb and Ec by the control signal.
5. The method of compositing color image video signals involving a colored backing, which comprises the method steps of:
(a) forming a first control signal as a function of the brightness and visibility of the colored backing and also as a function of the colored secondary illumination on foreground subjects being received from the colored backing, (b) removing the chroma of the colored backing in the backing area and removing discoloration chroma from the foreground subjects resulting from secondary illumination received from the colored backing by vector subtraction as determined by the first control signal, (c) forming a second control signal as a function of the brightness and visibility of the colored backing but not as a function of the backing illumination being reflected from foreground subjects, (d) removing the luminance of the colored backing from the fore-ground scene video signal in the backing area by subtraction as determined by the first or second control signal, and (e) combining the foreground scene video signal as controlled by the first control signal or as selectively controlled by the first and second control signals with the background scene video signal, the level of which has been controlled by the second control signal, to form a composite image.
(a) forming a first control signal as a function of the brightness and visibility of the colored backing and also as a function of the colored secondary illumination on foreground subjects being received from the colored backing, (b) removing the chroma of the colored backing in the backing area and removing discoloration chroma from the foreground subjects resulting from secondary illumination received from the colored backing by vector subtraction as determined by the first control signal, (c) forming a second control signal as a function of the brightness and visibility of the colored backing but not as a function of the backing illumination being reflected from foreground subjects, (d) removing the luminance of the colored backing from the fore-ground scene video signal in the backing area by subtraction as determined by the first or second control signal, and (e) combining the foreground scene video signal as controlled by the first control signal or as selectively controlled by the first and second control signals with the background scene video signal, the level of which has been controlled by the second control signal, to form a composite image.
6. The method of claim 5, in which:
(a) the foreground scene video and the background scene video are combined by simple addition.
(a) the foreground scene video and the background scene video are combined by simple addition.
7. The method of claim 5, in which:
(a) the foreground scene video and the background scene video are com-bined by non-additive mixing.
(a) the foreground scene video and the background scene video are com-bined by non-additive mixing.
8. The method of claim 5, in which:
(a) the first control signal is essentially linear.
(a) the first control signal is essentially linear.
9. The method of claim 5, in which:
(a) the first control signal is essentially non-linear.
(a) the first control signal is essentially non-linear.
10. The method of claim 5, in which:
(a) the second control signal is essentially linear.
(a) the second control signal is essentially linear.
11. The method of claim 5, in which (a) the second control signal is essentially non-linear.
12. The method of claim 5, in which:
(a) the foreground scene video channel is open at full level through-out the entire video frame.
(a) the foreground scene video channel is open at full level through-out the entire video frame.
13. An electronic image compositing system of the colored backing type, comprising:
(a) a source of foreground composite video signals containing the red, green and blue components of the foreground scene, (b) a source of background composite video signals, (c) means for generating a first control signal utilizing the red, green and blue components of the foreground video signal, said first control signal being a function of the intensity of these said components emanating directly from the colored backing and also indirectly as a reflection resulting from backing illumination of foreground subjects, (d) means for utilizing said first control signal to remove the chroma of the colored backing and to remove the chroma discoloration cast upon fore-ground subjects by the colored backing, by subtraction, (e) means for generating a second control signal utilizing the red, green and blue components of the foreground video signal, said second control signal being a function of the intensity of these said components emanating directly from the backing, but being unaffected by these said components reflect-ed from foreground subjects, (f) means for selectively subtracting the first and second control signals from the composte foreground video signal to remove the luminance com-ponent of the colored backing therefrom, (g) means for utilizing the second control signal to control the level of the background video signal, and (h) means for combining the foreground video signal and the back-ground video signal after said signals are subjected to control by said first and second control signals.
(a) a source of foreground composite video signals containing the red, green and blue components of the foreground scene, (b) a source of background composite video signals, (c) means for generating a first control signal utilizing the red, green and blue components of the foreground video signal, said first control signal being a function of the intensity of these said components emanating directly from the colored backing and also indirectly as a reflection resulting from backing illumination of foreground subjects, (d) means for utilizing said first control signal to remove the chroma of the colored backing and to remove the chroma discoloration cast upon fore-ground subjects by the colored backing, by subtraction, (e) means for generating a second control signal utilizing the red, green and blue components of the foreground video signal, said second control signal being a function of the intensity of these said components emanating directly from the backing, but being unaffected by these said components reflect-ed from foreground subjects, (f) means for selectively subtracting the first and second control signals from the composte foreground video signal to remove the luminance com-ponent of the colored backing therefrom, (g) means for utilizing the second control signal to control the level of the background video signal, and (h) means for combining the foreground video signal and the back-ground video signal after said signals are subjected to control by said first and second control signals.
14. An electronic image compositing system of the colored backing type, comprising:
(a) a source of foreground video signals having composite, and red, green and blue output signals, (b) a source of composite background video signals, (c) a source of color subcarrier frequency, (d) means for rotating the phase angle of the subcarrier frequency to match the phase angle of the chroma vector of the illuminated color backing, (e) means for generating a control signal Eb derived from the red, green and blue foreground signals, where said Eb is proportional to the colored light emanating directly from the backing and indirectly as a reflection result-ing from backing illumination of foreground subjects, (f) means for utilizing said control signal Eb to control the ampli-tude of the rotated phase angle vector of the subcarrier frequency, (g) means for subtracting the amplitude controlled subcarrier frequency from the foreground video signal over the entire foreground scene, (h) means for generating a control signal Ec utilizing the red, green ad blue foreground signals, where said Ec is proportional to the intensity of the colored light emanating directly from the backing, but is unaffected by the colored light reflecting from foreground subjects, (i) means to electively subtracting Eb and Ec control signals from the composite foreground video signal to remove the luminance component of the colored backing therefrom, (j) means for utilizing Ec control signal to linearly control the level of the background video signal, and (k) means for additively combining the foreground video signal and the background video signal after said signals are subjected to Eb and Ec control.
(a) a source of foreground video signals having composite, and red, green and blue output signals, (b) a source of composite background video signals, (c) a source of color subcarrier frequency, (d) means for rotating the phase angle of the subcarrier frequency to match the phase angle of the chroma vector of the illuminated color backing, (e) means for generating a control signal Eb derived from the red, green and blue foreground signals, where said Eb is proportional to the colored light emanating directly from the backing and indirectly as a reflection result-ing from backing illumination of foreground subjects, (f) means for utilizing said control signal Eb to control the ampli-tude of the rotated phase angle vector of the subcarrier frequency, (g) means for subtracting the amplitude controlled subcarrier frequency from the foreground video signal over the entire foreground scene, (h) means for generating a control signal Ec utilizing the red, green ad blue foreground signals, where said Ec is proportional to the intensity of the colored light emanating directly from the backing, but is unaffected by the colored light reflecting from foreground subjects, (i) means to electively subtracting Eb and Ec control signals from the composite foreground video signal to remove the luminance component of the colored backing therefrom, (j) means for utilizing Ec control signal to linearly control the level of the background video signal, and (k) means for additively combining the foreground video signal and the background video signal after said signals are subjected to Eb and Ec control.
15. The system of claim 14, in which the means for generating control signal Eb for a blue backing comprises:
(a) first means to provide a difference signal between the red and the green foreground signals of adjustable amplitude, (b) second means to provide a difference signal between the green and the red foreground signals of adjustable amplitude, (c) comparison means to cause only the higher of the first and second means signals to output said comparison means, and (d) subtractive means, which subtracts from the blue video signal the green video signal and the output of comparison means, to provide a control signal Eb for controlling the amplitude of the subcarrier vector.
(a) first means to provide a difference signal between the red and the green foreground signals of adjustable amplitude, (b) second means to provide a difference signal between the green and the red foreground signals of adjustable amplitude, (c) comparison means to cause only the higher of the first and second means signals to output said comparison means, and (d) subtractive means, which subtracts from the blue video signal the green video signal and the output of comparison means, to provide a control signal Eb for controlling the amplitude of the subcarrier vector.
16. The system of claim 15, in which:
(a) the terms blue and green video signals are interchanged and a green backing is employed.
(a) the terms blue and green video signals are interchanged and a green backing is employed.
17. The system of claim 15, in which:
(a) the terms blue and red video signals are interchanged and a red backing is employed.
(a) the terms blue and red video signals are interchanged and a red backing is employed.
18. The system of claim 14, in which the means for generating control signal Ec for a blue backing comprises:
(a) a linear OR gate, (b) a linear AND gate, (c) means to adjust the level of the green foreground signal to the OR and AND gates, (d) independent means to adjust the level of the red foreground signal to the OR and AND gates, (e) means for proportioning the outputs between the OR gate and AND
gate, (f) means for subtracting the selected proportion of OR and AND out-puts from the blue foreground video signal, and (g) means for subtracting a function of the blue video signal from the blue video signal to form the control signal Ec.
(a) a linear OR gate, (b) a linear AND gate, (c) means to adjust the level of the green foreground signal to the OR and AND gates, (d) independent means to adjust the level of the red foreground signal to the OR and AND gates, (e) means for proportioning the outputs between the OR gate and AND
gate, (f) means for subtracting the selected proportion of OR and AND out-puts from the blue foreground video signal, and (g) means for subtracting a function of the blue video signal from the blue video signal to form the control signal Ec.
19. The system of claim 14, in which said means for electively subtract-ing Eb and Ec control signals comprises:
(a) manual means for proportioning Eb and Ec, (b) a non-additive mixer for automatic proportioning of Eb and Ec, (c) means for selecting manual or automatic proportioning between Eb and Ec, and (d) means for integrating background scene brightness for effecting control of said non-additive mixer.
(a) manual means for proportioning Eb and Ec, (b) a non-additive mixer for automatic proportioning of Eb and Ec, (c) means for selecting manual or automatic proportioning between Eb and Ec, and (d) means for integrating background scene brightness for effecting control of said non-additive mixer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA000429718A CA1179051A (en) | 1983-06-03 | 1983-06-03 | Encoded signal color image compositing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA000429718A CA1179051A (en) | 1983-06-03 | 1983-06-03 | Encoded signal color image compositing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1179051A true CA1179051A (en) | 1984-12-04 |
Family
ID=4125404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000429718A Expired CA1179051A (en) | 1983-06-03 | 1983-06-03 | Encoded signal color image compositing |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA1179051A (en) |
-
1983
- 1983-06-03 CA CA000429718A patent/CA1179051A/en not_active Expired
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US4409611A (en) | Encoded signal color image compositing | |
| US4589013A (en) | Automated encoded signal color image compositing | |
| US4100569A (en) | Comprehensive electronic compositing system | |
| US4007487A (en) | Electronic composite photography with color control | |
| US4625231A (en) | Comprehensive electronic compositing system | |
| EP0790744B1 (en) | Compensation of backing colour and luminance non-uniformity for linear image compositing | |
| US4344085A (en) | Comprehensive electronic compositing system | |
| US3595987A (en) | Electronic composite photography | |
| US5424781A (en) | Backing color and luminance nonuniformity compensation for linear image compositing | |
| US4292649A (en) | Composite video signal combining system | |
| US4109278A (en) | Video traveling matte system | |
| US5343252A (en) | Method and apparatus for compositing video images | |
| US4496966A (en) | Method for mixing two color television signals | |
| GB1268936A (en) | Matrixing apparatus | |
| JP2556810B2 (en) | Chroma key device with secondary hue selector | |
| CA1179051A (en) | Encoded signal color image compositing | |
| US2850563A (en) | Processes for the reproduction of images in color | |
| GB2168564A (en) | Improvements in or relating to keying of video signals | |
| GB1581376A (en) | Comprehensive electronic compositing system | |
| Blom et al. | Adaptive highlight compression in today's CCD cameras | |
| Morris | Color Film Standards and Operating Practices for Television | |
| GB1405903A (en) | System for processing signals representing colour images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKEC | Expiry (correction) | ||
| MKEX | Expiry |