Apparatus and Method For Processing Previously
Processed Video Signals
This application is a continuation-in-part of my copending united States Application Serial No. 722,532, filed April 12, 1985.
This invention relates generally to video signal gain control, and more particularly, to the control of the gain of video signals in accordance with a plurality of independently related video signal gain determining key signals.
In television production it is often desirable to process a portion of a television signal less than the standard full field raster. For example, a common function of a digital special effects system is to compress a full field of video to a smaller size. In such a field of video, only the area corresponding to the compressed. video image is of interest. The surrounding display area contains no information of interest. When this video signal is outputted to Utilization devices, it is desirable for that device to know what area to process. This is the function of a matte or key signal that accompanies the compressed or otherwise processed video signal and defines the gain throughout the video signal interval.
A matte or video signal gain defining key signal effectively specifies the area of interest or the area to be processed. Key signals are commonly created during the processing of a video signals, but can also be independently created. For example, a key signal can be independently created that represents the selected shapes. Television signals are commonly created by television cameras, graphic art generators and other devices. Key signals can be created by dedicated key signal generators, digital special effects units, and other video signal processing devices. A chroma-key generator creates a key signal corresponding to a specific color found in an input video signal and can be used as a video signal gain defining key. For example, if the video signal from a camera receiving an image of an object against a blue background and chroma-keyed on the blue color background, the previously an object key could be produced. In the example above with the digital special effects system, a key signal can be created that corresponds to the area of the compressed field of video.
A video gain defining key signal is used to control the gain video signals over a range and with a resolution determined by the range and number of values the key signal can take. Normally, such keys are linear. so that there is a one to one correspondence between each part of the key signal and the video signal. Typically, the gain of a video signal is adjusted by the key signal by multiplying the level of the video signal by the key signal. When operating in the analog domain, the instantaneous level or amplitude of the video signal is multiplied with the instantaneous value of the key signal on a continuous basis. Ordinarily, an analog key signal has a value between zero and one.
A digital video signal is composed of discrete digitally encoded samples defining pixels that define the video image. Each pixel may be defined by a plurality of different components, such as a single luminance and a pair of chrominance components. In systems for such video signals a digital key signal is used having a value between zero and one for each pixel of the video image. When processed, each pixel of the video signal is processed with its corresponding key signal.
The value of a key signal often varies with time. In an analog system, the key value has an instantaneous value that changes with time, whereas in a digital system each pixel may have an associated key value.
One type of key signal is a bi-state key signal. This type of key signal has either a value of zero or one. When processed with a video signal, the portion of the key signal that has a value of zero is multiplied with the corresponding portion of the video signal to yield a processed or keyed video signal of zero. This portion of the keyed video signal appears as black when viewed on a monitor. The portion of the video signal that is processed with a key signal that has a value of one remains unchanged and is viewed on a monitor unchanged. In the aforediscussed object example, the portion of the key signal outside the representation of the object has a value of zero. The portion of the key signal corresponding to the inside of the object has a value of one.
The value of a key signal often is correspondingly processed proportionately with the gain of the video signal. If the values of two key signals are processed with their respective video signals representing two objects, the result would be a display of one of the objects shaped in the form of the other.
Another type of key signal is a linear key signal. Unlike the bi-state key signal which is restricted to either a value of zero or one, a linear key signal may
have several values between zero and one. Whereas, with the bi-state key signal, one typically indicates the presence of the video signal and zero the absence of the the video signal, the value of the linear key signal represents a continuum of change in gain of the video signal.
When a video signal is processed with a linear, key signal, the corresponding signals are multiplied, as with the bi-state key signal, but because the linear key can have any value between zero and one, the gain of the video signal can be adjusted to any value between zero and one. For example, if a point in a video signal is process with a linear key that has a value of 0.5, the gain of the video signal would be reduced in gain by one half. When viewed on a monitor, the video signal would look semi-transparent.
In combining typical video signals, the total gain of the two signals is never made more than one. Two video signals of nominal gain would not be directly added, as their total gain would be two. Adjusting the gain of a video signal with a key signal is one method of reducing the gain of video signals so that they can be combined with other video signals. Adding two gain adjusted video signals, each of which has been multiplied with a key signal of 0.5, would be permissible, as the gain of the combined video signals would only be one. The resulting video signal when viewed on a monitor would be both video signals. One method of combining two video signals whose gains are nominal, i.e., not reduced, is to use an independent or external key signal to process the video signals. In this method, the independent key signal determines the amount of each of the two signals present in the combined mixed video signals. The gain of one of the video signals is reduced directly by the key signal in a video processor, which multiplies the video signal by the key signal. The gain of the other
video signal is reduced by one minus the value of the key signal, which is determined by a 1-K key processor.
Generally, the gain of a video signal is processed by a key signal as a last step prior to output. There are situations, though, where it is necessary to process a video signal whose gain has been previously adjusted by another key signal. The way this is done is to adjust the gain of the video signal without regard to the previous gain adjustment.
If, in the above example, the gain of the video signal had been previously reduced at a point, by a key signal of 0.5, its gain would be 0.5 of its original value. When its gain is reduced further by an independent key of, for example, 0.3, the result gain of the signal is reduced to 0.15. When combined with the second video signal, the combined video signal gain is only 0.85. This is undesirable for two reasons. First, the independent key value of 0.3 is considered the desired value for the gain of the video signal. Because this video signal had already been reduced in gain, the processing by the independent key reduces its gain to a level below the desired level. Secondly, it is very desirable to fully use all the gain available for the combined video signal. The combined video signal, in this example, only has a gain of 0.85, not the desired 1.0.
A desirable method of combining one or more gain adjusted video signals, would take into account any previous gain adjustments of the video signals. There is a need for a method and apparatus which is able to combine previously gain adjusted video signals so as to obtain video signals with the desired gain.
The present invention overcomes the limitations of the previous method by taking into account the previous gain reduction of a video signal.
When gain adjusting a video signal with an independent key signal, the independent key signal is
considered the desired level of gain reduction. If the video signal has already been reduced in gain below the level of the independent key signal, it is preferred not to reduce its gain further. This is because when the gain on a video signal is reduce, information is lost. That lost information cannot be recovered by raising the gain of video signals. Raising the gain adds false information or noise to the video signal.
If the video signal has been reduced in gain by an amount less than the desired level called for by the independent key signal, then the video signal is processed with a new key signal that is the ratio of the independent key signal to the key signal used to previously adjust the gain of the video signal. The result of adjusting the gain with the new key signal is that the video signal is reduced in gain by the precise amount to bring the total gain reduction as specified by the independent key.
To process a previously processed video signal A' with an independent key signal, KI, the minimum between the key signal, KA, that was previously used to process video signal A' and independent key signal KI is determined as desired key signal, KD. Next, the ratio between key signals KD and KA is determined, KP, and used to process video signal A'. Key signal, KP, is multiplied with processed video signal A' to form twice processed video signal A".
An apparatus to process a previously processed video signal A' uses a minimum processor to determine the minimum between key signals KI and KA and outputs that minimum as key signal KD. A ratio processor outputs the ratio of KD to KA as key signal KP, which actually used to process video signal A' in a video processor.
To combine a previously processed video signal A' with an unprocessed video signal B, according to an independent key signal KI, the minimum between the key signal, KA, that was previously used to process
video signal A', and independent key signal KI is determined as desired key signal KD. Next, the ratio between key signals KD and KA is determined, KP, and used to process video signal A'. The minimum between key signals KA and KI is subtracted from one to determine the key signal used to process video signal B. Processed video signal B' and twice processed video signal A" are added to form a combined video signal C. Video signal C is considered an unprocessed video signal, even though it was created from two processed video signal, because its gain is always at one.
An apparatus to combine a previously processed video signal A' with an unprocessed video signal B uses a minimum processor to determine the minimum between key signals KI and KA and outputs that minimum as key signal KD. A ratio processor outputs the ratio of KD to KA as key signal KPA which is used to process video signal A'. The output of the ratio processor is fed to a first video processor where it is multiplied with processed video signal A' to form twice processed video signal A". The output of the minimum processor, KD, is fed to a 1-KD processor which subtracts KD from one to form key signal KPB used to process video signal B. That key signal is fed to a second video processor where it is multiplied with video signal B to form processed video signal B'. Processed video signal B' and twice processed video signal A" are fed to an adder to form combined signal C.
To combine a previously processed video signal A' with a previously processed video signal B' according to an independent key signal KI, the minimum between the key signal KA, that was previously used to process video signal A', and independent key signal KI is determined as the desired key signal KDA. Next, the ratio between key signals KDA and KA is determined, KPA, and used to process video signal A'.
Key signal KDA is subtracted from one and compared with key signal KB, the key signal previously used to process video signal B', to determine the minimum, K DB. The ratio of KDB to KA is next determined, KPB, and used to process video signal B' in to twice processed video signal B". Twice processed video signals A" and B" are added to form a combined signal C'.
Video signal C' is considered a processed video signal because both processed video signals A' and B' could have been reduced in gain, prior to the second processing, such that when combined, the total gain is less than one. By adding key signals KDA and KBA, the key signal KC is determined. The key signal KC indicates the previous processing of processed video signal C', even though KC was never actually used to process video signal C.
An apparatus to combine a previously processed video signal A' with previously processed video signal B' uses a first minimum processor to determine the minimum between key signals KI and KA and outputs that minimum as key signal KDA. A first ratio processor outputs the ratio of KDA to KA as key signal KPA, which is used to process video signal A'. The output of the first ratio processor is fed to a first video processor where it is multiplied with video signal A' to form twice processed video signal A". Key signal KDA is also fed to a 1-KDA processor where it is subtracted from one. The output of the 1-KDA processor is fed to a second minimum processor with key signal KB to form key signal KDB. Key signal KDB is fed to a second ratio processor where it is compared to key signal KB. The output of the second ratio processor, KPB, is fed to a second video processor where it is multiplied with video signal B' to form twice processed video signal B". Twice processed- video signal B" and twice processed video signal A"are fed to an adder to form a combined signal C'. A key
signal, KC, indicative of the previous processing of video signal C', is formed by a key added, from key signals KDA and KDB.
In each of these operations, the minimum between the key signal associated with the processed video signal and the independent desired key is determined. The minimum key signal is then divided by the associated key, the key previously used to process the video signal. The effect of this division is that if the minimum key signal is the associated key, the result of the division will be one. This means that when the processed video signal is multiplied by this independent key signal of one, there will be no reduction of gain. Where the video signal has already been processed with a key signal less than the independent desired key signal, then the best that can be done is not to reduce the gain any further.
Where the independent desired key calls for a reduction in gain that is less than that accomplished by the previous processing, then minimum between the two key signals will be the desired key. The desired key divided by the associated key will produce a key signal.that will reduce the gain of the processed video signal to the level called for by the independent desired key.
Various of the above-mentioned and further features and advantages of the present invention will be apparent from the specific examples of an exemplary apparatus and method described hereinbelow with reference to the accompanying drawings, in which:
Figure 1 is a schematic block diagram of one embodiment of a system for combining video signals of different possible gains in accordance with the present invention;
Figure 2 is a schematic block diagram of another embodiment of a system for combining video signals of different possible gains in accordance with the present invention;
Figure 3 is a schematic block diagram of a further embodiment of a system for combining video signals of different possible gains in accordance with the present invention;
Figure 4 is a schematic block diagram of one embodiment of a circuit for adjusting the gain of video signals in accordance with the present invention;
Figure 5 is a schematic block diagram of another embodiment of a circuit for adjusting the gain of video signals in accordance with the present invention; and
Figure 6 is a schematic block diagram of a further embodiment of a circuit for adjusting the gain of video signals in accordance with the present invention.
In accordance with the present invention, video and key signals may either be digital or analog, although the preferred system is digital. In an analog system, video signals are prooesβed continuously with their corresponding key signals, although, at any one point in time, only one point in the video signal is being processed with the corresponding, point in the key signal.
Further, the discussed embodiments use one or more of six basic circuit elements. Each element can easily be implemented in a variety of ways in either an analog or digital system by a person skilled in the art. Video processors used to process a video signal with a key signal can be implemented in a digital system as a digital multiplier. These are commonly available in standard integrated circuit packages. In an non-real time system, the video processor could be implemented with a microprocessor. An analog video processor can be implemented as a simple analog multiply circuit.
Another circuit element, is a video combiner. The video combiner is used to combine processed video signals to form an output video signal. In a digital system, this element is easily implemented as a digital adder, which is also commonly available in standard integrated circuit packages. In an analog system, the
video combiner can implemented as a simple mixing circuit.
A common element to each embodiment is a minimum processor. The minimum processor is used to determine the minimum value between two key signals. In a digital system, this is accomplished by a digital comparator which outputs the minimum value between the two input values. In a digital system, these values are discrete numbers fed serially to the minimum processor. In an analog system, a minimum circuit is conveniently implemented as an analog AND gate, which has the precise desired property.
Yet another element is a 1-K processor, which is used to determine the key signal remainder. In a digital system, a digital adder can be used to add the negative of the input key signal to one. In an analog system the key signal is routinely subtracted from a reference level signal to produce the desired output.
A key processor is used to determine the sum of two key signals. In a digital system, this can be implemented as a common digital adder. These are available in standard integrated circuit packages. In an analog system, the combiner can be implemented as analog add circuit.
Lastly, another element is a ratio processor. The ratio processor divides one key by another to form their ratio. The ratio processor determines the proper key signal with which to process the already processed video signal. In a digital system, this can be implement by arithmetic processor chip, which are commercially available. The preferred method of implementation is to use a look-up table in a ROM-type memory. While this implementation limits the resolution available, it is fast and low-cost. Analog division. circuits are known in the art and can be constructed by a person skilled in the art.
Three embodiments of the present invention will be discussed. The first is a system to process an already
processed video signal according to an independent key. This basic circuit is used in both other embodiments. The second system is for combining a processed video signal with an unprocessed video signal according to an independent key signal . The third embodiment combines two processed video signals according to an independent key signal. It is understood that these embodiments are merely exemplary of the present invention.
Referring now to Figure 1, a previously processed video signal A', 50, is fed to a video processor 52 to produce twice processed video signal A", 54. Video signal A' is processed with a processing key signal KP, 56.
Key signal KP, 56, is generated from the associated key signal KA, 58, of video signal A', 50, and independent key signal KI, 60. The associated key signal KA, 58, is not used to process the video signal A', 50, but rather is used as an indicator of previous processing of the video signal. The independent key indicates the desired level of gain reduction for the outputted video signal. If the processed video signal A', 50 has been processed with a key signal which has reduced its gain below the level of the independent key signal KI, then no further gain reduction should occur.
It is the task of minimum processor 62 to determine whether key signal KA, 58, indicate that the video signal A', 50, has already been reduced in gain by the same amount or more than the independent key signal KI. If key signal KA, 58, is less than KI, 60, then minimum processor 62 outputs key signal KA, 58, as key signal KD, 64. If key signal KA, 58, is more than KI, 60, then minimum processor 62 outputs key signal KI, 60, as key signal KD, 64. If the two signals are equal, then either signal may be outputted.
The output of minimum processor 62 if fed to ratio processor 66. Key signals KA, 58, and KD, 64, are
inputted to ration processor 66, which determines the ratio of KD divided by KA. That ratio is outputted as processing key signal KP, 56, which is used to process video signal A', 50 into video signal A", 54.
If the KD is KA because the video signal has already been reduced in gain by more than independent key signal A', 50, then the ratio will simply be KA divided by KA or one. Video signal A', 50, will then be processed by one, which will not change its value. This is exactly the result desired as video signal A', 50 was already reduced in gain by more than the independent key signal KI, 60, thus no further reduction of gain was desired.
In the case where KD is KI, then a ratio will be formed that reduce the gain of video signal A', 50, by an amount such that video signal A", 54, will have been reduced in gain as if it had been processed by key signal KI, 60, alone, instead of by KA, 58, and KP, 54, separately.
Referring now to Figure 2, which illustrates a system to combine a processed and unprocessed video signal according to an independent key. The system in Figure 2 is based largely on Figure 1, in that the processing of video signal A' into A" from key signals KA and KI is identical. A previously processed video signal A', 70, is fed to a video processor 72 to produce twice processed video signal A", 74. Video signal A' is processed with a processing key signal
KPA, 76.
Key signal KPA, 76, is generated from the associated key signal KA, 78, of video signal A', 70, and independent key signal KI, 80. It is the task of minimum processor 82 to determine whether key signal KA, 78, indicate that the video signal A', 70, has already been reduced in gain by the same amount or more than the independent key signal KI. If key signal KA, 78, is less than KI, 80, then minimum processor 82 outputs key signal KA, 78, as key signal KD, 84.
If key signal KA, 78, is more than KI, 80, then minimum processor 82 outputs key signal KI, 80, as key signal KD, 84. The output of minimum processor 82 is fed to ratio processor 86. Key signals KA, 78, and KD, 84, are inputted to ration processor 86, which determines the ratio of KD divided by KA. That ratio is outputted as processing key signal KPA, 76, which is used to process video signal A', 70 into video signal A", 74.
Twice processed video signal A", 74, is fed to video combiner 96, where it is combined with once processed video signal B, 88, to form an output video signal C, 97. Video signal B', 88 is processed with the remainder of key signal KD, 84, processing key signal KPB, 98. When combined, the video signals A", 74, and B', should have been reduced in gain such that the keys signals used to process them, KA, 78, KPA, 76, and KPB, should add up to be less or equal to one. By design, video signal A' is reduced in gain by at least the value of KI, 80, and therefore, video signal B, 94 should be reduced in gain by 1-KD.
1-KD processor 92, takes the output of minimum processor 82, KD, 84, and subtracts it from one to form KPB, 98, which it outputs to video processor 90. Using key signal KPB, 98, video processor 90, processed video signal B, 94, into processed video signal B', 88.
Referring now to Figure 3, which illustrates a system to combine two processed video signals according to an independent key. The system in Figure 3 is based largely on Figure 2, in that the processing of video signal A' into A" from key signals KA and KI is identical. A previously processed video signal A', 100, is fed to a video processor 102 to produce twice processed video signal A", 104. Video signal A' is processed with a processing key signal KPA, 106.
Key signal KPA, 106, is generated from the associated key signal KA, 108, of video signal A',
100, and independent key signal KI, 110. If key signal KA, 108, is less than KI, 110, then minimum processor
112 outputs key signal KA, 108, as key signal KD, 114.
If key signal KA, 108, is more than KI, 110, then minimum processor 112 outputs key signal KI, 110, as key signal KD, 114. The output of minimum processor
112 if fed to ratio processor 116. Key signals KA,
108, and KD, 114, are inputted to ration processor 116, which determines the ratio of KD divided by KA. That ratio is outputted as processing key signal KPA, 106, which is used to process video signal A', 100 into video signal A", 104.
Twice processed video signal A", 104, is fed to video combiner 126, where it is combined with twice processed video signal B", 118, to form an output video signal C', 127. Video signal B', 118 is processed with the remainder of key signal KD, 114, processing key signal KPB, 128. When combined, the video signals A",
104, and B", should have been reduced in gain such that the keys signals used to process them, KA, 108 , KPA, 106 , KB, 130 and KPB, 128 , should add to less or equal to one. By design, video signal A' is reduced in gain by at least the value of KI, 110, and therefore, video signal B', 124 should similarly be processed with 1-KDA, 114. The processing of video signal B', 114 is identical to that of video signal A', except that instead of using independent key signal KI, 110, 1-KDA is used as the independent key signal to the video signal B' processing part of the system.
1-KD processor 122, takes the output of minimum processor 112, KD, 114, and subtracts it from one to form 1-KDA, 122, which it outputs to minimum processor 136. Minimum processor 136, determines the minimum between KA and 1-KDA and outputs a key signal KDB to ratio processor 134. Ratio processor 134 divides KDB, 132, by K B, 130, to produce processing key signal KPB,
128, which is used by
video processor 120 to process video signal B', 114, into video signal B", 118. Video combiner 126 adds video signals A", 104, and B", 118, to form the output video signal C'.
In the system of Figure 2, the output video signal C, 97, is considered an unprocessed video signal by any downstream usage device. This is because, the amount that video signals A", 74, and B', 88, have been reduced in gain by an amount totaling one, because of the design of the system. If Figure 3, because both video signals A', 100 and B', 114 have been previously reduced in gain prior to being processed by the system, it is possible that either or both have been reduced below the level requested by the independent key signal KI. When this occurs, the amount that video signals A", 104, and B", 118 have been reduced in gain could total one or less. This is the value of key signal KC, 127, and is formed from the sum key signals KDA, 114, and KDB, 132, which represent the actual reductions in gain of video signals A", 104, and B", 118, respectively. The sum is determined by key combiner 138.
In summary, the above three embodiments all implement a form of the present invention. Common to each is the initial determination as to whether the processed video signal or signals have already been reduced in gain by more than that requested by the independent key signal. If it has been reduced in gain by more than that requested, then the video signal is processed by a key signal of one, which results in no further reduction of gain. If the video signal has been reduced in gain by less than that requested, then a processed key signal is determined which, when used to process the video signal, reduces the gain on the video signal by amount such that the total reduction in gain on the video signal is the amount requested by the independent key signal.
In the embodiment of Figure 2, the unprocessed video signal is processed with the remainder of the key signal not used to process the processed video signal. The results in a new video signal which is a combination of the the two video signals that has a net reduction in gain of one.
In the embodiment of Figure 3, the second processed video signal is processed with the remainder of the key signal not used to process the processed video signal as its independent key signal. The results in a new video signal which is a combination of the the two video signals that has a net reduction in gain of one or less. Thus this system also outputs an associated key signal indicating the total reduction in gain.
Referring to Figure 4, the input video signal A' has its gain adjusted in accordance with the value of a request key signal KI and the reciprocal of the value of an input key signal KA in successive and separate steps by a pair of multipliers in the communication line of the input video signal A'. The input key signal KA has a value that indicates the gain of the input video signal. This indicated gain signifies the gain of the input video signal A' relative to a reference gain, typically unity. As a result of the two multiplications, a output video signal A''' is provided having a selected gain determined by the value of the key signal KI. In the embodiment illustrated by Figure 4, the two multiplications are shown as having the video signal A' gain first adjusted by the reciprocal of the value of the input key signal KA and, thereafter, by the key signal KI. The sequence of multiplications however can be reversed.
Moreover, referring to Figure 6, the two key signals KI and KA can first be combined in a inversely proportional relationship relative to the input key
signal KA, whereby the input video signal A' only has to be multiplied one time to form the desired output video signal A'''.
Referring to Figure 5, an embodiment is shown similar to the embodiment of Figure 1. As illustrated therein, a minimum operation is performed on the two key signals KA and KI to determine whose value indicates a smaller video signal gain. Whichever key signal indicates the smaller gain value it is output by the minimum circuit to the first multiplier in the input video signal path. The second multiplier in that path receives the reciprocal of the input key value to multiply the results of the first multiplier, and thereby provide the output video signal A''' with the desired gain.
While the embodiments disclosed herein are preferred, it will be appreciated that they are merely examples, and that various alternatives, modifications, variations or improvements thereon may be made by those skilled in the art from this teaching, which are intended to be encompassed by the following claims.