JP2010532629A - Chroma key production method and apparatus - Google Patents

Abstract

A method and apparatus for generating an adaptive chroma key includes considering foreground object information during composition scene formation. The brightness and chromaticity of the region of interest between the foreground key and the background scene are considered during the formation of the composite scene.

Description

The present invention relates to chroma keying.
More particularly, the present invention relates to the production of adaptive chroma keys.
This application claims priority to the international patent application PCT / US2007 / 015254 entitled “Method and Apparatus for Chroma Key Production” filed on June 29, 2007.

  In general, a chroma key places a foreground object in a background scene. Because the chroma key parameters are calculated for each field based only on the foreground scene, the foreground object cannot adapt to the brightness of the background scene. Thus, the use of ambient light or artificial light changes.

  In fact, the chroma key parameters are based on a more uniform studio light / dark condition than light / dark in the background scene. This type of contrast between light and dark cannot produce a natural chroma key.

Aspects of the present invention provide an adaptive chroma key that overcomes the problems of the prior art.
This and other aspects use the foreground key to determine the area of interest (AOI) for the composite scene and to generate the chroma key taking into account the physical characteristics of the pixels in the determined AOI. This is achieved by a method for generating a chroma key.

  In accordance with another aspect of the invention, an apparatus for generating a chroma key uses a foreground key to determine an area of interest (AOI) for a composite scene and to determine the physical characteristics of the pixels in the determined AOI. Includes a source selection device configured to generate chroma keys in consideration. The source selection device includes a plurality of input sources, a processor, switching logic in communication with the processor, and an adaptive chroma key subsystem connected to the processor and the switching logic, the chroma key subsystem operating under control of the processor Then, two or more input scenes are selectively combined with respect to the composite scene.

  Other aspects and features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings. However, since the drawings are drawn for purposes of illustration and not as a limiting definition of the invention, the claims should be referred to. The drawings are not necessarily drawn to scale, and unless otherwise indicated, are intended to conceptually illustrate the structures and procedures described in this embodiment. In the drawings, like reference numerals indicate like components throughout the drawings.

A graphical depiction of a foreground scene in a green virtual studio set. A graphical depiction of foreground objects in the scene. A graphical depiction of a background scene intended for use with a foreground scene. A graphic depiction of the foreground key. A graphical depiction of the composite output of both the foreground and background scenes. A graphical depiction of an area of interest (AOI) in a composite scene. A graphical depiction of a composite scene with both brighter foreground and foreground objects. A graphical depiction of a composite scene with both darker foreground and foreground objects. FIGS. 9a to 9c are graphical depictions showing how different hues in the foreground scene form different but matching foreground objects. FIG. 6 is a flow diagram for determining brightness for display of a composite image, according to an implementation of the present invention. FIG. 6 is a flow chart for determining chroma key AOI according to the present invention; FIG. 4 is a flowchart of chromaticity determination and application for display of a composite image according to the implementation of the present invention. It is a block diagram of the switcher apparatus comprised in order to implement | achieve the adaptive chroma key of this invention. FIG. 3 is a block diagram of an adaptive chroma key subsystem according to the implementation of the present invention.

  In general, the foreground object can be placed in the background scene by the chroma key. Because the chroma key parameters are calculated for each field based solely on the foreground scene, the foreground object cannot adapt to changes in the brightness (eg, ambient and population) of the background scene. In fact, the chroma key parameters may be homogenized for studio light and dark conditions rather than light and dark in the background scene.

  The present invention provides an alternative to this chroma keying scheme by allowing the adaptive formation of chroma keys using luminance and chromaticity information from the background scene.

  1-5 show some background information to understand the concept of the present invention. FIG. 1 shows a foreground scene 10 (represented by a person in a picture) and a green virtual studio set 12. FIG. 2 shows the foreground object 10 without the virtual studio set 12. FIG. 3 shows a background scene 14 displayed on the virtual studio set. FIG. 4 shows a foreground key 16 representing an image of the foreground scene superimposed on the background scene. FIG. 5 is a depiction of a composite output image obtained from the combination of the background scene 14 and the foreground scene 10.

  First, according to the present invention, a determination is made regarding an area of interest (AOI) between a foreground scene and a background scene (step 102 of FIG. 10a). Referring to FIG. 10b, to do this, the foreground key is superimposed on the background scene (step 110), and all pixels of the background scene entering the foreground key are identified (step 112). . This identification constitutes the AOI. FIG. 6 is a graphic depiction of the AOI obtained from this process.

  According to one implementation, information from the background scene is used by the chroma key logic to adaptively form the chroma key. In doing this, the method includes taking into account one or more physical characteristics of the pixel in the determined AOI to form a chroma key. In the example of the present embodiment, these physical characteristics include pixel brightness and chromaticity.

  FIG. 10a shows a method 100 that takes into account the brightness of the AOI, and FIG. 11 shows a method 150 that takes into account the chromaticity of the AOI.

  Referring to FIG. 10a, first, a determination regarding the AOI of the composite scene is made (step 102). The average brightness of the pixels in the AOI is then calculated (step 104), and the average brightness of the sample area in the foreground scene is also calculated (step 106). Once these luminance calculations are made, the luminance in the foreground scene is linked to the luminance in the AOI (step 108). In other words, for each field, the difference in brightness (delta) in AOI relative to the brightness in the foreground scene is used. If the background has a sharp light-dark change like a video clip, for example, the video clip shows a light and dark road in Manhattan. A foreground object (eg, a new leader or report) adapts to the background scene and changes its brightness accordingly. FIG. 7 shows an example of a bright foreground with a bright foreground object, and FIG. 8 shows an example of a dark foreground with a dark foreground object.

  FIG. 11 shows an example of a method 150 in which chromaticity signals are considered. After AOI determination 102, it is determined whether a constant vector is applied to all pixels in the AOI (step 120). If not, the foreground scene remains unchanged (step 124). If there is a constant vector applied to all the pixels in the AOI, a small percentage of the same constant vector is applied to the resulting foreground scene. For example, if the background scene is a disco club with a rotating multicolored light beam, the foreground object will adapt to hue changes in the background scene (ie if the red light beam is included in the AOI) Similarly, a slightly red color appears in the foreground object). Therefore, although different hues in the background scene are different, matching objects can be formed in the foreground. This concept is illustrated in the exemplary images of FIGS. 9a-9c. In each of the figures, the hue is different and results in a change in the color of the foreground object, which results in an overall change in the overall displayed composite image. FIG. 9b shows the effect of the reddish hue (represented by an array of very small dots covering the entire figure 9b) added to the foreground contrast, and the reddish hue on the foreground object. FIG. 9c shows the overall effect (ie, reddish hue matching with the foreground) and green in the foreground scene (represented by an array of very small dashes covering the entire FIG. 9c). Shows the effect of hues.

  FIG. 12 shows a block diagram of a switcher system 200 that is programmed to operate in accordance with the present invention. The switcher 202 includes a plurality of inputs 208, a processor 204, and switching logic 206 in communication with the processor. The processor 208 may include on-board memory 210 or be linked to an external storage medium such as a hard disk drive, compact disk drive, flash memory or other solid state memory device, or other memory storage means. The adaptive chroma key subsystem 207 communicates with the processor 204 and the switching logic 206 to perform the method of the present invention, receive one of the inputs 208 having a foreground scene, and pass it on to another having a foreground scene. And, at its output 212, provide the desired composite scene.

  FIG. 13 shows a block diagram of an adaptive chroma key subsystem 207 according to the implementation of the present invention. Foreground video 250 and foreground key 252 are interpolated by interpolator 258. After selection of hue (via first hue selector 262 and second hue selector 266) and suppression (via first suppression 264 and second suppression 268), video 270 is the next in switcher 202. To a logical subsystem (eg, switching logic 206). The second hue selector 266 outputs foreground information that is processed (chip and gain 278) before the foreground change is applied (280). The foreground video 254 and the foreground key 256 are interpolated by the interpolator 260 and then the AOI is determined. As described above, once the AOI is determined, the luminance change 274 and chromaticity change 276 in the AOI are determined and provided to the foreground (280). At this stage, the offset 282 is added to the foreground key signal and the foreground key 212 is output.

  Various aspects, implementations and features may be implemented in one or more different ways. For example, various aspects, implementations, and features may be used to implement a program or set of instructions using an apparatus or processing device that performs the method, program, or other set of instructions using, for example, one or more methods, apparatuses. May be implemented using a device including, and using a computer-readable recording medium.

  The device may include, for example, discrete or integrated hardware, firmware and software. By way of example, an apparatus may include a processor that represents a processing apparatus that typically includes, for example, a microprocessor, an integrated circuit, or a programmable logic device. As another example, an apparatus may include one or more computer readable media having instructions for executing one or more processors.

  A computer readable recording medium may include, for example, a software carrier or other storage device such as a hard disk, compact disk, random access memory (RAM) or read only memory (ROM). A computer-readable recording medium may also include a formatted electromagnetic wave that encodes or transmits instructions, for example. The instructions can be, for example, hardware, firmware, software, or electromagnetic waves. The instructions may be found, for example, in the operating system, individual applications, or combinations thereof. A processor can be characterized as, for example, both an apparatus configured to perform a process and an apparatus that includes a computer-readable recording medium having instructions to perform the process.

  A number of implementations have been disclosed. Nevertheless, various changes may be made. For example, elements of different realizations may be combined, supplemented, changed or removed to generate other realizations. Accordingly, other implementations are within the scope of the following claims.

Claims (19)

A method for generating a chroma key,
Determining an area of interest (AOI) for the composite scene using foreground keys;
Generating a chroma key taking into account the physical characteristics of the pixels in the determined AOI;
A method comprising the steps of: The determining step includes:
Superimposing the foreground key on the background scene;
Identifying pixels in the background scene included in the foreground key;
The method of claim 1 comprising: The generating step includes
Calculating an average luminance of pixels in the AOI;
Calculating an average brightness of a sampled region in the foreground scene;
Associating brightness in the foreground scene with brightness in the AOI;
The method of claim 1 further comprising: The generating step includes
Further comprising considering a chromaticity signal in the AOI.
The method of claim 1. The step of considering includes
Determining whether a constant vector applies to all pixels in the AOI;
Applying a small percentage of the constant vector to the resulting foreground scene when a constant vector is applied to all pixels;
The method of claim 4 further comprising: The step of considering includes
Determining whether a constant vector applies to all pixels in the AOI;
Not applying a change to the resulting foreground scene when a constant vector is not applied to all pixels;
The method of claim 4 further comprising: The small percentage includes 5-10%,
The method of claim 5. A device for generating a chroma key,
Means to determine the area of interest (AOI) for the composite scene using the foreground keys;
Means for generating a chroma key taking into account the physical characteristics of the pixels in the determined AOI;
A device characterized by comprising: Means for overlaying the foreground keys on the background scene;
Means for identifying pixels in the background scene included in the foreground key;
9. The apparatus of claim 8, further comprising: Means for calculating the average luminance of the pixels in the AOI;
Means for calculating an average brightness of a sampled region in the foreground scene;
Means for associating luminance in the foreground scene with luminance in the AOI;
9. The apparatus of claim 8, further comprising: The means for generating is
Further comprising means for taking into account the chromaticity signal in the AOI;
The apparatus of claim 8. Said means for considering are:
Means for determining whether a constant vector is applied to all pixels in the AOI;
Means for applying a small percentage of the constant vector to the resulting foreground scene when it is determined that the constant vector is applied to all pixels;
The apparatus of claim 11 comprising: Said means for considering are:
Means for determining whether a constant vector is applied to all pixels in the AOI;
Means for not changing the resulting foreground scene when a constant vector is not applied to all pixels;
The apparatus of claim 11 comprising: The small percentage includes 5-10%,
The apparatus of claim 11. A device for generating a chroma key,
Using a foreground key to determine an area of interest (AOI) for a composite scene and having a source selection device that generates chroma keys in consideration of the physical characteristics of the pixels in the determined AOI;
A device characterized by that. The source selection device is:
Multiple input sources,
A processor;
Switching logic in communication with the processor;
An adaptive chroma key subsystem connected to the processor and the switching logic, the adaptive chroma key subsystem operating under control of the processor to selectively combine two or more input sources for the composite scene; ,
16. The apparatus of claim 15, comprising: The physical characteristics of the pixel include a luminance signal and a chromaticity signal.
The apparatus of claim 15. The source selection device calculates an average luminance of the pixels in the AOI, calculates an average luminance of a sampled region in the foreground scene, and links data that associates the luminance in the foreground scene with the luminance in the AOI. Generating the chroma key by supplying,
The apparatus of claim 17. The source selection device determines whether a constant vector is applied to all pixels in the AOI, and when a constant vector is applied to all pixels in the AOI, a small percentage of the constant vector is applied. Applying to the resulting foreground scene in the composite scene;
The apparatus of claim 17.

Images