JP3366023B2 - Digital video special effects apparatus and method - Google Patents

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to the field of digital video special effects processing. More particularly, this invention relates to a video special effects device of the type that produces an output image that includes a background produced by a background color generator and an input image processed.

2. Description of the Related Art As a digital image special effect device,
Takes in an input image, processes the input image, and creates an output image consisting of a partially processed input image and a background that partially fills the output image portion not occupied by the processed input image. Things are known.

As this processing (special effect processing), scaling (reduction or enlargement of the image, etc.), rotation (rotation of the target image), clipping (cutting of the image) or a processed input image is a small part of the output image. There are other processes to include in.

The material to be included in the background must be defined in some way. One simple way to deal with this is to allow the user to select a color that should fill the background.

[0004] This color is a technical reason such as the aesthetic reason that the background color matches the input image, or whether the background from both inside and outside the input image needs to be detoned. It could be selected for a reason.

[0005]

However, there are problems with this simple method. The background of the displayed input image may appear to the viewer in a stable color, but in reality it is a small change in the lighting effect of the image and / or
Fluctuations in cameras and other devices cause small fluctuations in the background color.

No background color produced by the background color generator exhibits the same change in background color of the input image. If all that was displayed was the input image, such changes would not be particularly noticeable.
However, although the change is small, when the backgrounds from the two sources are juxtaposed, there is a problem that an annoying effect is produced particularly when the backgrounds are used for tone removal described later.

There is also a problem that noise in the color determination information of the sample position causes an unnecessary change in the color generated by the background color generator.

In image processing, another problem is how to process a large amount of image data originally at a sufficient speed.

It is an object of the present invention to provide a digital image special effect device and method capable of eliminating these problems.

SUMMARY OF THE INVENTION The present invention processes an input image in cooperation with background generation means and produces an output image containing background information generated by the background generation means and a processed version of the input image. The means for generating, the background generating means,
The color determination information is read from the sample portion of the input image, and the background color is adjusted to match the color of the sample portion of the input image using the read color determination information.

The present invention also includes the steps of processing an input image and generating a background to generate an output image including the background obtained by processing the input image and the generated background. This involves reading the color determination information from the sample portion of the image and using the read color determination information to adjust the background color to match the color of the sample portion of the input image.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the digital video special effect apparatus and method of the present invention will be described in detail below with reference to FIGS.

FIG. 1 shows a single background 6
And the input image 2 composed of the figure 4 of the person above this. This input image 2 is rotated 90 ° clockwise as shown in image 8 by an operation. Since the input image 2 is not square, the processed portion of the input image 2 does not completely fit in the output image. Therefore, the background 10 must be added to both ends of the processed input image 9. This background 1
The requirement for adding zeros is determined by the digital video special effects device, as is well known. The material displayed in the background 10 is generated by a background color generator.

The following final output image 12 shows that the color of the background 10 is adjusted to a predetermined color by the background color generator. The operator can select this background color for each frame or for each reference time. The predetermined color can be selected to match the color of the background 6 in the input image 2.

As described above, the background 6 of the input image 2 is
Subject to slight variations in color due to lighting effects, or the like. As a result, there will be a slight color imbalance between the predetermined color of the background 10 and the background of what the input image was processed 9. Arranging the materials from these two sources will accentuate this slight color imbalance. This is indicated by a broken line in the output image 12.

FIG. 2 illustrates a sample portion 16 of the input image 14.
3 illustrates the operation of the system used to control the background color for which the color determination information read from is generated. The input image 14 undergoes the same processing as in FIG.
Resulting in a similar need for the occurrence of In this case, the color of the background 20 of the input image 14 is the sample portion 16
Read from inside.

The system presets a fixed number of alternative sample portions 22 that the user can switch to when the currently active (active) sample portion 16 is or is about to be infested with non-background material. You can keep it.

The background color generator sets the background color 20 of the input image 14.
Is dynamically tracked so that a substantially exact color balance between the background 18 and the processed input image 17 is maintained in the final output image 24. In contrast to the output image 12 of FIG.
The output image 24 is not lined with areas of slightly different colors.

FIG. 3 illustrates the process of using the output image generated according to FIG. 1 for subsequent tone removal. The output image 26 includes a block 28 located in the background consisting of the background 30 of the input image and the background 32 generated by the background color generator. As mentioned above, there may be a slight color imbalance between the different backgrounds indicated by the dashed lines.

Tonal removal is the act of removing the background color from an image by passing the image through a very narrowly adjusted color filter. The filter response is made as narrow as possible to alleviate the problem that the filter also removes color from within block 28. Therefore, for slight color imbalances between different background portions, the filter cannot remove both areas 30 and 32 all at the same time. As a result, in the removed image 33, the background 30 is removed, but the background 32 is not completely removed. Image 33 that has been removed
Includes the desired block 28 with opposite side areas 34 having an undesired light shade.

The stripped image 33 is then added to the second image 36 to produce a composite output image 38. Thus, the slight color imbalance in the background 32 generated by the background color generator will eventually result in unwanted color changes on both sides of the composite image 38.

FIG. 4 shows tonal removal with an image generated according to FIG. Briefly, the background is consistent in color as a result of its removal in its entirety, resulting in no undesired shade corresponding to the background produced by the background color generator.

FIG. 5 shows a system for background color control used by a background color generator. The control system shown in FIG. 5 relates to one of RGB components obtained by separating RGB signals. Thus the entire system contains three such devices. The individual component input video signals are supplied from the input video circuit 40 to the minimum / maximum value detection circuit 42 in the form of a serial data stream. The minimum value / maximum value detection circuit 42 samples the input data stream and detects the extreme value of the color component of the data stream portion corresponding to the sample in the input image. The extreme values of the detected color components are averaged by the averaging circuit 44, and this average value is supplied to the comparing circuit 46. In this comparison circuit 46,
The color determination information is compared with that of the immediately preceding image in the video sequence held in the latch circuit 48.

Minimum / maximum value detection circuit 42, averaging circuit 4
It is important that 4 and the comparison circuit 46 respond only to specified sample portions in the input image. This is guaranteed by the timing generator 50. The user uses the sample controller 52 to select the currently active sample portion. This information is stored in the sample position / size circuit 54.
Is converted into data representing the raster line number and sample number of the currently active sample. The timing generator 50 converts the output of the sample position / magnitude circuit 54 into a timing signal, and supplies this timing signal to the minimum / maximum value detection circuit 42, the averaging circuit 44 and the comparison circuit 46, respectively.

Since the input stream of serial data is in the form of a raster-scanned image, and the sample portion is in the upper left corner of the input image in this case, the timing generator 50 causes the first few lines of the image frame to fall into the first few lines. The timing signal is applied during the first few pixels in each line. This timing signal causes the minimum / maximum value detection circuit 42, the averaging circuit 44, and the comparison circuit 46 to be differential at the appropriate number of times required to sample the upper left corner of the image.

The output from the comparison circuit 46 is supplied to an error limiter 56 which performs attenuation control. The error limiter 56 can perform a certain number of damping operations. The function of the error limiter 56 is to suppress excessive reaction to transient changes in the color decision information within the sample portion. The various damping operations that can be used are described below with reference to FIG.
The output from the error limiter 56 matches changes in the background color component suitable for tracking changes in the background color component within the sample portion.

The adder circuit 58 adds this change to the current color component value supplied from the background color generator 60, and supplies this value to the latch circuit 48 to hold it. The background color generator 60 then applies the value held in the latch circuit 48 to the background that needs to be created.

It is convenient to use software to control the general purpose computer in broken line section 62 and to use hardware in broken line section 64. The hardware-based section 64 enables quick analysis of high-frequency input data, while the software-implemented section 62 relatively easily performs background color generation and attenuation control that does not require high-speed operation. I am considering what I can do.

FIG. 6 illustrates the various damping operations used by the error limiter 56. For convenience of explanation, it is assumed that the sample values of the color components within the sample portion undergo a step change at time T0. Any change of the sampled value to a new level V'less than the preset threshold level change VT will be made immediately. A change larger than the threshold level VT is often due to noise.
Will receive one.

In the damping operation B, the error limiter is in the period T.
It does not respond to any changes during d, but after this it responds completely. Therefore, a certain gradual change occurs in the period Td.
If it is a temporary glitch that has happened inside, there is no unwanted change in the background color. In the decay operation shown in FIG. 6C, the error limiter 56 responds immediately to changes, but only allows a certain maximum change in the background color between each frame of the image being generated. This is shown by the maximum slope M of the braking response graph following time T0. This action C
Then, a glitch on the input will cause a relatively small spurious change in the background color.

The damping operation shown in FIG. 6D is the same as in FIGS. 6B and 6C.
It is a combination of the actions of. There is no response during the period Td, and the subsequent response undergoes the maximum inter-frame change corresponding to the slope M.

FIG. 7 shows the decay response to a glitch input into the blue component of the video signal. Strong or weak, damped response is shown.

[0032]

As described above, in the present invention, the background color generator 60 reads the color determination information from the sample portion of the input image, and the read color determination information is used to find the color of the sample portion of the input image. Since the background color is adjusted so as to match, a good image with no color imbalance can be obtained, and image deterioration due to the inclusion of noise in the color determination information can be prevented.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a series of steps using a predetermined color for a background.

FIG. 2 is an explanatory diagram showing a series of steps using a sample of an input image for controlling a background color.

FIG. 3 is an explanatory diagram showing a problem when a color tone removing process is used.

FIG. 4 is an explanatory diagram showing an improved color tone removal process.

FIG. 5 is a block diagram showing an embodiment of a digital video special effect device and method according to the present invention.

FIG. 6 is a waveform diagram showing various examples of a damping operation.

FIG. 7 is a waveform diagram showing an example of an attenuation level added to input noise.

[Explanation of symbols]

14 Input image 16 sample parts 17 Processed input image 18 background 24 Output image 42 Minimum / maximum value detection circuit 44 averaging circuit 52 Sample controller 56 Error limiter 60 background color generator

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-61-289386 (JP, A) JP-A-64-36285 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 9/74 H04N 5/272

Claims (10)

(57) [Claims] 1. A means for generating an input image subjected to special effect processing in a part of an output image area and an output image having a background for the partial area generated by a background generating means in another part. The background generation means reads color determination information for determining the color of the background inside the input image from the sample portion set in the background portion inside the input image before the special effect processing is performed, and preset. A digital video special effect device, wherein the background color for the partial area is dynamically adjusted to match the color of the sample area. 2. The digital image special effect device according to claim 1, further comprising color change attenuating means for limiting a response of the background generating means to a change in the read color determination information. . 3. The digital video special effect according to claim 2, wherein the color change attenuation means suppresses a response to a change in the color determination information read by the background generation means within a predetermined time. apparatus. 4. The color change attenuating means limits the rate of change of the background color with respect to the partial area adjusted by the background generating means to a predetermined maximum rate. 3. The digital video special effect device according to any one of 3 above. 5. The digital video special effect apparatus according to claim 1, further comprising means for changing a sample portion set in a background portion inside the input image before the special effect processing is performed. 6. The means for changing the sample part selectively switches, in response to a user input, to another part which is set in advance as a sample part inside the input image before the special effect processing is performed. 6. The digital video special effect device according to claim 5, wherein: 7. The background generation means detects the extreme value of the read color determination information and adjusts the background color for the partial area according to the average of the extreme values. The digital video special effect device according to claim 1. 8. The background generation means reads the color determination information from a passing stream of a serial data string representing an input image before the special effect processing is performed. Digital video special effect device. 9. The digital video special effect device according to claim 1, wherein the sample portion is at a position in the input image corresponding to a tone-removed background area of the input image. 10. A process of generating an input image subjected to special effect processing in a part of the output image area and an output image having a background for the partial area generated by a background generating means in the other part. In the processing step, the color determination information for determining the color of the background inside the input image is read from the sample portion set in the background portion inside the input image before the special effect processing is performed, and preset. A digital image special effect method comprising dynamically adjusting a background color for the selected partial area to match a color of the sample portion.