JPH0632445B2 - Background separation device for image signals - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to a background separation device for image signals, and more particularly to separating a background portion from an image signal for displaying a subject in a predetermined background. Background separation device.

[Problems to be Solved by the Related Art and Invention] In the field of image processing, only the image of a target object is cut out from the image signal, and the object is measured or recognized, or combined with another image signal. It may happen.
BACKGROUND ART Conventionally, a chroma key method is known as a method of removing a background portion of a color image and outputting only an image of a target object. In the chroma key system, a screen of a specific color is prepared, and the target object is photographed against the screen. By removing the signal component corresponding to the specific color from the captured video signal, the background video is erased, and a video signal displaying only the target object can be obtained. However, in this chroma key method, as described above,
It is necessary to prepare a dedicated background screen to which a specific color is applied prior to shooting, which has a disadvantage of increasing the size of equipment for that purpose.

On the other hand, as a method that does not require a background screen, a video of only the background that does not include the target object is stored in advance as a background image, and the background image and the image including the target object are compared to determine a portion where the two match. There is a method of removing it with the background. In this method, there are the following two methods for comparing the above two images. In the first method, the color video signals are R (red), G (green) and B.
It is expressed by RGB expression consisting of three components of (blue), and comparison is performed based on the value of each signal component. In the second method, each R, G, and B signal component is changed into an HSV expression composed of three components of H (hue), S (saturation), and V (lightness), and the value of each component is changed. The two are compared based on each other.

However, in the first method, since the three primary colors are used as they are, when the brightness of the background changes due to a change in illumination or a shadow of the target object, the value of each color signal component also largely changes. Therefore, there is a drawback in that it is not possible to recognize the part where the brightness has changed as the target object and perform accurate background removal. Further, in the second method, the comparison is performed using the hue component representing the original color of the target object regardless of the shadow. Error is added in the portion where the color intensity is low when converting from the to the HSV representation. As a result, the error becomes noise, which causes a drawback that an accurate comparison cannot be performed.

The present invention has been made to solve the above problems, and it is an object of the present invention to provide a background separating device capable of accurately separating a background portion from an image signal displaying a subject placed in the background. To aim.

[Means for Solving the Problem] A background separation device according to the present invention includes a background storage unit that stores a background image signal that displays only a background and a first image signal that displays a subject in the background. In response to the color intensity indicated by the first image signal, the first image signal is separated into a second image signal indicating a high color image region and a third image signal indicating a low color image region. Area separating means, first difference detecting means for detecting a difference between the separated second image signal and the stored background image signal, and the separated third image signal and the stored background image signal Second to detect the difference with
Difference detecting means and the determining means for determining the background partial signal in the first image signal in response to the difference signals generated from the first and second difference detecting means, and for responding to the determination result by the determining means. And a background separating means for separating the background partial signal from the first image signal. In the background separating device according to the invention of claim 1, the first difference detecting means converts the red, green and blue color signals included in the second image signal into respective signals of hue, saturation and lightness. A second converting means for converting the red, green and blue color signals contained in the background image signal into respective signals of hue, saturation and lightness; and first and second converting means.
A first distance calculating means for calculating the distance for each pixel between the signals converted by the converting means.

In the background separating device according to the invention of claim 2, the second difference detecting means includes second distance calculating means for calculating the distance of each pixel between the third image signal and the background image signal.

[Operation] In the background separating device according to the present invention, the first image signal for displaying the subject in the background is divided by the area separating means into the second image signal indicating the high color image area and the third image signal indicating the low color image area. It is separated into image signals. Since the difference from the background image signal is detected for each of the separated second and third image signals, the difference can be accurately detected. In the background separating device according to the first aspect of the invention, the first converting means converts the red, green and blue color signals included in the second image signal into respective signals of hue, saturation and lightness. The second conversion means converts the red, green, and blue color signals included in the background image signal into hue, saturation, and lightness signals. The first distance calculation means calculates a distance for each pixel between the signals converted by the first and second conversion means,
The calculation result is output as a difference signal output from the first difference detecting means.

In the background separating device according to the invention of claim 2, the second distance calculating means calculates the distance of each pixel between the third image signal and the background image signal, and the calculation result is output from the second difference detecting means. Output as a difference signal. Therefore, the background partial signal can be accurately determined from the first image signal based on the obtained difference signal. As a result, the background separation is accurately performed.

[Embodiment of the Invention] FIG. 1 is a block diagram of a background removing apparatus showing an embodiment of the present invention. With reference to FIG. 1, this background separation device receives an input unit 1 which receives a color video signal S1 and performs signal processing necessary for a subsequent arithmetic process, and digital image signals S2 and S3 generated from input 1. An arithmetic processing unit 10 for receiving and performing arithmetic processing is included. The arithmetic processing unit 10 includes a switching unit 2 that selects the image signals S2 and S3 generated from the input unit 1. The background storage unit 3 receives the image signal S2 for displaying only the background not including the target object via the switching unit 2 and stores it. The image area separating unit 4 and the background removing unit 9 are configured to transmit the image signal S3 including the target object via the switching unit 2.
Receive. The image area separation unit 4 separates the image signal S3 including the target object into an image signal S4 of a high color area and an image signal S5 of a low color area. The high color area difference signal detection unit 5 receives the image signal S4 and the background image signal S6, and detects the difference signal S7 for each pixel. The low color area difference signal detection unit 7 receives the image signal S5 and the background image signal S6 and outputs a difference signal S8. High color area background determination unit 6
Determines based on the difference signal S7 whether or not each pixel displays a background portion. As a result of this determination, the mask signal S9 that defines the mask for background removal of the high color area
Is output. Similarly, the low color area background determination unit 8 determines whether or not each pixel displays the background portion based on the difference signal S8. Based on the determination result, the mask signal S10 that defines the mask for background removal in the low color area is output. The background removing unit 9 receives the image signal S3 including the target object and the mask signals S9 and S10, and outputs the image signal S11 from which the background portion is removed from the image signal S3.

FIG. 2 is a block diagram of the high color area difference signal detection unit 5 shown in FIG. With reference to FIG. 2, this difference signal detection unit 5 converts a background image signal S6 into an HSV representation in the same way as a conversion unit 51 that converts the RGB-represented image signal S4 in the high color region into an HSV representation. Converter 52 and converter 5
1 and 52, and a distance calculator 53 for calculating the distance for each pixel with respect to the signals S4 'and S6'.

Next, the processing in the background removing apparatus shown in FIG. 1 will be specifically described. First, the input unit 1 receives the color video signal S1. Normally, a video signal forming one screen is divided by frames, and a temporally continuous image can be obtained by updating the frame. However, in the following description, in order to simplify the description, The processing of the image signal of one screen, that is, one frame will be described. The input unit 1 is provided with a video signal S for a given one screen.
1 is separated into R, G, and B color signal components. The separated color signal components are A / D converted to generate three image matrices Rhv, Ghv and Bhv for each R, G and B. Each image matrix has H in the horizontal direction.
Pixel, each color signal pixel element indicating V pixel (H and V are positive integers) in the vertical direction. The input section 1 is alternately supplied with a video signal of only the background not including the target object and a video signal of the target object. Therefore, the input unit 1
Image matrices BRhv, BGhv displaying only the background
And BBhv (these are indicated by the signal S2 in FIG. 1) and the image matrix Rhv of the image containing the target object,
Ghv and Bhv (these are shown by the signal S3 in FIG. 1) are output.

First, the switching unit 2 is connected to the terminal B side, and the background image matrix signal S2 is stored in the background storage unit 3. next,
The switching unit 2 is connected to the terminal A side, and the image matrix signal S3 including the target object is given to the image region separating unit 4 and the background removing unit 9.

In the image area separating unit 4, the given image matrix Rh
The following processing is performed for each pixel for v, Ghv, and Bhv.

However, Dhv = max (Rhv, Ghv, Bhv) -min
(Rhv, Ghv, Bhv) (7) max (Rhv, Ghv, Bhv): maximum value of each color component for each element (h, v) of the image matrix min (Rhv, Ghv, Bhv): each of the image matrix Minimum value of each color component for element (h, v) Here, Dhv defined by the equation (7) indicates the intensity of color for each pixel. Further, T represents a predetermined threshold value, and in this embodiment, T = 10 is set. Therefore, for each image matrix Rhv, Ghv and Bhv, by applying the equations (1), (2) and (3), respectively, the high color image matrix HR can be obtained.
hv, HGhv and HBhv are generated. In the high-color image matrix, the elements that do not correspond to the high-color area in each element of the image matrix are replaced with the value "-1" indicating the outside of the area. Similarly, by applying the equations (4), (5) and (6), the low color image matrix L
Rhv, LGhv and HBhv are generated. That is, the image area separation unit 4 separates the supplied image matrix signal S3 into a high color image matrix signal S4 and a low color image matrix signal S5, and generates the high color image matrix signal S5.

FIG. 3 is a matrix diagram showing an example of a high-color image matrix generated in the image area separating section 4. Third
The matrix as shown in the figure is generated for each of R, G and B. As shown in FIG. 3, the high color image matrix shows the high color area D1 and the area D2 outside the area.

FIG. 4 is a matrix diagram showing an example of a low color image matrix. This matrix is also generated for each R, G and B. As shown in FIG. 4, a low color area D3,
Outside that area D4 is shown.

The high color area difference signal detection unit 5 receives the high color image matrix signal S4 and the background image tomalix signal S6. The high color image matrix signal S4 is the matrix HRhv, H
Includes Ghv and HBhv. On the other hand, the background image signal S6
Contains the matrices BRhv, BGhv and BBhv.

The conversion unit 51 shown in FIG. 2 converts the high color image matrix signal S4 into HSV representation. Similarly, the conversion unit 52
Converts the background image matrix signal S6 into an HSV representation. That is, in each converter 51 and 52, RG
The image matrix represented by B is converted into a matrix showing H (hue), S (saturation) and V (lightness) components. A conversion formula for that purpose will be described below.

For example, if the R, G, and B component values of one pixel are r, g, and b, respectively, then H for that pixel is
Each value h, s, v of the SV expression is obtained by the following equation.

However, r ′ = (v−r) / (v−min (r, g, b)) ...
(11) g '= (v-g) / (v-min (r, g, b)) ...
(12) b ′ = (v−b) / (v−min (r, g, b)) ...
(13) The above conversion formula is, for example, “COLOR GAMUT
"TRANSFORM PAIRS" (Computer Graphics 12, 3 1
August 1978, pages 12-19).

By applying the above equations (8) to (13) to the high color image matrix and the background image matrix, the following hue value, saturation value and lightness value are obtained as the elements of each matrix.

H1hv: Hue value of high color matrix element (h, v) H2hv: Hue value of background image matrix element (h, v) S1hv: Saturation value of high color image matrix element (h, v) S2hv: Background Saturation value of image matrix element (h, v) V1hv: lightness value of high color image matrix element (h, v) V2hv: lightness value of background image matrix element (h, v) (14) HSV representation The high color image difference signal matrix HEhv is generated by calculating the following equation for each element (that is, each pixel) of the generated high color image matrix and background image matrix.

That is, in the distance calculation unit 53 shown in FIG.
The distance is calculated for each pixel represented by HSV in 5), and a matrix indicating the distance, that is, a high color image difference signal matrix HEhv is generated. This matrix HEhv is given to the high color area background determination unit 6 shown in FIG. 1 as a signal S7. An example of a high color pixel difference signal matrix is shown in FIG.

On the other hand, the low color area difference signal detection unit 7 performs the following processing in parallel with the processing in the high color difference signal detection unit 5. The low color area difference signal detection unit 7 receives the low color image matrix signal S5 expressed in RGB and the background image matrix signal S6, and compares the distances of corresponding pixels based on the signal components expressed in RGB. That is, RG
When the B-represented low color image signal is converted into the HSV representation, an error in the conversion is large, so that the difference signal cannot be accurately detected as in the high color area difference signal detection unit 5 described above. Therefore, the low color region difference signal detection unit 7 detects the difference signal in the RGB representation without converting it into the HSV representation. Specifically, the low color image difference signal matrix Ehv is generated by performing the calculation according to the following equation.

The low color image difference signal matrix LEhv generated by the equation (16) is given to the low color area background determination unit 8 shown in FIG. 1 as the difference signal S8. An example of the low color image difference signal matrix LEhv is shown in FIG.

The high color area background determination unit 6 determines the high color difference signal matrix H.
Upon receiving Ehv, it is determined by the following calculation formula whether or not each element of the matrix should belong to the background, and the high color background area mask matrix HMhv is generated.

In the matrix HMhv, the element value “1” indicates that the pixel corresponding to the element belongs to the background, and the value “0”.
Indicates that it does not belong to the background. Here, TH is a predetermined threshold value, and TH = 10 is set in this embodiment.

Similarly, the low color background determination unit 8 determines whether or not each element in the matrix should belong to the background by performing the calculation by the following formula for the low color difference signal matrix LEhv, and A background area mask matrix LMhv is generated.

In the matrix LMhv, an element value “1” indicates that the pixel belongs to the background, and a value “0” indicates that the pixel does not belong to the background. Incidentally, TL is a predetermined threshold value, and in this embodiment, TL = 10 is set.

Examples of the matrices HMhv and LMhv generated in the background determination units 6 and 8 are shown in FIGS. 7 and 8.
As shown in the figure. Signals S9 and S10 indicating the matrices HMhv and LMhv are provided to the background removal unit 9.

The background removing unit 9 shown in FIG. 1 includes image matrices Rhv, Ghv and Bhv (signal S
3), the high color background area mask matrix HMhv (signal S9), and the low color background area mask matrix LMhv
(Signal S10) is received. In the background removing unit 9, among the elements of the mask matrices HMhv and LMhv, when the value of the element is “0”, it is determined that the corresponding pixel displays the target object, and each matrix Rhv of the corresponding pixel, The value of each element of Ghv and Bhv is left as it is. On the other hand, when the value of each element is "1", the value of the corresponding element of the image matrices Rhv, Ghv and Bhv is replaced with a value indicating no signal, for example, the value "0".
By the above process, the background removal unit 9 replaces the value of the element determined to indicate the background portion of the elements of the image matrices Rhv, Ghv, and Bhv with "0", and thus the background portion is removed. It has been done.

The signal S11 output from the background removal unit 9 is converted into an analog video signal by a D / A converter (not shown), and an image of the target object from which the background portion is removed can be obtained.

As described above, in the background removing apparatus shown in FIG. 1, the background is detected by dividing the image into the two regions of high color and low color, and the background detection method suitable for each characteristic is applied. Therefore, the background image can be removed with high accuracy. Moreover, it is not necessary to use dedicated background equipment, and processing can be performed in any background. In addition to this, since the image processing of the high color area and the image processing of the low color area are performed in parallel, the background can be removed at high speed.

[Effects of the Invention] As described above, according to the present invention, the area separation means for separating the image signal displaying the subject in the background into the image signal of the high color image area and the image signal of the low color image area is provided. Since it is provided, an error is unlikely to occur in detecting the difference from the background image signal, and a background separating device capable of accurately separating the background portion is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram of a background removing apparatus showing an embodiment of the present invention. FIG. 2 is a block diagram of the high color area difference signal detection unit shown in FIG. FIG. 3 is a matrix diagram showing an example of a high color image matrix. FIG. 4 is a matrix diagram showing an example of a low color image matrix. FIG. 5 is a matrix diagram showing an example of a high color image difference signal matrix. FIG. 6 is a matrix diagram showing an example of a low color image difference signal matrix. FIG. 7 is a matrix diagram showing an example of a high color background area mask matrix. FIG. 8 is a matrix diagram showing an example of a low color background area mask matrix. In the figure, 1 is an input unit, 2 is a switching unit, 3 is a background storage unit, 4 is an image region separation unit, 5 is a high color region difference signal detection unit, 6 is a high color region background determination unit, and 7 is a low color region. A difference signal detector, 8 a low color area background determiner, 9 a background remover,
Reference numeral 10 is an arithmetic processing unit.

Continuation of the front page (56) Reference JP-A-60-190078 (JP, A) JP-A-1-228288 (JP, A) JP-A-2-127886 (JP, A) JP-A-3-241993 (JP , A)

Claims (2)

[Claims] 1. A background separation device for separating a background portion signal for displaying the background portion from a first image signal for displaying an object in a predetermined background, wherein the subject is removed. Background storing means for storing a background image signal for displaying the background, and a first image signal for receiving the first image signal and responding to the intensity of the color indicated by the first image signal, the first image signal is converted into a high color image area. Area separating means for separating into a second image signal indicating a low color image area and a third image signal indicating a low color image area, a second image signal separated by the area separating means, and stored by the background storing means. First difference detecting means for detecting a difference from the background image signal, and second difference detecting means for detecting a difference between the third image signal separated by the area separating means and the background image signal stored by the background storing means. Difference detection means of In response to the difference signal generated from the first and second difference detecting means, in response to the determination means for determining the background partial signal in the first image signal, and in response to the determination result by the determination means. A background separating means for separating the background partial signal from the first image signal, each of the second image signal and the background image signal including each color signal of red, green and blue, Difference detecting means receives the second image signal, and converts the red, green and blue color signals included in the second image signal into respective signals of hue, saturation and lightness; Receiving the background image signal, red contained in the background image signal,
Second conversion means for converting the green and blue color signals into respective signals of hue, saturation and lightness, and calculating a distance for each pixel between the signals respectively converted by the first and second conversion means. A background separation device, comprising: a first distance calculation means. 2. A background separation device for separating a background portion signal for displaying the background portion from a first image signal for displaying an object in a predetermined background, wherein the subject is removed. Background storing means for storing a background image signal for displaying the background, and a first image signal for receiving the first image signal and responding to the intensity of the color indicated by the first image signal, the first image signal is converted into a high color image area. Area separating means for separating into a second image signal indicating a low color image area and a third image signal indicating a low color image area, a second image signal separated by the area separating means, and stored by the background storing means. First difference detecting means for detecting a difference from the background image signal, and second difference detecting means for detecting a difference between the third image signal separated by the area separating means and the background image signal stored by the background storing means. Difference detection means of In response to the difference signal generated from the first and second difference detecting means, in response to the determination means for determining the background partial signal in the first image signal, and in response to the determination result by the determination means. A background separating means for separating the background portion signal from the first image signal, each of the third image signal and the background image signal including each color signal of red, green and blue, The background detecting device, wherein the difference detecting means includes second distance calculating means for receiving the third image signal and the background image signal and calculating a distance for each pixel between the third image signal and the background image signal.