JPH0634232B2 - Video input device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a video input device, and in particular, when an image of an object is input using a photographing device, only an image of a specific object is input from the input image. The present invention relates to a video input device for extracting and inputting.

[Problems to be Solved by the Related Art and Invention] Conventionally, there is a chroma key method as a method for extracting and inputting only an image of a specific object from an input video imaged by an imaging device. In this chroma key method, a screen of a specific color is prepared, the target object is photographed with the screen as a background, and the background image is erased by removing the signal corresponding to the specific color from the input video signal, thereby removing the target object. The video of is being extracted. However, in the chroma key method, as described above, it is necessary to prepare a dedicated screen facility to which a specific color is applied at the time of input, and further it is difficult to extract when the extraction target object has the same color as the background screen. There is.

On the other hand, as another conventional method, there is a method of detecting a difference between two consecutive frames of an input image and extracting an image of only a portion where the difference occurs. This conventional method does not require a special background like the chroma key method. However, if the extraction target object does not move in place over time, not only will it not be detected as a difference between frames, but if a change occurs in the background part, this will also be detected as a difference, It has the drawback of being confused with the target and being extracted.

Therefore, a main object of the present invention is to provide a video input device which can solve the above-mentioned drawbacks and can extract and input only a video of a specific object from an input video.

[Means for Solving the Problems] The present invention is a video input device, which is provided with at least first and second photographing means which are arranged at regular intervals and simultaneously photograph an object, and these photographing means. Image storage means provided corresponding to each of the first and second image pickup means, which stores the background image of the object viewed in advance, and the image pickup means provided corresponding to each of the image pickup means and photographed by the corresponding image pickup means. In the screen of the difference detection means for comparing the image of the captured object with the background image read out from the image storage means and detecting the difference occurrence portion, and the difference occurrence portion detected by each difference detection means Parallax detection means for detecting the position of each of the parallaxes of each input image from each photographing means and detecting, and labeling according to whether or not the detected position is within a predetermined threshold value. Shooting method A distance detection unit that detects the distance from the object to the object, and an extraction unit that extracts an image of the object at a predetermined distance from each of the detected imaging units.

[Operation] The video input device according to the present invention detects a portion where a difference occurs between an image that does not include an object to be input and an image that includes the object, and the detected portion of the difference within the screen is displayed. Labeled according to whether or not the position detected by comparing the position at the parallax of each input image is within a predetermined threshold, and detecting the distance from the shooting position to the object,
Only the image of the object at a specific distance from the shooting position is extracted.

Embodiment of the Invention FIG. 1 is a schematic block diagram of an embodiment of the present invention.
FIG. 2 is a diagram schematically showing the positional relationship among the photographing device, the target object, and the background in one embodiment of the present invention.

First, the configuration of an embodiment of the present invention will be described with reference to FIGS. The image input is performed by using two or three or more image pickup devices arranged in parallel to the object to be photographed at regular intervals, but here, in order to simplify the description, it is shown in FIG. As described above, it is assumed that the input target object 2 is set in front of the background 1 and two image capturing devices 3 and 4 are provided so as to face the background 1 and are arranged in parallel in the left and right directions. The video signal output by the right photographing device 3 is input to the right input unit 11, and the video signal output by the left photographing device 4 is given to the left input unit 12.

Here, the video signal is a luminance signal in the case of a monochrome image, and an R (red) component, a G (green) component in the case of a color image,
It is a luminance signal of B (blue) component, or Y (luminance signal) component, I, Q (chromaticity signal) component in the NTSC system. In the following description, for the sake of convenience, only the case of a monochrome image will be described, but also in the case of a color image, the operation in the monochrome image will be described for each component of the color image.
Since it can be realized by applying them in parallel or repeatedly, it does not matter whether the image is a monochrome image or a color image.

The input units 11 and 12 perform the same operation, and the photographing device 3,
The video signal output from the reference numeral 4 is converted into a video signal given in digital quantity for each pixel in a matrix of horizontal H pixels and vertical V pixels (H and V are positive integers), and the changeover switch It outputs to 21 and 22. The changeover switch 21 outputs the output signal of the input unit 11 to the background storage unit 31.
Alternatively, the changeover switch 22 outputs by switching to the difference detection unit 41 and the extraction unit 71, and the changeover switch 22 outputs the output of the input unit 12 to the background accumulation unit 32 or the difference detection unit 42 and the extraction unit 7.
It is for switching to 2 and outputting.

The background accumulating units 31 and 32 store the background image 1 shown in FIG. 2 in advance. The difference detection units 41 and 42 read out one frame of the background image from the background storage units 31 and 32, respectively, and compare the frame with the frame of the input image output from the input units 11 and 12 to detect the difference. is there. The respective detection outputs of the difference detection units 41 and 42 are given to the parallax detection unit 5. The parallax detection unit 5 is the difference detection unit 4
The positions of the difference regions in the frame are compared with each other from the two difference matrices sent from 1 and 42 to detect the parallax, and the detection result is given to the mask generation unit 6. The mask generation unit 6 creates image data to be masked based on the detection output of the parallax detection unit 5. The output of the mask generation unit 6 is given to the extraction units 71 and 72. Extraction unit 7
Reference numerals 1 and 72 denote the ones of the video signals supplied from the input units 11 and 12, respectively, which are masked with the image data from the mask generation unit 6 and output the necessary video signals.

FIG. 3 is a diagram showing a difference matrix output from the difference detection unit, FIG. 4 is a diagram for explaining a difference region, and FIG. 5 is a comparison result list created by the parallax detection unit 5. FIG. 6 is a diagram showing an example of a difference matrix switched by the mask generator.

Next, a more specific operation of the embodiment of the present invention will be described with reference to FIGS. First, prior to the input of the target object 2, the image of the background 1 not including the target object 2 is input. At this time, the changeover switch 21,
22 is switched to the A terminal side. Input section 11, 12
The video signal output from is supplied to the background storage units 31 and 32. The background accumulators 31 and 32 store one video signal each.
Accumulate for frames. The accumulated frame may be the first one frame sent from the input units 11 and 12, or one frame generated by averaging several frames.

Next, the changeover switches 21 and 22 are tilted to the B terminal side, the target object 2 is set in front of the background 1, and an image is input. The video signals output from the input units 11 and 12 are given to the difference detection units 41 and 42 and the extraction units 71 and 72 with one frame as a break, and temporarily stored. The difference detection units 41 and 42 read one frame of the background image from the background accumulation units 31 and 32, compare this with the frame of the input image, and detect the difference. The difference detection is
This is performed by determining whether the signal values of the pixels at the same position are the same or different between the horizontal video pixel and the vertical video pixel and the background video frame and the input video frame using a known image processing method. The match / mismatch is determined by, for example, determining that the difference between the signal values of the two is less than a given threshold value, and determining that the difference value is not less than the threshold value. That is, it is determined that there is no difference if they match and if there is a difference if they do not match.

The difference detection units 41 and 42 detect the presence or absence of difference for all H pixels × V pixels in one frame of the input video, and generate a value “1” when there is a difference and “0” when there is no difference. A difference matrix is created and output to the parallax detection unit 5. As shown in FIG. 4, the difference matrix is a matrix composed of H × V elements each having a value of 0 or 1 corresponding to the number of pixels in one frame.

The parallax detection unit 5 compares the positions of the difference regions in the frame from the two difference matrices sent from the difference detection units 41 and 42. The position comparison is performed by the following procedure. Here, description will be made with reference to FIG. 4 as an example.

A closed area composed of elements having a value of "1" in the difference matrix is detected as a difference area, and the size of the area (the number of elements included in the area) and the center position (the element at the upper left corner of the matrix is the origin). Let x be the row direction and y be the column direction
It is regarded as an axis and is represented by x, y coordinate values). The center position is obtained by regarding the mass of each element included in the region as 1, and calculating the position of its center of gravity. For example,
The position of the center G ₁₀₁ of the area 101 in FIG. 4 is given by the following equation. ₁₀₁ = (x ₁₀₁ , y ₁₀₁ ) = (1 / N ₁₀₁ ) Σ _{101 101} : Position of element P ₁₀₁ included in area 101 N ₁₀₁ : Number of elements included in area 101 Detected in left and right difference matrix The different areas are matched. This correspondence is performed by comparing the size of the area and the y-coordinate of the center position between the left and right difference areas, and determining the areas having similar values as the corresponding areas. For example, in the example of FIG. 4, it is determined that the areas 101 and 102 and the areas 201 and 202 have a correspondence relationship.

The difference value of the x-coordinate of the center position is calculated as the amount of parallax for each of the difference regions corresponding to the left and right. In the example shown in FIG. 4, the parallax amount d ₁ for the regions 101 and 102 is calculated as d ₁ = x ₁₀₁ −x ₁₀₂ .

The parallax detection unit 5 creates the comparison result list shown in FIG. 5 from the above comparison results, and supplies it to the mask generation unit 6 together with the difference matrix.

The mask generation unit 6 refers to the comparison result list and determines whether or not the parallax amount of each difference area is within a specified threshold value range. Rewrite the value of the element in the matrix to "2". It is generally known that the farther from the shooting position, the smaller the parallax amount, and the closer to the photographing position, the larger the parallax amount. This determination and rewriting operation is nothing but the operation of extracting an object at a specific distance from the shooting position. The mask generation unit 6 gives the rewritten difference matrix as shown in FIG. 6 to the extraction units 71 and 72.

The extraction units 71 and 72 refer to the values of the elements of the difference matrix, and based on the temporarily stored one frame of the input video,
Create a type of video signal. One of them is a pixel of the input image corresponding to an element other than the element value "2" of the difference matrix in the frame of the input image, that is, an input image pixel corresponding to an element having a value "0" or "1". An image in which the signal value of a pixel is replaced with a dummy signal value, the second is a pixel of the input image corresponding to an element in which the value of the element of the difference matrix is other than "1" in the frame of the input image, that is, the value is " It is an image in which the signal values of the pixels of the input image signal corresponding to the elements of "0" and "2" are replaced with dummy signal values. The above-mentioned first video is output as a video which is the processing result of the present invention in which the input target object is extracted.

On the other hand, the second video image is provided to the background storage units 31 and 32. The background accumulating units 31 and 32 combine the sent image and the accumulated image to update the background image. By doing so, even if a portion other than the target object 2 such as the background 1 changes while the image of the target object 2 is being input, a stable input operation can be performed without being affected by the change. I can continue.

As described above, the clipping and input operation of the object for one frame of the input image is completed. By repeating this operation for successive frames of the input video, it is possible to cut out and input even a moving target object.

In the above description, the case where two image capturing devices 3 and 4 are provided has been described, but the number of image capturing devices is not limited to this, and the number of image capturing devices may be increased to three or more, and an input unit, a changeover switch, and a difference detection unit connected thereto. By configuring by adding the background storage unit and the extraction unit, if the target object having a complicated shape is operated by using images captured from three or more positions, it may not be hidden from one image capturing position. It is obvious that it is possible to capture even a part that has occurred and cannot be captured, and it is possible to improve the cutting accuracy of the target object.

[Effects of the Invention] As described above, according to the present invention, a portion that causes a difference between an image that does not include an object that is an input target and an image that includes the object is detected, and similarly detected from a plurality of shooting positions. The position on the screen where the difference occurs is detected by comparing it with the parallax of each input image, and is labeled according to whether the detected position is within a predetermined threshold value, and the image is taken. By detecting the distance from the position to the object and cutting out and inputting only the image of the object at a specific distance from the shooting position,
It is not only necessary to set a special background, but also in an environment in which a part of the background changes, it is possible to cut out and input only the image of the object at a specific distance from the shooting position. In addition, it can be input regardless of whether the target object is stationary.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.
FIG. 2 is a diagram schematically showing the positional relationship among the photographing position, the target object, and the background when operating the embodiment of the present invention. FIG. 3 is a diagram showing an example of a difference matrix created by the difference detection unit. FIG. 4 is a diagram for explaining the different areas. 5 is a diagram showing an example of a comparison result list created by the parallax detection unit. FIG. 6 is a diagram showing an example of a difference matrix rewritten by the mask generator. In the figure, 1 is a background image, 2 is a target object, 3 and 4 are image capturing devices, 5 is a parallax detection unit, 6 is a mask generation unit, and 11, 1
Reference numeral 2 is an input unit, 31 and 32 are background storage units, 41 and 42 are difference detection units, and 71 and 72 are extraction units.

Continuation of the front page (56) References JP 62-125478 (JP, A) JP 63-76081 (JP, A)

Claims (1)

[Claims] 1. At least first and second photographing means for photographing an object at the same time, each of which is arranged at a constant interval, and is provided corresponding to at least the first and second photographing means, and the first And image storing means for storing in advance a background image of the object viewed from the position of the second photographing means, provided corresponding to each of the at least first and second photographing means, and photographed by the corresponding photographing means. Difference image detecting means for comparing the image of the object with the background image read out from the image storing means and detecting a portion where a difference occurs, in the screen of the portion having a difference detected by each of the difference detecting means The position of the parallax detection means for detecting by comparing between the parallax of each input image from each of the photographing means, the position detected by the parallax detection means within a predetermined threshold Labeling according to whether or not there is a distance detecting means for detecting a distance from each of the photographing means to the object, and an image of an object at a predetermined distance from each of the photographing means detected by the distance detecting means An image input device including extraction means for extracting the image.