JP2709066B2 - A method of synthesizing a camera image and a computer-generated image - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing method, and more particularly, to a method of synthesizing a camera photographed image and a computer generated image suitable for generating a non-existent image.

[Conventional technology]

As a conventional method for inputting another image to a camera input image, a composition method using a wipe or a composition method using a chroma key has been used. In these methods, in order to combine another two-dimensional image on a two-dimensional image, it is necessary to precisely specify a combination area. For this reason, for example, in the method using the chroma key, the synthesis area is specified by the background color. These image synthesizing methods are described in, for example, “Computer Imaging”, edited by Masahiko Machida, Corona Publishing Co. (November 1984), No. 14.
It is described on pages 7 to 154.

[Problems to be solved by the invention]

In the above-described prior art, since an area to be synthesized is specified, it is difficult to perform synthesis such that the image before synthesis is complicatedly entangled, and no consideration is given to image synthesis including the perspective of the image before synthesis. The composition that maintains the context cannot be created. More recently, it has become possible to generate a complex image using a computer.However, a method of specifying a synthesis area has also been adopted for synthesizing this computer-generated image with an image captured by a camera. Atsuta.

An object of the present invention is to provide a method of synthesizing an image captured by a camera and an image generated by a computer, which can easily synthesize an image captured by a camera and an image generated by a computer.

[Means for solving the problem]

The above-mentioned object is to capture images taken by the cameras as images having a depth as seen by a human using two cameras separated by a specified distance constituting a stereoscopic camera, and two images taken by the two cameras Depth is calculated based on how far the same point is apart from each other in an image, and an image generated by one computer is also obtained as an image having depth by a method called a so-called depth buffer method, and for each pixel of the screen, This is achieved by a method for synthesizing a camera-captured image and a computer-generated image in which a depth of a camera-captured image is compared with a depth of a computer-generated image, and image information (color information) on a side closer to the depth is left to generate a composite image. .

[Action]

The method of synthesizing the camera-captured image and the computer-generated image includes:
The distance (depth) to the point is calculated from the displacement of the position of the same point on the two images taken by the two left and right cameras separated by the specified size of the stereoscopic camera from the image, and this is calculated as an image. Is performed for each pixel to form the color information and depth information for each pixel, and an image generated by one computer is also generated from an object described in three dimensions. The depth information can be output for each pixel at the same position in the image captured by the camera and the image generated by the computer, and the color information on the side closer to the depth is left so that the image captured by the camera and the image generated by the computer can be output. By synthesizing with the image, it is possible to easily obtain a synthesized image without a sense of discomfort.

〔Example〕

An embodiment of the present invention will be described below with reference to FIGS.

FIG. 1 is a flow chart showing one embodiment of a method for synthesizing a camera photographed image and a computer generated image according to the present invention. In step 1a of FIG. 1, an image is generated by a computer to generate a so-called computer graphics image, and in step 1b, the image is stored in the frame buffer 1 and the depth buffer 1 as color information and depth information for each image. . On the other hand, in step 1c, an image with depth information is photographed by a stereoscopic camera, and in step 1d, color information and depth information for each image of the image are stored in the frame buffer 2 and the depth buffer 2, respectively. Next, the depth buffer 1 is set in step 1e.
Then, the depth information is compared for each pixel having the same number in the depth buffer 2 and the color information of the frame buffer 1 or 2 on the near side is stored in the frame buffer 3. In this way, a composite image is generated on the frame buffer 3 in step 1f.

FIG. 2 is an apparatus configuration diagram showing one embodiment of a method of synthesizing a camera-captured image and a computer-generated image according to the present invention. Second
In accordance with a photographing signal SGN from a photographing switch 1 shown in the figure, photographing is performed by a stereoscopic camera 2 composed of two cameras arranged at a predetermined distance from each other, and two cameras, left and right, of the stereoscopic camera 2 Photoelectrically converts the image information taken by the camera of
The left-eye output L-DATA and the right-eye output R-DATA are output. Here, the left-eye output L-DATA and the right-eye output R-DATA output to the left-eye memory 3 and the right-eye memory 4, respectively, which have been initialized by the reset signal C-RST from the stereoscopic camera 2 in advance, are output to the stereoscopic camera 2. Is stored in accordance with the synchronization signal C-CLK. On the other hand, the data K- input from the data input means 6 such as a keyboard, a mouse, and a joystick.
DATA is converted into an image by a computer 7, and the image information is output to a frame buffer 8 and a depth buffer 9 as color information CLR2 and depth information DPT2 for each pixel. The color information CRT2 and the depth information DPT2 for each pixel are simultaneously synchronized with the timing signal G-CLK from the computer 7 to the memory positions of the frame buffer 8 and the depth buffer 9 specified by the pixel address ADR2 output from the computer 7. And are respectively stored.

In this state, when the computer 7 outputs the image synthesis start signal STRT to the pixel-by-pixel depth calculation circuit 5, the pixel-by-pixel depth calculation circuit 5 outputs the edit signal E-CLK to the left-eye memory 3 and the right-eye memory 4, and The left-eye output L-DATA and the right-eye output R-DATA are output from the memory 3 and the right-eye memory 4 to the depth calculation circuit 5 for each pixel. These left eye outputs L-DA
The depth information DPT1 for each pixel is calculated by the depth calculation circuit 5 for each pixel from the TA and the right-eye output R-DATA and output to the comparison circuit 10, and at the same time, the left-eye output L- is output as color information CLR1 for each pixel.
DATA is output to the frame buffer 8 as it is. However, the color information CLR1 for each image includes left-eye output L-DATA and right-eye output R
Only one of -DATA should be selected. At the same time as these outputs, the depth AD calculating circuit 5 outputs the address AD of the output pixel.
R1 is output to the frame buffer 8 and the depth buffer 9, and the depth information DPT3 of the same address is output from the depth buffer 9 to the comparison circuit 10 in synchronization with the output valid signal OE1. These two types of depth information DPT1 and depth information DPT3
Are compared by the comparison circuit 10, and when the depth information DPT1 is smaller, that is, when the corresponding pixel of the camera captured image is closer to the corresponding pixel of the computer-generated image, the memory rewrite signal CHG is output from the comparison circuit 10. The rewriting synchronization signal S of the AND circuit 11 is output in synchronization with the synchronization signal OE2 output to the AND circuit 11 and output from the depth buffer 9 to the AND circuit 11.
The LCT is output to the frame buffer 8, whereby the color information CLR2 of the corresponding pixel on the frame buffer 8 is rewritten to the color information CLR1 of the corresponding pixel. By repeating these processes over the entire screen, a composite image OUT of the camera photographed image and the computer-generated image can be obtained on the frame buffer 8. In this embodiment, one frame buffer 8 and one depth buffer 9 are provided.

[Effects of the Invention] According to the present invention, it is possible to easily combine a scene or real object image captured by a camera with a virtual image generated by a computer. And two are advantageous in that the input work of huge object data required when an image is generated only by a computer can be reduced by substituting an image taken by a camera.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a flowchart showing one embodiment of the present invention, and FIG. 2 is a device configuration diagram showing one embodiment of the present invention. 1 photographic switch, 2 stereoscopic camera, 3 left eye memory, 4 right eye memory, 5 depth calculation circuit for each pixel, 6 data input means, 7 calculator, 8 frame Buffer 9, depth buffer 10, comparison circuit 11, AND circuit.

 ──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-59-106071 (JP, A) JP-A-62-2223720 (JP, A) JP-A-61-162085 (JP, A)

Claims (1)

(57) [Claims] An image information composed of depth information and color information is calculated for each pixel from a left-eye image and a right-eye image obtained by photographing with a stereoscopic camera, and an image generated by a computer is also generated for each pixel. A method for synthesizing a camera-captured image and a computer-generated image, comprising: image information and color information; and a synthesized image is formed by color information on a side having a smaller depth value of corresponding pixels of both images.