JP3087236B2 - Stereoscopic image synthesis display device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for generating and displaying a stereoscopic image by synthesizing a real image picked up by two left and right video cameras and a three-dimensional computer graphics (hereinafter referred to as CG) image. The present invention relates to a synthesized display device for stereoscopic images.

[0002]

2. Description of the Related Art Conventionally, for example, at a construction site such as a building, a three-dimensional image having a sense of depth is used to capture the appearance of the entire building under construction or a partial shape of the building from a remote place. When monitoring or observing, images of a building or the like are taken with two left and right video cameras, and the captured images are reproduced and displayed on two monitors for the left eye and the right eye,
One that enables stereoscopic viewing by viewing both screens of the left and right monitors through a stereoscopic mirror for aerial photography, or
The left and right images captured by the two left and right video cameras are 60H
2. Description of the Related Art There is known a stereoscopic image display device which switches at a speed of z, alternately reproduces and displays an image on one monitor, and allows an image alternately displayed on the monitor to be viewed stereoscopically by viewing the image through a stereoscopic mirror.

[0003]

However, in the conventional stereoscopic image display apparatus as described above, in any case, the real images captured by the two left and right video cameras are reproduced and displayed on the monitor, and the stereoscopic image is displayed. Since it is only generated, for example, in order to grasp how much the construction status of the construction work is progressing, it is necessary to observe the monitor screen while comparing it with the design drawing. For this reason, there is a problem in that the burden on the observer or the like increases, and the applicable range is limited to simple monitoring of a construction site or the like. SUMMARY OF THE INVENTION The present invention has been made in view of the above-described circumstances, and by combining a real image obtained by two video cameras and a three-dimensional CG image, it is possible to apply the present invention to various fields in a more advanced and practical sense. It is an object of the present invention to provide a stereoscopic image synthesizing display device capable of obtaining the obtained image information.

[0004]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides two left and right video cameras for imaging the same subject from different directions, and is provided corresponding to each of the left and right video cameras. Left and right image processing means for converting each video signal output from each video camera into a digital amount separately for left and right to generate real image data, and real image data generated by the left and right image processing means, respectively. Left and right real image frame memories for individually storing the image data, computer graphic means for generating left and right graphic image data corresponding to the subject, and a graphic memory for storing the left and right graphic image data generated by the computer graphic means. A frame memory, the left and right real image frame memories and the graphic frame. Left and right image synthesizing means for synthesizing graphic image data with left- and right-hand real image data for one frame output from the memory, and right and left synthesized image data synthesized by the left and right image synthesizing means. Conversion means for converting the signal into a signal and outputting the image signal; display means for displaying the image signal as a stereoscopic image; detection means for detecting a shooting position and the direction of the video camera at an imaging site; Control means for controlling so that a graphic image corresponding to the development of a shooting field of view indicated by a real image picked up by the video camera is created by the computer graphic means, based on the shooting position and direction data, The detection of the shooting position of the video camera and the direction of the video camera by the detection means includes detecting a geomagnetic sensor and a gyro. There characterized in that it is carried out.

[0005]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a stereoscopic image synthesis display device according to an embodiment of the present invention. In FIG. 1, 1 is a left-eye video camera, 2 is a right-eye video camera, and these video cameras 1 and 2 image the same subject in a work site at a predetermined distance from the different direction. I do.

Reference numerals 3 and 4 denote left and right A / D converters for converting respective video signals output from the left and right video cameras 1 and 2 into digital signals. These digital signals are input to the corresponding left and right image processing devices 6 and 7. The image processing devices 6 and 7 operate according to a predetermined method.
The data amount of the image data output from the corresponding A / D converters 4 and 5 is reduced.
The reduced left and right image data are stored in the corresponding left and right real image frame memories 7 and 8, respectively. The frame memories 7 and 8 have a capacity capable of storing one or more frames of image data. It is composed of things.

In FIG. 1, reference numeral 9 denotes a left-eye CG device for creating a graphic image corresponding to an image picked up by the video camera 1, and reference numeral 10 denotes a graphic image corresponding to an image picked up by the video camera 2. This is a CG device for the right eye to be created. The graphic image signals created by these CG devices 9 and 10 are stored in the corresponding left and right graphic frame memories 11 and 12. The frame memories 11 and 12 have a capacity capable of storing one or more frames of image data.

In FIG. 1, reference numeral 13 denotes one frame of the real image data read from the real image frame memory 7 for the left eye and the graphic image data read from the graphic frame memory 11 for the left eye. This is an image synthesizing device for the left eye that synthesizes with the image data of the left eye, and the synthesized image data is output to the DA converter 15. A right eye 14 combines the real image data read from the right eye real image frame memory 8 and the graphic image read from the right eye graphic frame memory 12 into one frame of image data. The composite image data is output to the DA converter 16. The left and right DA converters 15 and 16 convert the combined image data output from the corresponding image combining devices 13 and 14 into NTSC.
The left and right NTSC signals are output to the corresponding left and right monitor display devices 17 and 18, respectively. Reference numeral 19 denotes a stereoscopic mirror for aerial photography which is arranged on the front of the monitor display devices 17 and 18 with a predetermined space therebetween.

In FIG. 1, reference numeral 20 denotes video cameras 1 and 2
Is a position / direction sensor for detecting the imaging position in the work site and the direction of the video camera, and includes a geomagnetic sensor, a gyro, and the like. Signals of the imaging position and orientation detected by the position / direction sensor 20 are output to the control circuit 21. The control circuit 21 generates a CG image development command corresponding to the development of the field of view of the video cameras 1 and 2 based on the signal from the position / direction sensor 20 and outputs the command to the left and right CG devices 9 and 10. is there.

Next, the operation of this embodiment will be described.
The image signals output from the left and right video cameras 1 and 2 are taken into the respective A / D converters 3 and 4 and sampled in accordance with the horizontal scanning timing, and are sampled.
The digital signal is converted into a digital signal of a plurality of bits per pixel and input to the respective image processing devices 5 and 6. The image processing devices 5 and 6 reduce the data amount by executing a predetermined method, for example, a thinning-out process of extracting only odd or even pixel data in one line, and correspond to the reduced image data. To be stored in the respective real image frame memories 7 and 8.

On the other hand, when a CG image development command output from the control circuit 21 is input to the left and right CG devices 9 and 10 in accordance with the shooting positions of the video cameras 1 and 2 and the orientation data thereof, the respective CG devices 9 , 10 generate CG images corresponding to the field of view of the video cameras 1 and 2,
The G image data is sequentially stored in the corresponding graphic frame memories 11 and 12.

On the other hand, in the image synthesizing device 13 on the left side, real image data and graphic image data sequentially read out from the corresponding real image frame memory 7 and graphic frame memory 11 by the FIFO method or the like are, for example, odd-numbered pixels in odd-numbered lines. The image data is synthesized by arranging real image data at the same time, arranging graphic image data at even pixels, and arranging graphic image data at odd pixels and arranging real image data at even pixels on even lines. The image data synthesized by each of the left and right image synthesis devices 13 and 14 corresponds to the corresponding DA.
By outputting to the converters 15 and 16, for example, N
After being converted to the TSC signal, the corresponding monitor display device 1
7 and 18 and reproduced and displayed. Therefore, by viewing the images displayed on the screens of the monitor display devices 17 and 18 through the stereoscopic mirror 19, it is possible to stereoscopically view a video obtained by combining the graphic image with the real image.

According to such a stereoscopic image display apparatus of the present embodiment, the left and right real images whose data amounts are reduced are generated by performing image processing on the images picked up by the two left and right video cameras. The left and right graphic images related to the images are created by the CG device, and the left and right CG images and the left and right real images are combined and displayed on the corresponding left and right monitor display devices. By viewing the screen of the display device through a stereoscopic mirror, a stereoscopic image in which a graphic image is superimposed on a real image can be easily obtained. Along with this, it is possible to provide advanced image information suitable for observation, monitoring, and the like, which is suitable for actual use. Therefore, for example, a video having a three-dimensional effect can be obtained by superimposing a video of a planned building such as a building on a video of a planned construction site, so that a building or the like constructed at the planned construction site can be observed with a practical sense. At the same time, the construction plan status of the construction site can be monitored from a remote location with a realistic sense. In addition, if the view of the field of view captured by the video camera is developed over time, the view of the CG image can be simultaneously developed in synchronization with this, so that the composite image may be distorted or ghosted. Thus, a clear and realistic stereoscopic image can always be obtained.

In the above embodiment, the left and right monitor display devices 17 and 18 composed of a TV receiver and the like and a stereoscopic mirror are combined as the monitor display means for displaying a stereoscopic image. The invention is not limited to this. For example, a configuration in which image signals synthesized by the image synthesis devices 13 and 14 are alternately output to a monitor CRT at a speed of 60 Hz or 120 Hz, and an image displayed on the monitor CRT is viewed through liquid crystal shutter glasses.
Alternatively, a system in which a composite image alternately projected on a screen by a projector is viewed through polarized glasses may be used. Further, in the above-described embodiment, the case has been described in which the left and right CG images are created by the respective CG devices 9 and 10 and stored in the separate frame memories 11 and 12. However, the present invention is not limited to this. Left and right CG images can be created and stored by one CG device and frame memory.

[0015]

As described above, according to the present invention, a left and right real image with a reduced data amount is generated by performing image processing on images taken by two left and right video cameras.
The left and right graphic images related to the real image are created by the CG device, and the left and right CG images and the left and right real images are combined and reproduced and displayed on the corresponding monitor display means. A stereoscopic image obtained by superimposing a graphic image can be easily obtained, and image information suitable for actual observation and monitoring can be obtained. Also, based on the shooting position and direction data detected by the detection means, since a graphic image corresponding to the development of the shooting field of view indicated by the real image picked up by the video camera is created by the computer graphic means, When the appearance of the field of view captured by the video camera is developed, the appearance of the CG image can be developed at the same time in synchronism with the development, and a clear and realistic stereoscopic image can always be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of a stereoscopic image synthesis and display device according to the present invention.

[Explanation of symbols]

 1, 2 Video camera 5, 6 Image processing device 7, 8 Frame memory for real image 9, 10 CG device 11, 12 Frame memory for graphics 13, 14 Image synthesizing device 17, 18 Monitor display device 19 Stereoscopic mirror 20 Position / direction Sensor 21 control circuit

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-62-223720 (JP, A) JP-A-60-18067 (JP, A) JP-A-3-29472 (JP, A) JP-A-3-29 29473 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 13/00-15/00

Claims (1)

(57) [Claims] 1. Two left and right video cameras for imaging the same subject from different directions, and provided for each of the left and right video cameras, and separately output left and right video signals from each video camera. Left and right image processing means for converting to a digital amount to generate real image data, left and right real image frame memories for individually storing respective real image data generated by the left and right image processing means, A computer graphic means for generating left and right graphic image data corresponding to the subject; a graphic frame memory for storing the left and right graphic image data generated by the computer graphic means; Left and right for one frame output from the frame memory for graphics Right and left image synthesizing means for synthesizing real image data and graphic image data of right and left, converting means for converting left and right synthesized image data synthesized by the left and right image synthesizing means into an image signal and outputting the image signal, Display means for displaying the image signal as a stereoscopic image; detecting means for detecting the shooting position and direction of the video camera at the shooting site; and shooting position and direction data detected by the detecting means.
Based on the actual image taken by the video camera
The graphic image corresponding to the development of the shooting field of view is
Controlled to be created by computer graphic means
Control means to perform, the shooting position of the video camera by the detection means and
The direction is detected using a geomagnetic sensor and a gyro.
Te is performed, composite display device of the stereoscopic image, characterized in that.