JPH06105232A - Picture synthesis device - Google Patents

Abstract

PURPOSE:To surely obtain a CG picture coincident with a visual point, direction and image pickup magnification of a video camera with a zoom lens and to synthesize a real picture by a video camera and the CG picture corresponding to the real picture as expected. CONSTITUTION:A view point on a CG is set based on absolute position data of a video camera 1 detected by a position sensor 9, a drawing picture area is set on the CG from the view point based on absolute position data of the video camera 1 and horizontal angle and elevating angle data sensed by sensors 6, 7, an image pickup magnification is obtained based on a zoom of a zoom lens 2 and object distance information, the size of a set drawn picture area is framed by the pickup magnification to generate a CG picture and the CG picture is overlaid on the real picture from the video camera 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus for synthesizing and displaying a real image picked up by a video camera with a zooming function and a computer graphics image (hereinafter referred to as CG image), and more particularly to a video camera. The present invention relates to an image synthesizing device in which a CG image size is aligned with a zooming image size and superimposing is possible.

[0002]

2. Description of the Related Art For example, when the expected completion status of a building to be constructed at a construction site is to be displayed as a video, the expected completion status of the building is created by CG and this CG image is displayed on a monitor image. It is common to see. In addition, recently, the planned construction site was photographed with a video camera, and the C
A display method in which G images are combined and displayed on a monitor has also appeared.

[0003]

By the way, a video camera for photographing a landscape of a planned construction site is designed so that a zoom lens can be operated while the object is in focus to enlarge or reduce the object window. It is generally the case. Therefore, it is necessary to set the window size and the viewpoint of the CG image according to the size of the window by zooming the video camera and the distance and direction from the video camera to the viewpoint.

However, when this kind of conventional image synthesizing apparatus is applied to a large object to be photographed and a wide field of view such as a construction site, the video camera's viewpoint, position and Since the direction and the like cannot be reflected in the CG image, even if the real image captured by the video camera and the corresponding CG image are combined, the positional relationship and the size of the two images are slightly deviated, and the images are different from those projected. However, there was a problem that it became far from practicality. The present invention has been made in view of the above circumstances, and it is possible to reliably obtain a CG image that matches the viewpoint, direction, and shooting magnification of a video camera with zoom, and at the same time, obtain an actual image captured by the video camera. It is an object of the present invention to provide an image synthesizing device capable of synthesizing a CG image corresponding to this and a CG image corresponding to this according to a planned predicted image.

[0005]

In order to achieve the above object, the present invention provides a video camera with a zoom lens for picking up an object and an absolute position of the video camera with respect to an area where the object is present. Position detecting means for detecting, distance detecting means for detecting an object distance to an object imaged by the video camera, zoom amount detecting means for detecting a zoom amount of a zoom lens of the video camera, Direction detecting means for detecting an imaging direction of the video camera with respect to the object, computer graphics means for creating a computer graphics image corresponding to the object, and absolute position of the video camera detected by the position detecting means Setting the viewpoint on a computer graphics image created by computer graphics means based on the data A drawing area for the viewpoint on the computer graphics image based on the image pickup direction data detected by the direction detecting means, and the zoom amount detected by the zoom amount detecting means and the object detected by the distance detecting means. Processing means for setting a magnification of the computer graphics image based on a physical distance; image processing means for converting a video signal output from the video camera into a discrete digital signal to generate real image data; An image synthesizing unit for synthesizing a computer graphics image processed by the processing unit and output from the computer graphics unit and an actual image output from the image processing unit are provided.

[0006]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an overall configuration diagram showing an embodiment of an image synthesizing apparatus according to the present invention. In FIG. 1, reference numeral 1 denotes a video camera installed at a predetermined place in a relatively large area such as a construction site. The video camera 1 includes a zoom lens 2 having an autofocus mechanism built therein. The zoom lens 2 can be zoomed by a motor 3 such as a stepping motor, and the motor 3 is provided with an encoder 4 for detecting the zoom amount of the zoom lens 2.

The video camera 1 is installed on a pan head 5 for operating the video camera in the horizontal direction and the elevation direction, and the pan head 5 has a horizontal angle for detecting the horizontal angle of the video camera 1 from this movement. A sensor 6 and a depression / elevation angle sensor 7 that detects an angle in the depression / protrusion direction are provided.

In FIG. 1, 8 is a distance measuring unit for detecting distance information according to the distance to the object photographed by the video camera 1, and 9 is an absolute position and orientation of the video camera 1 at a construction site. This is a position sensor, and the position sensor 9 is composed of a combination of a GPS and a geomagnetic sensor. The encoder 4, the horizontal angle sensor 6, the elevation angle sensor 7, the distance measuring unit 8 and the position sensor 9 are connected to a signal processing circuit 10, and the signal processing circuit 10 corresponds to an actual image captured by the video camera 1. CG
A CG device 11 for creating an image is connected to this C
The G device 11 includes a graphics memory 12 that stores CG image data for drawing as shown in FIG.

The signal processing circuit 10 sets the view point (viewpoint) on the CG based on the absolute position data output from the position sensor 9, and the absolute position data and the horizontal angle sensor 6 and the elevation angle sensor 7 are used. An image calculated based on the detection signal by setting the drawing area on the CG for which the viewpoint is set, and further based on the zoom amount detected by the encoder 4 and the object distance information detected by the distance measuring unit 8. The image size of the drawing area is set to the magnification, and the CG device 11 creates a CG image according to each setting command signal output from the signal processing circuit 10. Further, the created CG image data is stored in the frame memory 13.

In FIG. 1, reference numeral 15 denotes an NTSC / RGB converter which converts an NTSC video signal output from the video camera 1 into an RGB signal suitable for computer processing. An AD converter 16 is connected to the NTSC / RGB conversion unit 15 and converts an analog RGB signal into a discrete digital signal. To the A / D converter 16, a real image frame memory 17 that stores the real image data converted into a digital signal is connected. Also,
Reference numeral 18 denotes an image synthesizing circuit for synthesizing the CG image data read from the graphic frame memory 13 and the actual image data read from the actual image frame memory 17, and the synthetic image data synthesized by the image synthesizing circuit 18 is Monitor display device 2 via D-A converter 19
It is output to 0.

Next, the operation of this embodiment configured as described above will be described. When a zoom command signal is externally input to the zooming motor 3 with the video camera 1 facing the predetermined object in the construction site, the zooming motor 3
Is started, and the zoom lens 2 is zoomed toward the wide end or the tele end. At this time, the zoom amount of the zoom lens 2 is detected by the encoder 4, and the signal processing circuit 10
Is output to. When the zoom lens 2 is zoomed to a desired zoom position, the focal length of the taking lens including the zoom lens 2 is also adjusted to the object in conjunction with this. Along with this, the object image photographed by the video camera 1 through the photographing lens is taken out as an image signal in accordance with the horizontal scanning timing, and is input to the NTSC / RGB conversion unit 15. In this NTSC / RGB conversion unit 15, the NTSC
The image signal captured by the method is converted into an RGB signal and A
-Output to the D converter 16. A-D converter 16
Then, the image signal taken in at the horizontal scanning timing is sampled at a predetermined frequency, converted into a discrete digital signal of a plurality of bits per pixel, and stored in the actual image frame memory 17. The frame memory 1
7 is composed of a capacity capable of storing one frame or more of image data.

On the other hand, the video camera 1 in the construction site
The absolute position of is detected by the position sensor 9, and the detection result is taken into the signal processing circuit 10 as initial data. Similarly, the horizontal angle of the video camera 1 from the reference point detected by the horizontal angle sensor 6 and the vertical angle of the video camera 1 from the reference point detected by the vertical angle sensor 7 are also captured in the signal processing circuit 10. . Further, the object distance information detected by the distance measuring unit 8 is also fetched by the signal processing circuit 10.

A signal processing circuit 10 which takes in each input information.
Then, based on the absolute position data, a signal for setting a viewpoint similar to that of the video camera 1 on the object on the CG is generated and output to the CG device 11. In addition, a signal for setting a drawing area on the CG from the viewpoint (viewpoint) is generated based on the absolute position data, the horizontal angle data, and the elevation angle data, and is output to the CG device 11. Further, a signal for setting the image size of the set drawing area is generated from the shooting magnification calculated based on the zoom amount and the object distance information, and is output to the CG device 11.

In the CG device 11, as shown in FIG. 2, a view point VP similar to that of the video camera 1 is set in the CG object data area on the graphics memory 12, and drawing on the CG from the view point VP is performed. The area corresponds to the direction of the video camera 1 and the rectangular frame 3 in FIG.
Set as indicated by 0 or 31. After that, square frame 3
The image size of the drawing area indicated by 0 or 31 is reduced to the photographing magnification calculated by the signal processing circuit 10, for example, when the zoom lens is at the tele end, the image size is reduced to the size indicated by the square frame 30 in FIGS. 2 and 3. When the zoom lens is at the wide end, it is enlarged to the size shown in the rectangular frame 31 'in FIG. Then, the CG image data reduced or enlarged by the CG device 11 is stored in the frame memory 13.

On the other hand, in the image synthesizing circuit 18, CG which is sequentially read from the graphics frame memory 13 and the actual image frame memory 17 by the FIFO method or the like.
Of the image data and the actual image data, for example, in the odd line, the CG image data is arranged in the odd-numbered image element, the actual image data is arranged in the even pixel, and in the even line, the actual image data is arranged in the odd pixel. CG for even pixels
The CG image and the real image are combined by arranging the image data. Then, the image data synthesized by the image synthesizing circuit 18 is converted into an NTSC analog signal by the DA converter 19 and output to the monitor display device 20 for reproduction and display.

In the image synthesizing apparatus of this embodiment as described above, the view point on the CG is set based on the absolute position data of the video camera, and the absolute position data of the video camera and the horizontal angle and the elevation angle are set. The drawing area on the CG from the set viewpoint is set, the shooting magnification is obtained from the zoom amount of the zoom lens and the object distance information, and the size of the set drawing area is framed to the obtained magnification to create a CG image. Since this is created and the CG image is overlaid on the actual image from the video camera, it is possible to reliably obtain the CG image that matches the viewpoint, direction and shooting magnification of the video camera, as well as the actual image from the video camera. It is possible to synthesize the image and the CG image opposite to the image according to the planned image without any deviation. Along with this, practical application is facilitated, and image information suitable for observation, monitoring, etc. can be provided.

In the above embodiment, the case where one video camera 1 with a zoom lens is used has been described, but the present invention is not limited to this. For example, a stereoscopic image using two left and right video cameras. It is also applicable to those who get. In this case, the left and right CG corresponding to each video camera
Images are required, and stereoscopic viewing is possible by combining a stereoscopic mirror with the monitor display device. Further, the present invention is not limited to the configurations shown in the above embodiments, and various modifications can be made without departing from the scope of the claims.

[0018]

As described above, according to the present invention, a video camera with a zoom lens for picking up an object and position detection for detecting the absolute position of the video camera with respect to the area where the object exists. Means, a distance detecting means for detecting an object distance to an object imaged by the video camera, a zoom amount detecting means for detecting a zoom amount of a zoom lens of the video camera, and an object of the video camera. Based on direction detection means for detecting an imaging direction with respect to the object, computer graphics means for creating a computer graphics image corresponding to the object, and absolute position data of the video camera detected by the position detection means. Setting the viewpoint on the computer graphics image created by the computer graphics means, and the direction A drawing area for a viewpoint on the computer graphics image is set based on the imaging direction data detected by the output means, and based on the zoom amount detected by the zoom amount detection means and the object distance detected by the distance detection means. Processing means for setting the magnification of the computer graphics image, image processing means for converting a video signal output from the video camera into a discrete digital signal to generate real image data, and processing by the processing means. Since it is configured to include an image synthesizing unit for synthesizing the computer graphics image output from the computer graphics unit and the actual image output from the image processing unit, the viewpoint, direction, and A CG image that matches the shooting magnification can be reliably obtained, and the video The camera can be synthesized in plan expected imagewise without displacement and CG image corresponding to the real image captured by an effect that practically can be easily realized.

[Brief description of drawings]

FIG. 1 is a block diagram of an image synthesizing apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a framing state of a viewpoint and CG drawing with respect to a CG data area during zooming according to the present embodiment.

FIG. 3 is an explanatory diagram showing a framing state of CG drawing during zooming according to the present exemplary embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Video camera 2 Zoom lens 4 Encoder (zoom amount detecting means) 6 Horizontal angle sensor 7 Depression angle sensor 8 Distance measuring section (distance detecting means) 9 Position sensor 10 Signal processing circuit (processing means) 11 CG device

Claims (1)

[Claims] 1. A video camera with a zoom lens for picking up an object, position detecting means for detecting an absolute position of the video camera with respect to an area where the object exists, and an object picked up by the video camera. Distance detecting means for detecting an object distance to an object, zoom amount detecting means for detecting a zoom amount of a zoom lens of the video camera, and direction detecting means for detecting an image pickup direction of the video camera with respect to the object. A computer graphics means for creating a computer graphics image corresponding to the object, and a computer graphic created by the computer graphics means based on absolute position data of the video camera detected by the position detecting means. The viewpoint on the video image is set and the imaging direction data detected by the direction detecting means is set. A drawing area for a viewpoint on the computer graphics image based on the computer graphics image, and the computer graphics image based on the zoom amount detected by the zoom amount detecting means and the object distance detected by the distance detecting means. , A processing means for setting a magnification, an image processing means for converting a video signal output from the video camera into a discrete digital signal to generate actual image data, and a computer graphics means processed by the processing means. An image synthesizing apparatus comprising: an image synthesizing unit that synthesizes a computer graphics image output from the image processing unit and an actual image output from the image processing unit.