JPH11271060A - Video camera position angle measuring apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To measure the position and angle of a video camera moving at photographing, and to collect and use data thereof for composing moving pictures with a picture of a body formed by the computer graphics. SOLUTION: The video camera position angle measuring apparatus obtains data of the position and angle of a video camera 1 used for composing moving pictures with pictures of a body formed by the computer graphics, using a picture composer. A moving position measuring unit 2 measures the absolute position of the moving video camera to output its latitude, longitude and height data. A camera angle measuring unit 3 measures the roll, pitch and bearing angles of the video camera being moving to output its roll, pitch and bearing angle data. A data collector 4 collects measured data from the camera position measuring unit 2 and camera angle measuring unit 3 synchronously with the forming of each picture frame the video camera 1 photographs.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used to create a composite video image used in a landscape simulation or the like, that is, a landscape image taken by a video camera and an image of an object created by CG (computer graphic). The present invention relates to a video camera position and angle measurement device for obtaining data on the position and angle of a video camera.

[0002]

2. Description of the Related Art Conventionally, for example, in order to view a predicted completion state of a building to be constructed on a construction site on a simulation screen, a predicted appearance of the building is created as a three-dimensional image by CG, and the scenery of the construction site is constructed. An apparatus has been proposed in which an image is captured by a video camera, and an image captured by the video camera and a CG image are combined and displayed (for example, see Japanese Patent Application Laid-Open No. HEI 5 (1993) -520).
-011727 gazette).

[0003]

[Problems to be solved by the invention]

However, in a conventional image synthesizing apparatus, when a moving image such as a landscape is photographed while moving a video camera at a certain speed, the direction and position data of the camera are recorded at a speed corresponding to each frame image of the moving image. There is a defect that can not be obtained.

That is, when a video camera is mounted on an automobile, for example, and it is desired to capture a surrounding landscape image as a moving image while moving, the position data and direction data of the camera, which constantly changes while moving, are stored in each frame image of the camera. (Eg 1
/ 30 seconds) has not yet been proposed.

[0006] For this reason, for example, when performing a landscape evaluation of a planned construction area where a structure such as a steel tower is to be constructed, while moving the area by car and photographing the surrounding landscape, the structure can be visually recognized from any position. There is a problem that it is not possible to create an image of a landscape simulation for evaluating how the landscape changes.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and measures the position and angle of a video camera moving at the time of photographing, collects their data, and records the captured moving image and CG.
It is an object of the present invention to provide a video camera position and angle measuring device that can use the data when combining the images of the object created by the method.

[0008]

In order to achieve the above object, a video camera position and angle measuring apparatus according to the present invention is an image synthesizing apparatus which combines a moving image taken by a moving video camera and an image of an object created by CG. In a video camera position and angle measurement device for obtaining data of the position and angle of the video camera used when synthesizing, the absolute position of the moving video camera is measured, and its latitude data, longitude data,
Camera position measuring means for outputting altitude data; camera angle measuring means for measuring the roll angle, pitch angle and azimuth angle of the video camera when moving, and outputting the roll angle data, pitch angle data and azimuth angle data; Data collection means for collecting measurement data from the camera position measurement means and camera angle measurement means in synchronization with the creation of each image frame taken by the video camera; and Data storage processing for capturing latitude data, longitude data, and altitude data output from the camera position measuring means through the collecting means, and capturing and storing roll angle data, pitch angle data, and azimuth angle data output from the camera angle measuring means Means, latitude data, longitude data, and altitude data for each predetermined time temporarily stored by the data storage processing means. , Wherein the video camera is equipped with a data interpolation means for performing interpolation and correction of the data at the timing of each frame of the captured image.

Here, as the camera position measuring means, for example, a moving position measuring device using a GPS can be used. When a video camera is mounted on a car and moving, the moving position measuring device is similarly mounted on the car. Mount. As the camera angle measuring means, for example, three gyros for measuring angles rotating around three axes can be used. Further, a processing device having a CPU as a main part can be used for the data collection means, and a general-purpose personal computer can be used for the data storage processing means and the data interpolation processing means.

[0010]

The video camera position / angle measuring device having the above-mentioned configuration is used by being mounted on an automobile or the like together with the video camera. It measures the absolute position of the moving video camera and outputs its latitude, longitude and altitude data. The camera angle measuring means measures a roll angle, a pitch angle, and an azimuth of the video camera when moving, and outputs the roll angle data, the pitch angle data, and the azimuth data.

At this time, the data collecting means collects the measurement data from the camera position measuring means and the camera angle measuring means in synchronization with the creation of each image frame photographed by the video camera, and sends them to the data storage processing means. Send data. The data storage processing means takes in and stores these latitude data, longitude data, altitude data, roll angle data, pitch angle data, and azimuth angle data.

After storing these data, the data interpolation processing means interpolates the latitude data, the longitude data, and the altitude data for each fixed time at the timing of each frame of the image taken by the video camera. Make corrections.

The data of the position and angle of the video camera collected and corrected as described above are input to a CG device together with the photographed image data when a photographed moving image and an image of an object created by CG are combined. Are used as data of the viewpoint position and the line-of-sight angle of the captured image and the CG image, and the image of the moving image is synthesized.

As described above, in the image synthesis, for each frame of the captured image data, the position data of the latitude, longitude, and altitude serving as the imaging viewpoint, the roll angle of the line of sight,
Since the image of the object when the pinch angle and the azimuth angle data are given to the CG image is synthesized, it is possible to synthesize the CG object image on the captured moving image so as to stick to the correct position. it can.

In addition, since the image synthesizing operation for each frame can be automatically and efficiently performed by the CG device because the viewpoint position data and the line-of-sight angle data are provided for each frame, furthermore, the video camera can be used. Since the position data and the angle data of each frame, which are sometimes collected and appropriately interpolated and corrected, are used, a smooth moving image without a sense of incongruity can be created, and can be effectively used for landscape simulation and the like.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration diagram of an apparatus according to an embodiment of the present invention. As shown in FIG. 1, this video camera position and angle measuring device is a video camera 1 mounted on an automobile for photographing a landscape or the like and recording a moving image on a magnetic tape or the like.
And the absolute position of the video camera 1, which is mounted on the vehicle, is measured at regular time intervals, and its latitude data,
Moving position measuring device 2 that outputs longitude data and altitude data
And

Furthermore, the video camera position and angle measuring device is
A camera angle measuring device 3 for measuring a roll angle, a pitch angle and an azimuth angle of the video camera 1 at regular intervals and outputting the roll angle data, the pitch angle data and the azimuth data, and each image frame taken by the video camera 1 A data collection device 4 that collects measurement data from the movement position measurement device 2 and the camera angle measurement device 3 in synchronization with the creation of the data, and the latitude data, longitude data, altitude data, and roll angle transmitted from the data collection device 4. A data processing device 5 is provided to fetch and store data such as data, pitch angle data, and azimuth angle data and perform data correction.

The video camera 1 is an integrated video camera having a built-in VTR. The video camera 1 receives an image light incident through an optical system at the tip and converts the image light into an electric signal. A process amplifier for amplifying and adjusting a signal, an A / D converter for converting a video signal output from the process amplifier into a digital signal,
It has a data compression circuit for compressing the digital video signal output from the / D converter by a predetermined method, and a VTR section for recording the compressed video data signal on a magnetic tape.

When taking a picture, the video camera 1 records a signal (time code signal) indicating time on a part of the magnetic tape, and outputs the time code signal to the data collection device 4 as a digital signal. The time code signal is output when a frame image is generated, that is, every 1/30 seconds when the number of frames of the video to be shot is, for example, 30 frames per second. The video camera 1 is fixed to a base plate 7 together with the camera angle measuring device 3 and mounted on a car with its front facing.

The mobile position measuring device 2 uses a satellite positioning system (GPS), has a GPS receiver for receiving radio waves transmitted from three or more satellites, calculates the arrival time of the radio waves, The absolute position, that is, latitude, longitude, and altitude data are obtained. Further, the measuring device 2 includes, for example, map data in a storage medium such as a CDROM in order to improve the measurement accuracy. When there is no measurement position on a map road by a so-called map matching method, the measurement device 2 displays the map data on a nearby road. Compensate and correct the measurement position data to correct GPS errors. Therefore, the moving position measuring device 2 measures the absolute position of the moving vehicle, that is, the video camera 1, and outputs the latitude data, the longitude data, and the altitude data.

In order to increase the GPS measurement accuracy, a combination with a self-contained navigation of a car can be adopted in addition to the map matching method. In self-contained navigation, the azimuth is measured using a geomagnetic sensor mounted on a car, a gyro, a wheel speed sensor that calculates an azimuth based on a rotation difference between left and right wheels, and the like.
The positioning data by the PS is corrected. A reference station whose precise position is known receives a GPS signal from a GPS satellite,
By comparing the positioning data calculated from the GPS signal with the known position data, the error data is calculated, and the error data is transmitted on a broadcast wave or the like to an automobile, and the GPS device on the automobile converts the error data into the error data. So-called DGPS, which corrects GPS positioning data based on it, can also be employed.

As shown in FIG. 2, the camera angle measuring device 3 includes three gyros 31, 32, and 33 for measuring angles about three-dimensional three axes (XYZ axes). 3
The gyros 31, 32, 33 are fixed on a base plate 34 arranged horizontally with respect to the moving direction and two vertical walls 35, 36 erected vertically therewith.

That is, the gyro 31 is fixed to a vertical wall (a wall perpendicular to the X axis) 36 in order to measure the right and left tilt angles (roll angles) with respect to the moving direction of the video camera 1.
A gyro 32 is fixed to a vertical wall (a wall perpendicular to the Y axis) 35 for measuring the tilt angle (pitch angle) before and after, and a gyro 33 is mounted on a base plate (vertical to the Z axis) for measuring the azimuth. Surface 34). Each gyro has a structure using, for example, gas, optical fiber, piezoelectric element, etc., and integrates the turning speed of the moving object to obtain its direction, and obtains measurement data of roll angle, pitch angle, and azimuth angle. Is output.

As shown in FIG. 3, the data collection device 4
CPU 41, memory 42, interfaces 43 to 46
And a video camera 1, a movement position measuring device 2, a camera angle measuring device 3, and a data processing device 5 are connected via interfaces 43 to 46. The data collection device 4 synchronizes with the creation of each image frame captured by the video camera 1, that is, by using a time code signal input from the video camera 1 as a trigger,
And captures the measurement data sent from the camera angle measuring device 3 and outputs to the data processing device 5 the position data of latitude, longitude and altitude, and the camera angle data of roll angle, pinch angle and azimuth together with the time code signal. . The transmission path from the data collection device 4 to the data processing device 5 includes, for example, an RS
232C is used, and each position data and each camera angle data are transmitted as serial data together with a time code signal.

The data processing device 5 is composed of, for example, a personal computer. While the video camera 1 is shooting, the time code signal, latitude, longitude, and altitude position data sent from the data collection device 4 are processed in an on-line process. , Roll angle, pinch angle, and azimuth angle are taken in through the RS232C port and stored in a temporary memory. Thereafter, the stored latitude data, longitude data, and altitude data are read off-line, and data correction and interpolation are performed.

The data interpolation is performed at the timing of each frame of the image captured by the video camera. The moving position measuring device 2 using the GPS normally outputs one to several pieces of position data every second, and there is often no update of data at the generation timing (1/30 second) of each frame of a captured image.

For this reason, when photographing a camera while the vehicle is continuously moved, the moving position measuring device is arranged such that the latitude data, the longitude data and the altitude data change according to the change of the position of the continuously moving camera. Latitude, longitude from 2,
Each position data of the altitude is smoothly interpolated by using linear interpolation or non-linear interpolation at each generation timing (1/30 second) of each frame of the captured image. If the latitude, longitude, and altitude position data are significantly different from the preceding and following data, the data is not used, and the position data is corrected based on the preceding and following data.

Next, the operation of the video camera position / angle measuring apparatus having the above-described configuration will be described by taking a landscape photographing for a landscape simulation of a power transmission tower as an example. All the devices shown in FIG. 1, that is, a video camera 1, a moving position measuring device 2,
The data collection device 4 and the data processing device 5 are wired and connected as shown in FIG. 1, and are mounted in an automobile. The video camera 1 is fixed horizontally with the camera angle measuring device 3 facing forward in the vehicle, and the X-axis (FIG. 2) for detecting the roll angle of the camera angle measuring device 3 is set in front of the vehicle (straight forward direction).
Join.

In order to use a video camera to capture a landscape image of the site where the power transmission tower is to be constructed, the vehicle is moved to a road passing near the site, the vehicle is stopped just before the shooting location, and the device is reset. The camera angle measuring device 3 using the gyro measures only the relative movement angle and cannot detect the direction (azimuth) of the camera. Therefore, the direction (absolute direction) of the camera at the time of starting is measured in advance and input to the data processing device 5. I do. The video camera 1, that is, the automobile may be directed to a predetermined direction (for example, true north), and movement and photographing may be started therefrom.

After turning on the power to all the devices, the camera angle measuring device 3 is initialized with the vehicle completely stopped. That is, the output values of the gyros 31, 32, and 33 are converged, the origins of the roll angle, the pitch angle, and the azimuth angle are initialized, and then the output values of the gyros are measured with the vehicle engine started. Then, the value is initialized as an offset value (step 100 in FIG. 4).

Next, the data collecting device 4 and the data processing device 5 take in the position data from the moving position measuring device 2 and the angle data from the camera angle measuring device 3 to measure the position and angle of the camera. Is started (step 110). Subsequently, photographing by the video camera 1 is started (step 120), and the scenery near the planned place is photographed while moving the car along the road.

When the video camera 1 starts photographing, the video camera 1 outputs a time code signal for each frame (for example, every 1/30 second) of the photographed image, and the data collection device 4 uses the time code signal as a synchronization signal (trigger). Signal)
The measurement data at this time sent from the movement position measurement device 2 and the camera angle measurement device 3 is taken in (step 130).
~ 140). Then, the data collection device 4 processes the position data of latitude, longitude, and altitude, and the camera angle data of roll angle, pinch angle, and azimuth together with the time code signal as serial data in a predetermined format. Output to the device 5 (step 150).

While the video camera 1 is taking a picture, the data processing device 5 performs time processing of the time code signal, latitude, longitude, and altitude position data, and roll angle and pinch angle sent from the data collection device 4 in online processing. , Azimuth angles, and stores them in a temporary memory in order.

With the operation of the data collection device 4 and the data processing device 5 as described above, while the surroundings are photographed by the video camera 1 while the vehicle is running at a low speed, each frame of the photographed image (for example, 1 / Every 30 seconds), the position data of the latitude, longitude, and altitude of the video camera 1 as the viewpoint of shooting, and the roll angle of the camera as the gaze direction of shooting,
Each data of the pinch angle and the azimuth is stored in the data processing device 5.

After the photographing is completed, the data of the camera position is interpolated and corrected using the data processing device 5. The interpolation / correction of the data is carried out by the moving position measuring device 2 using the GPS, which normally outputs one to several position data every second, and outputs the data at the generation timing (1/30 second) of each frame of the captured image. Conducted because there was no update of
The collected position data (latitude, longitude, and altitude) are converted into each frame of the captured image so that the position (latitude, longitude, and altitude) changes continuously according to the change in the position of the continuously moving camera. At each occurrence timing (1/30 second), linear interpolation or nonlinear interpolation is used for smooth interpolation. Also,
If the latitude, longitude, and altitude location data differ significantly from the data before and after it, do not use that data.
Correct the position data based on the preceding and following data.

The video camera 1 thus photographed
For example, the image data of the image as shown in FIG. 5A is, for each frame, together with its camera position data and angle data,
Input to the CG device. And in the CG device, FIG.
An image of the power transmission tower is created in advance, and a three-dimensional graphic image of the power transmission tower and the above-mentioned captured image are combined to form a composite video image as shown in FIG. Video footage is created for

In the image synthesis, for each frame of captured image data, latitude, longitude, and altitude position data, which are the viewpoints, and roll data, pinch angle, and azimuth angle data of the line of sight are converted into a CG image. Since the 3D image of the tower at the time of giving is synthesized, the power transmission tower can be synthesized so as to be stuck at a correct position on the captured moving image landscape image.

Further, such image synthesizing operation for each frame can be automatically and efficiently performed by the CG device because there is viewpoint position data and line-of-sight angle data for each frame. Since the position data and angle data for each frame, which are collected at the time of photographing by the camera 1 and appropriately interpolated and corrected, are used, a smooth moving image without a sense of incongruity can be created.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a video camera position / angle measurement apparatus according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a camera angle measuring device.

FIG. 3 is a block diagram showing the configuration and connection of a data collection device.

FIG. 4 is a flowchart illustrating an operation of the data collection device.

FIG. 5 is an explanatory diagram of an image (a) captured by a video camera, an image (b) created by CG, and an image (c) obtained by combining them.

[Explanation of symbols]

 1-Video camera 2-Moving position measuring device 3-Camera angle measuring device 4-Data collecting device 5-Data processing device

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI G03B 37/04 G03B 37/04 G06T 1/00 H04N 5/225 Z H04N 5/225 G06F 15/64 330 15/66 450

Claims (2)

[Claims] 1. A video camera position for obtaining data of a position and an angle of a video camera used when a moving image captured by a moving video camera and an image of an object created by CG are combined by an image combining device. In an angle measuring device, a camera position measuring means for measuring an absolute position of a moving video camera and outputting its latitude data, longitude data, and altitude data, and determining a roll angle, a pitch angle, and an azimuth angle of the video camera when moving. A camera angle measuring means for measuring and outputting the roll angle data, the pitch angle data and the azimuth angle data; and a camera position measuring means and a camera angle measuring means synchronized with the creation of each image frame taken by the video camera. Data collecting means for collecting measurement data from the camera, and the camera through the data collecting means while the video camera performs shooting. Data storage processing means for taking in latitude data, longitude data, and altitude data output from the position measuring means, and taking in and storing roll angle data, pitch angle data, and azimuth angle data output from the camera angle measuring means; Data interpolation processing means for interpolating and correcting the latitude data, the longitude data, and the altitude data for each predetermined time temporarily stored by the storage processing means at the timing of each frame of the image captured by the video camera. A video camera position and angle measurement device. 2. The video camera records a time code signal on a recording medium at the time of shooting, sends the time code data to the data storage processing means, and stores the time code data together with the camera position data and angle data. 2. The video camera position / angle measuring apparatus according to claim 1, wherein the data is stored in said data storage processing means.