JP5527710B2 - Calibration data acquisition apparatus and computer program - Google Patents

Description

  The present invention relates to a calibration data acquisition apparatus and a computer program for acquiring calibration data when a plurality of cameras for taking a stereo image of a predetermined space area are installed in the field.

  Conventionally, it is necessary to set the positions of two cameras in order to perform stereo imaging of a predetermined spatial region with high measurement accuracy, and to perform calibration (calibration) with high accuracy after setting.

As high-precision calibration, for example, a technique called “8-point calibration” is employed. In this 8-point calibration, eight vertices of a rectangular parallelepiped (cube), which is a model accurately set by measurement, are illuminated on the photographing space, and light is received (photographed) to align the stereo photographing. For example, the coordinates of eight vertices in the cube are input to the computer. Alternatively, after the cube is photographed by two cameras and two camera images are acquired, the coordinates of each vertex of the cube and the coordinates of the two camera images are calibrated.
In order to execute calibration (calibration) with high accuracy and efficiency, techniques as disclosed in Patent Documents 1 to 3 are disclosed.

By the way, as a part of the environmental assessment, a flying object such as a bird with respect to a predetermined space area (for example, a place where a building is to be built) is detected, and the detected information is analyzed. To detect the flying object, a plurality of (usually two) cameras acquire image data including the flying object, and extract matching particles and pixels in the synchronized plurality of image data (for example, PTV stereo) Pair matching) is used. That is, the coincident particles and pixels are converted into three-dimensional velocity and position information, and the trajectory of the flying object is obtained as three-dimensional data.
By applying such a flying object detection method, it is possible to obtain a three-dimensional flight trajectory, for example, for bird observation and behavior investigation.

  In a flying object survey for a predetermined space area, the distance between two cameras installed for performing stereo shooting for shooting a target space is often 50 meters or more. In the case of a wind farm or the like where multiple wind power generators are planned to be installed, it may reach several kilometers.

JP-A-8-14828 Japanese Patent Laid-Open No. 10-122819 JP 2007-64836 A

Now, when the space area to be subjected to stereo shooting is vast, the calibration work becomes large and troublesome. In other words, in order to reduce the error in the space area as the subject, it is necessary to make the virtual cube (calibrator) as large as possible for calibration, and the setting of the eight vertex coordinates becomes large. .
It is more difficult to set the vertex coordinates as a physical structure as the cube becomes larger, and it is also difficult to move and store the structure.

  As a further problem, there are restrictions on the location where two cameras for stereo shooting are installed. That is, in many cases, it is difficult to select an ideal place where a camera is to be installed and to set a calibration cube in order to take a stereo image of a space area to be imaged.

This will be described more specifically.
The place where the two cameras are installed at an appropriate distance must be the subject of the space area to be photographed, so it is often considered in advance using aerial photographs or the like. However, when I arrived at the site, it turned out that there were more cases than expected, such as plants and buildings that obstructed the space area that was the subject, or the scaffolding was bad and it was difficult to fix the camera.
Needless to say, it is difficult to install a calibration cube in the field, even though it is difficult to install the camera alone.

  In view of the above problems, an object of the present invention is to provide a technique capable of easily acquiring calibration data necessary for stereo shooting in a wide shooting area.

(First invention)
The first invention in the present application is to acquire calibration data necessary for the photographed image data using the photographing condition data obtained when the photographed image data is obtained by stereo photographing the space area (50) in the field. According to the device (10).
That is, on-site shooting condition data input means for inputting shooting condition data relating to two cameras shooting from different directions with respect to the space area (50) of the spot, and stereo shooting reduced in scale using the shooting condition data is executed. A scale model photographing means for computing scale model data for performing, a scale model photographing means for photographing a calibration sample in stereo using two cameras spaced apart using the scale model data, and the scale model photographing A calibration data acquisition device comprising calibration data calculation means for calculating calibration data using a calibration sample photographed by the means.
The space area (50) in the field is a space where the distance between the two cameras is 600 meters, and the two cameras are cameras using a C-MOS sensor, and use a photographing control device. The space area (50) of the field is simultaneously photographed, and the position data of the camera and the orientation data of the photographing direction in the photographing condition data are acquired by a GPS device provided in each camera, and the scale model photographing means includes Stereo shooting with two cameras separated by 10 to 40 meters.

  The “shooting condition data” includes camera position data, camera altitude data, camera elevation angle, camera lens optical data, and shooting direction azimuth data. “Camera position data” is acquired by, for example, a GPS device mounted on the camera (“GPS” is a global positioning system = Global Positioning System). The “camera altitude data” is acquired by, for example, an altimeter mounted on the camera. “Camera lens optical data” includes at least the angle of view. The “azimuth data in the photographing direction” is, for example, an orientation in which the center line of the camera lens is oriented at the time of photographing (for example, 360 degrees clockwise with true north as 0 degree), which is measured by an azimuth magnetic needle (compass) mounted on the camera. Any one). The “azimuth data in the shooting direction” may be measured by GPS.

  The “scale model data” is a value suitable for making a distance model between two cameras that have performed on-site stereo shooting a scale model. For example, when the distance between two cameras that have performed on-site stereo photography is 600 meters, and the distance that separates the two cameras for stereo photography of the calibration sample is 20 meters. Becomes “1/30”.

(Function)
First, two cameras are separated from each other, and the space area (50) in the field is photographed from different directions by the two cameras, so that the photographed image data is obtained by stereo photographing. At this time, photographing condition data for the camera is acquired.
The photographing conditions are input to field photographing condition data input means in the calibration data acquisition device. Using the shooting condition data, the scale calculation means calculates scale model data for performing a reduced stereo shooting.
Using the scale model data, the calibration sample is stereo-photographed by two cameras set apart in the scale model photographing means.
Using the calibration sample photographed by the scale model photographing means, the calibration data calculating means calculates the calibration data. Using the calculated calibration data and the above-described scale model data, it is possible to calibrate the captured image data obtained by photographing the space area (50) in the field.

(Variation 1 of the first invention)
The first invention can be formed as follows.
That is, comprising three-dimensional analysis means for three-dimensionally analyzing the photographed image data using the calibration data and the scale model data,
The three-dimensional analysis means may include a PTV analysis algorithm.

(Function)
Since it is equipped with a three-dimensional analysis means, it is possible to calibrate the captured image data obtained by photographing the space area (50) in the field, and further to the three-dimensional analysis.

(Variation 2 of the first invention)
The first invention can also be formed as follows.
That is, the calibration sample photographed by the scale model photographing means may emit light by changing the identification color for each sample point.

(Second invention)
The second invention of the present application is a computer program for acquiring calibration data necessary for the photographed image data by using the photographing condition data obtained when the photographed image data is obtained by stereo photographing of the space area in the field. Related.
The program includes an on-site shooting condition data input procedure for inputting shooting condition data relating to a camera using two C-MOS sensors for shooting at a distance of 600 meters from different directions with respect to the space area of the site; A scale calculation procedure for calculating scale model data for performing stereo shooting scaled using the shooting condition data, and two cameras installed 10 to 40 meters apart using the scale model data Let the computer execute a scale model shooting procedure for stereo shooting of the calibration sample and a calibration data calculation procedure for calculating calibration data using the calibration sample shot by the scale model shooting procedure,
The position data of the camera and the direction data of the shooting direction in the shooting condition data are computer programs that are acquired by the GPS device provided in each camera.

(Variation of the second invention)
The second invention may be a computer program having a three-dimensional analysis procedure for three-dimensionally analyzing the captured image data using the calibration data and the scale model data.

  The second invention can also be provided by being stored in a recording medium (for example, a hard disk, CD-R, DVD-R, etc.). It can also be transmitted via a communication line.

According to the first to third aspects of the invention, it is possible to provide a calibration data acquisition apparatus that can easily acquire calibration data necessary for stereo imaging in a wide imaging area.
According to the invention described in claims 4 and 5, it is possible to provide a computer program capable of easily acquiring calibration data necessary for stereo shooting in a wide shooting area.

It is a hardware block diagram concerning the embodiment of the present invention. It is a conceptual diagram which shows the data acquisition apparatus for calibration of embodiment of this invention. It is a schematic explanatory drawing which shows the variation of the reference | standard model for performing calibration.

Hereinafter, the present invention will be described in more detail based on embodiments.
As shown in FIG. 2, a predetermined space region 50 is shot in stereo using two cameras 11. Of the cameras 11 in FIG. 2, the camera located on the left is the left camera 11L, and the camera located on the right is the right camera 11R. The cameras 11L and 11R are cameras using C-MOS sensors. The use of the left and right cameras 11L and 11R equipped with the C-MOS sensor is easier to control the exposure time than the CCD element, and is suitable for capturing images of birds having a high flight speed. In addition, it is easy to shoot in cloudy weather and the like, and equipment such as laser light irradiation is not necessary.

The predetermined space area 50 is an area that needs to be investigated, for example, as to whether or not a wild bird to be protected lives on the planned construction site as part of an environmental assessment at the planned construction site of the structure.
In FIG. 2A, the two cameras 11L and 11R are arranged in a triangular shape on a plane as indicated by the one-dot chain line in FIG. Moreover, the lens orientations 12L and 12R (imaging direction) of the two cameras 11L and 11R are substantially located on the above-described triangular line.

Each of the cameras 11L and 11R is provided with a GPS device, an altimeter, and a magnetic stylus, and when photographing data is acquired, it is recorded along with the photographing data.
Further, each of the cameras 11L and 11R needs to shoot at the same time, and thus shoots via the shooting control device 15. In the photographing control device 15, the distance between the two cameras is calculated based on the position data obtained by the GPS device in each of the cameras 11L and 11R. This distance is assumed to be “LA”. Specifically, 600 meters will be used later.

The shooting direction 12L (lens direction) of the left camera 11L shown in FIG. 2A is on the right side with respect to the left azimuth reference line 14L orthogonal to the line 13 connecting the two cameras 11L and 11R. The shooting angle θL is established. Further, the shooting direction 12R (lens orientation) of the right camera 11R in FIG. 1 forms a shooting angle θR on the left side with respect to the right azimuth reference line 14R perpendicular to the line 13 in the horizontal direction. The left azimuth reference line 14L and the right azimuth reference line 14R are parallel to each other.
Shooting condition data including camera position data, camera altitude data, camera elevation angle, camera lens optical data, and shooting direction orientation data is recorded in the image recording device 16 together with the shot image data.

As described above, shooting condition data and on-site image data are obtained by the on-site imaging means described with reference to FIG. Next, description will be made based on FIG.
The photographing condition data is input to the on-site photographing condition data input means, and the scale model data is calculated by the scale calculating means. Specifically, the scale is calculated in relation to data relating to experimental equipment that can be photographed with a scale model, in particular, distance data of two cameras used for stereo photography with a scale model photographing means. Since the scale model photographing means in this embodiment can perform stereo photographing with two cameras separated by 10 to 40 meters, the scale model data is set to “1/30”. That is, in the scale model photographing means in the present embodiment, two cameras are separated by 20 meters to perform stereo photographing.

The scale model photographing means in FIG. 1 is means for obtaining calibration data shown in FIG.
According to the scale model data described above, LB in FIG. 2B is 20 meters. Other conditions are set using the shooting condition data. Sample points 21A, 21B,... Of eight vertices of, for example, a rectangular parallelepiped (cube), which becomes the reference model 20 accurately set in the photographing space by the two cameras 11L, 11R set at this distance. 21H is photographed, and photographing data for calibration is acquired.
In other words, if it is possible to secure a shooting location where stereo shooting can be performed by two cameras separated by 20 meters, calibration can be performed to obtain calibration data.

  Note that the two cameras used for the scale model photographing means are preferably the same as those used for the photographing means in the field. However, if the same cannot be used, the correction value is calculated using the shooting condition data, and the same condition is set in a pseudo manner.

The spatial coordinates (X axis, Y axis, Z axis) of the sample points 21A to 21H, which are the eight vertices in the cube, are input to the computer. Alternatively, the sample points 21A to 21H at the eight vertices in the cube are photographed by the two cameras 11L and 11R to obtain the two camera images, and then the space of the sample points 21A to 21H that are the vertices of the cube. The coordinates correspond to the coordinates of the two camera images.
The two cameras 11L and 11R control the shooting timing by the shooting control device 15, and the images shot by the two cameras 11L and 11R are recorded in the image recording device 16. The image recording device 16 is provided with a calculation device (calibration data calculation means) 17 for executing the above calibration.

It is necessary to install sample points 21A to 21H at eight vertices of a rectangular parallelepiped (cube) to be the reference model 20.
In FIG. 2, sample points 21A and 21E are positioned on a vertical line from the ground point 22A toward the sky, and sample points 21B and 21F are positioned on a vertical line from the ground point 22B toward the sky. 21C and 21G are located on a vertical line from the ground point 22C toward the sky, and sample points 21D and 21H are located on a vertical line from the ground point 22D to the sky.

Each dimension of the sample 20 can be grasped in advance, and a camera image of a spatial coordinate by eight sample points 21A to 21H of each vertex in the cube is acquired and recorded in the image recording device 16. Thereafter, the arithmetic unit 17 can associate the spatial coordinates of the sample points 21A to 21H at the respective vertices in the cube with the coordinates of the two camera images to obtain calibration data.
Using the calibration data obtained here, as shown in FIG. 1, the on-site photographed image data is analyzed by the photographed data analysis means, and the image data analyzed by the analysis image output means is output.

In the above embodiment, since the scale model data is “1/30”, the calibration data is used after being multiplied by 30 in the calibration of the on-site captured image data.
If the scale model data is set to “1/20” and “1/10”, the error of the calibration data acquired based on the scale model photographing means becomes small, but the scale model photographing means is installed. The area to do becomes wide.

  FIG. 3 shows a case in which the reference model 20 is not installed as a structure on the ground but a subject 30 that can move and stop a desired spatial coordinate is used. The subject 30 is moved to sample points 21A to 21H at eight vertices of, for example, a rectangular parallelepiped (cube) to be the reference model 20, for example, using a radio controlled helicopter having a GPS function and balloons 34A, 34B,. Thus, the subject 30 stopped at each of the sample points 21A to 21H is photographed by the two cameras 11L and 11R.

  The radio control helicopter and the balloon used here are mounted with LEDs capable of emitting light with different identification colors such as red, blue, yellow, and green. Then, each time the radio control helicopter stops at each of the sample points 21A to 21H, a different identification color is displayed by light emission. The radio controlled helicopter 31 can be photographed by the two cameras 11L and 11R while confirming the displayed identification color and the positions of the sample points 21A to 21H. This can reduce human error.

Also, for example, the LED may be able to generate several different trigger signals. Each time the radio control helicopter stops at each of the sample points 21A to 21H, a different trigger signal is generated, so that a command is issued to indicate that the timing for photographing the radio control helicopter with the two cameras 11L and 11R is ready. Can do. The spatial coordinates at this time can be grasped and confirmed by functioning in the remote control device.
If the stop at each of the sample points 21A to 21H and the movement operation to the next sample point are performed like a single stroke, photographing at the calibration point can be performed efficiently.

The above-mentioned LEDs are prepared in three different light emission colors, and a pattern in which one of the three lights is lit in combination of three kinds and three, and a pattern in which all three kinds are lit up. One type can be secured. For example, if a pattern is used in which all three are turned on at the two farthest points, the photographing data can be acquired at eight sample points.

  The present invention has applicability in an environmental assessment research business that requires stereo imaging of a wide area, a wind power generator manufacturing business, an aerospace maintenance business, a bird research business in environmental impact assessment, and the like.

10 Calibration data acquisition device 11 Camera (photographing means)
11L Left camera 11R Right camera 12L Camera 11L lens direction (shooting direction)
Direction of lens of 12R camera 11R (shooting direction)
13 Line 14L connecting cameras 11L and 11R Left azimuth reference line 14R Right azimuth reference line 15 Imaging control device 16 Image recording device 17 Computing device (calibration data computing means)
20 Reference model 21A-21H Sample point 22A-22D Point 30 Subject 34A-34H Balloon 36A-36H Identification color (of balloon)
50 Spatial region L Actual field distance θL Shooting angle θR in the shooting direction of the left camera 11L Shooting angle in the shooting direction of the right camera 11R

Claims (5)

A device for acquiring calibration data necessary for the captured image data, using the imaging condition data when the captured spatial image is obtained by stereo imaging of the space area of the site,
On-site shooting condition data input means for inputting shooting condition data on two cameras that shoot from different directions with respect to the space area of the site;
Scale calculation means for calculating scale model data for performing stereo shooting scaled using the shooting condition data;
A scale model photographing means for photographing a calibration sample in stereo with two cameras set apart from each other using the scale model data;
Calibration data computing means for computing calibration data using the calibration sample photographed by the scale model photographing means;
With
The space area of the site is a space where the distance between the two cameras is 600 meters ,
The two cameras are cameras using C-MOS sensors, and simultaneously photograph the space area of the site using a photographing control device,
The position data of the camera and the direction data of the shooting direction in the shooting condition data are acquired by the GPS device provided in each camera,
The scale model photographing means is a calibration data acquisition device that performs stereo photographing with two cameras separated by 10 to 40 meters. Using the calibration data and the scale model data, comprising three-dimensional analysis means for three-dimensional analysis of the captured image data,
2. The calibration data acquisition apparatus according to claim 1, wherein the three-dimensional analysis means includes a PTV analysis algorithm. The calibration data acquisition apparatus according to claim 1, wherein the calibration sample photographed by the scale model photographing unit emits light by changing an identification color for each sample point. A computer program for obtaining calibration data necessary for the captured image data, using the imaging condition data when the captured spatial image is obtained by stereo imaging of the space area of the site,
The program includes an on-site shooting condition data input procedure for inputting shooting condition data relating to a camera using two C-MOS sensors for shooting at a distance of 600 meters from different directions with respect to the space area of the site;
A scale calculation procedure for calculating scale model data for performing stereo shooting scaled using the shooting condition data;
Using the scale model data, a scale model photographing procedure for photographing a calibration sample in stereo with two cameras set 10 to 40 meters apart from each other;
Let the computer execute a calibration data calculation procedure for calculating calibration data using a calibration sample imaged by the scale model imaging procedure,
The computer program which the GPS apparatus with which each camera acquired the position data of the camera in the said imaging | photography condition data, and direction data of an imaging | photography direction was acquired.   The computer program according to claim 4, further comprising a three-dimensional analysis procedure for three-dimensionally analyzing the captured image data using the calibration data and the scale model data.

Images