JP2009168461A - Camera calibration device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera calibration device for calibrating a camera by a marker on a negligible position of a relative position relation in a horizontal direction with a vehicle. <P>SOLUTION: This device includes a common marker camera parameter optimization means 300 described as follows: by using one dimensional value of a position coordinate of a common marker 200 in a world coordinate system in a three-dimensional space as a known value, two unknown residual dimensional values of the position coordinate of the common marker are calculated from an image coordinate of the common marker 200 when being imaged by the camera A, a known dimensional value of the position coordinate, and a camera parameter applied to the camera A; an image coordinate of the camera B of the common marker 200 is calculated from the calculated position coordinate of the common marker 200 and a camera parameter of the camera B; and each camera parameter of the two cameras is adjusted so that a distance between the calculated image coordinate of the camera B of the common marker 200 and the image coordinate of the common marker 200 when being imaged by the camera B becomes shorter than a threshold. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a camera calibration apparatus, and more particularly to a camera calibration apparatus that calibrates a camera installed in a vehicle or the like.

  In recent years, in the industrial field, in order to perform monitoring and object detection by a camera, a camera calibration apparatus for obtaining a distance and direction from a camera to an imaging object has been developed.

  For example, when a camera is mounted on a vehicle and a vehicle travel guideline is drawn on an image, camera calibration is required to accurately determine the positional relationship between the road surface and the camera.

  In the case of a camera mounted on a mass-produced vehicle, since the camera is installed at a predetermined position of the vehicle, the positional relationship between the road surface and the camera is basically the same, but when attaching the camera to the vehicle, The camera installation position varies. FIG. 5 is an explanatory diagram of installation errors when a camera is attached to a vehicle.

  FIG. 5 shows a camera 100 installed in a vehicle 500 at a factory. The solid line starting from the camera 100 indicates the ideal optical axis of the camera 100 in design, and the broken line is the optical axis after installation, which causes an installation error. As described above, the camera calibration device eliminates an installation error caused when the camera 100 is attached to the vehicle 500.

  As a conventional camera calibration device, a device that calibrates a camera using camera parameters for eliminating an installation error is known (see, for example, Patent Document 1 and Patent Document 2). FIG. 6 is a configuration diagram of a conventional camera calibration apparatus. As shown in FIG. 6, the conventional camera calibration apparatus includes a camera 100, an individual marker 600, and an individual marker camera parameter optimization unit 700. A similar conventional camera calibration device is disclosed in Patent Document 2.

  FIG. 7 is an explanatory diagram of the positional relationship between the vehicle 500, the camera 100, and the individual marker 600 in the conventional camera calibration apparatus. FIG. 8 shows an image when the camera 100 images the individual marker 600. FIG. 9 shows the positional relationship between the world coordinate system and the camera coordinate system.

Hereinafter, the operation of the conventional camera calibration apparatus will be described. As shown in FIG. 7, a world coordinate system (x _w , y _w , z _w ) in which the parallel direction to the actual road surface is the x axis, the vehicle traveling direction is the y axis, and the vertical direction to the road surface is the z axis. And the camera coordinate system (x _c , y _c , z _c ) expressed by a pinhole camera model or the like can be expressed by the following [Equation 1].

Here, r ₁ to r ₉ are parameters representing rotation from the world coordinates, and T _x , T _y , and T _z are parameters representing parallel movement from the world coordinates, and these parameters are camera parameters. It is.

The camera coordinate system (x _c , y _c , z _c ) is based on the lens center as the origin and the z axis is based on the optical axis direction of the camera. Also represent the camera coordinate system _{(x c, y c, z} c) _{In, z} c = f (focal length) of the sensor image coordinate system to the coordinate system is parallel to the XY plane at a position _(X u, _{Y u)} and .

The sensor image coordinate system (X _u , Y _u ) can be expressed by the following [Equation 2] and [Equation 3] using the camera coordinate system (x _c , y _c , z _c ).

The relationship between the image coordinates (X _f , Y _f ) that are two-dimensional coordinates corresponding to the image on the camera monitor and the sensor image coordinates (X _u , Y _u ) is expressed by the following [Equation 4] and [Equation 5]. Can be expressed as

Here, d _x and _dy are coefficients for performing scale conversion, and C _x and C _y are image center coordinates.

  As shown in FIG. 7, for example, the center axes of the vehicle 500 and the individual marker 600 are aligned, and the individual marker 600 is arranged 1 m ahead of the bumper of the vehicle 500, and then the relative positional relationship between the individual marker 600 and the vehicle 500. Measure. The feature points of the individual marker 600 use four rectangular vertices of the individual marker 600. The position coordinates of the four vertices of the individual marker 600 in the world coordinate system are all known.

After the camera 100 is installed in the vehicle 500, the individual marker 600 is imaged. The description will be made with attention paid to the first point among the four vertices of the individual marker 600. The known position coordinate of the one point is assumed to be ( _Xw1 , _Yw1 , _Zw1 ).

As shown in FIG. 8, the conventional camera calibration apparatus uses [Equation 1] to [Equation 5] to calculate the marker position (X X) from the position coordinates (X _w1 , Y _w1 , Z _w1 ) of the feature points. _f1 , _Yf1 ) is obtained. Further, the marker position (X _f1T , Y _f1T ) on the image is specified from the captured actual image. It should be noted that the vertex of the marker position on the image is designated by a cursor or the like from the actual captured image as disclosed in FIG. 10 of Patent Document 2, for example.

  Here, if the camera parameters are appropriate, the calculated image coordinates are equal to the actual image coordinates (the distance between the coordinates is 0). However, the conventional camera calibration apparatus uses [Equation 6]. The camera parameters are calculated so that the value obtained in (1) is minimized.

Here, the calculated marker position (X _fn , Y _fn ) is the vertex of the n th marker obtained by calculation, and the marker position (X _fnT , Y _fnT ) on the image is the n th It is at the vertex of the marker position on the actual image. As a method for calculating camera parameters so that the value obtained by [Equation 6] is minimized, there is a steepest descent method.
JP 2006-135621 A JP 2004-4043 A

  However, the above-described conventional camera calibration device has a problem that it is necessary to obtain the relative positional relationship between the marker and the vehicle, which is troublesome.

  An object of the present invention is to provide a camera calibration apparatus that can calibrate a camera using a marker at a position where the relative positional relationship between the marker and a vehicle can be ignored.

A camera calibration apparatus according to the present invention includes a plurality of cameras that capture images by applying camera parameters including predetermined design values, a shared marker that is in a common imaging range of the plurality of cameras, and world coordinates in a three-dimensional space. When the one-dimensional value of the position coordinate of the shared marker in the system is known, and the plurality of cameras are an A camera and a B camera, the image coordinates of the shared marker when captured by the A camera, From the value of the known dimension of the position coordinate and the design value of the camera parameter applied to the A camera, the value of the remaining two dimensions of the position coordinate of the shared marker that is not known is calculated, and the calculated shared marker Image coordinates of the B camera of the shared marker are calculated from the position coordinates of the camera and the design values of the camera parameters applied to the B camera. The camera parameters of the A camera and the B camera are set so that the distance between the image coordinates of the B camera of the shared marker and the image coordinates of the shared marker when captured by the B camera is equal to or less than a predetermined threshold. And a camera parameter optimization unit for adjustment.
With this configuration, when the value of one dimension of the position coordinate of the shared marker in the world coordinate system of the three-dimensional space is known, the image coordinate of the shared marker when captured by the A camera, the known dimension of the position coordinate , And the camera parameters applied to the A camera, values of the remaining two dimensions of the unknown shared marker position coordinates are calculated. From the calculated shared marker position coordinates and the B camera camera parameters, the shared marker is calculated. The image coordinates of the B camera are calculated, and the A camera is set such that the distance between the calculated B camera image coordinates of the shared marker and the image coordinates of the shared marker when captured by the B camera is equal to or less than a predetermined threshold. In order to adjust the camera parameters of camera B and B Camera calibration can be.

  As described above, the present invention provides a camera calibration apparatus that can calibrate a camera with a marker at a position where the relative positional relationship between the marker and the vehicle can be ignored.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a camera calibration apparatus according to an embodiment of the present invention. The camera calibration apparatus shown in FIG. 1 includes a camera 100, a shared marker 200, and a shared marker camera parameter optimization unit 300.

  FIG. 2 is a plan view showing the positional relationship between the vehicle 500 and the shared marker 200. A vehicle 500 is provided with a front camera 110 and a side camera 120. The vehicle 500 is arranged so that the shared marker 200 enters the imaging range of the front camera 110 and the side camera 120. Further, the shared marker 200 is a square and placed on the road surface.

In the embodiment of the present invention, in the world coordinate system (X _w , Y _w , Z _w ) shown in FIG. 2, the XY plane including the X axis and the Y axis is equal to the road surface, and Z _w is 0 on the XY plane. As an example, the case where the Z-axis is taken upward is described. Here, it is assumed that the front camera 110 is the A camera 110 and the side camera 120 is the B camera 120.

  FIG. 3 is an explanatory diagram of images taken by the A camera 110 and the B camera 120. The image 210 is obtained by being imaged from the A camera 110, and the image 220 is obtained by being imaged from the B camera 120. A shared marker 200 is captured in each image.

Here, in the embodiment of the present invention, the four vertices of the shared marker 200 are feature points. The position coordinates of the shared marker 200 in the world coordinate system are assumed to be (X _w , Y _w , Z _w ). Note that the position coordinate _Zw of the shared marker 200 becomes 0 because the shared marker 200 is placed on the road surface.

Also, _let the image coordinates of the feature points in the image 210 when captured by the A camera 110 be (AX _f , AY _f ), and the image coordinates of the corresponding feature points in the image 220 when captured by the B camera 120. (BX _f , BY _f ). It is assumed that the design value of the camera parameter is determined in advance from the position / angle attached to the vehicle 500.

  The shared marker camera parameter optimizing means 300 executes a process for obtaining a camera parameter to be optimized based on a given design value of the camera parameter.

  The operation will be described with reference to FIGS. FIG. 4 is a flowchart of processing performed by the shared marker camera parameter optimization unit 300.

After the camera 100 is installed in the vehicle 500, the shared marker 200 is captured by the A camera 110 and the B camera 120, and the image coordinates (AX _f , AY _f ) on the actual image at the feature points of the shared marker 200 and the actual Image coordinates (BX _f , BY _f ) on the image are processed by the shared marker camera parameter optimization means 300. Note that the feature point is designated by a cursor or the like from the image captured as described above.

First, for one of the feature points, from the image coordinates (AX _f , AY _f ) of the A camera 110, the position coordinates Z _w (= 0), and the design values of the camera parameters of the A camera 110, [Equation 1] Then, the position coordinates (X _w , Y _w ) are calculated using [Equation 5] (step S1).

Next, based on the position coordinates (X _w , Y _w , Z _w ) of the shared marker 200 calculated in step S1 and the design values of the camera parameters of the B camera 120, [Equation 1] to [Equation 5] are used. The calculated image coordinates (CalculatedBX _f , CalculatedBY _f ) are calculated (step S2).

Next, the distance between the image coordinates (CalculatedBX _f , CalculatedBY _f ) calculated in step S2 and the actual image coordinates (BX _f , BY _f ) of the B camera 120 is calculated (step S3), and all feature points are calculated. An average distance is calculated from the distance obtained as a result of the execution of steps S1 to S3.

  It is determined whether the average distance calculated in step S3 is equal to or smaller than a predetermined threshold value (step S4). If the average distance is equal to or smaller than the threshold value, the process is terminated, and the camera parameters of the A camera 110 and the B camera 120 at that time are finished. Is the optimized camera parameter. If the average distance is larger than the threshold value in step S4, the camera parameters of the A camera 110 and the B camera 120 are adjusted so that the distance becomes smaller than the threshold value (step S5), and the process returns to step S1.

  Further, an individual marker 600 whose position coordinates are all known may be used in combination. When used together, the distance between the image coordinates obtained from the position coordinates of the individual marker 600 and the actual image coordinates is obtained as in the conventional example, and the average value of the distance of the shared marker 200 is equal to or less than a predetermined threshold in step S4. In addition, the camera parameters of the A camera 110 and the B camera 120 may be adjusted, and the distance between the image coordinates obtained from the position coordinates of the individual marker 600 and the actual image coordinates and the distance between the shared marker 200 is equal to or less than a predetermined threshold. The camera parameters of the A camera 110 and the B camera 120 may be adjusted so that

Further, the shared marker camera parameter optimization unit 300 includes the image coordinates (AX _f , AY _f ) of the shared marker 200 of the A camera 110, the Z _w (= 0) of the position coordinates of the shared marker 200, and the camera of the A camera 110. The position coordinates (X _w , Y _w ) are obtained from the parameters, and similarly, the image coordinates (BX _f , BY _f ) of the shared marker 200 of the B camera 120, Z _w (= 0) of the position coordinates of the shared marker 200, and The position coordinates (X _w , Y _w ) are obtained from the camera parameters of the B camera 120, and the camera parameter that minimizes the distance between these two position coordinates may be used as the optimized camera parameter.

  As described above, the camera calibration apparatus according to the embodiment of the present invention has the image coordinates when the shared marker in the common imaging range of the A camera 110 and the B camera 120 is imaged, the A camera 110 and the B camera 120. In order to adjust the optimal camera parameter from the camera parameter applied to the Z and the known value of the position coordinate Z of the shared marker 200, the marker at a position where the horizontal relative positional relationship between the shared marker 200 and the vehicle 500 can be ignored. Can calibrate the camera.

  As described above, the present invention has an effect that the camera can be calibrated with the marker at a position where the relative positional relationship between the marker and the vehicle can be ignored, and the camera calibration device that calibrates the camera installed in the vehicle or the like. Useful as.

The figure which shows the structure of the camera calibration apparatus which concerns on embodiment of this invention Plan view showing the positional relationship between the vehicle and the shared marker Explanatory drawing of an image when it is imaged with A camera and B camera Flow chart of processing performed by shared marker camera parameter optimization means Illustration of installation error when a camera is attached to the vehicle Configuration of conventional camera calibration device Explanatory drawing of the positional relationship of vehicles, cameras, and individual markers in a conventional camera calibration device The figure which shows an image when a camera images an individual marker Diagram showing the positional relationship between the world coordinate system and the camera coordinate system

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Camera 110 Front camera, A camera 120 Side camera, B camera 200 Shared marker 210, 220 Image 300 Shared marker camera parameter optimization means (camera parameter optimization means)
500 Vehicle 600 Individual marker 700 Individual marker camera parameter optimization means

Claims (2)

A plurality of cameras for imaging by applying camera parameters including predetermined design values;
A shared marker in a common imaging range of the plurality of cameras;
When the value of one dimension of the position coordinate of the shared marker in the world coordinate system of the three-dimensional space is known, and the plurality of cameras are an A camera and a B camera,
From the image coordinates of the shared marker when imaged by the A camera, the value of the known dimension of the position coordinate, and the design value of the camera parameter applied to the A camera, the position coordinate of the shared marker that is not known Calculate the values of the remaining two dimensions of
From the calculated position coordinates of the shared marker and the design values of the camera parameters applied to the B camera, the image coordinates of the B camera of the shared marker are calculated,
The distance between the calculated image coordinates of the B camera of the shared marker and the image coordinates of the shared marker when captured by the B camera is equal to or less than a predetermined threshold value. A camera calibration device comprising camera parameter optimization means for adjusting camera parameters. An individual marker in which all of the three-dimensional world coordinates within the imaging range of each of the A camera and the B camera are known;
The camera coordinates of the A camera and the B camera are calculated such that the image coordinates are calculated from the camera parameters using the three-dimensional world coordinates of the individual markers, and the distance between the image coordinates and the shared marker is equal to or less than a predetermined threshold. The camera calibration device according to claim 1, further comprising camera parameter optimization means for adjusting parameters.

Images