JP2006349443A - Camera calibration device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera calibration device capable of accurately estimating the center of an image based on linear feature points of a part of the image. <P>SOLUTION: An image center calculating means 12 sets feature points (feature points arranged linearly in the lateral direction of a marker 20) arranged linearly in the horizontal direction of the image as feature points for determining the linearity, and determines an approximate straight line along these feature points. The calculating means sets the average of distances between the approximate straight line and the feature points as a linearity cost, and classifies the straight lines (parallel straight lines passing through feature points in the vertical direction of the marker 20) passing through feature points arranged in the perpendicular direction of the image into two right and left groups. The calculating means determines vanishing points from a plurality of straight lines in each group, and calculates the distance between two determined vanishing points. The calculating means sets this distance as a vanishing point identity cost, calculates the linearity cost and vanishing point identity cost while changing the image centers C<SB>x</SB>and C<SB>y</SB>within a set range, and determines the image center at which the cost calculated based on the linearity cost and vanishing point identity cost is minimum. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a camera calibration device that calibrates the imaging direction of a camera.

  In recent years, in the industrial field, in order to perform monitoring and object detection by a camera, a camera calibration device that calibrates the imaging direction of the camera by obtaining the distance and direction from the camera to the imaging object has been developed.

  For example, when a camera is mounted on a vehicle and a vehicle travel guideline is drawn on an image, camera calibration is required to accurately determine the positional relationship between the road surface and the camera and calibrate the imaging direction of the camera.

  As such a camera calibration device, after being installed in a vehicle, a target bar placed on the floor surface is imaged by a camera, and a roll angle is set so that the target bar in the captured image substantially matches a predetermined window. In some cases, the tilt angle and the pan angle are adjusted, and the obtained angle information is stored as camera parameters in a storage unit inside or outside the camera calibration apparatus (for example, see Patent Document 1).

  In addition, a camera calibration image having a camera calibration reference position and identification information for identifying the reference position is captured by the camera, and the camera calibration parameter is automatically set based on the reference position and the identification information in the captured image. There is also one that calculates (for example, see Patent Document 2).

  Here, the image center obtained by the camera calibration device will be described.

  FIG. 7 is a diagram for explaining a coordinate system used for camera calibration. FIG. 7A shows a camera coordinate system (x, y, z) expressed by a pinhole camera model.

A point P in the camera coordinate system is projected onto a projection plane at a focal length f from the camera origin. If this projection plane is a sensor coordinate system (x _u , _yu ) with no distortion, the camera coordinate system has the relationship of (Equation 1).

Actually, since the lens has distortion, the relationship between the sensor coordinate system (x _d , y _d ) with distortion and the sensor coordinate system without distortion in consideration of the lens distortion is (Equation 2), (Equation 2) There is a relationship 3).
_{x d + D x = x u} y d + D y = y u ( Equation 2)

  Here, κ is a constant representing the degree of distortion.

FIG. 7B is a diagram showing an image projected on the projection plane. The real image coordinate system (x _f , y _f ) is a coordinate system corresponding to the image on the monitor, and is a two-dimensional coordinate system with the upper left corner of the image as the origin. The actual image coordinate system and the sensor coordinate system with distortion have the relationship of (Equation 4).
x _f = s _x d _x ^-1 x _d + C _x y _f = d _y ^-1 y _d + C _y (Formula 4)
Here, d _x and _dy are distances between pixels in the x and y directions. The inter-pixel distance is an actual distance between pixels converted on the projection plane. Further, s _x is called a scale factor, and is a parameter generated during image capture and can be regarded as a constant. C _x and _Cy are image centers.

The parameters to be obtained by the camera calibration device are the image centers C _x and C _y . f and κ are determined by the camera specification, and κ is given as a constant and a distortion ratio corresponding to the distance from the image center.

  FIG. 8 is a block diagram showing a conventional camera calibration apparatus. As shown in FIG. 8, the conventional camera calibration apparatus is an image obtained from an image of a marker 101 that includes linearly aligned portions, a camera 102 that is a calibration target that images the marker 101, and a marker 101 that the camera 102 captures. Image center calculation means 103 for calculating the center.

  FIG. 9 is a perspective view showing the positional relationship between the marker 101 and the camera 102 in the conventional camera calibration apparatus.

  FIG. 10 is a diagram illustrating a marker image in a sensor coordinate system with distortion and a marker image in a sensor coordinate system without distortion.

In such a conventional camera calibration apparatus, the marker image in the sensor coordinate system with distortion as shown in FIG. 10A is converted into sensor coordinates without distortion as shown in FIG. Using the fact that it can be converted like a marker image of the system, the square vertex of the marker 101 is used as a feature point, and the coordinates of the feature point in the real image coordinate system are converted into sensor coordinates without distortion according to (Equation 2) to (Equation 4). The coordinates C in the system are converted, and the centers C _x and C _y are obtained so that the coordinates of the converted feature points are arranged in a straight line.

Specifically, the image center calculation unit 103 calculates the actual image coordinates of n feature points arranged in a straight line from the image obtained by capturing the marker 101 with the camera 102 with the square vertex of the marker 101 as the feature point. . The actual image coordinates (X _i, Y _i) When i = 1 to n, not the center C _x of the initial value, based on C _y from equation (2) the actual image coordinates distortion by (Equation 4) Sensors The coordinates (X _ui , Y _ui ) i = 1 to n of the coordinate system are converted.

From the converted (X _ui , Y _ui ), an approximate straight line equation is obtained by the least square method, and the sum (linearity cost) of the distance between this approximate straight line and (X _ui , Y _ui ) is calculated.

The linearity cost is calculated while changing the values of C _x and _Cy within a set range, and the center that minimizes the linearity cost is obtained.
JP 2001-245326 A JP 2003-194520 A

  However, in the conventional camera calibration device, if the feature point of the marker is not captured in the entire image, the estimation error of the image center becomes large, and the image center can be accurately determined only by the linearity of a part of the feature points. There was a problem that it could not be estimated.

  The present invention has been made to solve the conventional problems, and an object of the present invention is to provide a camera calibration apparatus that can accurately estimate the center of an image from a linear feature point of a part of the image.

  The camera calibration apparatus according to the present invention includes an imaging means for imaging a feature point group having at least one linear portion arranged in a straight line and a plurality of parallel line portions arranged on at least four parallel lines, Image center calculating means for obtaining an image center based on a linearity evaluation value of the straight line portion and an identity evaluation value of a plurality of vanishing points obtained from the parallel line portion of an image captured by the image pickup means have.

  With this configuration, the image center is obtained based on the linearity evaluation value of the straight line part and the identity evaluation value of a plurality of vanishing points obtained from the parallel line part, and the feature points arranged in a straight line are included in a part of the image. Thus, the center of the image can be estimated with high accuracy.

  Here, the image center calculation means is configured to use an average value of the distance between the approximate straight line obtained from the feature point of the straight line portion and the feature point of the straight line portion as the linearity evaluation value.

  With this configuration, the linearity evaluation value can be easily obtained.

  Further, the image center calculating means uses a sum of distances of the plurality of vanishing points as the identity evaluation value.

  With this configuration, the vanishing point identity evaluation value can be easily obtained.

  According to the present invention, since the image center is obtained based on the linearity evaluation value of the straight line portion and the identity evaluation values of a plurality of vanishing points obtained from the parallel line portion, the feature points arranged in a straight line are imaged. If it is in a part of the image center, the image center can be estimated with high accuracy.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a camera calibration apparatus according to a first embodiment of the present invention.

  In FIG. 1, the camera calibration apparatus according to the present embodiment includes a camera 11 that is a calibration target for capturing an image, and an image center calculation unit 12 that calculates an image center from an image of a marker 20 captured by the camera 11. Yes.

  The marker 20 includes a plurality of feature points arranged in at least one straight line and a plurality of feature points arranged on each of at least four parallel straight lines.

  Then, the camera 11 is disposed obliquely above the marker 20, and the marker 20 is imaged by the camera 11. At this time, the camera 11 and the camera 11 are arranged such that the parallel straight lines extend from the front of the camera to the back of the front so that at least four parallel straight lines passing through the plurality of feature points disposed on the marker 20 have vanishing points. The marker 20 is arranged.

  FIG. 2 is a perspective view showing the positional relationship between the camera 11 and the marker 20 in the present embodiment.

  In the present embodiment, two markers 20 (first marker 20a, second marker 20) each having six intersections of two grid-like straight lines in the horizontal direction and three in the vertical direction are arranged. 2), the horizontal lines of the markers 20b are arranged so that the horizontal lines thereof are the same straight line, and the vertical lines of the respective horizontal lines are parallel to each other, as shown in FIG. The camera 11 was arranged such that the straight line in the vertical direction of the marker 20 extends from the front of the camera to the back of the front so that the straight line is parallel to the horizontal direction of the image captured by the camera 11.

  If the center of the image is accurate, a straight line in the real world becomes a straight line in the camera coordinate system, and therefore a straight line even in a sensor coordinate system without distortion. If the image center is accurate, the vanishing points of the straight lines in the sensor coordinate system without distortion are the same point.

  FIG. 3 is a diagram illustrating an image of the marker 20 captured by the camera 11.

  The image center calculation means 12 uses the feature points arranged in a straight line in the horizontal direction of the image (feature points arranged in a straight line in the horizontal direction of the marker 20) as the feature points for determining the linearity. Convert image coordinates to undistorted sensor coordinates, find an approximate line based on the undistorted sensor coordinates of each feature point, and linearize the average of the distance between the approximate line and sensor points without distortion Cost.

  Further, the image center calculating unit 12 sets a straight line passing through the feature points arranged in the vertical direction of the image (a parallel straight line passing through the feature points in the vertical direction of the marker 20) as a group passing through the feature points of the first marker 20a. Grouped into groups passing through the feature points of the second marker 20b (grouped to the left and right), converted the actual image coordinates of these feature points into sensor coordinates without distortion, and sensor coordinates without distortion of each feature point The vanishing point is obtained from a plurality of straight lines of each group based on the above, the distance between the obtained two vanishing points is calculated, and this distance is defined as the vanishing point identity cost. In addition, what is necessary is just to make it the sum total of the distance of the vanishing point of each group, when making it into three or more groups.

Image center calculating unit 12 obtains an image center C _x, the cost of the C _y in (Equation 5).
Cost = A × linearity cost + B × vanishing point identity cost (Formula 5)
Here, A and B are constants satisfying A> 0 and B> 0.

The image center calculation means 12 calculates the cost by (Equation 5) while changing the image centers C _x and _Cy within the set range, and obtains the image center that minimizes the cost.

  FIG. 4 is a flowchart for explaining processing for calculating the image center.

As shown in FIG. 4, the image center calculating means 12, C _x, set the initial value C _y (S11), and converts the sensor coordinate undistorted real image coordinates of the feature points (S12).

  Then, an approximate straight line is obtained by the least square method from the sensor coordinates without distortion of the feature points arranged in a straight line (S13). The average of the distance between the obtained approximate straight line and each feature point is obtained to obtain the linearity cost (S14).

  Next, parallel straight lines passing through the feature points arranged in the vertical direction of the image are divided into two left and right groups including a plurality of straight lines, the vanishing points of the straight lines of each group are obtained (S15), and the distances of the vanishing points of the obtained groups are obtained. And the vanishing point has the same cost (S16).

From linearity cost and vanishing point identity costs determined to calculate the cost by (Equation 5), C _x, it is stored together with the value of C _y (S17).

Adding a search step size to C _y (S18), C _y determines whether less than (initial value + calculation width of C _y) upper limit of the search (S19).

If the added C _y is smaller than the upper limit of the search back to S12, and converts the search step size C _y obtained by adding the actual image coordinates of the feature points without the sensor coordinate distortion, subsequent processing Do.

If the added C _y is greater than or equal to the upper limit value of the search by adding the search step size to C _x (S20), (initial value + calculation width of C _x) C _x is the upper limit of the search determines smaller (S21).

If the added C _x is smaller than the upper limit value of the search, the process returns to S12, and the actual image coordinates of the feature points are converted into sensor coordinates without distortion for C _x with the added search step size, and the subsequent processing is performed. Do.

If the added C _x is equal to or higher than the upper limit of the search, the calculated cost is the lowest C _x, and the image around the C _y (S22).

  FIG. 5 is a graph showing the relationship between the value of the Y coordinate and the linearity cost when the X coordinate of the image center in the present embodiment is fixed and the Y coordinate is changed.

  As shown in FIG. 5, it can be seen that the value of the Y coordinate of the image center where the linearity cost of the feature point along the X axis is the minimum value is substantially equal to the true value of the Y coordinate of the image center.

  FIG. 6A is a graph showing the relationship between the value of the X coordinate and the linearity cost when the Y coordinate of the image center in the present embodiment is fixed and the X coordinate is changed.

  FIG. 6B is a graph showing the relationship between the value of the X coordinate and the vanishing point identity cost when the Y coordinate of the image center is fixed and the X coordinate is changed in the present embodiment.

  Comparing the two graphs, the vanishing point identity cost is more correlated with the image center X coordinate than the linearity cost, and the image center X coordinate value at which the vanishing point identity cost is minimum is It can be seen that it is almost equal to the true value of the X coordinate.

  Thus, in the present embodiment, the linearity cost is calculated from the feature points arranged in a line in the horizontal direction, the feature points arranged on a straight line parallel to the vertical direction are grouped on the left and right, and The vanishing point is calculated from parallel straight lines that pass through the feature points, the vanishing point identity cost is calculated from the vanishing point of each group, the image center cost is calculated from the linearity cost and the vanishing point identity cost, and this cost is minimized. Therefore, the image center can be accurately estimated from the feature points arranged in a straight line in a part of the image.

  As described above, the camera calibration device according to the present invention has an effect that the center of the image can be accurately estimated if the feature points arranged in a straight line are in a part of the image, and the imaging direction of the camera is calibrated. It is useful as a camera calibration device.

1 is a block diagram of a camera calibration apparatus according to a first embodiment of the present invention. The perspective view which shows the example of the positional relationship of the camera and marker of the camera calibration apparatus in the 1st Embodiment of this invention The figure which shows the example of an image of the marker imaged with the camera of the camera calibration apparatus in the 1st Embodiment of this invention The flowchart for demonstrating operation | movement of the camera calibration apparatus in the 1st Embodiment of this invention The graph which shows the relationship between the value of Y coordinate and linearity cost when X coordinate of the image center of the camera calibration apparatus in the 1st Embodiment of this invention is fixed and Y coordinate is changed. (A) A graph showing the relationship between the value of the X coordinate and the linearity cost when the Y coordinate of the image center of the camera calibration apparatus according to the first embodiment of the present invention is fixed and the X coordinate is changed. The graph which shows the relationship between the value of X coordinate and vanishing point identity cost when fixing the Y coordinate of the image center of the camera calibration apparatus in 1st Embodiment of an invention, and changing X coordinate (A) The figure which shows the camera coordinate system expressed with a pinhole camera model (b) The figure which shows the image projected on the projection surface Block diagram of a conventional camera calibration device The perspective view which shows the example of the positional relationship of the camera and marker of the conventional camera calibration apparatus (A) The figure which shows the marker image of a sensor coordinate system with distortion (b) The figure which shows the marker image of a sensor coordinate system without distortion

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Camera 12 Image center calculation means 20 Marker 20a 1st marker 20b 2nd marker 101 Marker 102 Camera 103 Image center calculation means

Claims (3)

An image pickup means for picking up a feature point group having at least one straight line portion and a plurality of parallel line portions respectively arranged on at least four parallel lines; and an image of the image picked up by the image pickup means, An apparatus for calibrating a camera, comprising: an image center calculating means for determining an image center based on a linearity evaluation value of the straight line portion and an identity evaluation value of a plurality of vanishing points obtained from the parallel line portion. The image center calculation means uses, as the linearity evaluation value, an average value of a distance between an approximate straight line obtained from a feature point of the straight line portion and a feature point of the straight line portion. The camera calibration device described. The camera calibration apparatus according to claim 1, wherein the image center calculation unit uses a sum of distances of the plurality of vanishing points as the identity evaluation value.

Images