JP2003259357A - Calibration method for camera and attachment of camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a calibration method for a camera capable of performing calibration of a monitor camera without the need for installing a marker on a road and performing traffic regulation and to provide a camera attachment. <P>SOLUTION: A normal vector of a plane including feature quantities is calculated from coordinates of a plurality of the feature quantities detected by a feature quantity detection section 4 and a road plane parameter is calculated from the normal vector of the plane and a distance L from the camera 1 to the road plane measured along the optical axis of the camera 1. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for calibrating a surveillance camera installed on a road or a facility, and a camera attachment. Here, the calculation of the road plane parameter viewed from the camera coordinate system is called calibration. The road plane parameters in the camera coordinate system are the height and direction (tilt angle, pan angle) of the camera on the road plane.
Can also be paraphrased as.

[0002]

2. Description of the Related Art In a road monitoring system, in order to measure the position and speed of a vehicle, it is necessary to calculate in advance the relationship between a camera and a road plane, which is a reference plane of a monitoring area. Japanese Unexamined Patent Publication No. 4-150591 discloses a camera calibration method for calibrating a surveillance camera by using a white line on a road. In this calibration method, the white line on the road must be a straight line and cannot be used for curved roads. In some cases, roads do not have clear features such as white lines.

Therefore, recently, a marker (characteristic point) is installed on the road, the distance between the markers is measured on the road, and the coordinate value on the image of the marker is extracted and used to monitor. I am trying to calibrate the camera. A specific method of calculating the calibration is, for example, “application of face orientation detection method using monocular image to instruction input” (IEICE Transactions D
-II, Vol. J72-D-II, No. 9, pp.
1441-1447, 1989).

That is, by using the method described in the above document, the relationship between the road plane and the surveillance camera can be obtained as follows. A camera coordinate system (X, Y, Z) whose origin is the principal point of the camera lens and an image plane (x, y) are set as shown in FIG. It is assumed that the focal length of the camera is f and the optical axis coincides with the Z axis. Here, the projected images of the three markers P _n (X _n , Y _n , Z _n ) on the road plane are Q _n.
Assuming that (x _n , y _n ) (n = 1, 2, 3), P _n exists on a straight line passing through the origins O and Q _n . Therefore, the position vector _{p n} of _{P n} can be expressed using the unit vector _{u n} scalar _{a n} representing the direction of the straight line OQ _n. p _n = u _n a _n

Where u_nIs the projected image Q_nCoordinate value of
(X_n, Y_n) Is used, it becomes as follows. u_n= (X_n, Y_n, F) / (x^Two _n, Y^Two _n, F^Two)
^1/2 Also, the scalar a_nIs origin O to P_nRepresents the distance to
And a_nP is decided_nThe three-dimensional position of
You can

Where P_nTwo pieces of information about
Maker P_m, P_nDistance l_mnCan be measured in advance
Therefore, it can be expressed as follows. ｜ p_m-P_n│ = 1_mn Also, P_m, P_nThe distance between them can also be expressed as follows. ｜ p_m-P_n｜^Two= A^Two _m+ A^Two _n-2 (u_m, U_n)
a_ma_n

Therefore, the following equation is established. a^Two _m+ A^Two _n-2 (u_m, U_n) A_ma_n= L^Two _mn Similarly, deriving the equation at each two points of the three markers,
The following simultaneous equations are obtained. a^Two ₁+ A^Two _Two-2 (u₁, U_Two) A₁a_Two= L^Two ₁₂ a^Two _Two+ A^Two _Three-2 (u_Two, U_Three) A_Twoa_Three= L^Two ₂₃ a^Two _Three+ A^Two ₁-2 (u_Three, U₁) A_Threea₁= L^Two ₃₁

By solving this simultaneous equation, the solutions a ₁ , a ₂ ,
Substituting a ₃ into p _n = u _n a _n , the three markers P _n
The three-dimensional coordinate value of can be obtained. Next, by calculating the plane passing through the three markers, the road plane in the camera coordinate system is obtained.

[0009]

Since the conventional camera calibration method is configured as described above,
If sufficient traffic control can be implemented, a marker is installed on the road, the distance between the markers is measured on the road, and the coordinate values on the image of the marker are extracted to calibrate the camera. Can be carried out. However, there was a problem that sufficient traffic regulation could not be implemented depending on the surveillance location, and the surveillance camera could not be calibrated. There was also a problem that the work required a lot of time and labor.

The present invention has been made in order to solve the above problems, and a camera capable of calibrating a surveillance camera without installing markers on the road or restricting traffic. It is intended to obtain a calibration method and a camera attachment.

[0011]

A camera calibration method according to the present invention uses a road plane parameter based on a distance from the camera to a road plane measured along an optical axis of the camera and coordinate values of a plurality of feature points. Is calculated.

The camera calibration method according to the present invention searches for a rectangular area on a plane parallel to the road plane on the vehicle from the image of the vehicle and detects the end points of the rectangular area as feature points. is there.

In the camera calibration method according to the present invention, two parts on the vehicle which are installed horizontally are mutually detected as feature points from the images of the two vehicles having different image capturing times. is there.

The camera calibration method according to the present invention detects the left and right tail lamps or the left and right headlights as feature points.

In the camera calibration method according to the present invention, the road plane parameter calculation process is repeatedly executed while changing the vehicle to be imaged, and the calculated plurality of road plane parameters are averaged. .

The attachment of the camera according to the present invention is such that a cylindrical member is attached to the lens mount of the camera, and the distance from the camera to the road plane is measured on the central axis of the cylindrical member along the optical axis of the camera. It is designed to install a meter.

In the attachment of the camera according to the present invention, the distance meter measures the distance from the camera to the road plane along the optical axis of the camera via the half mirror installed inside the cylindrical member. It is a thing.

In the camera attachment according to the present invention, the range finder is attached to the camera so that the center of the field of view of the camera and the center of the image viewed through the finder coincide.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below. Embodiment 1. FIG. 1 is a configuration diagram showing a calibration device to which a camera calibration method according to a first embodiment of the present invention is applied. In the figure, 1 is a monitoring camera, and 2 is a distance from the camera 1 to a road plane. A laser range finder (range finder) 3 that measures along the optical axis of the camera 1 controls the imaging of the camera 1 to receive the image of the vehicle captured by the camera 1 and a laser range finder 2 The control unit 4 which controls the distance measurement of the vehicle and receives the distance measured by the laser range finder 2 is a feature point detection unit which detects a plurality of feature points on the vehicle from the image of the vehicle captured by the camera 1. Reference numeral 5 is a normal vector calculation unit that calculates a normal vector of a plane including the feature points in the coordinate system of the camera 1 from coordinate values of the plurality of feature points detected by the feature point detection unit 4, and 6 is a normal line. According to the vector calculation unit 5 From the distance from the calculated normal vector and the camera 1 to the road plane is a parameter calculator for calculating a road plane parameter.

FIG. 2 is an explanatory diagram showing the relationship between the camera and the road plane, and FIG. 3 is a flowchart showing the camera calibration method according to the first embodiment of the present invention.

Next, the operation will be described. First, the control unit 3 outputs a distance measurement start command to the laser distance meter 2. When the laser distance meter 2 receives a distance measurement start command from the control unit 3,
As shown in FIG. 2, the distance L from the camera 1 to the road plane
Is measured along the optical axis of the camera 1, and the distance L as the measurement result is output to the control unit 3 (step ST1). When the control unit 3 receives the measurement result distance L from the laser range finder 2, the control unit 3 outputs the measurement result distance L to the parameter calculation unit 6.

Here, since the laser distance meter 2 measures coaxially with the optical axis of the camera 1, the distance measuring point P on the road is inevitably (0, 0, L) in the camera coordinate system. Can be approximated, and its projection point Q substantially corresponds to the image center (0, 0). Therefore, it is not necessary to install a marker for clarifying the distance measuring point P on the road, and it is not necessary to detect the coordinate value of the image corresponding to the distance measuring point P.

Next, the control section 3 outputs an image pickup start command to the camera 1. When the camera 1 receives the image capturing start command from the control unit 3, as shown in FIG. 2, the camera 1 captures an image of a general vehicle traveling in the monitoring area and outputs the image of the vehicle, which is the image capture result, to the control unit 3 (step ST2). The control unit 3
When receiving the image of the vehicle from the camera 1, outputs the image to the feature point detection unit 4.

When the feature point detection unit 4 receives the image of the vehicle from the control unit 3, the feature point detection unit 4 detects four feature points on the vehicle from the image of the vehicle (step ST3). Specifically, as shown in FIG. 4, a rectangular area of the plane on the vehicle parallel to the road plane (see the broken line in FIG. 4) is searched from the image of the vehicle, and the end points of the rectangular area are detected as feature points. To do. Note that, in general, the feature point detection unit 4 performs image processing to detect the feature points.

The normal vector calculation unit 5 includes a feature point detection unit 4
There detects the four characteristic points, the coordinate values of the four characteristic points are calculated in the coordinate system of the camera 1, the normal vector of the plane containing the feature point _{(V x, V y, V} z) (steps ST4 ). A specific method of calculating the normal vector is, for example, “Measurement of position and orientation of marker from monocular image for artificial reality interface” (IEICE Transactions A, Vol. J79-A, No. 3, pp. 804-
812, 1996).

The parameter calculating section 6, the vector generator 5 is the normal vector _{(V x, V y, V} z) calculating the camera 1 the normal vector _{(V x, V y, V} z) and
A road plane parameter is calculated from the distance L from the road plane to the road plane (step ST5). Here, the distance measuring point P
Including (0, 0, L), the normal vector (V _x , V _y , V
_{It is sufficient} to calculate a plane having _{z 1} ) and the equation of the plane is as follows. _{V x x + V y y +} V z (z-L) = 0

As is clear from the above, the first embodiment
According to the above, the normal vector of the plane including the feature points is calculated from the coordinate values of the plurality of feature points detected by the feature point detection unit 4, and the normal vector of the plane and the optical axis of the camera 1 are measured. Since the road plane parameter is calculated from the distance L from the camera 1 to the road plane, the surveillance camera can be calibrated without setting markers on the road or restricting traffic. There is an effect that can be done. Further, since the characteristic points on the general vehicle are used, it is possible to save the trouble of preparing a dedicated vehicle and the like, and reduce the load on the worker. Further, since the end points of the rectangular area are used as the feature points, the calculation can be simplified as compared with the case of using a complicated shape.

In the first embodiment, the end points of the rectangular area of the vehicle upper surface are selected as the four characteristic points, but the present invention is not limited to this. The rectangular points are parallel to the road plane. If so, it may be another part on the vehicle. In addition, as the calculation method in step ST4, for example, “estimation of optimal posture of arbitrary polygon by monocular”
(The IEICE General Conference, D-12-132, p.
p. 339, 1997), the calculation time is longer, but it is not limited to a rectangle,
It is also possible to select the end points of any polygon of known shape. From the viewpoint of calculation error, it is desirable to select the feature points that are separated from each other, and it is better to extract the feature points from a large vehicle such as a panel truck.

Embodiment 2. In the above-described first embodiment, the rectangular area of the plane on the vehicle parallel to the road plane is searched from the image of the vehicle and the end points of the rectangular area are detected as the feature points. It is also possible to detect two parts on the vehicle, which are installed horizontally from each other and perpendicular to the traveling direction, as feature points from the image of the vehicle. Although specifically described below, FIG. 5 is a flowchart showing a camera calibration method according to the second embodiment of the present invention.

When the laser range finder 2 measures the distance L from the camera 1 to the road plane, the control section 3 outputs an image pickup start command to the camera 1 as in the first embodiment.
When the camera 1 receives an image capturing start command from the control unit 3, as shown in FIG. 6, the camera 1 continuously captures images of ordinary vehicles traveling in the monitoring area, and the control unit 3 receives continuous images of the vehicles as the image capturing results. Output (step ST11). That is, two images with different imaging times are output to the control unit 3. When the control unit 3 receives two images from the camera 1, the control unit 3 outputs the two images to the feature point detection unit 4.

When the characteristic point detection unit 4 receives the two images from the control unit 3, it detects the characteristic points on the vehicle from the two images (step ST12). Specifically, first,
The left and right tail lamps attached to the tail of the vehicle are detected as feature points from the first image. next,
Left and right tail lamps attached to the tail of the same vehicle are detected as feature points from the second image.
As a result, four feature points have been detected.
As shown in FIG. 6, the four feature points form a rectangular area of a plane parallel to the road plane.

The following description is omitted because it is similar to that of the first embodiment, but in the second embodiment, two feature points are detected from two images, so that the above-mentioned embodiment is performed. There is no need to simultaneously detect four feature points from one image as in the first embodiment. Therefore, camera 1
This is effective when the installation position is low and the upper surface area of the vehicle is not shown in the image. Further, even at night when it is generally difficult to detect feature points, at least only the tail lamps that are turned on can be detected, so that the calibration can be reliably performed.

In the second embodiment, the left and right tail lamps attached to the tail of the vehicle are detected as the characteristic points, but it is sufficient if they are two parts on the vehicle that are installed horizontally with respect to each other. For example, the left and right headlights attached to the vehicle head may be detected as the characteristic points.

Embodiment 3. In the first and second embodiments, the road plane parameter is calculated only once. However, as shown in FIG. 7, the road plane parameter calculation process is repeatedly executed while changing the vehicle to be imaged. The calculated plurality of road plane parameters may be averaged.

That is, until the number of times the road plane parameter is calculated reaches a preset number, the road plane parameter calculation process is repeatedly executed while changing the vehicle to be imaged (step ST21). Then, the road plane parameters obtained by the plurality of calculation processes are statistically smoothed to determine the final road plane parameters (step ST22). As a result, the calculation error of the road plane parameter can be statistically reduced, and the calibration accuracy can be improved.

In the third embodiment, the road plane parameters obtained by a plurality of calculation processes are statistically smoothed. However, after smoothing the normal vector obtained by a plurality of calculation processes, The road plane parameter may be calculated.

Fourth Embodiment 8 is a perspective view showing an attachment of a camera according to Embodiment 4 of the present invention. In the figure, 11 is a main body of the camera 1, 12 is a lens mount of the camera 1, and 13 is attached to a lens mount 12 of the camera 1. A cylindrical cylinder (cylindrical member), 14 is a laser range finder (rangefinder) installed on the center axis of the cylinder 13 and measuring the distance from the camera 1 to the road plane along the optical axis of the camera 1. is there.

Next, the operation will be described. When measuring the distance from the camera 1 to the road plane, the cylinder 13 having the laser range finder 14 built in on the central axis is attached to the lens mount 12 of the camera 1. In this way, the tube 13 is attached to the camera 1
Attached to the lens mount 12 of the laser rangefinder 1
4 measures the distance L coaxially with the optical axis of the camera 1. Therefore, since the coordinate value of the image corresponding to the distance measuring point P becomes the center of the image, the calibration methods of the above first to third embodiments can be applied.

Further, by using the attachment, it is not necessary to incorporate a range finder for calibration into the surveillance camera for commercialization, and it is possible to correspond to the surveillance camera already installed. Moreover, since the camera 1 and the attachment are attached to the lens mount 12, the positional relationship between the camera 1 and the attachment is uniquely determined. It is easy to install and can be easily reused for multiple cameras.

Embodiment 5. 9 is a perspective view showing an attachment of a camera according to a fifth embodiment of the present invention. In the figure, the same reference numerals as those in FIG. 8 indicate the same or corresponding parts, and therefore the description thereof will be omitted. Reference numeral 15 denotes a cylindrical cylinder (cylindrical member) which is attached to the lens mount 12 of the camera 1 and has a hollow interior, 16 is a half mirror installed inside the cylinder 15 at an angle of 45 degrees, and 17 is a cylinder 15. A laser range finder (range finder) that is installed on the side surface and measures the distance from the camera 1 to the road plane via the half mirror 16 along the optical axis of the camera 1.

In the fourth embodiment, the laser range finder 14 is used.
It was shown that the is installed on the central axis of the cylinder 13,
The laser range finder 17 may be installed on the side surface of the tube 15, and the laser range finder 14 may measure the distance L via the half mirror 16 installed inside the tube 15 at an angle of 45 degrees. . As a result, the distance L can be measured coaxially with the optical axis of the camera 1 as in the fourth embodiment.
Further, in this case, the camera 1 can capture an image of the vehicle with the attachment attached.

Sixth Embodiment FIG. 10 is a perspective view showing an attachment of a camera according to a sixth embodiment of the present invention. In the figure, reference numeral 21 denotes a laser attached to the camera 1 so that the center of the field of view of the camera 1 and the center of the image seen through the viewfinder 22 are aligned. A range finder (range finder), 22 is a finder for determining a range finder point P of the laser range finder 21, and 23 is an aiming mark indicating the center of the image.

As shown in FIG. 10, the attachment of the sixth embodiment has a laser range finder 21 and a finder 22.
It can be attached to the camera 1. Aiming mark 2 is in the center of the image seen through the viewfinder 22.
3 is overlapped, and calibration is performed in advance so that the target object that overlaps this aiming mark 23 matches the distance measuring point P. When the camera 1 is calibrated, this attachment is attached beside the camera 1 such that the center of the field of view of the camera 1 and the center of the image viewed through the finder 22 coincide with each other. The method of attachment is not particularly limited.

As a result, the laser range finder 21 measures the distance L substantially coaxially with the optical axis of the camera 1, so that the calibration methods of the above-described first to third embodiments can be applied. . Further, by using the attachment, it is not necessary to incorporate a range finder for calibration into the surveillance camera for commercialization, and it is possible to correspond to the surveillance camera already installed. In addition, since the laser range finder 21 is calibrated in advance so as to measure the distance to the target object that can be seen in the center of the finder 22, the mounting work during calibration is easy and it can be reused for multiple cameras. Will be easier.

[0045]

As described above, according to the present invention, the road plane parameter is calculated from the distance from the camera to the road plane measured along the optical axis of the camera and the coordinate values of a plurality of feature points. Since it is configured as described above, there is an effect that the surveillance camera can be calibrated without setting a marker on the road or implementing traffic regulation.

According to the present invention, since the rectangular area of the plane on the vehicle parallel to the road plane is searched from the image of the vehicle and the end points of the rectangular area are detected as the characteristic points, a complicated shape is formed. This has the effect of simplifying the calculation rather than using it.

According to the present invention, two parts on the vehicle which are horizontally installed mutually are detected as the feature points from the images of the two vehicles whose image capturing times are different from each other. Therefore, the structure is complicated. There is an effect that the characteristic point on the vehicle can be detected without inviting.

According to the present invention, the left and right tail lamps,
Alternatively, since the left and right headlights are configured to be detected as feature points, there is an effect that feature points can be reliably detected even at night when it is generally difficult to detect feature points.

According to the present invention, the calculation processing of the road plane parameters is repeatedly executed while changing the vehicle to be imaged, and the calculated plurality of road plane parameters are averaged, so that the calibration accuracy is improved. There is an effect that can be.

According to the present invention, the cylindrical member is attached to the lens mount of the camera, and the range finder for measuring the distance from the camera to the road plane along the optical axis of the camera is installed on the central axis of the cylindrical member. With this configuration, the distance from the camera to the road plane can be measured along the optical axis of the camera without incorporating the rangefinder for calibration into the surveillance camera for commercialization.

According to the present invention, since the range finder measures the distance from the camera to the road plane along the optical axis of the camera through the half mirror installed inside the cylindrical member, There is an effect that the distance from the camera to the road plane can be measured along the optical axis of the camera without incorporating a rangefinder for calibration into a surveillance camera for commercialization. Further, there is an effect that the camera can capture an image of the vehicle with the attachment attached.

According to the present invention, since the range finder is attached to the camera so that the center of the field of view of the camera and the center of the image viewed through the finder coincide, the range finder for calibration is incorporated in the surveillance camera. There is an effect that the distance from the camera to the road plane can be measured along the optical axis of the camera without commercialization.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a calibration device to which a camera calibration method according to a first embodiment of the present invention is applied.

FIG. 2 is an explanatory diagram showing a relationship between a camera and a road plane.

FIG. 3 is a flowchart showing a camera calibration method according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram showing an image of a vehicle.

FIG. 5 is a flowchart showing a camera calibration method according to a second embodiment of the present invention.

FIG. 6 is an explanatory diagram showing an image of a vehicle.

FIG. 7 is a flowchart showing a camera calibration method according to a third embodiment of the present invention.

FIG. 8 is a perspective view showing an attachment of a camera according to Embodiment 4 of the present invention.

FIG. 9 is a perspective view showing an attachment of a camera according to Embodiment 5 of the present invention.

FIG. 10 is a perspective view showing an attachment of a camera according to Embodiment 6 of the present invention.

FIG. 11 is an explanatory diagram showing a relationship between a camera and a road plane.

[Explanation of symbols]

1 camera, 2 laser range finder (rangefinder), 3 control section, 4 feature point detection section, 5 normal vector calculation section, 6
Parameter calculation unit, 11 camera body, 12 lens mount, 13 tube (cylindrical member), 14 laser range finder (rangefinder), 15 tube (cylindrical member), 16 half mirror, 17 laser range finder (rangefinder) 21) Laser rangefinder (rangefinder), 22 finder, 23 sighting mark.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kazuhiko Sumi             2-3 2-3 Marunouchi, Chiyoda-ku, Tokyo             Inside Ryo Electric Co., Ltd. F term (reference) 5B057 AA16 CA12 CH01                 5C022 AA04 AB62 AC27 AC78                 5C054 AA01 CA04 CE11 CE15 CF06                       CH01 EA01 EA05 ED07 FC03                       FC15 GB11 HA30                 5H180 AA01 CC03 CC04 DD01

Claims (8)

[Claims] 1. A method of measuring a distance from a camera to a road plane along an optical axis of the camera, detecting a plurality of feature points on the vehicle from an image of the vehicle captured by the camera, A camera calibration method for calculating a road plane parameter from a distance to a plane and coordinate values of the plurality of feature points. 2. The calibration of the camera according to claim 1, wherein a rectangular area of a plane on the vehicle parallel to the road plane is searched from the image of the vehicle, and an end point of the rectangular area is detected as a feature point. Method. 3. The calibration of the camera according to claim 1, wherein two parts on the vehicle which are installed horizontally are mutually detected as feature points from images of two vehicles having different imaging times. Method. 4. The camera calibration method according to claim 3, wherein the left and right tail lamps or the left and right headlights are detected as feature points. 5. The road plane parameter calculation process is repeatedly executed while changing the vehicle to be imaged, and the calculated plurality of road plane parameters are averaged. The method for calibrating a camera according to claim 1. 6. A cylindrical member mounted on a lens mount of a camera, and a range finder installed on a central axis of the cylindrical member and measuring a distance from the camera to a road plane along an optical axis of the camera. Camera attachment with. 7. A cylindrical member attached to a lens mount of a camera, a half mirror installed inside the cylindrical member, and a distance from the camera to a road plane through the half mirror is set to an optical axis of the camera. A camera attachment with a range finder that measures along. 8. An attachment of a camera equipped with a range finder for measuring a distance from a camera to a road plane and a finder for determining a range finder point of the range finder, which is visible through the view center of the camera and the finder. An attachment for a camera, characterized in that the range finder is attached to the camera so that the centers of images coincide with each other.