JP2011047725A - Method for estimating fixation position of imaging device and calibration system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an estimation method, along with a calibration system, capable of estimating the fixation position of an imaging device, which is more accurate than the conventional method thereof. <P>SOLUTION: The method of estimating the fixation position of an imaging camera includes: the first step of providing a target board TG with a circle pattern C at the predetermined position in a vehicle; the second step of imaging the target board TG by the imaging camera; the third step of converting the image data picked up in the second step into bird's-eye view data; the fourth step of calculating the coordinate position of a pattern on the target board on the basis of the bird's-eye view data; the fifth step of converting the coordinate position calculated in the fourth step into a coordinate position on the image data; and the sixth step of estimating the fixation position of the imaging camera on the basis of the coordinate position converted in the fifth step. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a method for estimating an attachment position of an imaging device (imaging camera) attached to a vehicle and a calibration system.

  The mounting rate of in-vehicle cameras has increased, and along with this, applications using imaging cameras have been put into practical use. As a typical application, there is an application that displays an image around a vehicle captured by an imaging camera on an in-vehicle display to assist driving or call attention.

  An imaging camera attached to a vehicle requires calibration for each vehicle in order to keep the accuracy constant. In Patent Document 1, a calibration pattern is captured by an in-vehicle imaging camera, the contour of the calibration pattern is extracted from the captured image, the position information of the calibration pattern is extracted from the contour, and the in-vehicle is extracted from the extracted position information. Discloses a calibration method in which the mounting height and mounting angle of an imaging camera are estimated, and the estimated results are stored in a memory and used during operation.

JP 2001-116515 A

  However, the calibration of a conventional imaging camera has the following problems. FIG. 1 is a diagram illustrating a positional relationship between the imaging camera and the target board when the imaging camera is calibrated. In the figure, an imaging camera 10 is attached in the vicinity of a front bumper of a vehicle M. The imaging camera 10 uses a fisheye lens with a large viewing angle so that a wide range in front of the vehicle M can be imaged, and its optical axis is substantially parallel to the ground 12. A target board TG is installed at a known distance L in front of the vehicle M. The target board TG is used to calibrate the imaging camera and is synonymous with a calibration board or a calibration pattern.

  As shown in FIG. 2, the target board TG includes a rectangular frame pattern F and six circular patterns C in the frame pattern F on a thin plate member. The frame pattern F and the circle pattern C are black. As shown in FIG. 3, a plurality of (three in the drawing) such target boards TG are installed side by side, and the center line in the longitudinal direction of the center target board TG is the center of the imaging camera 10. It almost coincides with the optical axis O of the XY plane. When the calibration is performed, the target board TG is imaged by the imaging camera 10, the corner coordinates of the frame pattern F of the target board TG and the center coordinates of the circular pattern C are calculated from the imaging data, and the imaging camera is calculated from the coordinate information. 10 mounting positions and mounting angles are estimated.

  However, the imaging camera 10 often uses a fisheye lens with a large viewing angle in order to realize a wide view, and a large distortion occurs in the image of the target board TG imaged through the fisheye lens, resulting in a frame pattern F or a circle. In some cases, the coordinates of the pattern C cannot be calculated accurately. Although it is possible to correct the distortion of the imaging data, even if such correction is performed, an accurate coordinate position may still not be obtained.

  FIG. 4 shows image data (hereinafter referred to as fish-eye image or fish-eye image data for convenience) when the target board TG is imaged via the fish-eye lens. As shown in the figure, the fish-eye image has a greater distortion in the periphery than in the center, and a greater distortion in the vicinity than in the distance. For this reason, the distortion of the images of the left and right target boards TG is large, and the distortion on the near side of the images of the target boards TG is large. If the coordinates of the frame pattern F and the circle pattern C are obtained using such fish-eye image data having a large distortion, an error of the obtained coordinates becomes large, and it becomes impossible to accurately estimate the mounting position of the imaging camera. . This causes a failure in calibration, and operations such as estimation of the mounting position of the imaging camera must be repeated again, leading to deterioration in the work efficiency of calibration.

  An object of the present invention is to solve such a conventional problem and to provide a method for estimating the mounting position of an imaging apparatus and a calibration system thereof that can estimate the mounting position of the imaging apparatus more accurately than in the past. And

  According to the present invention, an estimation method for estimating an attachment position of an imaging device attached to a vehicle includes a first step of installing a target provided with a pattern at a predetermined position with respect to the vehicle, and the imaging device. A second step of imaging the target, a third step of converting the imaging data imaged in the second step into overhead image data, and a coordinate position of the pattern on the target based on the overhead image data. A fourth step of calculating, a fifth step of converting the coordinate position calculated in the fourth step into a coordinate position on the imaging data, and the imaging device based on the coordinate position converted in the fifth step And a sixth step of estimating the mounting position.

  Preferably, the pattern includes at least one circular pattern, and the fourth step calculates a center of gravity of the circular pattern. Preferably, the pattern includes a rectangular frame pattern, and the fourth step calculates corner coordinates of the frame pattern. Preferably, the third step converts a part of the imaging data including the target pattern into overhead image data. Preferably, the pattern includes a rectangular frame pattern and at least one circular pattern in the frame pattern. In the third step, the imaging data is trimmed along the frame pattern, and the trimmed imaging data is obtained. Convert to a bird's-eye view image.

  A system for calibrating an imaging device attached to a vehicle according to the present invention includes an imaging device attached to the vehicle, a target installed in a predetermined position with respect to the vehicle, and a pattern formed thereon, Estimation means for estimating the mounting position of the imaging device based on imaging data obtained from the imaging device, the estimation means causing the imaging device to image the target, and the target imaged by the imaging means Image conversion means for converting imaging data including the pattern into overhead image data, calculation means for calculating the coordinate position of the target pattern based on the converted overhead image data, and calculating the calculated coordinate position on the imaging data Coordinate conversion means for converting to the coordinate position.

  Preferably, the pattern includes at least one circular pattern, and the calculation unit calculates a center of gravity of the circular pattern. Preferably, the pattern includes a rectangular frame pattern, and the calculation means calculates corner coordinates of the frame pattern. Preferably, the image conversion means converts a part of the imaging data including the target pattern into overhead image data. Preferably, the pattern includes a rectangular frame pattern and at least one circular pattern in the frame pattern, and the image conversion unit trims the imaging data along the frame pattern, and the trimmed imaging data is obtained. Convert to a bird's-eye view image. Preferably, the image conversion means converts the imaging data into overhead image data using viewpoint conversion means for converting the viewpoint of the captured image data into an image looking down from directly above the vehicle.

  According to the present invention, the coordinate position of the pattern is calculated by converting the imaging data into the overhead image data, and the coordinates are converted into the coordinates on the imaging data, so that the coordinate position of the pattern can be obtained more accurately. As a result, the mounting position of the imaging device can be estimated with high accuracy.

It is a figure which shows the example of the positional relationship of the imaging camera and target board at the time of performing calibration. It is a top view which shows the example of the pattern of a target board. It is a top view which shows the relationship between a target board and a vehicle. It is an example of a fisheye image when the target board shown in FIG. 3 is imaged. It is a top view which shows the example of the vehicle carrying the imaging camera preferably used by this Embodiment. It is a figure which shows the structure of the calibration system which concerns on the Example of this invention. It is a functional block diagram regarding the calibration of the control apparatus shown in FIG. It is a flowchart for demonstrating the calibration method which concerns on a present Example. It is a figure which shows the bird's-eye view image of a target board.

  It will be described in detail with reference to the drawings, embodiments of the present invention. As a preferred embodiment, as shown in FIG. 5, four imaging cameras 110A, 110B, 110C, and 110D are attached to the front, rear, and left and right of the vehicle M, and the four imaging cameras each have a viewing angle θ (preferably The image of the entire circumference of the vehicle is captured using a fisheye lens of 160 degrees or more. In the vehicle M, a system for displaying a so-called top view image in which fisheye image data captured by four imaging cameras is converted into a bird's-eye view image data as if looking down from directly above the vehicle and displayed on the in-vehicle display. Is installed.

  FIG. 6 is a block diagram showing the configuration of the calibration system according to the embodiment of the present invention. The calibration system 100 according to the present embodiment includes an imaging camera 110 (110A to 110D) attached to a vehicle, an input / output device 120, a memory 130, and a control device 140 connected to each unit via a bus 150. The calibration system 100 may be a part of a system that displays a top view image.

  The imaging cameras 110 (110A to 110D) are mounted on the front, rear, left and right of the vehicle M in order to display the top view image as described above. The four imaging cameras are calibrated when attached to the vehicle, and the information obtained by the calibration can be used when displaying the top view image.

  The target board TG can be configured to include various patterns. For example, as shown in FIG. 2, a rectangular frame pattern F and six circular patterns C of the same size are included in the frame pattern F. Consists of including. The target board TG is not particularly limited as long as it is a thin plate member, and a frame pattern F and a circular pattern C are drawn on the surface thereof, or those patterns are attached. Preferably, the frame pattern F and the circular pattern C are represented in black so that the difference from the surrounding environment becomes clear. Furthermore, the target board TG is not necessarily a portable member, and the patterns F and C may be fixedly drawn on the ground.

  The input / output device 120 includes means for inputting an instruction from the user, such as a keyboard and a mouse, and further includes means for outputting an image and sound, such as a display and a speaker. The memory 130 stores data necessary for performing calibration, information regarding the mounting position of the imaging camera estimated by the calibration, and the like. The control device 140 is connected to each unit via the bus B, and is configured by, for example, a computer device, a server device, and other electronic devices. The control device 140 is not necessarily installed in the vehicle, and may be installed outside the vehicle and connected to each part via the bus B. Preferably, the control device 140 includes a memory for storing the program and a processing device for executing the program, and the memory includes a program for performing calibration and imaging data captured by the imaging camera as a top view image. Stores a program for converting the viewpoint and displaying it.

FIG. 7 is a functional block diagram relating to calibration of the imaging apparatus 140.
The control device 140 converts an image conversion unit 142 that converts fisheye image data captured by the imaging camera 110 into overhead image data, and a coordinate position calculation unit that calculates a coordinate position of a pattern on the target board TG based on the overhead image data. 144, coordinate conversion means 146 for converting the calculated coordinate position into coordinates on the fisheye image data, and position estimation means 148 for estimating the mounting position of the imaging camera based on the converted coordinate information.

  Next, a calibration method of the imaging camera will be described with reference to the flowchart of FIG. Here, an example of calibration of the imaging camera 110A mounted in front of the vehicle M will be described. First, as shown in FIG. 1 or FIG. 3, three target boards TG are installed on the ground at a known distance L with respect to the vehicle M (step S101). The interval between the target boards TG is determined in advance, and the interval is known.

  Next, the control device 140 causes the imaging camera 110A to image the target board TG (step S102). In the fish-eye image data imaged by the imaging camera 110A, as shown in FIG. 4, three target boards TG are shown.

Next, the image conversion means 142 receives the fisheye image data imaged by the imaging camera 110A (step S103), and converts it into overhead image data (step S104).
For the conversion to the overhead image data, it is preferable to use a conversion formula or mapping table for viewpoint conversion processing prepared in a system for displaying a top view image. These conversion formulas or mapping tables are well known and will not be described in detail here.

  Further, it is not necessary to convert all the fisheye image data into the overhead image data, and it is desirable to convert a part of the fisheye image data on which the target board TG is projected into the overhead image data. An area to be converted in the fisheye image data may be determined in advance, and the area may be trimmed and converted into overhead image data, or the frame pattern F of the target board TG displayed in the fisheye image data is extracted. Then, the fish-eye image data may be trimmed along the frame pattern F and converted into overhead image data. By reducing the image data unnecessary for conversion, the capacity of the image memory can be reduced, and the time required for the image conversion process can be shortened.

  FIG. 9 shows an image of the target board TG when the trimmed fish-eye image data is converted into overhead image data. Since the overhead image data is projected onto the XY plane so as to look down from directly above, the frame pattern F and the circular pattern C of the target board TG hardly show any distortion, as is apparent from the drawing.

  Next, the coordinate position calculation unit 144 calculates the coordinate position of the frame pattern F and / or the circle pattern C of the target board based on the overhead image data projected on the XY plane (step S105). An example of how to obtain the center coordinates of the circle pattern C will be described. The coordinate position calculation means 144 identifies the frame pattern F of the overhead image data, extracts black pixels in the frame pattern F, counts the number of these pixels, and calculates the center of gravity of the circular pattern C from the total number of pixels. This is obtained and used as the center coordinates of the circle pattern C. In the same way, the center coordinates of all the circular patterns C of the target board TG are calculated. Further, the coordinate position calculation means 144 calculates the coordinates of the four corners of the frame pattern F if necessary. For example, the outline of the frame pattern F is extracted, the position where the line intersects perpendicularly is calculated, and this is set as the corner coordinates.

  Next, the coordinate conversion means 146 converts the calculated center coordinates of the circle pattern C and / or corner coordinates of the frame pattern F into the coordinates of fisheye image data (step S106). This coordinate conversion is realized by performing a conversion reverse to the conversion from fisheye image data to overhead image data.

  Next, the position estimation means 148 attaches the imaging camera 110A based on the relative positional relationship between the center coordinates of the circular pattern C and / or the corner coordinates of the frame pattern F on the fisheye image data and the imaging camera 110A. Information on the position, for example, the height and angle (optical axis) of the mounting of the imaging camera is estimated (step S107) and stored in the memory 130 (step S108). This information on the attachment position is used when displaying the top view image. The imaging camera 110A is calibrated as described above, and the remaining imaging cameras 110B to 110D are also calibrated in the same manner (step S109).

  As described above, according to the present embodiment, the fisheye image data is converted into the overhead image data, and the corner coordinates of the circle pattern C and / or the frame pattern F are calculated. There are fewer errors than coordinates calculated from a distorted image such as eye image data, and as a result, the mounting position of the imaging camera can be estimated more accurately than in the past.

  The preferred embodiment of the present invention has been described in detail above, but the present invention is not limited to the specific embodiment, and various modifications can be made within the scope of the present invention described in the claims. Deformation / change is possible.

  In the above embodiment, the target board TG includes the frame pattern F and the circular pattern C. However, the number, shape, and the like of these patterns can be changed as appropriate according to the purpose. Furthermore, in the said Example, although the calibration method when image pick-up camera 110A-110D was attached to four places of front and rear, right and left of a vehicle was shown, this is an example and the attachment number and attachment position of an image pickup camera are required. It can be changed as appropriate according to the situation. Further, in the above embodiment, a vehicle equipped with a display system that displays a top view image is used as an example. However, the present invention can be applied to any vehicle that displays imaging data from an imaging camera.

TG: target board F: frame pattern C: circular pattern O: optical axis θ: viewing angle M: vehicle 100: calibration system 110, 110A to 110D: imaging camera 120: input / output device 130: memory 140: control device

Claims (11)

An estimation method for estimating an attachment position of an imaging device attached to a vehicle,
A first step of installing a target provided with a pattern at a predetermined position with respect to the vehicle;
A second step of imaging the target by the imaging device;
A third step of converting the image data captured in the second step into overhead image data;
A fourth step of calculating the coordinate position of the pattern on the target based on the overhead image data;
A fifth step of converting the coordinate position calculated in the fourth step into a coordinate position on the imaging data;
A sixth step of estimating an attachment position of the imaging device based on the coordinate position converted in the fifth step;
An estimation method comprising: The estimation method according to claim 1, wherein the pattern includes at least one circular pattern, and the fourth step calculates a center of gravity of the circular pattern. The estimation method according to claim 1, wherein the pattern includes a rectangular frame pattern, and the fourth step calculates corner coordinates of the frame pattern. The estimation method according to claim 1, wherein the third step converts part of the imaging data including the target pattern into overhead image data. The pattern includes a rectangular frame pattern and at least one circular pattern in the frame pattern. In the third step, the imaging data is trimmed along the frame pattern, and the trimmed imaging data is obtained as an overhead image. The estimation method according to claim 4, wherein the estimation method is converted into: A system for calibrating an imaging device attached to a vehicle,
An imaging device attached to the vehicle;
A target installed in a predetermined position with respect to the vehicle and formed with a pattern;
Including estimation means for estimating an attachment position of the imaging device based on imaging data obtained from the imaging device,
The estimation means includes
Means for causing the imaging device to image the target;
Image conversion means for converting imaging data including the target pattern imaged by the imaging means into overhead image data;
Calculation means for calculating the coordinate position of the pattern of the target based on the converted overhead image data;
A calibration system for an imaging apparatus, comprising: coordinate conversion means for converting the calculated coordinate position into a coordinate position on the imaging data. The calibration system according to claim 6, wherein the pattern includes at least one circular pattern, and the calculation unit calculates a center of gravity of the circular pattern. The system according to claim 6, wherein the pattern includes a rectangular frame pattern, and the calculation means calculates corner coordinates of the frame pattern. The system according to claim 6, wherein the image conversion unit converts a part of imaging data including the target pattern into overhead image data. The pattern includes a rectangular frame pattern and at least one circular pattern in the frame pattern, and the image conversion unit trims the imaging data along the frame pattern, and the trimmed imaging data is an overhead image. The system according to claim 9, which converts to The system according to claim 6, wherein the image conversion unit converts the imaging data into bird's-eye image data using viewpoint conversion unit for converting the viewpoint into an image looking down from directly above the vehicle. .

Images