JP4905812B2 - Camera calibration device - Google Patents

Description

  The present invention relates to a camera calibration apparatus that performs camera calibration using the principle of planar projective transformation.

  As a device for calibrating a camera based on planar projective transformation, a planar calibration index including at least four points of coordinate information is arranged on a first plane, and a captured image plane of the camera including the calibration index is set to a second plane. To identify points that are present on the second plane and that correspond to a predetermined portion of the calibration index, and that are at least four points that are commonly included in the first plane and the second plane. There is known a camera calibration device that calculates a homography between the first plane and the second plane based on the correspondence (for example, see Patent Document 1). In this apparatus, a calibration index is arranged in the imaging region, and the correspondence between the coordinates of the captured image (captured image system coordinate position) and the coordinate of the converted image (planar coordinate system coordinate position) is determined based on the captured calibration index. The proofreading work of obtaining the homography which is the transformation matrix shown is performed. This camera calibration does not require camera external parameters or camera internal parameters due to the principle of plane projective transformation, and the corresponding coordinates between the captured image and the converted image are specified based on the actually captured calibration index. Therefore, there is an advantage that it is not affected by the installation error of the camera.

JP 2006-148745 A (paragraph number [0011]-, figure *)

  The homography for projecting the captured image of each camera onto the road surface used in the above-described camera calibration apparatus according to Patent Document 1 can be calculated based on at least four or more calibration indices whose coordinate positions are known. In order to calculate the coordinate position of the calibration index in the captured image system, it is necessary to detect the calibration index from the captured image using an image processing technique. However, in order to detect at least four calibration indexes from a captured image in a short time, there is a problem that the calculation load of image processing is high and a high-performance calculation device is required, which increases costs.

  An object of the present invention is to provide a technique for easily and accurately obtaining a necessary number of calibration indexes from a photographed image in camera calibration using homography. Another object of the present invention is to provide a technique for easily calibrating a plurality of cameras using homography.

In order to achieve the above object, a camera calibration apparatus according to the present invention is arranged in a planar priority calibration index arranged in a specific area defined in a first plane in a shooting field of view of the camera and in an area other than the specific area. An image acquisition unit that acquires a captured image including a calibration index representing at least four coordinates including the planar non-priority calibration index, and a positional relationship between the priority calibration index and the non-priority calibration index is preset. And an index information storage unit for storing the coordinate positions of the priority calibration index and the non-priority calibration index on the first plane as actual coordinate positions, and priority calibration for calculating the coordinate position of the priority calibration index on the image. An index position calculation unit; a non-priority calibration index position calculation unit that calculates a coordinate position of the non-priority calibration index with reference to the positional relationship and the calculated coordinate position of the priority calibration index; A homography calculation unit for calculating a homography between a captured image plane of the captured image and the first plane from a real coordinate position and a calculated coordinate position of a calibration index representing at least four coordinates; And a calibration unit that calibrates the camera , wherein the specific area is a near area where the shooting distance from the camera is short, and the non-priority calibration index is arranged in a common area where the shooting distance from the camera is long. It is.

  According to this configuration, first, the coordinate position of the photographic image system of the priority calibration index arranged in the specific area in the first plane is calculated from the acquired photographic image. Since this specific area is a limited area of the captured image, the processing load is less than when the calibration index is detected from the entire captured image. When the coordinate position of at least one priority calibration index is calculated, the remaining calibration index on the photographed image plane is referred to with reference to a preset positional relationship between the calculated coordinate position of the priority calibration index and the non-priority positive index. Estimate the existence area of a certain non-priority positive indicator. Examples of the positional relationship information set in advance include, for example, information that a non-priority positive index exists in the right region of a straight line connecting the priority calibration index and the optical center, and is drawn as a priority calibration index. Information on directionality is included in a certain pattern, and information indicating that a non-priority positive index exists on a straight line defined by the directionality is included. Thus, by estimating the existence area of the non-priority positive index, the calculated coordinate position of the non-priority index can be obtained efficiently. When the calculated coordinate position of the calibration index representing the coordinates of at least four points is obtained in this way, it is given to the homography calculation unit together with the corresponding actual coordinate position (the coordinate position of the first planar coordinate system of the calibration index). Therefore, the homography between the captured image plane of the captured image and the first plane is calculated. By using this homography, camera calibration can be performed without knowing camera parameters.

Further, the specific area is a near area where the shooting distance from the camera is short, and the non-priority calibration index is arranged in a far area where the shooting distance from the camera is long. By adopting such an arrangement of the calibration indices, the area defined by the priority calibration indices and the non-priority calibration indices can be distributed over a wide range in the captured image. The image in the captured image of the priority calibration index placed close to the camera is higher in image accuracy and higher in detection accuracy than placing it in other areas without using a high-quality lens. In addition, since the calibration indices representing the coordinates of at least four points that are the origins are distributed over the wide range of the photographed image as much as possible, there is an advantage that the finally obtained homography has high accuracy over the entire range of the photographed image. It is done.

In surveillance systems using cameras, attempts are made to install multiple cameras, display bird's-eye views and panoramic panoramic images from the captured images, and make it easier to grasp the situation around the monitored area of interest. . For this reason, it is required to calibrate a plurality of cameras so that a joint image or a tomogram does not occur in a composite image synthesized by converting captured images of the cameras. In order to satisfy this requirement, a camera calibration apparatus in the case where the present invention is applied to calibration of a plurality of cameras is a planar priority calibration arranged in a dedicated area defined in the first plane in the shooting field of view of each camera. Each captured image including a calibration index representing coordinates of at least four points composed of a planar non-priority calibration index arranged in a shared area shared in the first plane between the index and the captured images of adjacent cameras. An image acquisition unit that acquires from a camera, an index information storage unit that stores the coordinate positions of the priority calibration index and the non-priority calibration index on the first plane as actual coordinate positions, and calculates the coordinate position of the priority calibration index non-priority calibration target to calculate a priority calibration target position calculation unit, a coordinate position of the reference to the non-priority calibration target coordinate position before Kieu destination calibration indices were calculated and the actual coordinate position of the calibration target to And a homography between the captured image plane of each camera and the first plane from the actual coordinate position and calculated coordinate position of the calibration index representing the coordinates of at least four points for each captured image from each camera. And a calibration unit that calibrates the plurality of cameras using the homography calculated for each camera. In the calibration apparatus for a plurality of cameras configured in this way, the coordinate value on the planar image of the calibration index imaged in the area sharing the camera view matches the actual value from the homography characteristics, There is no occurrence of a seam or a fault in an image obtained by synthesizing a converted image obtained by converting an image of each camera into an image viewed from above on a coordinate system defined as a reference.

In particular, recent vehicles such as automobiles are equipped with multiple cameras, displaying bird's-eye views and panoramic images from the captured images, making it easier for drivers and other operators to understand the situation around the vehicle. Assuming that such an attempt is made, the camera is a camera attached to a moving body, the first plane is a road surface, and the calibration index is a pattern drawn on the road surface. It is preferable to use the marker. Similarly, it is also preferable here that the calibration index is a marker having a plane pattern arranged in parallel with the road surface. In other words, the calibration index used in the present invention includes not only a marker directly painted on the road surface with paint etc., but also a marker painted with paint etc. on a plate considered to be substantially the same level as the road surface. Various types suitable for homography calculation are included, such as a form that is placed on the screen.
Further, the camera includes a front camera having a shooting field in front of the moving body, a right side camera having a shooting field on the right side of the moving body, a left side camera having a shooting field on the left side of the moving body, and a rear side of the moving body. It is good to comprise from the rear camera made into a photography visual field. When this camera calibration is performed in the factory, the road surface is the floor of the factory.

  The present invention is not limited to the camera calibration apparatus described above, but also a camera calibration program that implements a camera calibration function in such an apparatus by a computer and a camera calibration method that regulates the flow of control by such a program. It is targeted. For example, one of the camera calibration methods for solving the above-described problem is that a planar priority calibration index arranged in a specific area defined in the first plane in the camera field of view and an area other than the specific area. An image acquisition step of acquiring a photographed image including at least four calibration indexes composed of arranged planar non-priority calibration indexes; a priority calibration index position calculating step of calculating a coordinate position of the priority calibration index on the image; Non-priority for calculating a coordinate position on the image of the non-priority calibration index with reference to a preset positional relationship between the priority calibration index and the non-priority calibration index and the calculated coordinate position of the priority calibration index From the calibration index position calculation step, the actual coordinate position of the calibration index representing the coordinates of the at least four points, and the calculated coordinate position, a homology between the captured image plane of the captured image and the first plane is obtained. A homography calculating step of calculating the Rafi, and a calibration step for calibrating the camera using the homography. The camera calibration method including such steps is also accompanied by the operational effects described in the above-described camera calibration apparatus, and various additional features described above can also be applied. The same applies to substantially the same camera calibration program.

It is explanatory drawing explaining the basic principle of the planar projective transformation utilized by this invention. It is explanatory drawing explaining the basic principle of the camera calibration apparatus by this invention. It is explanatory drawing which shows the relationship between the calibration parameter | index arrange | positioned on the road surface, a vehicle, and a camera view. It is a top view which shows a planar calibration parameter | index. It is a functional block diagram which shows the function constructed | assembled by the controller unit of a camera calibration apparatus. 5 is a flowchart illustrating an example of a calibration process flow for a plurality of cameras 1. It is a top view which shows the planar calibration parameter | index of another embodiment. It is a top view which shows the planar calibration parameter | index of another embodiment. It is a top view which shows the planar calibration parameter | index of another embodiment. It is a top view which shows the picked-up image of the planar calibration parameter | index by FIG. It is explanatory drawing explaining the positional relationship information of a priority calibration parameter | index and a non-priority positive parameter | index.

  First, the basic principle of planar projective transformation used in the present invention will be described with reference to FIG. A first plane Π exists in the shooting field of view of a camera having a shooting center C (which is also an optical center in this case), and a calibration index (points in FIG. 1) representing the coordinates of four points for camera calibration on this plane. M1, M2, M3, and M4 are arranged. Images of four calibration indices M1, M2, M3, and M4 are shown on the captured image plane π of the camera. The coordinate positions of the points P1, P2, P3, and P4 in the captured image coordinate system (here, xy coordinate system) of these images can be calculated by image processing. Further, since the calibration indices are arranged as set in advance, the calibration indices (indicated by dots in FIG. 1) M1, M2, M3, and M4 arranged on the first plane Π The coordinate position in the set plane coordinate system (here, XY coordinate system is assumed) is known.

  If four points whose arrangement on the plane in the three-dimensional space is known are used as calibration indices, the relationship with the four points of the captured image corresponding to each calibration index, so-called homography (planar projective transformation): H It is known from projection geometry that it is possible to calculate and restore the four points on the captured image plane as four points on the plane in the original three-dimensional space even if the external parameters of the camera are unknown. ing. Although detailed explanation is omitted here, for example, refer to Japanese Patent Application Laid-Open No. 2006-148745, “Computer Vision” written by Satoshi Sato, Corona Publishing Co., Ltd., October 10, 2001, first edition 3rd edition. The What should be noted here is that external parameters of the camera, which are information on the position and orientation of the camera in the three-dimensional space, such as the height and tilt angle of the camera, are not necessary, and these may be unknown. As shown in FIG. 1, the first plane coordinate positions of the four calibration indices: M1 (X1, Y1), M2 (X2, Y2), M3 (X3, Y3), M4 (X4, Y4), and their captured images The calculated photographic image coordinate positions of the four calibration indices at P1: x1 (y1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4) : H is calculated. Once the homography H 1 is calculated, a point Pn (xn, yn) of an arbitrary captured image coordinate position on the captured image plane: π can be converted to a coordinate position Mn (Xn, Yn) on the first plane. .

The principle of the camera calibration apparatus according to the present invention using the basic principle described above will be described with reference to FIG.
Here, in the first plane に お け る in the shooting field of view of the camera 1, a specific area that is a short shooting distance from the camera 1 is set as the exclusive area 専 m, and shooting from the camera 1 is performed in an area other than the specific area. A part of the area with a long distance is set as the shared area Πc. This shared area Πc is suitable as an area to be shared in captured images of adjacent cameras when a large area of the first plane Π is covered using a plurality of cameras. The exclusive area Πm is suitable as an area that is optimally captured by only one camera when a large area of the first plane Π is covered using a plurality of cameras. Note that the shared area Πc is divided into two areas that are largely spaced in the transverse direction of the camera optical axis. In order to calculate the homography, at least four planar priority calibration indices are required in the plane region Π1 in the field of view of one camera in the first plane Π. Accordingly, here, two priority calibration indices (hereinafter simply referred to as priority indices) M1 and M2 are arranged in the exclusive area Πm, and two non-priority positive indices (hereinafter simply referred to as non-priority indices) M3, M4 is arranged. In the case where it is not necessary to distinguish between the priority index and the non-priority index, the term “index” is simply used, and the figure number that collectively refers to the calibration index is 9. Further, the coordinate position of each index in the plane coordinate system of the first plane surface is a known value as the actual coordinate position of the index.

  When this partial first plane Π1 is photographed by the camera 1, the photographed image is obtained on the photographed image plane π. In this captured image, first, the coordinate positions in the captured image coordinate system of the two priority indexes M1 and M2 arranged in the exclusive area Πm are calculated as calculated coordinate positions through image processing technology. At this time, the two priority indicators M1 and M2 are arranged in the exclusive area Πm, which is a specified area that is a central area in the transverse direction with a short imaging distance, so that the captured image is also specified in the captured image. Therefore, the calculation load of the search process is small.

  When the priority indexes M1 and M2 are detected, the non-priority positive on the photographed image plane π is referred with reference to the positional relationship between the preset coordinate positions of the priority calibration indexes M1 and M2 and the non-priority positive indexes M3 and M4. Presence areas of the indicators M3 and M4 are estimated. For example, if it is defined that the non-priority positive index M3 exists in the right region of the vertical bisector of the priority calibration indexes M1, M2, and the non-priority positive index M4 exists in the left region thereof, The search area is halved. Further, it is assumed that the non-priority positive index M4 exists on the straight line connecting the optical center C and the priority calibration index M1, and the non-priority positive index M3 exists on the straight line connecting the optical center C and the priority calibration index M2. If defined, the search area is even narrower. Thus, the estimated coordinate positions of the non-priority indices M3 and M4 can be obtained efficiently by estimating the existence areas of the non-priority positive indices M3 and M4 that have not yet been detected.

  If the indexes M1, M2, M3, and M4 obtained in this way are set so as to define a region having the largest first plane Π1 as much as possible, the accuracy of homography calculated based on this is improved.

Calculated coordinate positions P1 (x1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4) in the captured image system of the priority indices M1, M2 and non-priority indices M3, M4, The coordinate position in the plane coordinate system (or road surface coordinate system) which is the actual coordinate position on the first plane plane: M1 (X1, Y1), M2 (X2, Y2), M3 (X3, Y3), M4 ( X4, Y4) and the homography H1 between the captured image plane π of the camera 1 and the first plane Π can be calculated.

As schematically shown in FIG. 2, when the first plane plane is covered by four cameras 1, four homography: H1, H2, H3, and H4 are calculated.
By calibrating the camera 1 using the homography constructed in this manner, the first plane plane such as a road surface can be obtained without requiring information on not only the internal parameters of the camera 1 but also the external parameters. Camera calibration that achieves appropriate correspondence with a display image such as a monitor based on the photographed image is realized.

An example in which the camera calibration apparatus according to the present invention is applied to calibrate a plurality of in-vehicle cameras 1 mounted on a vehicle as an example of a moving body will be described below as one embodiment.
FIG. 3 is an explanatory diagram showing the relationship between the vehicle stopped at a predetermined position on the road surface as the first plane fence and the calibration index drawn on the road surface and the camera field of view. Here, as the in-vehicle camera 1, a first camera 11 having a camera field of view in front of the vehicle, a second camera 12 having a camera field of view on the right side of the vehicle, a third camera 13 having a camera field of view on the rear side of the vehicle, and a left side of the vehicle. And a fourth camera 14 having a camera field of view. Note that the cameras 1 are commonly referred to as cameras 1 when they are not particularly distinguished.

  Each road surface within the field of view of each camera 1 includes a dedicated area Πm located in the center near the vehicle and shared areas Πc on both sides far from the vehicle. The common area Πc is a common area on the road surface that is within the field of view of the adjacent camera 1. Two priority indices M1 and M2 are disposed near the camera 1 in the exclusive area Πm, and one non-priority index M3 and M4 are disposed at positions far from the cameras in the shared area Πc on the left and right sides. The priority indicators M1 and M2 and the non-priority indicators M3 and M4 are all the same indicators collectively referred to as a diagram number 9 composed of a black and white checkered pattern called a marker as illustrated in FIG. . This index 9 is a checkered pattern calibrated by a total of four rectangles, two white rectangles and two black rectangles, and the coordinate position of the point Q at the center of the pattern is the coordinate position of the index 9 actually used. It is. Therefore, in order to detect the position of the point Q in terms of image processing, the edge of the checkerboard pattern is detected, and the intersection of the orthogonal lines obtained thereby is regarded as the point Q. The indicator itself may be drawn directly on the road surface, or the indicator may be drawn on the surface of the plate body and the plate body may be arbitrarily moved and installed.

  In the captured image by the first camera 11, priority indices M1 and M2 arranged in the exclusive area Πm and a non-priority index M3 arranged in the shared area Πc shared by the adjacent second camera 12 or the fourth camera 14 are used. And M4 images. Similarly, the images taken by the second camera 12 include images of priority indices M1 and M2 and non-priority indices M3 and M4, and the images taken by the third camera 13 also include priority indices M1 and M2 and non-priority indices M3 and M4. The images captured by the fourth camera 14 also include images of priority indices M1 and M2 and non-priority indices M3 and M4.

  In order to perform the camera calibration described with reference to FIG. 2 in the relationship between the vehicle, the calibration index, and the camera field of view, the camera calibration device mounted on the vehicle has a function as schematically illustrated in FIG. And a monitor 6 connected to the control unit 5. The four in-vehicle cameras 1 are connected to the control unit 5 so as to be able to transmit data, and send captured images of each field of view to the control unit 5.

  The functional units that are particularly relevant to the present invention built in the control unit 5 are an image acquisition unit 51, an index information management unit 52, an index position calculation unit 53, a homography calculation unit 54, a calibration unit 56, This is a display image generation unit 57. The image acquisition unit 51 selectively acquires a captured image sent from each camera 1, performs necessary preprocessing, and develops it in a working memory. The index information management unit 52 manages the road surface coordinate position, which is the actual coordinate position of the calibration index 9 including the priority indices M1 and M2 and the non-priority indices M3 and M4, arranged on the road surface for each field of view of each camera 1. In response to the request, the road surface coordinate position of the calibration index 9 to be processed is given to the function unit. When the stop position of the vehicle is determined and each calibration index 9 is drawn or set at a predetermined position, the road surface coordinate position may be stored in a nonvolatile memory or the like. Further, when the relative position on the road surface between the calibration index 9 and the vehicle fluctuates, the index information management unit 52 manages the road surface coordinate position of the calibration index 9 input each time.

  Further, the index information management unit 52 also stores information representing the positional relationship between the priority indexes M1 and M2 and the non-priority positive indexes M3 and M4 as described above, and after detecting the priority indexes M1 and M2 In order to limit the area that is the detection target of the non-priority index, the index position calculation unit 53 is provided as necessary.

  The index position calculation unit 53 performs processing for obtaining the coordinate position of the calibration index 9 on the captured image coordinates in the captured image developed in the working memory as the calculated coordinate position. In this process, in order to detect the point Q in FIG. 4, an edge detection filter for horizontal line detection and vertical line detection is applied, a straight line is obtained using RANSAC, and the coordinate position of the intersection of the straight lines is set as the calculated coordinate position. To do. The index position calculation unit 53 refers to the priority index position calculation unit 53a for calculating the coordinate position of the priority calibration index and the coordinate position of the non-priority calibration index with reference to the positional relationship information given from the index position calculation unit 53. It has two functions of the non-priority calibration index position calculation unit 53b to calculate.

  The homography calculation unit 54 calculates the photographic image plane π and the road surface と of the photographic image from the road surface coordinate positions of at least four calibration indices 9 and the calculated coordinate positions calculated by the index position calculation section 53 of the calibration indices 9. Calculate homography between.

  The calibration unit 56 obtains calibration data of each camera 1 from the precise homography H1, H2, H3, and H4 between each captured image plane π calculated by the homography calculation unit 54 and the road surface for each camera 1. Set. The display image generation unit 57 refers to the calibration data set by the calibration unit 56 and converts the captured image from each camera 1 into a display image for display on the monitor 6, and between the captured images of each camera 1. Providing a display image without any deviation to the user.

The flow of the calibration process of the multiple cameras 1 by the camera calibration apparatus configured as described above will be described with reference to the flowchart of FIG.
First, the camera 1 to be calibrated is first designated (# 01), and the captured image of the designated camera 1 is taken into the working memory (# 02). With reference to the position information of the exclusive area belonging to the field of view of the designated camera 1, the attention search area for the priority index arranged in the exclusive area is set for the captured image (# 03). Two priority indices are detected from the set attention search area, and their image coordinate positions are calculated (# 04). Obtain the road surface coordinate position that is the coordinate position in the road surface coordinate system of the two calculated priority indices and the positional relationship information between the priority index and the corresponding non-priority index finger from the report information management unit 52, and The attention retrieval area of the non-priority index is set from the estimated position of the index in the captured image coordinate system (# 05). Two non-priority indices are detected from the set attention search area, and their image coordinate positions are calculated (# 09). The road surface coordinate position which is the coordinate position in the road surface coordinate system of the two calculated non-priority indices is read from the index information management unit 52 (# 10). Homography: Hn (n is a camera number; 1, 2, 3, 4) is calculated using the calculated coordinate position and road surface coordinate position of the two priority indices and the non-priority index obtained in this way. (# 08). The calculated homography: Hn is registered as the homography: H1 of the designated camera 1 (# 092). Next, the precision homography of all four cameras 1 is checked whether registration of Hn is completed (# 10). If there are still unfinished cameras (# 10 No branch), step # 01 is performed again. The process described above is repeated. Precision homography of all cameras 1: When registration of Hn is completed (# 10 Yes branch), calibration data of each camera 1 is registered in the camera calibration table from all camera 1 homography: H1, H2, H3, and H4 Then, this camera calibration routine is finished (# 11).

[Another form of flat calibration index]
Hereinafter, a planar calibration index having a pattern other than the planar calibration index 9 shown in FIG. 4 will be exemplified.
(1) In the calibration index 9 shown in FIG. 7, the priority index is a black trapezoid (the color is not limited to black if it can be distinguished from the surroundings in terms of image processing), and the non-priority index is a black triangle It is. The trapezoid defining the priority index has two straight lines extending radially from the optical center position of the camera at the same radiation angle and two orthogonal to the optical axis of the camera (a line extending vertically from the camera position in FIG. 7). It is a trapezoid defined by parallel straight lines. The four corner points of this trapezoid become the four calibration index points, q1, q2, q3, and q4. By detecting the trapezoidal boundary side by edge detection and calculating the intersection of the four detected boundary lines, four calibration index points, q1, q2, q3, and q4 are obtained. The triangle that defines the non-priority index is defined by one radiation straight line extending from the optical center position of the camera toward the far region, one straight line orthogonal to the optical axis of the camera, and a straight line that can be arbitrarily set. It is a triangle. The calibration index point q5 or q6 is obtained by detecting two boundary sides of the triangle by lateral edge detection and calculating the intersection of the two detected boundary lines.

(2)
If the camera used is not of high quality, the position of the image may be shifted depending on the brightness of the subject. In such a case, even if the straight line is applied after detecting the edge as it is, the straight line may be set at a position different from the actual position. In order to solve this problem, the calibration index shown in FIG. 8 is designed to cancel the above-described deviation because the shading pattern is complementary. That is, compared with the calibration index according to FIG. 7, the pattern pattern of the priority index is substantially a trapezoid defined by two radiation lines and two parallel lines, and each of the trapezoids. Four similar trapezoidal trapezoids defined at the 180 ° rotation position of the apex are blacked out. The pattern pattern of the priority index is a pattern in which another dummy triangle is arranged on the opposite side of the triangle which is the non-priority index in FIG. In other words, the presence of a central trapezoid as a calibration index and a dummy trapezoid and a triangle that share a triangular reference line provide a complementary shading pattern.
(2)
The calibration index shown in FIG. 9 is similar to the calibration index shown in FIG. 8, but the trapezoidal and triangular sizes including the dummy are made larger as they are located farther from the camera 1. Each size on the photographed image is set to be substantially the same size. A photographed image of this calibration index is shown in FIG. As can be understood from FIG. 10, what is a radial straight line on the actual road surface appears as a vertical line on the photographed image, and what was a horizontal line appears as a horizontal line as it is. Becomes simple. As a result, the edge detection accuracy of each pattern is improved, and as a result, the accuracy of straight line detection leading to the calculation of calibration index points can be expected.

  An example of the positional relationship information between the priority calibration index and the non-priority positive index will be described with reference to FIG. Here, as the calibration index 9, two priority calibration indices M1 and M2 and two non-priority positive indices M3 and M4 are shown. Each calibration index 9 is a black and white checkered pattern shown in FIG. As described above, when calculating the index point Q of one of the priority calibration indices M1, two orthogonal straight lines are obtained by edge detection of the checkered pattern. Here, it is set in advance so that the non-priority positive index M4 exists on one of the straight lines L1. Similarly, the non-priority positive index M3 is set in advance so as to exist on the straight line L2 obtained when calculating the index point Q of the other priority calibration index M2. This is the positional relationship information between the priority calibration index and the non-priority positive index in the example of FIG. 11, and the checkerboard pattern drawn as the priority calibration index includes straight line information that is direction information. . After the priority calibration indices M1 and M2 are detected based on the positional relationship information, the non-priority positive indices M3 and M4 can be efficiently found by searching on the straight line L1 and the straight line L2.

One specific purpose of camera calibration (camera calibration) according to the present invention is to display a captured image acquired by the camera 1 on the monitor 6 and to display predetermined image information (for example, a vehicle parking space) on the captured image. When superimposing a well-known parking assistance device or a driving assistance device that predicts the course of a vehicle in a known parking assistance device or driving assistance device that assists the driver when parking or driving backward, This is to make the relationship with the superimposed image accurate. Since the actual road surface position and the road surface position on the photographed image are accurately associated by the camera calibration (camera calibration) according to the present invention, the captured image acquired by the camera 1 in the monitor 6 and the additional superimposed image are added. The positional relationship with is accurate. That is, the captured image is left as it is, and the superimposed image is corrected so that the positional relationship between the captured image and the additional superimposed image is accurate. The superimposed image is fixed and the captured image is corrected, The positional relationship between the captured image and the additional superimposed image can be made accurate.
In addition, when the actual road surface position and the road surface position on the captured image are accurately associated through camera calibration (camera calibration) according to the present invention, a display object (for example, included in the captured image acquired by the in-vehicle camera) The actual position of the lane, object, etc.) can be accurately specified through image processing of the captured image.

[Another embodiment]
(1) In the above-described embodiment, four calibration indices are used to calculate homography, but of course, four or more calibration indices may be used. In addition, the number of indices arranged in the exclusive area or the shared area may be increased, and only those that are most easily detected may be selected and used.
(2) Various modifications other than those described above are possible for the form of the calibration index. The formation of the pattern pattern is not limited to drawing using paint or the like, and for example, pattern pattern formation by an optical or illumination method may be employed.
(3) The positional relationship between the priority calibration index and the non-priority calibration index can be set to various relationships other than those described above, and the positional relationship is not necessarily stored in a memory or the like. It may be incorporated as an algorithm.

  The present invention is applicable to all fields where it is necessary to perform camera calibration for matching a relationship between a captured image plane and a specific plane in a camera view for a single camera or a plurality of cameras whose camera parameters are unknown. can do.

Π: First plane (road surface)
π: captured image plane H, H1, H2, H3, H4: homography 5: control unit 9: calibration index M1, M2: priority calibration index M3, M4: non-priority calibration index 51: image acquisition unit 52: index information management Unit 53: Index position calculation unit 53a: Priority index position calculation unit 53b: Non-priority index position calculation unit 54: Homography calculation unit 56: Calibration unit 57: Display image generation unit

Claims (6)

At least four calibrations comprising a planar priority calibration index arranged in a specific area defined within the first plane in the camera field of view and a planar non-priority calibration index arranged in an area other than the specific area. An image acquisition unit for acquiring a captured image including an index;
An index information storage unit that presets the positional relationship between the priority calibration index and the non-priority calibration index, and stores the coordinate position of the priority calibration index and the non-priority calibration index on the first plane as an actual coordinate position. When,
A priority calibration index position calculator for calculating a coordinate position on the image of the priority calibration index;
A non-priority calibration index position calculation unit that calculates a coordinate position on the image of the non-priority calibration index with reference to the positional relationship and the calculated coordinate position of the priority calibration index;
A homography calculating unit that calculates a homography between the captured image plane of the captured image and the first plane from the actual coordinate position and the calculated coordinate position of the calibration index representing the coordinates of the at least four points;
A calibration unit that calibrates the camera using the homography;
With
The specific area is a near area where the shooting distance from the camera is short, and the non-priority calibration index is arranged in a far area where the shooting distance from the camera is long.   The image acquisition unit acquires captured images from a plurality of cameras having different shooting fields of view, and the captured image from each camera is shared in the first plane between the captured images of adjacent cameras. And the non-priority calibration index is arranged in the exclusive area, and the calibration unit is provided for each camera. The camera calibration apparatus according to claim 1, wherein the plurality of cameras are calibrated using the homography calculated in step 1.   The said camera is a vehicle-mounted camera attached to the vehicle, The said 1st plane is a road surface, The said calibration parameter | index is a marker which has a plane pattern arrange | positioned in parallel with the said road surface. Camera calibration device. The camera calibration device according to claim 1, wherein the camera is a camera attached to a moving body, the first plane is a road surface, and the calibration index is a marker as a pattern drawn on the road surface.   The camera has a front camera with a shooting field in front of the moving body, a right side camera with a shooting field on the right side of the moving body, a left side camera with a shooting field on the left side of the moving body, and a rear image of the moving body. The camera calibration apparatus according to claim 1, wherein a rear camera serving as a field of view is included. The planar priority calibration index arranged in the exclusive area defined in the first plane in the shooting field of view of each camera and the shared area shared in the first plane between the captured images of the adjacent cameras. An image acquisition unit that acquires, from each camera, a captured image that includes a calibration index representing at least four coordinates of a flat non-priority calibration index;
An index information storage unit that stores the coordinate positions of the priority calibration index and the non-priority calibration index in the first plane as actual coordinate positions;
A priority calibration index position calculator for calculating a coordinate position on the image of the priority calibration index;
And the non-priority calibration target position calculating section for calculating a reference to a coordinate position on the image of the non-priority calibration target coordinate position before Kieu destination calibration indices were calculated and the actual coordinate position of the calibration target,
Homography for calculating homography between the captured image plane of each camera and the first plane from the actual coordinate position and calculated coordinate position of the calibration index representing the coordinates of the at least four points for each captured image from each camera A calculation department;
A calibration unit that calibrates the plurality of cameras using homography calculated for each camera,
With
The exclusive area is a near area where the shooting distance from the camera is short, and the non-priority calibration index is arranged in a common area where the shooting distance from the camera is long.

Images