JP2007033087A - Calibration device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a calibration device and a method capable of easily acquiring a precise parameter. <P>SOLUTION: A known object 12 is photographed by cameras 13<SB>1</SB>-13<SB>n</SB>from different respective viewpoints, and its images are stored in a storage part 14. An evaluation part 15 evaluates the position of the known object 12 in a three-dimensional space estimated from the images stored in the storage part 14. Accordingly, the position of the known object 12 to be acquired in the three-dimensional space can be identified, and the precise parameter can be easily acquired. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a calibration apparatus and method for calibrating parameters necessary for a stereo camera, for example, to realize distance image measurement (depth measurement).

  2. Description of the Related Art Conventionally, a technique for creating a stereoscopic image using parallax of images captured from different positions is known. In such a stereo vision system using image parallax, a plurality of cameras are provided, and parameters such as positions and postures between the cameras necessary for the stereo camera are calibrated in advance. In many cases, the position / orientation of the camera changes over time, and parameter calibration is required every time the position / orientation changes. However, it is difficult for a person who is not familiar with the technology to generate a new parameter.

  Therefore, the applicant of the present application has proposed a calibration method for newly calculating parameters using images obtained by capturing a plane on which a known pattern is drawn from each of three or more viewpoints with no spatial position constraint. (For example, refer to Patent Document 1).

JP 2002-27507 A

  The technique described in Patent Document 1 performs calibration by imaging a known pattern on a plane from different directions, but it is intuitive to determine from which direction accurate calibration is performed. Since it is very difficult to grasp, accurate parameters may not be obtained.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a calibration apparatus and method capable of easily obtaining accurate parameters.

  In order to solve the above-described problem, a calibration apparatus according to the present invention includes an imaging unit that captures images of known objects from different viewpoints, a storage unit that stores an image captured by the imaging unit, and a storage unit that stores the image captured by the imaging unit. The evaluation means for evaluating the position in the three-dimensional space of the known object estimated from the stored image, and the characteristics of the imaging means based on the known object placed at a high evaluation position in the evaluation means And calibrating means for calibrating the parameters.

  The calibration method according to the present invention includes an imaging step of imaging a known object from different viewpoints by an imaging unit, a storage step of storing an image captured by the imaging step in a storage unit, and a storage in the storage unit An evaluation step for evaluating the position in the three-dimensional space of the known object estimated from the obtained image, and a parameter indicating the characteristics of the imaging means based on the known object arranged at a high evaluation position in the evaluation step And a calibration process for calibrating.

  According to the present invention, each known object is imaged from different viewpoints, the position of the known object in the three-dimensional space is estimated, and the characteristics of the imaging means are evaluated. Since the position can be known, accurate parameters can be easily obtained.

  Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

  First, parameters in the stereo vision system will be described with reference to FIGS. Here, for simplicity, the stereo method is performed using only two viewpoints, that is, two cameras. The stereo method uses an image captured from a plurality of viewpoints (projection centers) having a predetermined positional relationship, and obtains the distance between each point in the captured image and the projection center based on a so-called “triangulation” principle. is there.

  FIG. 1 schematically shows the arrangement of the reference camera B and the detection camera D with respect to the imaging plane, and FIG. 2 shows a case where a substantially square pattern is captured by each of the reference camera B and the detection camera D. A reference image and a detected image are schematically shown. One of the two cameras is used as a reference camera B, which captures an object from a position facing the front and outputs a reference image. The other camera is used as a detection camera D, images a target object from an oblique direction, and outputs a detection image.

  As shown in FIG. 1, in the captured image of the reference camera B, the point M is at the intersection m between the straight line connecting the point M on the plane to be imaged and the projection center Cb of the reference camera B and the projection screen Sb of the reference camera. Is observed. The straight line connecting the point M and the projection center Cb of the reference camera B is the line of sight of the reference camera B. In the captured image of the detection camera D, the point M is observed at the intersection m ′ between the straight line connecting the point M and the projection center Cd of the detection camera and the projection screen Sd of the detection camera D. A straight line connecting the point M and the projection center Cd of the detection camera D is the line of sight of the detection camera D. Here, the line of sight of the reference camera B is observed as a straight line on the projection screen of the detection camera D. This straight line is hereinafter referred to as an “epipolar line”.

  In the example shown in FIGS. 1 and 2, the captured image captured by the reference camera B facing the substantially square pattern is a square, whereas the image captured by the detection camera D that squints this pattern is the viewpoint. As a result of the reduction of the side with the longer distance from, it appears as a trapezoid. This is due to the basic property of central projection, in which even an object of the same size is projected as a large image as it approaches the projection center C of the camera, and conversely, it is projected smaller as it moves away from the projection center C. It is.

  As described above, when the imaging target is a plane, the captured image of the detection camera D is an image obtained by projective conversion of the captured image of the reference camera B. That is, the equation (1) is established between the point m (xb, yb) in the captured image of the reference camera B and the corresponding point m ′ (xd, yd) in the captured image of the detection camera D. To do. In the above equation, F is a 3-by-3 weak calibration matrix (Fundamental Matrix), and m and m ′ are homogeneous vectors.

  The weak calibration matrix F is a matrix that implicitly includes the internal and external parameters of the camera and the plane equation, and has 8 degrees of freedom because the degree of freedom remains in the scale factor. In addition, Kenichi Kanaya's “Image Understanding” (Morikita Publishing, 1990) describes that corresponding points can be obtained by projective transformation between a base image and a reference image.

  The line of sight of the reference camera B appears as a straight line called an epipolar line on the projection screen of the detection camera D (see above and FIG. 1). The point M existing on the line of sight of the reference camera B appears on the same observation point m on the projection screen of the reference camera B regardless of the depth of the point M, that is, the distance from the reference camera B. On the other hand, the observation point m ′ of the point M on the projection screen of the detection camera D appears at a position corresponding to the distance between the reference camera B and the observation point M on the epipolar line.

  FIG. 3 illustrates the epipolar line and the state of the observation point m ′ on the projection screen of the detection camera D. As shown in this figure, as the position of the point M changes to M1, M2, and M3, the observation point in the reference image shifts to m′1, m′2, and m′3. That is, the position on the epipolar line corresponds to the depth of the observation point M.

  The distance from the reference camera B to the point P can be identified by searching the observation point m ′ corresponding to the observation point m of the reference camera B on the epipolar line using the above geometric optical properties. it can.

  Thus, in order to estimate the three-dimensional spatial position of an object from a plurality of images, it is necessary that the imaging optical system of the camera has a characteristic that completely matches the theory and that the image has no distortion. Therefore, a predetermined correction must be performed on the actually obtained image. For example, a camera lens generally has a distortion parameter, and an observation point is imaged at a position displaced from a theoretical point. Therefore, a camera-specific parameter is calculated, and image data according to this parameter is calculated in projective transformation. Without correction, an accurate projected image cannot be obtained from the front image.

  In addition to the lens distortion parameters, the parameters of the camera are classified into internal parameters representing camera characteristics such as camera focus and camera lens, and external parameters representing the three-dimensional spatial position of the camera. The calculation of these parameters is generally called “calibration”.

Next, the calibration apparatus in the present embodiment will be described with reference to FIG. The calibration device 10 stores a plurality of cameras 13 _{1 to} 13 _n that capture images of a known object 12 moved by the user 11 from different viewpoints, and a storage unit that stores images captured by the plurality of cameras 13 _{1 to} 13 _n. 14, an evaluation unit 15 that evaluates the position in the three-dimensional space of the known object 12 estimated from the image stored in the storage unit 14, and the movement of the known object 12 relative to the user 11 according to the evaluation result of the evaluation unit 15 And an output unit 16 that outputs the above-described instruction, and a calculation unit 17 that calculates parameters indicating the characteristics of the cameras 13 _{1 to} 13 _n according to the moved known object 12.

The cameras 13 _{1 to} 13 _n are installed at different positions and postures, and image the known object 12. As shown in FIG. 5, each camera 13 performs evaluation by setting the optical center as an origin, such as the X axis, the Y axis, and the Z axis. In the present embodiment, calibration is performed using a known object 12 as shown in FIG. Further, it is assumed that the calibration apparatus 10 has the previously calibrated parameters, and the position / orientation of the camera 13 changes slightly with time.

The storage unit 14 stores data of images captured by the cameras 13 _{1 to} 13 _n . Further, the position information of the known object 12 estimated by the evaluation unit 15 is stored.

  The evaluation unit 15 estimates the position of the known object 12 in the three-dimensional space from the image data stored in the storage unit 14. All the light rays are focused on the optical center of the camera 13 as shown in FIG. 7, and the area of the image of the known object 12 in the screen is proportional to the size of the known object in the three-dimensional space viewed from the optical axis direction. Therefore, the distance in the depth direction (Z-axis direction) from the camera 13 can be estimated from the area of the image of the known object 12 in the screen. That is, when the known object 12 is placed at a position A far from the position B and when the known object 12 is placed at a position B closer to the position A, the image of the known object 12 when placed at the position A is placed at the position B. Smaller than the image of the known object 12.

  Further, as shown in FIG. 8, a plane a parallel to the X-axis and Y-axis direction of the depth at which the known object 12 is estimated is arranged, and a straight line passing through the known object image is drawn from the optical center, thereby The intersection point with the image plane can be estimated as the position on the XY plane of the known object 12 in the three-dimensional space.

Further, the evaluation unit 15 evaluates the estimated position of the known object 12 on the XY plane and the position in the Z-axis direction. Since the accuracy of the calibration changes due to the variation in the position of the known object 12 in the three-dimensional space, which has been imaged a plurality of times, in the three-dimensional space, the variation of the known object 12 in the three-dimensional space The calculation is performed separately for the parallel component and the component in the depth direction with respect to the image plane. That is, in order to maintain the calibration accuracy between the cameras 13, the position where the known object 12 is arranged must have a certain degree of variation when viewed from the cameras 13 _{1 to} 13 _n . By having a certain degree of variation, it is possible to obtain a parameter with less error and perform highly accurate calibration.

The evaluation value E _XY regarding the component parallel to the image plane of the camera 13 is expressed by the following equation (2):

Where Corr _XY is the correlation value between the X-axis and Y-axis components of the data, V _X is the variance of the X-axis direction component of the data, and V _Y is the variance of the Y-axis direction component of the data.

The evaluation value E _Z with respect to the depth direction of the component with respect to the image plane is expressed by equation (3).

However, V _Z denotes the variance of the Z-axis data (depth) direction component.

Thus, by dividing the evaluation value based on the variance in the XY axis direction and the evaluation value based on the variance in the Z axis direction, the calibration accuracy can be improved. The calculated evaluation value E _XY for each _camera, E _V is equal to or greater than a predetermined threshold value, it is possible to maintain the accuracy of the calibration. This accuracy range can be arbitrarily set by the user. However, the evaluation value depends on the set accuracy, the range of the three-dimensional space where the accuracy is required, and the camera internal parameters, and when the accuracy is high or the range of the three-dimensional space where the accuracy is required is large. The threshold value of the evaluation value becomes high.

  As will be described later, the evaluation unit 15 obtains the position of the known object 12 to be arranged in the three-dimensional space based on the calculated evaluation value and the previously calibrated parameter.

  The output unit 16 instructs the user 11 on the position in the three-dimensional space where the known object 12 obtained by the evaluation unit 15 is to be placed. For example, as shown in FIG. 9, the contour of the known object 12 is displayed on the screen, and the user 11 is guided to move the known object 12 to the contour position. The position and size of the contour can be calculated based on the position in the three-dimensional space where the known object 12 obtained by the evaluation unit 15 is to be placed.

  The calculation unit 17 calculates a parameter of the camera based on the known object 12 moved to a predetermined position, and calibrates the previous parameter. The calculated parameters are stored in the storage unit 14 and output to, for example, a parallax image generation unit (not shown) in the stereo vision system.

  With this configuration, the user 11 can calibrate the camera 13 in the stereo vision system simply by moving the known object 12 in accordance with an instruction from the output unit 16.

  Next, the operation of the calibration apparatus 10 will be described in detail. Here, it is assumed that the previous parameter is stored in the storage unit 14 and the position / orientation of the camera changes little by little over time. That is, the external parameters are slightly changed, and the internal parameters such as the focus of the camera 13 and the camera lens characteristics are not changed. Further, it is assumed that the stereo vision system includes two cameras, ie, a left camera and a right camera serving as a reference, and the XY plane of the three-dimensional coordinate system of the camera is parallel to the image plane.

  FIG. 10 is a flowchart illustrating an example of the operation of the calibration apparatus 10. First, in step S11, the stereo vision accuracy and the initial setting of the range that maintains the accuracy are input. That is, the accuracy related to epipolar constraint when generating a distance image in stereo vision and the depth of a range that maintains the accuracy are input by an input device (not shown). Here, the accuracy depends on the resolution of the camera image, the angle of view, the distance between the cameras, and the like. For example, the error between the epipolar line and the corresponding point is set within several pixels. Further, the range for maintaining the accuracy is a three-dimensional space in which the shooting ranges of the left and right cameras are superimposed.

  In step S <b> 12, the evaluation unit 15 reads out the parameters when the stereo image was previously generated from the storage unit 14.

  The evaluation unit 15 obtains a location where the left and right images are shared in a certain plane (depth) from the parameters read from the storage unit 14. That is, a range in which the imaging range of the left camera that is the reference camera and the imaging range of the right camera that is the detection camera overlap is calculated.

The left camera projection transformation matrix P _left and the right camera projection transformation matrix P _right are respectively obtained by the equations (4) and (5) and correspond to the pixel positions (m _Ileft , m _Jleft ) of the left camera image plane. The relationship between the pixel positions (m _Iright , m _Jright ) of the right camera is _expressed by _equations (6) to (8).

Here, the matrix A _left and the matrix A _right are internal parameters of the left camera and the right camera, respectively. The matrix [R | t] is an external parameter, and R and t represent the rotation and parallel progression of the image, respectively. Z _3D indicates the depth in the three-dimensional space with respect to the left camera, and α is a scale factor.

Next, in step S13, based on the accuracy input in step S11 and the range that maintains the accuracy, the known object 12 is placed within the overlapping range of the left and right cameras calculated by the above equations (4) to (8). Calculate the position. Here, as shown in FIG. 11, it is assumed that the depth of the range that maintains the accuracy set in step S11 is Z ₀ and Z ₁ (Z ₀ <Z ₁ ). It should be noted that Z ₀ and Z ₁ are within the range where the shooting ranges of the left and right cameras are superimposed, and the range where the shooting ranges of the left and right cameras are overlapped is larger as Z ₀ and Z ₁ are larger, that is, farther from the camera The more you go, the wider. Further, if the accuracy cannot be sufficiently maintained, the depth of the range that maintains the accuracy may be further set.

Since the known object 12 is imaged the same number of times in each plane at the depth Z ₀ and the depth Z ₁ , the average depth of the position of the acquired known object 12 is (Z ₀ + Z ₁ ) / 2. Therefore, the evaluation value is determined by the interval between Z ₀ and Z ₁ . In this example, Z ₀ is fixed, and the depth Z ₁ that satisfies the evaluation value E _Z is obtained by the equation (9).

Here, Th _Z is a value based on the variation error of the pixel data by the parameter when the stereo image was generated last time.

From the above equation (9), the position at which the known object 12 is arranged for each XY plane of the depth Z ₁ that satisfies the evaluation value is obtained. That is, based on the above equation (2), a position where the variation of each axis of the X axis and the Y axis is large and the correlation value between the X axis and the Y axis of the pixel data is small is obtained, and the evaluation value as shown in FIG. The arrangement position of the known object 12 is obtained so that _EXY is increased. Furthermore, as shown in FIG. 13, it is preferable to arrange at the four vertices of the rectangle with the largest area in the region where the imaging regions of the left and right cameras overlap. In this case, the rectangle is arranged in consideration of the size of the known object 12 with respect to the depth.

After the information of the known object 12 at the depth Z ₀ is obtained, the position where the known object 12 is arranged at the depth Z ₁ is obtained. Here, when the evaluation value is sufficiently large, the size of the rectangle may satisfy the evaluation value threshold. As a result, the rectangular shape does not become larger than the determination of the arrangement position of the known object 12 as in the case of the depth Z ₀ , so that the trouble of arranging the known object 12 far away can be saved.

First, X axis position to be placed in a known object 12 obtained in the depth Z _0, obtains the dispersion of the Y-axis direction. Next, in order to simplify the calculation, the average of the positions of the four known objects 12 at the depth Z ₀ that instructs the user to place them (the center of the rectangle with the largest area) Ac is a known object that guides the depth Z _1. It is calculated so as to coincide with the average of 12 positions. That is, as shown in FIG. 14, the arrangement position of the known object 12 in the depth Z ₁ is obtained from the variance of the known object 12 in the X axis / Y axis direction at the average position Ac of the known object 12. Then, a rectangle whose area is the maximum within the range of the overlapping positions of the imaging regions of the left and right cameras with Ac as the center and whose sides are parallel to the X axis and the Y axis is obtained, and one vertex of the rectangle is defined as Ab ′. When a vector subtracted from Ac to Ab ′ is a unit vector, equation (10) is obtained.

Since the rectangle is symmetrical with respect to the X-axis and Y-axis directions with respect to Ac, the variation of each vertex as seen from Ac is the same. Therefore, when considering the dispersion of the four vertices in the X-axis and Y-axis directions, a deviation sum of squares is obtained from the distance between a certain vertex and the center of the rectangle, and is multiplied by four to obtain a known object at each point of the rectangle The sum of square deviations in the X-axis and Y-axis directions when 12 is arranged is obtained. That is, the evaluation value when the known object 12 is arranged in a rectangle at the depth Z ₁ is as follows.

Here, S _XX and S _YY indicate variations in the X-axis and Y-axis directions of the position of the known object 12 obtained at the depth Z ₀ . It should be noted that the location where the known object 12 is to be placed is the four vertices of the rectangle, so the correlation value is assumed to be zero.

Above (11) to (13), when obtaining the α type the threshold of evaluation value, obtained in a range of known objects 12 required depth Z _1. However, when α is 1 or more, there is no overlapping portion of the imaging areas of the left and right cameras, and therefore the depth Z ₁ may be expanded to a range where the imaging ranges of the left and right cameras are overlapped by the value of α. At this time, since the variation becomes large, the evaluation value in the depth direction does not decrease.

  In step S14, the user is guided to arrange the known object 12 at the position obtained in step S13. For example, the display is placed under the left and right cameras, and the position of the known object 12 to be placed as viewed from the left camera is displayed on the video taken by the left camera as shown in FIG. The position in the X-axis / Y-axis direction of the image plane of the camera is designated by the position in the screen, and the depth (position in the Z-axis direction) is designated from the size of the frame.

  In step S15, it is determined whether or not the known object 12 is arranged at the guided position. The evaluation unit 15 performs the guide in step 14 until the known object 12 is arranged at the guided position.

  In step S15, when the known object 12 is arranged at the guided position, the process proceeds to step S16, and calibration is started. That is, a parameter is calculated based on the known object 12 placed at a predetermined position, and the previous parameter is calibrated. The calibration can be performed based on, for example, “Three-dimensional Computer Vision (A Geometric Overview)”, Oliver Faugeras, The MIT Press (1993).

  Thus, by estimating the position of the known object 12 in the three-dimensional space, evaluating the characteristics of the camera 13, and calculating the position where the known object 12 is arranged according to the evaluation result, the user can move to that position. If the known object 12 is arranged, an accurate parameter can be easily obtained.

In step S14, guidance using a display is performed. However, guidance may be performed by voice as shown in FIG. For example, the depth direction may be guided by the height of the sound range, and the position in the X axis / Y axis direction may be guided by the strength of the sound. That is, when the known object 12 is far from the position Z _{0 in the} depth direction to be arranged, a low-pitched sound is output, and when close to the position Z ₀ , a high-pitched sound is output. If the known object 12 is far from the position in the X-axis / Y-axis direction to be arranged, the sound output is reduced, and if it is close to the position in the X-axis / Y-axis direction to be arranged, the sound output is increased. To do. Then, the user moves to a position where the known object 12 is to be placed based on the level and strength of the output voice.

  Moreover, as shown in FIG. 17, the evaluation value of each position in the image when the known object 12 is arranged may be obtained, and the level of the evaluation value may be displayed. Thereby, the user can know at which position in the image the evaluation value increases when the known object 12 is displayed.

  Further, according to the present invention, not only external parameters of the camera 13 but also internal parameters can be calibrated. For example, by using the known plane 18 as shown in FIG. 18, the calibration apparatus 1 detects the plane, obtains the positional relationship of the four corners of the known plane 18, and determines the known plane 18 in the three-dimensional space. The posture can be estimated, and the characteristics of the camera 13 can be evaluated based on the variation in posture. When the estimated posture variation is large, the calibration accuracy can be maintained.

  A specific method for guiding internal parameters will be described with reference to FIG. The calibration apparatus 10 obtains the homography of the known plane 18 and the known plane image 19 in the image when the known plane 18 as shown in FIG. 18 is arranged in a three-dimensional space as shown in FIG. This can be done based on "Computer Vision-Visual Geometry-" (by Satoshi Sato). According to this, the HomographyH between the known plane 18 and the known plane image 19 in the image can be developed as shown in Expression (14).

  However, it is assumed that the X-axis and Y-axis of the world coordinate system in Expression (14) are parallel to the known plane 18 and the Z-axis is perpendicular to the known plane 18. M ~ is the position of the pixel of the known planar image 19 in the image, and M ~ (the value in the Z-axis direction is always 0, so it is omitted. Therefore, it is a three-row homogeneous vector) is a three-dimensional space. The position of the above known plane 18 is shown, A is a camera internal parameter, r1 and r2 are row vectors of a rotation matrix representing the known plane (world coordinate system) 18 and the attitude of the camera 13 shown in equation (15), t Indicates the positional relationship between the camera 13 and the known plane (world coordinate system) 19. As shown in Expression (14), the relationship between the position and orientation of the camera 13 and the known plane 18 can be obtained by calculating Homography based on the previous calibration.

  In order to obtain the internal parameters, three or more known planes having different postures are required. On the other hand, if the imaging data of the known object 18 with the same posture is acquired, the internal parameters are not calibrated. Therefore, in order to perform calibration with high accuracy, the posture of the known plane 18 is obtained using the equation (14) and guided so that the posture is changed as much as possible. The degree to which the known plane 18 is tilted is calculated from the data given at the previous calibration.

  When the calibration apparatus 10 guides the user, for example, the known plane 18 tilted as shown in FIG. 20 is output to the screen. The inclination of the known plane 18 in the screen is obtained by inputting r1 and r2 representing the posture and the position t into Expression (14). By outputting the tilted known plane 18 to the screen and tilting the known plane 18 so that the user has the same shape, the internal parameters can be accurately calibrated.

  Further, in the above embodiment, when performing calibration, the position where the known object 12 is arranged is guided to the user. However, the present invention is not limited to this, and the calibration is performed by freely moving the known object 12. May be performed.

  Next, a method of performing calibration by freely moving the known object 12 will be described with reference to a flowchart shown in FIG.

  In step S21, as in step S11, an initial setting of an accuracy that assumes stereo vision and a range that maintains accuracy such as depth is input. That is, the accuracy related to epipolar constraint when generating a distance image in stereo vision and the range in which the accuracy is maintained are input by an input device (not shown).

  In step S <b> 22, the evaluation unit 15 reads the parameters when the stereo image was generated last time from the storage unit 14, and the evaluation unit 15 reads the left and right images on a certain plane (depth) from the parameters read from the storage unit 14. Find the places where the images appear in common. That is, a range in which the imaging range of the left camera as the reference camera and the imaging range of the right camera as the detection camera are superimposed is calculated by the above formulas (4) to (8).

  In step S23, the total number N of position information of the known object 12 is set by an input device (not shown). This position information is the position of the known object 12 estimated in the three-dimensional space. The total number N is an arbitrary number set empirically.

The evaluation unit acquires the position information of the known object 12 from the storage unit 14 (step S24), and acquires the position information of the known object 12 until the number of acquisition reaches the total number N set in step S23 (step S25) N When the pieces of position information are acquired, the evaluation unit 15 compares evaluation values for each combination of the N pieces of position information set in step S23 and the newly acquired N + 1th piece of position information. _In all combinations of _{N + 1} C _N = N + 1, the evaluation values E _XY and E _Z are calculated based on the above formulas (2) and (3). Then, the combination having the highest evaluation value is set as new N pieces of position information, and one piece of position information that is out of the combination is deleted and selected (step S26).

  In step S27, it is determined whether or not the evaluation value of the combination newly created in step S26 is higher than a threshold value. This threshold is set based on the evaluation value based on the N pieces of position information set in step S23.

  If the evaluation value calculated by selection in step S27 is smaller than the threshold value, the process returns to step S24 to acquire the position information of the known object 12 again. If the evaluation value is equal to or larger than the threshold value, the process proceeds to step S28 and calibration is started. To do. That is, the parameter is calculated based on the position information of the new combination, and the previous parameter is calibrated.

  As described above, by evaluating the characteristics of the camera based on the position information of the known object, it is possible to obtain parameters that should be easily acquired.

It is a figure which shows typically arrangement | positioning of the reference | standard camera and detection camera with respect to an imaging target. It is a figure which shows the image which imaged the substantially square pattern with each of the reference | standard camera and the detection camera. The epipolar line and the state of the observation point in the reference image are illustrated. It is a block diagram which shows the structure of the calibration apparatus in one embodiment of this invention. It is a figure for demonstrating a camera coordinate system. It is a figure which shows an example of a known object. It is a figure which shows the relationship between the position of a known object, and depth. It is a figure which shows the relationship between the position of a known object, and an X-axis Y-axis plane. It is a figure which shows the example of a guide of a known object. It is a flowchart which shows the calibration method performed by the guide of a known object. It is a figure which shows the relationship between a camera and depth. It is a figure which shows the example arrange | positioned in the area | region which the imaging | photography area | region of a right-and-left camera overlaps. It is a figure which shows the example arrange | positioned at the four vertexes of the rectangle with the largest area within the area | region which the imaging | photography area | region of a left-right camera overlaps. It is a view for explaining an arrangement position of a known object 12 in the depth Z _1. It is a figure which shows the example of a guide by a display. It is a figure which shows the example of a guide by an audio | voice. It is a figure which shows the example of a guide by showing an evaluation value. It is a figure which shows the known plane in calibration of an internal parameter. It is a figure explaining calibration of an internal parameter. It is a figure which shows the example of a guide of a known plane. It is a flowchart which shows the calibration method performed by moving a known object freely.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Calibration apparatus, 11 User, 12 Known object, 13 Camera, 14 Storage part, 15 Evaluation part, 16 Output part, 17 Calculation part, 18 Known plane, 19 Known plane image

Claims (10)

Imaging means for imaging known objects from different viewpoints;
Storage means for storing an image captured by the imaging means;
Evaluation means for evaluating the position in the three-dimensional space of the known object estimated from the image stored in the storage means;
A calibration apparatus comprising: calibration means for calibrating a parameter indicating characteristics of the imaging means based on the known object placed at a highly evaluated position in the evaluation means. Instructing means for instructing to place the known object at a highly evaluated position in the evaluating means,
The calibration apparatus according to claim 1, wherein the calibration unit calibrates the parameter based on the known object arranged according to an instruction from the instruction unit.   The calibration apparatus according to claim 2, wherein the instruction unit instructs to place the known object at a vertex of a maximum rectangle in an image region where an imaging region of the imaging unit overlaps.   2. The calibration means calibrates the parameter by a combination having the highest evaluation value in each evaluation value based on a combination of positions of the known object in the three-dimensional space estimated by the evaluation means. The calibration device described.   The calibration apparatus according to claim 1, wherein the evaluation unit evaluates characteristics of the imaging unit based on a variance of the known object at a position in a three-dimensional space estimated by the evaluation unit. The known object has a plane;
The evaluation means detects the plane from an image stored in the storage means, and evaluates the characteristics of the imaging means by estimating the orientation of the plane in a three-dimensional space. The calibration device described. An imaging step of imaging a known object from different viewpoints by an imaging means;
A storage step of storing an image captured by the imaging step in a storage unit;
An evaluation step for evaluating the position in the three-dimensional space of the known object estimated from the image stored in the storage means;
A calibration step of calibrating a parameter indicating characteristics of the imaging means based on the known object placed at a highly evaluated position in the evaluation step. An instruction step for instructing to place the known object at a highly evaluated position in the evaluation step;
8. The calibration method according to claim 7, wherein, in the calibration step, the parameter is calibrated based on the known object arranged according to an instruction in the instruction step.   8. The calibration method according to claim 7, wherein in the instruction step, the arrangement of the known object is instructed to the vertex of the largest rectangle in the image area where the imaging area of the imaging means is superimposed.   8. In the calculation step, the parameter is calibrated by a combination having the highest evaluation value in each evaluation value based on a combination of positions in the three-dimensional space of the known object estimated in the evaluation step. The calibration method described.

Images