JPH10122819A - Method and device for calibration - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simply perform calibration irrespective of the dimension of a measuring space. SOLUTION: Calibration processing is executed by using images from two cameras 1a, 1b having equal focal lengths whose optical axes are arranged in parallel. The images from the cameras 1a and 1b are stored in picture image memories 4a and 4b, respectively. The image of the second camera 1b is displayed on a monitor 9. When an operator specifies, on the displayed image, a plurality of image points of an object point whose spatial coordinates are known, a coordinate processing part 6 extracts points corresponding to the specified points, on the input image from the camera 1a. By using the coordinated results and the positional relation of the cameras 1a, 1b, a calibration processing part 8 calculates stereo coordinates of each object point. By using the calculated results and the known spatial coordinates, parameters expressing the positional relation between a stereo coordinate system and a spatial coordinate system are calculated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image obtained by imaging a predetermined observation position with two or more imaging means.
The present invention relates to a calibration method for calculating a parameter required for a measurement process for an apparatus for performing dimension measurement, and to the apparatus.

[0002]

2. Description of the Related Art When performing three-dimensional measurement using two or more television cameras (hereinafter simply referred to as "cameras"), a spatial coordinate system having a predetermined spatial position as an origin in an actual space is set in advance. The position information of the object in the space coordinate system is output.

[0003] Specifically, the position information is obtained by extracting a feature point indicating an object point of an object from an image from each camera,
These feature points are associated between the images, and the two-dimensional coordinates of each associated feature point and a parameter representing the relationship between the camera coordinate system on the image plane of the camera and the spatial coordinate system (generally, “camera parameter”) ), And it is necessary to carry out calibration for determining the camera parameters prior to the measurement processing.

Now, for an object point P whose spatial coordinates are (X, Y, Z), an image point P 'obtained by imaging with any camera is located at the position (X', Y ') on the image. Assuming that an image is formed, the relationship represented by the following equations (1) and (2) is established between these coordinate values X, Y, Z, X ', and Y'.

[0005]

(Equation 1)

[0006]

(Equation 2)

In the above equations (1) and (2), c _{1 to} c ₁₂ are the camera parameters described above. In order to obtain these parameters, a plurality of object points whose spatial coordinates are known are sampled. It is necessary to substitute the coordinates of the image points of the sample points on the image and the known spatial coordinates into the equations (1) and (2) for each image captured by each camera. is there. In this case, in order to obtain twelve unknown parameters, it is necessary to set at least six points whose spatial coordinates are known and are not located on the same plane as the sample points.

The setting of the sample points is generally performed using a calibration work in which regular marks such as grids and dots are drawn on the surface of a work such as a cube having a fixed shape and size. The center of gravity and the like of each mark described above are extracted and used as the two-dimensional coordinates of the sample points.

[0009]

However, in the above-described calibration method, it is necessary to prepare at least six sample points, and it is necessary to calculate parameters for each camera. There is such a problem. In addition, since this calibration method limits the measurement space to a narrow range of several cm to several m square, it is possible to perform calibration using a work having a size of about 10 cm to 1 m, When a large measurement space is required, such as when measuring the position of a vehicle on the road, it is difficult to identify each sample point mark on an image with a work of this size, and the parameters are calculated accurately. There is a problem that it becomes impossible.

On the other hand, it is theoretically possible to set a large work in accordance with the size of the measurement space and perform calibration, but a great deal of labor is required to create and install the work. Also, in order to obtain each parameter with high accuracy, it is necessary to set a large number of sample points. However, there is a problem that it takes a lot of time until the measurement conditions are set. Further, when it is impossible to enter a measurement area such as a dangerous area, it becomes impossible to use the above-described calibration method.

The present invention has been made in view of the above-mentioned problems. The relative positional relationship between two or more image pickup means is obtained in advance, and the image points of a plurality of object points are designated on any one of the images. Then, based on the correspondence between these designated points and the object points on the other image, stereo coordinates are calculated, and the positional relationship between the spatial coordinate system for indicating the actual spatial position and the stereo coordinates is obtained. A first technical problem is to easily perform calibration regardless of the size of a measurement space.

Further, according to the present invention, at least three image points of object points located on the same plane are designated on an image obtained by any one of the image pickup means, stereo coordinates of each object point are calculated, and the plane is used as a reference. A second technical problem is that by setting a spatial coordinate system to determine the positional relationship with the stereo coordinate system, calibration can be performed without measuring the position of the sample point in the measurement space. .

[0013]

The invention of claims 1 to 3 is
A predetermined observation position is imaged by two or more imaging means and 3
A method for performing calibration on an apparatus for performing dimension measurement, wherein in the invention according to claim 1, a first step of storing a relative positional relationship of the image capturing means and an image captured by each image capturing means are obtained. On one of the images,
A second step of designating a plurality of image points of an object point whose position in the spatial coordinate system having a predetermined spatial position as the origin is known, and, on an image obtained by another imaging means, A third step of extracting corresponding points, and calculating spatial coordinates of each object point in a stereo coordinate system having an image pickup position as an origin based on the correspondence between the designated points and the relative positional relation of the image pickup means; A fourth step of obtaining a positional relationship between the spatial coordinate system and the stereo coordinate system using the calculated stereo coordinates of each object point and the known spatial coordinates is performed in series.

According to the second aspect of the present invention, in the second step, at least three image points of object points whose spatial coordinates are known on the image are specified.

According to a third aspect of the present invention, in the calibration method, the first method stores a relative positional relationship of the imaging means.
And a second step of specifying image points of at least three object points located on the same plane on any one of images obtained by imaging by each imaging unit;
On the image obtained by the other imaging means, corresponding points of the respective designated points are respectively extracted, and based on the correspondence between these designated points and the relative positional relation of the imaging means, stereo coordinates with the imaging position as the origin. A third step of calculating spatial coordinates of each object point in the system, and setting a spatial coordinate system with reference to a plane on which each object point is located, and using the calculated stereo coordinates of each object point to obtain the spatial coordinates. A fourth step of obtaining a positional relationship between the system and the stereo coordinate system is performed in series.

The invention according to claims 4 to 10 relates to an apparatus for performing calibration for an apparatus for performing three-dimensional measurement by imaging a predetermined observation position by two or more imaging means.
First, a calibration device according to a fourth aspect of the present invention includes an image input unit that inputs an image from each of the imaging units; a storage unit that stores a relative positional relationship between the imaging units; Specifying means for specifying a plurality of image points of an object point whose position in a spatial coordinate system having a spatial position as an origin is known, and extracting a corresponding point of each point specified by the specifying means on another input image; Calculating means for calculating the spatial coordinates of each object point in the stereo coordinate system with the imaging position as the origin, based on the correspondence relationship and the stored relative positional relationship of each imaging means; and A parameter calculating unit that calculates a parameter representing a positional relationship between the spatial coordinate system and the stereo coordinate system using the stereo coordinates and the known spatial coordinates.

According to a fifth aspect of the present invention, the specifying means specifies at least three image points of object points whose spatial coordinates are known on the input image.

According to a sixth aspect of the present invention, there is provided a calibration apparatus comprising the same image input means and storage means as described above, and at least three image points of an object point located on the same plane on any input image. A designation unit for designating, and, on another input image, a corresponding point of each point designated by the designation unit is extracted, and an imaging position is determined based on the correspondence and the stored relative positional relationship of each imaging unit. Calculating means for calculating the spatial coordinates of each object point in the stereo coordinate system having the origin as the origin, setting means for setting the spatial coordinate system based on the plane on which each object point is located, and calculated stereo coordinates of each object point And a parameter calculating means for calculating a parameter representing a positional relationship between the spatial coordinate system and the stereo coordinate system set by the setting means.

According to a seventh aspect of the present invention, the setting means sets a spatial coordinate system such that any two axes are included on a plane on which the object points are located.

In the invention according to claim 8, the designation means designates, on any of the input images, an image of one line segment and one object point located on the same plane, and the setting means comprises: A spatial coordinate system is set such that one axis is located in the direction of the line segment and another axis is located in a direction perpendicular to this axis on the plane.

In the ninth aspect of the present invention, the designating means designates an image of a plurality of line segments located on the same plane on the image, and the setting means designates an image in a direction of one of the line segments. The spatial coordinate system is set such that one axis is located and another axis is located on the plane in a direction perpendicular to this axis.

In the tenth aspect of the present invention, the specifying means includes:
The image processing apparatus further includes a feature extracting unit that extracts a feature indicating a line segment from the input image, and is configured to determine an image of the designated line segment from the extraction result.

[0023]

According to the first and fourth aspects of the present invention, the observation position is imaged by two or more imaging means whose relative positional relationship is known in advance, and the spatial coordinates are determined on the image from any one of the imaging means. A plurality of image points of the object point are designated, and corresponding points of these designated points are extracted on an image from another imaging means. Next, after the stereo coordinates of each object point (that is, the spatial position of the object point viewed from the imaging position) is calculated based on the correspondence relationship and the relative positional relationship of each of the imaging units, the known spatial coordinates of each object point are calculated. From the relationship with the calculated stereo coordinates, the positional relationship between the spatial coordinate system and the stereo coordinate system is obtained. This eliminates the need to calculate parameters for each camera, greatly reducing the time required for calibration.

According to the second and fifth aspects of the present invention, at least three image points of points whose spatial coordinates are known are specified as the specified points.
The labor required for calibration is greatly reduced as compared with the conventional method in which it is necessary to specify individual points.

According to the third and sixth aspects of the present invention, at least three image points of an object point on the same plane are designated on an image from any one of the imaging means, and stereo coordinates are calculated for each of them. The spatial coordinate system is set based on the plane on which the object point is located, and the positional relationship between the spatial coordinate system and the stereo coordinate system is calculated using the calculated stereo coordinates. Can be implemented.

According to the seventh aspect of the present invention, the setting process of the coordinate system is simplified by setting the real space coordinate system so that two axes are included on the plane where each designated point is located.

According to the eighth and ninth aspects of the present invention, an image of a predetermined line segment is designated, and one axis of the spatial coordinate system is set in the direction of the line segment, and is set in a direction perpendicular to the line segment on a plane. Another one
Since the axes are set, the setting processing of the spatial coordinate system can be further simplified. In addition, by setting the space coordinate system based on the line segment, the position of the target object in the actual space can be easily recognized.

According to the tenth aspect of the present invention, the feature indicating the line segment is extracted from the input image, and the image of the specified line segment is determined based on the extraction result. To perform high-precision calibration.

[0029]

FIG. 1 shows a preferred embodiment of a calibration apparatus according to the present invention. Hereinafter, an embodiment according to claims 1, 2, 4, and 5 according to FIGS.
FIG. 11 shows an embodiment according to claims 3, 6, and 7.
The embodiments according to claims 8 and 9 will be described with reference to FIG. Further, a specific example of the calibration device according to the tenth aspect of the present invention will be described with reference to FIGS.

[0030]

FIG. 1 shows an example of the configuration of a calibration apparatus according to the present invention. The calibration device 2 performs calibration necessary for measurement on a measurement device (not shown) that performs a three-dimensional measurement process using the imaging unit 1 including the two cameras 1a and 1b. A / D converters 3a and 3b for processing images from the cameras 1a and 1b; image memories 4a and 4b for storing digitally converted input image data; It includes an attachment processing unit 6, a calibration data memory 7, a calibration processing unit 8, a monitor 9, and the like.

Each of the cameras 1a and 1b has a lens having the same focal length, and is arranged in a state where each optical axis is parallel to each other at a predetermined interval. FIGS. 2 (1) and (2)
FIG. 3 shows a specific configuration example of the image pickup unit 1, and shows each camera 1.
By fixing and disposing the units a and 1b in the housing 10 and on the support plate 11, the relative positional relationship between the cameras 1a and 1b is maintained.

FIG. 3 shows the relationship between the spatial coordinate system (hereinafter referred to as "stereo coordinate system") determined by the positional relationship between the cameras 1a and 1b and the camera coordinate systems of the cameras 1a and 1b. In the figure, O ', each camera 1a, 1b of the projection center point C _L, a center point of the base line connecting the C _R, the point O' to the origin, the direction of the base line X 'axis, the height A stereo coordinate system is set in which the direction is the Y 'axis and the depth direction is the Z' axis. Each camera 1a as described above also, since the optical axis of the 1b are set in parallel, where each projection center point C _L, the camera coordinate system as the origin C _R X _L,
Y _L , Z _L and X _R , Y _R , Z _R are all parallel to the axes X ′, Y ′, Z ′ of the stereo coordinate system.

In the figure, reference numerals 12a and 12b denote each camera 1
FIGS. 1A and 1B show image forming planes. For simplicity, the image forming planes 12a and 12b are shown on the opposite side of the actual image forming position, and the image forming plane of the first camera 1a is shown. 12a
The 2-dimensional coordinate system by the axes of x _l, y _l in the second
The two-dimensional coordinates on the imaging surface 12b of the camera 1b by the axes of x _r, y _r, represents respectively.

[0034] Now each projection center point C _L, stereo coordinates C _R, respectively (-B, 0,0), (B , 0,0), each camera 1a, the focal length of the 1b is f, the imaging Surface 12a,
12b origin o _l of each of the two-dimensional coordinate system, expressed by the o _r using stereo coordinates, the origin o _l, o _r respectively (-B, f, 0), point (B, f, 0) Is represented. In this example, the cameras 1a and 1b
Is assumed to be placed in an ideal state,
In practice, each camera 1a, 1
It is necessary to carry out calibration for the positional relationship b.

By arranging the imaging unit 1 in such a state at a predetermined position in the actual space and imaging the observation position, the spatial position of each object point at the observation position can be determined by the spatial coordinates in the stereo coordinate system. (Hereinafter referred to as “stereo coordinates”).

Returning to FIG. 1, one of the image data from the cameras 1a and 1b (the image from the camera 2b in the illustrated example) is also output to the monitor 9. Input processing unit 5
Designates a predetermined image point on the input image displayed on the monitor 9 using a pointing device such as a mouse.
These designated points are image points of points whose spatial coordinates are known in a space coordinate system defined in advance with a predetermined position in the space as the origin, and the coordinates of the designated points are together with the input image. The information is displayed on the monitor 9 and output to the association processing unit 6.

The associating section 6 is for extracting a corresponding point in the other input image for each of the designated points, and the associating result is output to the calibration processing section 8.

The calibration data memory 7 has
As data indicating the relative positional relationship between the cameras 1a and 1b, parameters such as the distance 2B between the cameras and the focal length f (hereinafter referred to as "observation system parameters") are stored. The calibration processing unit 8 associates these observation system parameters with the association processing unit 6
Using the result of the association of each designated point obtained by the above, a parameter representing the positional relationship between the spatial coordinate system and the stereo coordinate system (hereinafter referred to as “coordinate system parameter”) is calculated. This calculation result is filed together with the observation system parameters and stored in the calibration data memory 7.
Is stored within.

FIG. 4 shows a series of procedures in the above-mentioned calibration apparatus 2, and the calibration procedure will be described in accordance with the flow of FIG. 4 with reference to FIGS. .

First, in the first step 1 (each step is indicated by "ST" in the figure), when images from the cameras 1a and 1b are taken into the apparatus, the A / D converters 3a and 3b output the analog signals. The amount of image data is digitally converted and output to the image memories 4a and 4b.

FIGS. 5A and 5B show the input images 13a and 13b when the camera 1a is arranged in the left direction and the camera 1b is arranged in the right direction. In addition to the images 14a and 14b of the object having the shape and the images 15a and 15b showing the support surface of the object, the images 16a, 16b, 17a and 1 of the marks written on the support surface
7b appears. At this time, the input image 13b from the camera 2b is displayed on the monitor 9 (hereinafter, this image is referred to as a "reference image"). In the next step 2, the operator refers to this reference image and adjusts the spatial coordinates. Specify three or more image points of known object points.

FIG. 6 shows an example of a display screen of the monitor 9. When the operator sets windows W _RA , W _RB , and W _RC at predetermined vertex positions of the mark on the reference image 13b, input processing is performed. The unit 5 includes these windows W _RA , W _RB ,
Image point from W in _RC indicate the vertices of the marks of a _R,
Extract b _R and c _R and specify them as designated points. Here, the image point of the object point on the same plane is specified, but the object point at any position in the space may be specified as long as the spatial coordinates of the object point are known. Instead of specifying discrete points, specify at least one line segment image and an image point of an object point in any space, or specify two or more line segment images. Is also good.

Next, the association processing section 6 extracts image points corresponding to each designated point on the input image 13a from the camera 1a (hereinafter, this image is referred to as "corresponding image") (step 3). FIG. 7 shows the designated points a _R ,
b _R, of c _R, an example of extracting a _L point corresponding to the point a _R. Processor 6 First association is on the corresponding image 13b, sets epipolar lines EP of the designated point a _R. In this case, since the cameras 1a and 1b are arranged with the optical axis parallel and side by side, the epipolar line EP
It is set parallel to the x _l-axis at a position corresponding to the y-coordinate of the point a _R.

Further, the associating processing unit 6 scans the window W _LA having the same size as the window W _RA on the epipolar line EP, while simultaneously scanning the image data in the window W _{LA at} each scanning position with the window W _LA . by using the image data in the W _RA execute the following equation (3), calculates the dissimilarity D between each window. In the following equation, g _ri and g _li are windows W _RA and W _LA , respectively.
Indicates the luminance value of the i-th pixel, and n indicates the number of pixels in each window.

[0045]

(Equation 3)

[0046] Figure 8 is a calculated value of the dissimilarity D at each scan position, an illustration in correspondence with the x coordinate of the center point of the window W _LA, difference D is a window W _LA at the time when the minimum center point (point x in the figure coordinate is x _LA) is considered the a corresponding point a _L to the specified point a _R. When the scanning of the window W _LA on epipolar line EP is finished, the correspondence processing section 6 checks the degree of difference D at each scanning position based on the above principle, to identify the corresponding points a _L of the to the specified point a _R .

When the above-described association processing is performed for all the designated points, the calibration processing unit 8
Calculates the stereo coordinates of the object point corresponding to each designated point using the two-dimensional coordinates of these associated points, and further uses the calculated stereo coordinates and the known spatial coordinates to calculate the coordinates. Calculate system parameters (Step 4,
5).

Now, in each of the first and second input images, the coordinates of the image points a _R and a _L are respectively expressed as (x _RA , y _RA ) (x
_LA, When y _LA), stereo coordinates of those points A corresponding to these image point _{(X A ', Y A'} , Z A ') is given by the following equation (4).

[0049]

(Equation 4)

FIG. 9 shows the relationship between the spatial coordinate system (represented by the X, Y, and Z axes) and the stereo coordinate system in the measurement processing shown in FIGS. Each axis X ', Y', Z 'is defined by each axis X, Y,
Z is set to a position shifted and rotated by a predetermined amount, respectively. Note that the spatial coordinate system in this case is
A predetermined position on the support surface is set in advance as the origin. Points A, B, and C in the figure are designated points a _R , b
_R and c _R are object points, and in this case, each of the object points A, B and C is located on the XZ plane.

Now, each axis X ', Y', Z of the stereo coordinate system
′ Is α, for each axis X, Y, Z in the spatial coordinate system, respectively.
Assuming that the rotation is shifted by β and γ and the position is shifted by Hx, Hy and Hz, the stereo coordinates (X ′, Y ′, Z ′) and the space coordinates (X, Y, Z), the following relational expression holds.

[0052]

(Equation 5)

The nine unknown parameters sin α, sin β, sin γ, cos α, cos β,
In order to obtain cosγ, Hx, Hy, and Hz, at least three points where both the spatial coordinates and the stereo coordinates are known are required. Therefore, as described above, for the three points A, B, and C whose spatial coordinates are known, after specifying the image points a _R , b _R , and c _R on the reference image displayed on the monitor, these specified points a _R, b _R, the corresponding point on the coordinates and the corresponding image of the c _R a _L, b _L, each point by and c _L coordinates a, B,
By obtaining the stereo coordinates of C and substituting the calculation result and the known space coordinates into the equation (5), each parameter can be calculated. In order to improve the parameter calculation accuracy, four or more image points of points whose spatial coordinates are known on the reference image are specified, and the stereo coordinates and spatial coordinates of these points are substituted into the equation (5). Then, the least square method may be executed.

Returning to FIG. 4, the rotational deviation amounts α, β, γ and the positional deviation amounts Hx, H obtained by the above equation (5)
The values y and Hz are output to the calibration data memory 7 as the above-described coordinate system parameters, and stored together with the observation system parameters (step 6). This file is extracted as needed and stored in a memory or the like of the measuring device.

The above-described calibration process uses an object point whose spatial coordinates are known. However, when it is difficult to measure the spatial coordinates at the observation position and to set the spatial coordinate system, the calibration of the target object is performed. It is possible to set a spatial coordinate system on the basis of the surface and calculate the coordinate system parameters using the stereo coordinates of at least three object points on this support surface.

FIG. 10 shows a specific example of setting the spatial coordinate system. A, B, and C in the figure are points a _R , b _R , and designated on the reference image by the second camera 2b, as in the above embodiment.
c _R to a point corresponds, these object point A, B, and C is the XZ plane on the plane (hereinafter referred to as "reference plane") that is located shall be located. Next, a perpendicular VL lowered from the origin O 'of the stereo coordinate system onto the reference plane is set as the Y axis of the spatial coordinate system, and an intersection between the perpendicular VL and the reference plane is set as the origin O. , The Z axis in the direction in which the Z ′ axis of the stereo coordinate system is projected,
The X axis is set in a direction perpendicular to the axis.

In the illustrated example, between the spatial coordinate system and the stereo coordinate system, the rotational deviation β in the y-axis direction and the positional deviations Hx and Hz in the x- and z-axis directions are all zero.
If the amount of rotational deviation α in the axial direction, the amount of rotational deviation γ in the z-axis direction, and the amount of positional deviation Hy in the y-axis direction are known, all the parameters of the above equation (5) are calculated.

Assuming that the reference plane exists at a position that does not pass through the origin of the stereo coordinate system, the reference plane is expressed by the following equation (6) in the stereo coordinate system.

[0059]

(Equation 6)

Here, by substituting the calculated stereo coordinates of the object points A, B, and C into the equation (6),
The constants u, v, and w in the equation (6) are calculated, and the calculated results are substituted into the equations (7) to (9) to calculate the α, γ, and Hy.

[0061]

(Equation 7)

[0062]

(Equation 8)

[0063]

(Equation 9)

Note that, of the three minimum designated points required for calculating the coordinate system parameters, two points may be image points of points located on a predetermined straight line on the reference plane. In this case, if the spatial coordinates are set such that any one axis of the spatial coordinate system is located on this straight line, the measurer can easily recognize the spatial coordinate system.

FIG. 11 shows an example in which, on the reference image 13b displayed on the monitor 9, an image of a line segment corresponding to the contour of the support surface of the object and an image point a _R of the object point A on the support surface are designated. Is shown. Note that here, in order to reduce the line segment extraction error,
Set a plurality of windows W _R1 to _W-R5 on the line, and to extract the center point of the line segments included in each window W _R1 to _W-R5 as specified point.

The coordinates of each designated point are output to the associating processor 6 in the same manner as in the previous embodiment, and the corresponding points on the corresponding image 13a are extracted. Stereo coordinates of an object point corresponding to each associated point are calculated. Further, the calibration processing unit 8 executes the least squares method by substituting the respective stereo coordinates obtained for the constituent points of the straight line into the following equation (10) to specify the straight line including the actual line segment. Are the intercepts φ in the x, y, and z directions.
x, φy, φz and gradients ψx, ψy, ψz are calculated.

[0067]

(Equation 10)

FIG. 12 shows an example of setting a spatial coordinate system based on the specified straight line and the reference plane.
3b in the direction of the straight line L1 corresponding to the line segment specified on
The axis is set. In addition, the X axis is set in a direction perpendicular to the Z axis through an intersection O ″ between a perpendicular drawn from the origin O ′ of the stereo coordinate system onto the reference plane and the reference plane. Further, these X and Z axes are determined. At the origin O position, the Y axis is set in a direction perpendicular to the reference plane.

On the other hand, since the stereo coordinates of point A corresponding to the designated point a _R other than the line segment are also calculated in the same manner as described above, a reference is obtained from the stereo coordinates and the equation (10) of the straight line. Each parameter u, v, w of the above equation (6) indicating the plane is calculated. Further, by substituting these parameters into the above equations (7) to (9), α, γ,
Each parameter of Hy is calculated. The rotation deviation amount β in the y-axis direction is a parameter ψ indicating the inclination of the straight line.
It is calculated by the following equation (11) using x, ψy, ψz.

[0070]

[Equation 11]

Further, when the respective coordinate systems have the relationship as shown in FIG. 12, the parameters Hx and Hz indicating the amounts of displacement of the x and z in the respective axial directions are Hx = φx and Hz = 0, respectively. As a result, all the coordinate system parameters necessary for calculating the spatial coordinates are calculated, so that the conditions for the three-dimensional measurement processing can be adjusted. Instead of specifying an image of one line segment and one object point, an image of two or more line segments on the same plane may be specified.

As described above, if stereo coordinates are calculated for three or more points located on the same plane, a spatial coordinate system based on this plane is set, and the coordinate system parameters are calculated. Therefore, even when measurement is performed in a large measurement space such as outdoors or when it is difficult to measure a position for calibration, calibration can be performed without any problem. in this case,
If a mark or a line segment on the support surface is determined as an index at the observation position, and a feature corresponding to the index is extracted from the input image, calibration can be performed more simply and accurately.

FIG. 13 shows an example of the configuration of a calibration apparatus having a function of extracting the feature indicating the index. The configuration is the same as that of FIG. 1 (in this case, the same reference numerals are assigned to the components as in FIG. 1). ), And a feature extraction unit 18 as a configuration. This feature extraction unit 18 is used for each camera 1
Among the input images a and 1b, an input image from the camera 2b used as a reference image is fetched and processed. Here, a feature indicating a line segment on the image is extracted.

FIGS. 14A and 14B show input images 19a and 19b from the cameras 1a and 1b when a predetermined position on the road is to be observed.
Scans a Laplacian filter as shown in FIG. 15 on the reference image 19b in FIG. 14 (2) to extract edge constituent points on the image. FIG. 16 shows the result of edge extraction for the reference image 19b, in which the boundary of the road and the outline of the lane at the center of the road are extracted.

Further, the feature extraction unit 18 performs a Hough transform on the edge extraction result to extract a feature that is a line segment candidate on the image. When the extraction result of the line segment candidates is displayed on the monitor 9, the operator looks at this display screen, selects a line segment necessary for calculating the coordinate system parameters, and uses the input processing unit 5 in the same manner as described above. To execute the specified process. FIG.
FIG. 18 shows a screen displaying the feature extraction result by the Hough transform, and FIG. 18 shows a screen displaying the result of specifying the line segment by the operator. The specified line segment m1 is recognizable by the operator. , M2 are displayed in bold lines.

The specified result is also output to the associating processing section 6, and after the points corresponding to the constituent points of each line segment are extracted,
Further, the calibration processing unit 8 obtains three-dimensional data on the specified actual line segment. Thereafter, in the same manner as in the embodiment of FIG. 12, after the spatial coordinate system in which the Z axis is located on any of the line segments is set, the coordinate system parameters are set using the stereo coordinates of the constituent points of each line segment. Calculation processing is performed.

FIG. 19 shows an example of the configuration of an observation apparatus that performs three-dimensional measurement using the calibration data generated by the above-described methods. In addition to the imaging unit 1, the A / D conversion unit 20a , 20b, image memory 21a,
21b, feature extraction unit 22, association processing unit 23, calibration data memory 24, three-dimensional measurement processing unit 2
5, an output unit 26 and the like.

The A / D converters 20a and 20b are for taking in the images from the cameras 1a and 1b of the image pickup unit 1 and performing digital conversion, and the digital images are stored in the image memories 21a and 21b. The feature extracting unit 22 is for extracting the edge constituent points from the input images in each of the image memories 21a and 21b, and the extraction result is output to the association processing unit 23.

The associating processing unit 23 executes an associating process between the images for each of the extracted edge constituent points, and outputs the associating result to the three-dimensional measurement processing unit 25. Instead of the above configuration, similarly to the calibration device of FIG. 13, the edge constituent points are extracted from only one of the input images, and the points corresponding to the respective edge constituent points are extracted on the other input image. May be configured.

The calibration data memory 24 stores a calibration data file including the observation system parameters of the imaging unit 1 and the coordinate system parameters calculated by any of the above embodiments. The three-dimensional measurement processing unit 25 calculates the actual object point corresponding to each edge composing point, that is, the spatial coordinates of the contour composing point of the object, using these parameters and the association result.

The output unit 26 is provided with the three-dimensional measurement processing unit 25.
It outputs the spatial coordinates of each contour composing point calculated by the above, and is constituted by a monitor, a printer, or a transmission device to another device.

FIG. 20 shows a series of procedures of a three-dimensional measurement process by the observation device. First, in the first step 1, when the image from the imaging unit 1 is stored in the image memories 21a and 21b, the feature extraction unit 22 extracts the edge constituent points corresponding to the contour of the object from each input image.

FIG. 21 shows an example of a Laplacian filter for extracting edge constituent points in the feature extraction unit 18.
FIG. 22 shows the results of extraction by the Laplacian filter. Note that FIG. 22 shows an extraction result for the same input image as that shown in FIG. 5 (2), in which vertical component points on the image are extracted by the Laplacian filter.

Returning to FIG. 20, when the edge composing points are extracted, in the next step 3, the associating processing section 23 performs the associating process between the images of the respective edge composing points.
Specifically, the associating process focuses on a predetermined edge composing point on one of the input images as a reference image, and detects an edge located on the epipolar line of the point of interest on the other image. This is performed by selecting, from among the constituent points, those having the closest image data to the image data near the target point on the reference image as the corresponding points. When all the edge constituent points have been associated, the process proceeds to the measurement process by the three-dimensional measurement processing unit 25.

First, the three-dimensional measurement processing unit 25 uses the two-dimensional coordinates and the observation system parameters stored in the calibration data memory 24 for each set of edge constituent points associated with each input image. The equation (4) is executed to calculate the stereo coordinates of the contour constituting point of the object (step 4). Further, the three-dimensional measurement processing unit 25 reads the coordinate system parameters from the calibration data memory 24, substitutes these parameters and the stereo coordinates into the above equation (5), and calculates the spatial coordinates of each contour constituent point ( Step 5). The calculated spatial coordinates are output from the output unit to the outside (step 6), and a series of procedures is completed.

Note that the above-described measuring device corresponds to FIG. 1 or FIG.
The measurement processing is executed by fetching the calibration data generated by the calibration device 2 shown in (1). However, the present invention is not limited to this. Is also good.

[0087]

According to the first and fourth aspects of the present invention, the observation position is imaged by two or more imaging means whose relative positional relationship is known in advance, and the known spatial coordinates of a plurality of object points and these object positions are obtained. Since the positional relationship between the spatial coordinate system and the stereo coordinate system is obtained using the stereo coordinates calculated from the correspondence between the image points of the points on each image, it is necessary to calculate the parameters for each camera. And the time required for calibration can be significantly reduced.

According to the second and fifth aspects of the present invention, since the positional relationship between the spatial coordinate system and the stereo coordinate system is obtained by using at least three points whose spatial coordinates are known, at least six points are used for calibration. The labor required for the calibration can be greatly reduced as compared with the conventional method in which it is necessary to specify.

According to the third and sixth aspects of the present invention, for each object point on the same plane, the stereo coordinates are calculated based on the correspondence between the image points on each image, and based on the plane where these object points are located. Since the spatial coordinate system is set and the positional relationship between the spatial coordinate system and the stereo coordinate system is obtained using the calculated stereo coordinates, there is no need to prepare object points whose spatial coordinates are known for calibration, and measurement is performed. Calibration can be performed even when actual measurement in space is difficult.

According to the seventh aspect of the present invention, since the spatial coordinate system is set so that two axes are included on the plane where each object point is located, the setting process of the coordinate system can be simplified.

According to the eighth and ninth aspects of the present invention, one axis of the spatial coordinate system is set in a direction of a predetermined line segment on a plane, and another axis is set in a direction perpendicular to the line segment on the plane. As a result, the setting process of the spatial coordinate system can be further simplified. In addition, by setting the space coordinate system based on the line segment, the position of the target object in the actual space can be easily recognized.

According to the tenth aspect, a feature indicating a line segment is extracted from the input image, and the image of the line segment designated for calibration is determined based on the extraction result. Can be easily performed, and highly accurate calibration can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a calibration device according to one embodiment of the present invention.

FIG. 2 is a perspective view illustrating a specific configuration of an imaging unit.

FIG. 3 is an explanatory diagram showing a relationship between a stereo coordinate system and a camera coordinate system.

FIG. 4 is a flowchart illustrating a procedure of a calibration process.

FIG. 5 is an explanatory diagram showing an example of an input image from each camera.

FIG. 6 is an explanatory diagram showing an example of specifying an image point on a display screen.

FIG. 7 is an explanatory diagram showing a principle for extracting a corresponding point of a designated point.

FIG. 8 is an explanatory diagram showing a principle for extracting a corresponding point of a designated point.

FIG. 9 is an explanatory diagram showing a positional relationship between a stereo coordinate system and a spatial coordinate system.

FIG. 10 is an explanatory diagram showing a setting method of a spatial coordinate system.

FIG. 11 is an explanatory diagram showing an example of specifying an image point on a display screen of a monitor.

FIG. 12 is an explanatory diagram showing a setting method of a spatial coordinate system.

FIG. 13 is a block diagram showing another configuration of the calibration device.

FIG. 14 is an explanatory diagram illustrating an example of an input image from each camera.

FIG. 15 is an explanatory diagram showing a configuration of a Laplacian filter for edge extraction.

FIG. 16 is an explanatory diagram showing an edge extraction process for the input image of FIG. 14 (2).

FIG. 17 is an explanatory diagram showing a result of performing Hough transform on the edge extraction result of FIG. 16;

FIG. 18 is an explanatory diagram illustrating a display example of a designated result of a line segment.

FIG. 19 is a block diagram illustrating a configuration example of a measurement device.

FIG. 20 is a flowchart illustrating a procedure of a three-dimensional measurement process.

FIG. 21 is an explanatory diagram showing a configuration of a Laplacian filter for edge extraction.

FIG. 22 is an explanatory diagram showing an example of an edge extraction result.

[Explanation of symbols]

 1a, 1b TV camera 5 Input processing unit 6 Correlation processing unit 7 Calibration data memory 8 Calibration processing unit 18 Feature extraction unit

Claims (10)

[Claims] 1. A method for calibrating an apparatus that performs three-dimensional measurement by imaging a predetermined observation position by two or more imaging means, wherein a first positional relationship between the imaging means is stored. And specifying a plurality of image points of an object point whose position in a spatial coordinate system having a predetermined spatial position as an origin is known on one of the images obtained by the imaging means. Second
And extracting the corresponding points of each of the designated points on the image obtained by the other imaging means, and determining the imaging position as the origin based on the correspondence between these designated points and the relative positional relationship of the imaging means. A third step of calculating the spatial coordinates of each object point in the stereo coordinate system, and using the calculated stereo coordinates of each object point and the known spatial coordinates to calculate the spatial coordinate system and the stereo coordinate system. And a fourth step of obtaining the positional relationship of the above. 2. The calibration method according to claim 1, wherein the second step is a step of specifying at least three image points of object points whose spatial coordinates are known on the image. 3. A method for calibrating an apparatus for performing three-dimensional measurement by imaging a predetermined observation position by two or more imaging means, the method comprising: storing a relative positional relationship between the imaging means. A second step of designating image points of at least three object points located on the same plane on any one of the images obtained by the imaging means. On the image obtained by the imaging unit, the corresponding points of the respective designated points are respectively extracted, and based on the correspondence between these designated points and the relative positional relationship of the imaging unit, a stereo coordinate system having an imaging position as an origin is used. A third step of calculating the spatial coordinates of each object point, and setting a spatial coordinate system with reference to a plane on which each object point is located, and using the calculated stereo coordinates of each object point and the spatial coordinate system. stereo Calibration method according to claim 1, which comprises carrying out the fourth step of obtaining a positional relationship between the target system in a series. 4. An apparatus for performing calibration for an apparatus which performs three-dimensional measurement by imaging a predetermined observation position by using two or more imaging means, and wherein an image input for inputting an image from each of the imaging means is provided. Means, storage means for storing a relative positional relationship between the respective image pickup means, and a plurality of image points of an object point whose position in a spatial coordinate system having a predetermined spatial position as an origin is known on any one of the input images. Specifying means for specifying, and, on another input image, extracting a corresponding point of each point specified by the specifying means, based on the corresponding relation and the stored relative positional relation of each imaging means, Calculating means for calculating the spatial coordinates of each object point in the stereo coordinate system having the origin as the origin, using the calculated stereo coordinates of each object point and the known spatial coordinates, the spatial coordinate system and the stereo coordinate system. Location Calibration apparatus comprising a parameter calculating means for calculating parameters representing. 5. The calibration apparatus according to claim 4, wherein the specifying unit specifies at least three image points of an object point whose spatial coordinates are known on the input image. 6. An apparatus for performing calibration for an apparatus that performs three-dimensional measurement by imaging a predetermined observation position by using two or more imaging units, and an image input unit that inputs an image from each of the imaging units. Means, storage means for storing a relative positional relationship between the respective imaging means, designating means for designating at least three image points of object points located on the same plane on any of the input images, and other input means On the image, a corresponding point of each point specified by the specifying unit is extracted, and based on the correspondence and the stored relative positional relationship of each imaging unit, each point in a stereo coordinate system whose origin is the imaging position is defined. Calculating means for calculating the spatial coordinates of the object point; setting means for setting the spatial coordinate system with reference to the plane on which each object point is located; and setting by the setting means using the calculated stereo coordinates of each object point Is And a parameter calculating means for calculating a parameter representing a positional relationship between the spatial coordinate system and the stereo coordinate system. 7. The calibration apparatus according to claim 6, wherein the setting unit sets the spatial coordinate system such that any two axes are included on a plane where each of the object points is located. 8. The designating means designates an image of one line segment and one object point located on the same plane on any of the input images, and the setting means designates a direction of the line segment. 7. The calibration apparatus according to claim 6, wherein a spatial coordinate system is set such that any one axis is located on the plane and another axis is located in a direction perpendicular to the axis on the plane. 9. The designating means designates an image of a plurality of line segments located on the same plane on the image, and the setting means designates that any one axis is located in any direction of the line segment. 7. The spatial coordinate system is set such that another axis is positioned on the plane in a direction perpendicular to the axis.
The calibration device described in 1. 10. The apparatus according to claim 6, wherein said designating means includes a feature extracting means for extracting a feature indicating a line segment from said input image, and determines an image of said designated line segment from said extraction result. The calibration device according to any one of the above.