JPH07270124A - Method and apparatus for measurement of imaging system parameter - Google Patents

Abstract

PURPOSE:To easily install an object, to be tested, for an imaging-system parameter measurement, to measure and process the object, to be tested, at high speed and to avoid a measuring error or the like being propagated to a processed result. CONSTITUTION:An image display device 1 which is flat and in which pixels are arranged with high accuracy is installed in front of a visual sensor 2 for an object to be measured. Thereby, the object to be measured is installed with high flexibility and easily. On the basis of input information from the visual sensor 2, a graphic form on the image display device 1 is changed sequentially, and an imaging-system parameter is computed on the basis of geometric information on an observed graphic form by the visual sensor 2 and on the graphic form displayed on the image display device 1. By means of a purely geometrical computation in this manner, the convergence of a solution, the problem of an initial value and the like are eliminated, and the solution can be found at one blow. In addition, when the image display device 1 is used, the graphic form can be detected by processing only the difference between a plurality of images by an inverting display, a brightness change or the like, the detection accuracy of the graphic form is enhanced, an error is reduced, and the influence of the error on a processed result can be reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging system such as position and orientation information of the visual sensor, image center, etc., which is necessary for performing measurement and work in an actual environment from information obtained using the visual sensor as an input means. The present invention relates to a method and apparatus for calculating parameters and the like.

[0002]

2. Description of the Related Art Generally, a method used to calculate an imaging system parameter is to detect a plurality of points or line segments arranged at known positions in space as a test pattern and calculate them as a problem of projective geometry. It is a thing.

In this case, an ideal image pickup system having no distortion can be treated as a pure geometrical problem, but in reality, the distortion aberration of the lens, the distortion of the image pickup element, or the like causes an error. . The method that has been used to solve the error due to distortion is to add a distortion model to the imaging system model,
This is a method of iteratively calculating each parameter so as to minimize the error between the calculated imaging system model and the actual input image.

[0004]

However, in this method, the arrangement of the test pattern to be tested and the way of giving the initial value to the imaging model influence the convergence of the solution. Here, it is difficult to obtain a stable result because the appropriate initial value is completely unknown. Normally, an ideal model without distortion is set as an initial value, but it is difficult to apply it to an optical system with a strong distortion such as a wide-angle lens or a fisheye lens because the solution cannot be expected to converge. In addition, the larger the distortion, the longer iterative calculation takes.

Furthermore, the detection of points and line segments requires an appropriate binarization process, and there is a problem that sufficient accuracy cannot be obtained when the input image is accompanied by noise. Further, since these detections include a quantization error, this also has a problem of affecting the calculation result.

Strict accuracy is also required for setting the test pattern to be tested, and this accuracy also directly affects the result. Therefore, if high accuracy is sought, the simplicity of measurement becomes poor.

An object of the present invention is to solve the problems of the conventional method, such as the ease of setting a test object, the high speed of processing, and the problem of propagation of measurement error to the processing result. It is to provide a measuring method and its device.

[0008]

In order to achieve the above object, in the image pickup system parameter measuring method according to the first aspect of the invention, an image display means having a plane and highly accurate pixel array is used as a measuring object of a visual sensor. A step of installing the image in front of the visual sensor, taking an image with the visual sensor, and changing the figure on the image display means observed by the visual sensor as information; The method includes the step of determining the position and orientation of the visual sensor in space and the parameter caused by the image pickup system including the lens and the image pickup element from the information of the figure, and the above step is repeated once or plural times.

According to the second aspect of the present invention, the image display means having a plane and highly accurate pixel array is installed in front of the visual sensor to be measured, and a graphic is drawn on the display screen of the image display means. And a calculation processing means for inputting the drawn figure from the visual sensor, and determining the position and orientation of the visual sensor in the space and the parameters caused by the image pickup system including the lens and the image pickup element from the drawn figure and the input figure. The device configuration has ,.

According to a third aspect of the invention, in the method of the first aspect, the step of determining the parameters includes a first processing procedure of drawing a figure on the image display means and detecting the figure with the visual sensor. The second processing procedure for calculating the imaging system parameter from the graphic information in the first processing procedure, and the error between the imaging system parameter and the true value obtained in the second processing procedure are detected, and if there is an error, the condition is changed. And a third processing procedure for returning to the first processing procedure and a fourth processing procedure for outputting the imaging system parameter when the error in the third processing procedure disappears.

According to the invention of claim 4, the above-mentioned claim 3 is adopted.
In the present invention, in the second processing procedure, the image center is obtained from the intersection of the straight lines of each set which is displayed by displaying at least two sets of parallel line groups on the image display means.

According to the invention of claim 5, the above-mentioned claim 3 is used.
In the invention, the second processing procedure draws on the image display means points equidistant on a straight line orthogonal to the image plane from the image center obtained as the imaging system parameter, and the points on the image display means 4. The imaging system parameter measuring device according to claim 3, wherein the position and orientation of the visual sensor are calculated from the angle formed by the intersection of the straight lines and the distance of the corresponding point from the intersection.

Further, in the sixth aspect of the present invention, in each of the above measuring methods, the process of detecting a line segment, a point or a figure is performed by reversing the displayed figure on the image display means or changing the brightness of the plurality of images. The difference processing will be performed.

[0014]

In the imaging system parameter measuring method and the apparatus therefor according to the present invention, the installation of the image display means to be tested need only be arranged in front of the visual sensor of the object to be measured. As a result, it is easy to install the test object.

Further, by obtaining the imaging system parameters by pure geometric calculation, it is possible to eliminate the convergence of the solution, the initial value problem, etc., which have been problems in the past, and to obtain the solution in a single stroke. Achieve high speed.

Further, by displaying figures, line segments, and points of the test object or the like by the image display means, they can be displayed in reverse,
By changing the brightness and detecting figures etc. only by the difference processing with multiple images, the accuracy of figure detection etc. is improved and the error is reduced, and the influence of the figure detection error on the processing result is reduced. Allows measurement of system parameters.

[0017]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a diagram for explaining an embodiment of the present invention, in which 1 is an image display device as a test object, 2 is a visual sensor as a measurement object, 3 is a visual sensor 2 and an image display device 1. A computing device that receives and processes input from
Reference numeral 4 is a monitor that displays the input of the visual sensor 2. The image display device 1 is assumed to have a flat and highly accurate image display element array such as a liquid crystal, and is installed while looking at the monitor 4 so as to keep an appropriate inclination with respect to the visual sensor 2. To do.

FIG. 2 is a diagram showing the outline of the processing procedure of this embodiment. <1> is a process of drawing a graphic on the image display device 1 and detecting the graphic with the visual sensor 2, <2> is a process of calculating and acquiring an imaging system parameter from the graphic information in the graphic detection process <1>, <3> is the imaging system parameter acquisition process <
2> is a process of detecting an error between the imaging system parameter and the true value, and <4> is a process result output process of outputting the imaging system parameter when the error in the error detection process <3> is eliminated. .

FIG. 3 is a block diagram of a processing procedure showing a concrete example of the figure detection processing <1> in FIG. 5 is a lens distortion detection process, 6 is a normal lens image center detection process, 7 is a distortion aberration center detection process, 8 is a drawing process, 9 is a figure detection process, and 10 is a figure determination process. Hereinafter, each process will be described.

FIG. 5 is a diagram for explaining the lens distortion detection processing 5, in which 13 and 14 are input images from the visual sensor. In the lens distortion detection process 5, a straight line group is drawn on the image display device 1 and is observed by the visual sensor 2 in FIG. At this time, when all the straight lines are observed as perfect straight lines as in the input image 13 shown in FIG. 5A, the normal image lens center detection processing 6 is performed, and the input image 1 shown in FIG.
When it is observed as a curve like 4, the process shifts to distortion aberration center detection processing 7.

FIG. 6 is a diagram for explaining the normal image lens center detection processing 6, wherein in FIG. 6A, 16 is a group of parallel lines drawn on the image display device, and 15 is a group of parallel lines 16 thereof. It is a vanishing point. In the normal image lens center detection processing 6, a parallel line group 16 intersecting at an arbitrary angle θ is drawn, and a vanishing point 15 of each parallel line is calculated. The same process is performed three times while changing the angle of the image display device 1 with respect to the visual sensor 2.

As shown in FIG. 6 (b), three sets of vanishing points obtained by the three processings are set to 15a and 15 respectively.
b and 15c, the crossing angle on the image display device 1 is θa,
Let θb and θc. Here, each intersection angle of θa to θc is 90.
In terms of degrees, the viewpoint position of the CCD camera with respect to a set of vanishing points on the image plane P can be limited to the surface of a sphere having a straight line connecting the vanishing points as a diameter. Three sets of these spheres can be set independently, and a point O where all the spheres intersect is the viewpoint position.
Therefore, the point O ′ where the perpendicular line drawn from the viewpoint position O intersects on the image plane P is the image center. Further, the length f of OO 'is the focal length.

FIG. 7 is a diagram for explaining the distortion aberration center detection processing 7, wherein 17 and 18 are straight lines observed by the visual sensor, and 19 is an intersection of these straight lines. In the distortion aberration center detection processing 7, as shown in FIG.
Drawing is repeated while moving 7 in parallel, and the position observed as a perfect straight line 18 is detected. As shown in FIG. 7B, this process is performed at least twice from different directions to obtain the intersection point 19 thereof. This intersection 19 is the center of distortion, that is, the center of the image.

FIG. 8 is a diagram for explaining a target drawing figure in the drawing process 8, the figure detecting process 9, and the figure judging process 10. In this series of processes, corresponding points 21 to 24 are projected on the image display device 1 from the image center 19 on the monitor 4 at equal distances n in the horizontal and vertical directions.

FIG. 9 is a diagram for explaining a graphic drawing method in the drawing processing 8, the graphic detection processing 9, and the graphic determination processing 10. Taking the point 21 as an example, when an arbitrary area 26 of the image display device 1 shown in FIG. 9A is blinked, if the change is observed at the point 21 on the monitor 4 in FIG.
Is a candidate area, and if not, a part other than the area 26 is a candidate area. Further, as shown in FIG. 9B, the area on the image display device 1 which has become the candidate area is divided into two areas, and similarly, only one area 27 is blinked to limit the candidate area. The corresponding point 21 is determined by repeating the above-mentioned processing a plurality of times. Do the same processing from point 22
Do about 24.

FIG. 10 is a diagram for explaining a drawing result on the image display device 1, and P is a display portion of the image display device 1. As a result of the above processing, the image center 19 on the monitor 4 and the corresponding points 21 to 24 are drawn on the display portion P of the image display device of FIG. Here, FIG. 10B
As shown in, the distance between the points 21 and 19 on the image display device 1 is a, the distance between the points 22 and 19 on the image display device 1 is b, and the distance between the points 23 and 19 is on the image display device 1. , The distance on the image display device 1 between the points 24 and 19 is d,
The angle at which the straight lines connecting the points 21 and 23 and the points 22 and 24 intersect is θ.

FIG. 4 is a processing procedure configuration diagram showing a specific example of the imaging system parameter acquisition processing <2> in FIG. The imaging system parameter acquisition process <2> includes the viewpoint position calculation process 11 and
It is composed of an imaging system parameter derivation process 12. Hereinafter, each of these processes will be described.

FIG. 11 is a diagram for explaining the principle of viewpoint position measurement. In the visual sensor 2, points 21 to 24 observed at equal intervals from the image center 19 are points on the conical surface of the cone 28 having the apex at the position 29 of the visual sensor 2 in the three-dimensional space. Is the axial direction of the cone. Point 21
And the line connecting point 23 and point 22 and point 24 are projected as straight lines orthogonal to each other on the bottom surface of this cone. Here, when there is a difference in the aspect ratio of the output from the light receiving element of the visual sensor, it is an elliptical cone instead of a cone.

FIG. 12 is a diagram for explaining the visual sensor viewpoint position estimation by the line segment connecting the points 21 and 23. The cone or elliptical cone with the point 29 as the apex is shown in FIG.
As shown in (a), when cut by a plane including a straight line connecting the points 29 and 19 which are the central axes, the cross section becomes a triangle, and the central axis is an angle two-dotted line. When the existence range of the viewpoint position 29 is obtained from this condition, as shown in FIG. 12B, when a> b, the line segment connecting the points 21 and 23 is extended in the direction of the point 23, Distance from 19 ab / (a-
Sphere g with radius ab / (ab) centered on point 30 in b)
It becomes the surface of 1. Similarly, the sphere g2 can be set for the line segment connecting the points 21 and 23. Therefore, the viewpoint position is
As shown in FIG. 12 (c), it is constrained on the circle S where these two spheres intersect.

However, when the image display device 1 is placed perpendicular to the optical axis, the radius of the sphere g becomes infinite, and the viewpoint position cannot be calculated. In this case, a method of instructing to place the image display device 1 at an angle and then performing the viewpoint position calculation again, or a method of calculating the camera parameters while keeping the vertical position is conceivable.

When the calculation is performed in the vertical position as it is, the conventionally proposed method can be applied. For example, the following method can be applied.

Method when aberration distortion is large: Y. Nom
ura, H .; Naruse, M .; Sagara and
A. Ide. "Simple calibratio
nalgorithm for high-disto
region-lens camera ”(Reference 1: IE
EE Trans. on Pattern Anal
isys and Machine Intellige
nce, Vol PAMI-14, No. 11, pp.
1095-1099, Nov. 1992. ) Method when aberration distortion is small: J. Weng. "Camera C
calibration with Distortio
n Models and Accuracy Eva
luation ”(Reference 2: IEEE Trans.
onPattern Analysys and Ma
chin Intelligence, Vol PAM
I-14, No. 10, pp. 965-980, Oc
t. 1992. ) Here, in the conventional method, there is no method other than manually determining the correspondence between the input image and the actual three-dimensional image. With regard to this point, in the present method, since the drawing and the image input are simultaneously performed, it is not necessary to manually determine the corresponding points, and the problem of the conventional method is solved.

FIG. 13 is a diagram for explaining the visual sensor viewpoint position estimation under the condition that the straight lines connecting the points 21 and 23 and the points 22 and 24 are observed at right angles. The viewpoint position observed so that these two straight lines intersect at right angles is point 19
The ratio of the major axis to the minor axis with c as the apex is cos (sin ⁻¹ (tan
(Θ / 2))): The surface of an elliptical cone whose bottom surface is an ellipse of 1. The surface of this elliptical cone and the above-mentioned circle intersect at two points, and the point 29 located in the surface direction of the inner image display device becomes the viewpoint position of the visual sensor. As described above, the image forming point position of the visual sensor 2 with respect to the image display device 1 can be calculated.

The process <3> is a process for verifying the error in the image forming point position of the visual sensor obtained in the process <2>. Specifically, another figure is drawn on the image display device, and verification is performed by the difference between this actual figure shape calculated from this figure and the figure shape calculated only from the visual sensor information. If an error is detected here, the parameters such as the image center are changed and the process returns to the process <1> of FIG. 2. If there is no error, the polar coordinate conversion is performed for each point on the image sensor when the optical axis is the zenith. The table is calculated and the process <4> is entered. The calculation of the polar coordinate conversion table can be performed by the same operation as the processing 8 to 10. This is because when an arbitrary position on the display surface such as a liquid crystal of the image display device is blinked as shown in FIG. 15, when the optical axis is set to the zenith from the positional relationship on the monitor and the display surface, the blinking direction is changed. The principle is that the latitude and longitude are uniquely obtained.

In process <4>, the imaging system parameters are calculated using the result obtained in process <3>. The imaging system parameters mentioned here are optical parameters such as focal length, lens aberration, and image center, and external parameters such as the position and orientation of the camera in real space. Among these, since the image center has already been derived, the focal length, lens distortion, and the like may be obtained. This is performed by obtaining each coefficient of the optical model so that the derived conversion system table fits to the optical model as shown in the above-mentioned reference 2.

Regarding the external parameters, since the relationship between the display surface of the image display device and the camera position is obtained, the direction, distance, and attitude of the display surface from the reference position in the real space can be accurately measured. It can be derived by doing.

In FIG. 14, an image is obtained by calculating the tangents of the bisectors of a triangle connecting the points 30, 21, 23 and a vertex angle θ2 of the triangle connecting the points 30, 22, 24. It is possible to calculate the aspect ratio of the device.

Therefore, the necessary imaging system parameters can be derived.

In the processes described above, when detecting a figure or the like displayed on the image display device, the process of detecting a line segment, a point or a figure is performed by reversing the displayed figure or changing the brightness. By performing the difference processing of the plurality of images obtained in this way, the detection error can be easily reduced.

As described above, in the present embodiment, the image display device is installed in front of the visual sensor with an appropriate inclination, and the figure etc. on the image display device is sequentially changed by the input information from the visual sensor. , Calculate the imaging system parameters from the geometric information of both the visual figure of the visual sensor and the visual figure on the image display device. The main feature is that the difference image is detected.

Due to such a characteristic configuration, measurement is possible even in the case where the image display device to be tested is highly flexible from the prior art, strong aberration distortion in the image pickup system or the aspect ratio of the image is different. It is possible, it realizes high speed and high accuracy of calculation, and even in a noisy situation,
Different effects such as high-precision figure detection are obtained.

If the processing of the above embodiment is repeated a plurality of times for the same image pickup system, the accuracy can be further improved, and if a plurality of visual sensors are used as inputs, the positional relationship between the visual sensors can be measured. Therefore, there is an effect of improving accuracy even in the three-dimensional measurement by the visual sensor or the navigation system by the visual sensor.

[0044]

As described above, according to the imaging system parameter measuring method and the apparatus thereof according to the present invention represented by claims 1 and 2, the imaging parameters can be derived only by pure geometric calculation. Therefore, there is no solution convergence, initial value problem, etc., and the solution can be obtained in one shot, and the processing speed is high. Further, the image display means for the test need only be arbitrarily arranged with respect to the visual sensor, the flexibility of the installation is very high, and the test object can be easily installed.

Further, according to the present invention represented by claims 1 and 2, particularly according to the invention of claim 6, by using the image display means, the display can be reversed and the brightness can be changed. Therefore, the figure or the like can be detected only by the difference processing using a plurality of images, the detection accuracy of the figure or the like can be improved, and the corresponding region of each light receiving element can be detected, so that the quantization error is small. Therefore, optical system distortion, or quantization error,
It is possible to measure the imaging system parameters while reducing the influence of the figure detection error on the processing result.

According to the third aspect of the invention, in particular, the error is detected and the imaging system parameter having no error is output, so that the accuracy can be further improved.

According to the fourth aspect of the invention, in particular, the image center, which is one of the imaging system parameters, can be easily obtained.

Further, according to the invention of claim 5, in particular,
Imaging system parameters relating to the position and orientation of the visual sensor can be easily obtained.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of the present invention.

FIG. 2 is a diagram showing an outline of a processing procedure of the above embodiment.

FIG. 3 is a diagram showing a specific example of figure detection processing in the processing procedure of FIG. 2;

FIG. 4 is a diagram showing a specific example of imaging system parameter acquisition processing in the processing procedure of FIG. 2;

5A and 5B are views for explaining the lens distortion detection process in the figure detection process of FIG.

6A and 6B are views for explaining a normal lens image center detection process in the figure detection process of FIG. 3;

7A and 7B are views for explaining distortion center estimation processing in the figure detection processing of FIG.

8 is a diagram for explaining a target figure in the drawing process, the figure detecting process, and the figure determining process in the figure detecting process of FIG. 3;

9A and 9B are diagrams for explaining a graphic drawing method in the drawing processing, the graphic detection processing, and the graphic determination processing during the graphic detection processing of FIG.

10A and 10B are diagrams for explaining a drawn image by the graphic drawing method of FIG. 9.

11 is a diagram for explaining the principle of viewpoint position measurement in the imaging system parameter acquisition processing of FIG.

12A, 12B, and 12C are views for explaining visual sensor viewpoint position estimation in the imaging system parameter acquisition processing of FIG.

13 is a diagram for explaining visual sensor viewpoint position estimation under specific conditions in the imaging system parameter acquisition process of FIG.

14 is a diagram for explaining the derivation of the aspect ratio of the output image in the processing result output processing of FIG.

15 is a diagram for explaining the principle of calculation of a polar coordinate conversion table in the error detection process of FIG.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Image display device 2 ... Visual sensor 3 ... Calculation processing device 4 ... Monitor 5 ... Lens distortion detection processing 6 ... Normal lens image center detection processing 7 ... Distortion aberration center detection processing 8 ... Drawing processing 9 ... Graphic detection processing 10 ... Graphic Judgment process 11 ... Viewpoint position calculation process 12 ... Imaging system parameter derivation process 13, 14 ... Input image from viewpoint sensor 15 ... Vanishing point 16 ... Parallel line group 17, 18 ... Straight line 19 ... Straight line intersection (image center) 21 to 21 24 ... Points observed equidistant from the center of the image 26, 27 ... Flashing area 28 ... Cone 29 ... Viewpoint position of visual sensor

Claims (6)

[Claims] 1. An image display means having a flat and highly accurate pixel array is installed in front of a visual sensor to be measured, and an image is taken by the visual sensor, and drawing on the image display means observed by the visual sensor. Due to the step of changing the figure by using the figure as information, the position and orientation of the visual sensor in the space from the information of the input figure of the visual sensor and the drawn figure on the image display device, and the imaging system including the lens and the image sensor. A method for measuring an imaging system parameter, comprising the step of determining a parameter to be performed, and repeating the above steps once or a plurality of times. 2. An image display means having a flat and highly accurate pixel array and installed in front of a visual sensor to be measured,
An image is drawn on the display screen of the image display means, and the drawn figure is input from the visual sensor, and the drawing figure and the position and orientation of the visual sensor in space from the input figure, and an image including a lens and an image sensor. An image pickup system parameter measuring device, comprising: a calculation processing unit that determines a parameter caused by the system. 3. The step of deciding the parameter includes a first processing procedure for drawing a figure on the image display means and detecting the figure with a visual sensor, and calculating an imaging system parameter from the figure information in the first processing procedure. The second processing procedure for
The error between the imaging system parameter and the true value obtained in the processing procedure is detected, and if there is an error, the condition is changed and the procedure returns to the first procedure. A fourth processing procedure for outputting imaging system parameters at the time
The imaging system parameter measuring method according to claim 1, wherein 4. The second processing procedure is to display at least two sets of parallel line groups on the image display means and obtain the image center from the intersection of the detected straight lines. 3. The imaging system parameter measurement device according to item 3. 5. The second processing procedure draws on the image display means points equidistant on a straight line orthogonal to the image plane from the center of the image obtained as the imaging system parameter, and draws the points on the image display means. 4. The imaging system parameter measuring device according to claim 3, wherein the position and orientation of the visual sensor are calculated from the angle formed by the intersection of the two straight lines and the distance of the corresponding point from the intersection. 6. The process of detecting a line segment, a point or a figure is performed by a difference process of a plurality of images by reversing or changing the brightness of a figure drawn on an image display means. The parameter measuring method of an imaging system according to claim 4 or 5.