JP3315240B2 - Imaging system parameter measurement method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging system for obtaining position and orientation information of a visual sensor and image center necessary for performing measurement and work in a real environment from information obtained using a visual sensor as input means. The present invention relates to a method and an apparatus for calculating parameters and the like.

[0002]

2. Description of the Related Art Usually, a technique used to calculate an imaging system parameter is to detect a plurality of points or line segments arranged at known positions in space as a test pattern, and calculate it as a projective geometry problem. Things.

In this case, an ideal image pickup system without distortion can be treated as a pure geometrical problem. However, in reality, an error is caused by the lens distortion and the image sensor distortion. . A technique that has been used to resolve errors due to distortion is to add a distortion model to the imaging system model,
In this method, each parameter is repeatedly calculated so as to minimize the error between the calculated imaging system model and the actual input image.

[0004]

In this method, however, the convergence of the solution is affected by the arrangement of the test pattern to be tested and the way of giving initial values to the imaging model. Here, it is difficult to obtain a stable result because an appropriate initial value is completely unknown. Normally, an ideal model without distortion is set as an initial value. However, in an optical system having strong distortion such as a wide-angle lens or a fish-eye lens, convergence of a solution cannot be expected and application is difficult. Also, the larger the distortion, the longer it takes to perform the iterative calculation.

Further, detection of points and line segments requires an appropriate binarization process, and there is a problem that sufficient accuracy cannot be obtained when an input image is accompanied by noise. In addition, since these detections include a quantization error, there is a problem that this also affects the calculation result.

[0006] Strict precision is also required for setting the test pattern to be tested, and this precision directly affects the results. Therefore, the simplicity of the measurement is poor if high accuracy is to be obtained.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging system parameter which solves problems such as easiness of installation of a test object, high-speed processing, and propagation of a measurement error or the like to a processing result, which are problems in the conventional method. An object of the present invention is to provide a measuring method and an apparatus therefor.

[0008]

In order to achieve the above object, an image pickup system parameter measuring method according to the first aspect of the present invention is a method of measuring an image display means having a planar and high-precision pixel array by a visual sensor to be measured. A first stage in which the image is input by photographing with the visual sensor installed in front, and a drawing figure on the image display means observed by the visual sensor is changed by an angle or a translation; And the change
And a second step of determining the parameters resulting from the imaging system including the position and orientation and lenses and the image sensor in a space of the visual sensor from the information of the drawing shapes on the image display device with the above stage Repeat one or more times.

Further, the inventions of claim 2, an image display means placed in front of the visual sensor to be measured has a flat and accurate pixel array, the graphics on a display screen of said image display means position and orientation and the lens and the image sensor in a space of a visual sensor drawing figures obtained by the change from the input graphic image drawing figures and the visual sensor and input from the visual sensor with drawing by changing an angle or parallel movement And a calculation processing means for determining a parameter caused by the imaging system.

Further, the inventions of claim 3, The method of claim 1, the second stage, the input figure in the first stage
First calculating the imaging system parameters from the information in the drawing figures and
And the error between the imaging system parameter obtained in the first processing procedure and the true value is detected, and if there is an error, the condition is changed.
A second processing procedure returns to the first step, a third procedure that outputs an imaging system parameters at the time the error is gone in the second procedure, in configuring.

Further, the inventions of claim 4, in the invention of the claim 3, in a first stage, drawing on the image display means
Draw the figure at the angle of a set of parallel lines or at least two different
Varied translation straight line from direction, the second step first
In the processing procedure of at least, at least
From the vanishing point of the input figure of the three parallel lines with different angles,
Or from at least two different intersections of the moved <br/> linear translation in the direction of the input figure, the imaging system images a center parameter
To be determined as data.

Further, the inventions of claim 5, in the invention of the third aspect, in the first processing procedure, from the image center determined as the imaging system parameters equidistant straight line perpendicular on the imaging surface draw processing point to the image display means, the imaging system the position and orientation of the vision sensor from the distance of the corresponding point of the point from the 2 angle of the straight line of intersection and intersection points on the image display means
And Rukoto be calculated as a parameter.

Furthermore, the inventions of claim 6, in each measurement method described above, the process of detecting a line segment or point or shape is drawn figures to the image display unit as input figure of the visual sensor, the image display means It is assumed that the image to be drawn is displayed in a reversed manner or the brightness is changed to perform the difference processing of a plurality of images.

[0014]

According to the method and the apparatus for measuring the parameters of the imaging system of the present invention, the installation of the image display means to be tested need only be arranged in front of the visual sensor to be measured. To realize the easiness of installation of the test object.

Further, by obtaining the imaging system parameters by pure geometric calculation, the convergence of the solution, the initial value problem, and the like, which have been problems in the past, can be eliminated, and the solution can be obtained at a stroke. Achieve high speed.

Further, by displaying figures, line segments, and points of the test object or the like by the image display means, they can be inverted and displayed.
Imaging that enables brightness change and detects figures etc. only by differential processing with multiple images, improves the detection accuracy of figures etc., reduces errors, and reduces the influence of figure detection errors on processing results Enables measurement of system parameters.

[0017]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a view for explaining an embodiment of the present invention. 1 is an image display device to be tested, 2 is a visual sensor to be measured, 3 is a visual sensor 2 and an image display device 1 A processing unit that receives input from and performs processing
Reference numeral 4 denotes a monitor for displaying the input of the visual sensor 2. The image display device 1 has a flat and high-precision image display element array such as a liquid crystal, for example, and is installed while looking at the monitor 4 so as to maintain an appropriate inclination with respect to the visual sensor 2. I do.

FIG. 2 is a diagram showing an outline of the processing procedure of this embodiment. <1> is a process of drawing a graphic on the image display device 1 and detecting the graphic by the visual sensor 2, <2> a process of calculating and acquiring an imaging system parameter from graphic information in the graphic detection process <1>, <3> is an imaging system parameter acquisition process <
A process for detecting an error between the imaging system parameter obtained in 2> and the true value, and a processing result output process for outputting the imaging system parameter when the error in the error detection process <3> disappears. .

FIG. 3 is a configuration diagram of a processing procedure showing a specific example of the graphic detection processing <1> in FIG. Reference numeral 5 denotes lens distortion detection processing, 6 denotes normal lens image center detection processing, 7 denotes distortion aberration center detection processing, 8 denotes drawing processing, 9 denotes figure detection processing, and 10 denotes figure determination processing. Hereinafter, each process will be described.

FIG. 5 is a diagram for explaining the lens distortion detection processing 5, and 13 and 14 are input images from a visual sensor. In the lens distortion detection processing 5, a straight line group is drawn on the image display device 1, and this is observed by the visual sensor 2 in FIG. At this time, when all the straight lines are observed as perfect straight lines as in the input image 13 shown in FIG. 5A, the normal image lens center detection processing 6 is performed and the input image 1 shown in FIG.
If it is observed as a curve as in 4, the process proceeds to distortion aberration center detection processing 7.

FIG. 6 is a diagram for explaining the normal image lens center detection processing 6. In FIG. 6A, reference numeral 16 denotes a group of parallel lines drawn on the image display device, and 15 denotes a group of the parallel lines 16. It is a vanishing point. In the normal image lens center detection processing 6, a parallel line group 16 intersecting at an arbitrary angle θ is drawn, and a vanishing point 15 of each parallel line is calculated. The same processing is performed three times by changing the angle of the image display device 1 with respect to the visual sensor 2.

As shown in FIG. 6B, three sets of vanishing points obtained by these three processes are respectively denoted by 15a and 15a.
b, 15c, and the intersection angle on the image display device 1 is θa,
θb and θc. Here, each intersection angle of θa to θc is 90
In terms of degrees, the viewpoint position of the CCD camera with respect to a set of vanishing points on the image plane P can be limited to the surface of a sphere whose diameter is a straight line connecting the vanishing points. The three spheres can be set independently, and one point O where all the spheres intersect is the viewpoint position.
Therefore, the point O 'where the perpendicular line lowered from the viewpoint position O on the image plane P intersects is the center of the image. The length f of OO 'is the focal length.

FIG. 7 is a diagram for explaining the distortion aberration center detecting process 7, wherein 17 and 18 are straight lines observed by the visual sensor, and 19 is the intersection of these straight lines. In the distortion center detection process 7, as shown in FIG.
The drawing is repeated while moving 7 in parallel, and the position observed as a complete straight line 18 is detected. As shown in FIG. 7B, this processing is performed at least twice from different directions, and the intersection 19 is obtained. This intersection 19 is the center of the distortion, that is, the center of the image.

FIG. 8 is a diagram for explaining a target drawing figure in the drawing processing 8, the graphic detection processing 9, and the graphic determination processing 10. In this series of processing, corresponding points 21 to 24 projected at equal distances n in the horizontal and vertical directions from the image center 19 on the monitor 4 are drawn on the image display device 1.

FIG. 9 is a diagram for explaining a graphic drawing technique in the drawing processing 8, the graphic detection processing 9, and the graphic determination processing 10. Taking the point 21 as an example, when an arbitrary area 26 of the image display device 1 shown in FIG. 9A is blinked, if the change is observed at the point 21 on the monitor 4 in FIG.
Is a candidate area, otherwise, a part other than the area 26 is a candidate. Further, as shown in FIG. 9B, the area on the image display device 1 which has become a candidate area is divided into two areas, and only one area 27 is similarly turned on and off, thereby limiting the candidate area. The corresponding point 21 is determined by repeating the above process a plurality of times. The same processing is performed at points 22 to
24.

FIG. 10 is a diagram for explaining a drawing result on the image display device 1, where P is a display portion of the image display device 1. As a result of the above processing, the image center 19 and the corresponding points 21 to 24 on the monitor 4 are drawn on the display portion P of the image display device in FIG. Here, FIG.
As shown in the figure, the distance between the points 21 and 19 on the image display device 1 is a, the distance between the points 22 and 19 on the image display device 1 is b, and the points 23 and 19 are on the image display device 1. Is the distance at c, the distance between the points 24 and 19 on the image display device 1 is d,
The angle at which the straight line connecting the points 21 and 23 and the line connecting the points 22 and 24 intersects is θ.

FIG. 4 is a processing procedure block diagram showing a specific example of the imaging system parameter acquisition processing <2> in FIG. The imaging system parameter acquisition process <2> includes a viewpoint position calculation process 11,
It comprises an imaging system parameter deriving process 12. Hereinafter, each of these processes will be described.

FIG. 11 is a diagram for explaining the principle of viewpoint position measurement. In the visual sensor 2, points 21 to 24 observed at equal intervals from the image center 19 are points on the conical surface of the cone 28 having the vertex at the position 29 in the three-dimensional space of the visual sensor 2. Is the axial direction of the cone. Point 21
The straight line connecting the point 23 and the point 22 and the point 22 and the point 24 is projected as a straight line orthogonal to the bottom of the cone. Here, when there is a difference in the aspect ratio with respect to the output from the light receiving element of the visual sensor, the output is not a cone but an elliptical cone.

FIG. 12 is a diagram for explaining the visual sensor viewpoint position estimation based on the line segment connecting the points 21 and 23. A cone or elliptical cone with the vertex at point 29 is shown in FIG.
As shown in (a), when cut along a plane including a straight line connecting the points 29 and 19, which are the central axes, the cross section becomes a triangle, and the central axis is a two-pointed line of an angle. When the existence range of the viewpoint position 29 is obtained from this condition, as shown in FIG. 12B, when a> b, the point extending on the line 23 connecting the points 21 and 23 in the direction of the point 23 is obtained. Distance ab / (a-
sphere g of radius ab / (ab) centered on point 30 of b)
1 surface. Similarly, a sphere g2 can be set for a line segment connecting the points 21 and 23. Therefore, the viewpoint position is
As shown in FIG. 12C, the sphere is constrained on a circle S where the two spheres intersect.

However, when the image display device 1 is placed perpendicular to the optical axis, the radius of the sphere g becomes infinite and the viewpoint position cannot be calculated. In this case, a method of instructing the image display apparatus 1 to be inclined and placing it again to calculate the viewpoint position, or a method of calculating the camera parameters while maintaining the vertical position can be considered.

When the calculation is performed with the vertical position unchanged, a conventionally proposed method can be applied. For example, the following method can be applied.

Method when aberration distortion is large: Nom
ura, H .; Naruse, M .; Sagara and
A. Ide. "Simple caliblatio
nalgorithm for high-disto
rotation-lens camera "(Reference 1: IE
EE Trans. on Pattern Anal
isys and Machine Intellige
nce, Vol PAMI-14, No. 11, pp.
1095-1099, Nov. 1992. Method when aberration distortion is small: Weng. "Camera C
Alibration with Distortion
n Models and Accuracy Eva
luation "(Reference 2: IEEE Trans.
onPattern Analysys and Ma
chin Intelligence, Vol PAM
I-14, No. 10, pp. 965-980, Oc
t. 1992. Here, in the conventional method, there is no method other than manually determining the correspondence between the input image and the actual three-dimensional image. In this regard, in the present method, since drawing and image input are performed at the same time, there is no need to manually determine a corresponding point, and the problem of the conventional method is solved.

FIG. 13 is a diagram for explaining the visual sensor viewpoint position estimation on the condition that a straight line connecting the points 21 and 23 and the points 22 and 24 is observed at a right angle. The viewpoint position at which the two straight lines are observed to be orthogonal to each other is point 19
Is defined as cos (sin ^-1 (tan
(Θ / 2))) is the surface of an elliptical cone with the ellipse of 1 as the base. The surface of the elliptical cone and the circle intersect at two points, and the point 29 located in the surface direction of the inner image display device is the viewpoint position of the visual sensor. As described above, the imaging point position of the visual sensor 2 with respect to the image display device 1 can be calculated.

The process <3> is a process for verifying the error of the imaging point position of the visual sensor obtained in the process <2>. Specifically, another graphic is drawn on the image display device, and verification is performed based on a difference between the actual graphic shape calculated from the graphic and the graphic shape calculated only from the visual sensor information. Here, if an error is detected, the parameters such as the center of the image are changed and the process returns to the process <1> in FIG. 2. If there is no error, each point on the image sensor is transformed into a polar coordinate with the optical axis at the zenith. The table is calculated, and the process proceeds to <4>. The calculation of the polar coordinate conversion table can be performed by the same operation as in processes 8 to 10. This is because, when an arbitrary position on a display surface such as a liquid crystal of an image display device is blinked as shown in FIG. The principle is that the latitude and longitude are uniquely determined.

In the process <4>, the imaging system parameters are calculated using the result obtained in the process <3>. The imaging system parameters referred to here are optical parameters such as a focal length, a lens aberration, an image center, and external parameters such as a position and a posture of a camera in a real space. Since the center of the image has been derived, the focal length, lens distortion, etc. may be obtained. This is performed by obtaining each coefficient of the optical model such that the derived conversion system table fits the optical model as shown in the above-mentioned document 2.

As for the external parameters, since the relationship between the display surface of the image display device and the camera position is determined, the direction, distance, and orientation of the display surface from a reference position in the real space can be accurately measured. Can be derived.

In FIG. 14, the image is obtained by calculating the tangent of each bisecting angle of the vertex angle θ1 of the triangle connecting the points 30, 21, and 23 and the vertex angle θ2 of the triangle connecting the points 30, 22, and 24. Calculation of the aspect ratio of the element is possible.

Therefore, necessary imaging system parameters can be derived.

In each of the processes described above, when detecting a figure or the like displayed on the image display device, the process of detecting a line segment, a point, or a figure is performed by reversely displaying a figure to be drawn on the image display means or by changing the brightness. By performing the difference processing of a plurality of images obtained by this, it is possible to easily and reduce the detection error.

As described above, in the present embodiment, the image display device is installed at an appropriate inclination in front of the visual sensor, and the graphics and the like on the image display device are sequentially changed according to the input information from the visual sensor. Calculating the imaging system parameters from geometric information of both the observation figure etc. of the visual sensor and the display figure etc. on the image display device. When the figure etc. is detected, the inverted display or brightness change of the display figure etc. is performed multiple times. The main feature is that the difference image is detected.

With such a characteristic configuration, the present invention is different from the prior art in that it has high flexibility in installing the image display device to be tested and measures even when the imaging system has a strong aberration distortion or a different aspect ratio of the image. It is possible to realize high-speed and high-precision calculation, even under noisy conditions,
Different effects such as high-accuracy figure detection can be obtained.

It should be noted that if the processing of the above embodiment is repeated a plurality of times for the same image pickup system, the accuracy can be further improved. If the input is made to a plurality of visual sensors, the positional relationship between the visual sensors can be measured. Therefore, there is an effect of improving accuracy even in three-dimensional measurement using a visual sensor or in a navigation system using a visual sensor.

[0044]

As described above, according to the imaging system parameter measuring method and the apparatus according to the present invention represented by claims 1 and 2, it is possible to derive the imaging parameters only by pure geometric calculations. There is no convergence of the solution, no initial value problem, etc., and the solution can be obtained at a stroke, resulting in high processing speed. Further, the image display means for the test need only be arbitrarily arranged with respect to the visual sensor, so that the flexibility of the installation is very high and the test object can be easily installed.

Further, according to the present invention represented by the first and second aspects, in particular, according to the sixth aspect of the present invention, it is possible to reverse the display and change the brightness by using the image display means. Therefore, a figure or the like can be detected only by the difference processing using a plurality of images, the detection accuracy of the figure or the like can be improved, and the corresponding area for each light receiving element can be detected, so that the quantization error is small. Therefore, optical system distortion or quantization error,
It is possible to measure the imaging system parameters with less influence on the processing result due to the figure detection error.

According to the third aspect of the present invention, in particular, an error is detected and an imaging system parameter having no error is output, so that the accuracy can be further improved.

According to the fourth aspect of the invention, particularly, the center of the image, which is one of the imaging system parameters, can be easily obtained.

Further, according to the invention of claim 5, in particular,
Imaging system parameters relating to the position and orientation of the visual sensor can be easily obtained.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing one embodiment of the present invention.

FIG. 2 is a diagram showing an outline of a processing procedure of the embodiment.

FIG. 3 is a diagram showing a specific example of a graphic detection process in the processing procedure of FIG. 2;

FIG. 4 is a diagram showing a specific example of an imaging system parameter acquisition process in the procedure of FIG. 2;

FIGS. 5A and 5B are diagrams for explaining the lens distortion detection processing in the graphic detection processing of FIG. 3;

6A and 6B are diagrams for explaining a normal lens image center detection process in the graphic detection process of FIG. 3;

7A and 7B are diagrams for explaining a distortion center estimation process in the graphic detection process of FIG. 3;

8 is a diagram for explaining a target graphic in the drawing processing, the graphic detection processing, and the graphic determination processing in the graphic detection processing in FIG. 3;

FIGS. 9A and 9B are diagrams for explaining a graphic drawing technique in the graphic processing, graphic detection processing, and graphic determination processing in the graphic detection processing of FIG. 3;

10A and 10B are diagrams for explaining a drawing image by the graphic drawing method of FIG. 9;

11 is a view for explaining the principle of viewpoint position measurement in the imaging system parameter acquisition processing of FIG.

12A, 12B, and 12C are diagrams for explaining visual sensor viewpoint position estimation in the imaging system parameter acquisition processing of FIG. 4;

FIG. 13 is a view for explaining the visual sensor viewpoint position estimation based on specific conditions in the imaging system parameter acquisition processing of FIG. 4;

14 is a view for explaining derivation of an aspect ratio of an output image in the processing result output processing of FIG. 2;

15 is a view for explaining the principle of calculation of a polar coordinate conversion table in the error detection processing of FIG. 2;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Image display apparatus 2 ... Visual sensor 3 ... Calculation processing apparatus 4 ... Monitor 5 ... Lens distortion detection processing 6 ... Normal lens image center detection processing 7 ... Distortion aberration center detection processing 8 ... Drawing processing 9 ... Graphic detection processing 10 ... Graphic Judgment process 11: viewpoint position calculation process 12: imaging system parameter derivation process 13, 14, input image from viewpoint sensor 15: vanishing point 16: parallel line group 17, 18, straight line 19: intersection of straight lines (image center) 21 to 21 24: points observed equidistant from the center of the image 26, 27 ... blinking area 28 ... cone 29 ... viewpoint position of visual sensor

Continuation of front page (58) Fields investigated (Int.Cl. ⁷ , DB name) G01B 11/00-11/30 102 G06T 1/00 G06T 3/00 G06T 7/00

Claims (6)

(57) [Claims] 1. An image display means having a planar and high-precision pixel array is installed in front of a visual sensor to be measured, an image is taken and input by the visual sensor, and an image displayed by the visual sensor is displayed. Angle the drawn figure on the
Others a first step of changing a parallel movement, the position and orientation and lenses and the image sensor in a space of the visual sensor from the information of the drawing shapes on the image display device with the input graphic and said change of said visual sensor Determine the parameters caused by the imaging system including
And a second step, the imaging system parameter measurement method and repeating one or more times the above steps. 2. An image display means provided in front of a visual sensor to be measured having a planar and highly accurate pixel arrangement,
Angle or parallel figures on the display screen of the image display means
Drawing figure obtained by the change over to draw varied Or type from said visual sensor image drawing figures and the visual cell
An image processing system parameter measuring device, comprising: a calculation processing means for determining a position and orientation in a space of a visual sensor and a parameter caused by an imaging system including a lens and an imaging device from an input graphic of the sensor. 3. The method according to claim 1, wherein the second step is an input in the first step .
The change in the first procedure of calculating the imaging system parameters from the graphic information of the drawing figures, the detected condition if there is an error the error between the obtained imaging system parameters and true value in the first processing procedure a second processing procedure returns to one stage, according to claim 1, characterized in that it consists of a third procedure for outputting the imaging system parameters at the time the error is gone in the second procedure Imaging system parameter measurement method. In the first stage, drawing on an image display means is performed.
Draw the figure at the angle of a set of parallel lines or at least two different
Varied translation straight line from direction, the second step first
In the processing procedure of at least, at least
From the vanishing point of the input figure of the three parallel lines with different angles,
Or from at least two different intersections of the moved <br/> linear translation in the direction of the input figure, the imaging system images a center parameter
Imaging system parameter measuring method according to claim 3, wherein the benzalkonium determined as data. 5. In a first processing procedure , a point at an equidistant point on a straight line orthogonal to an image pickup plane from an image center obtained as an image pickup system parameter is drawn on an image display means, and the point on the image display means is drawn. imaging system the position and orientation of the distance of the corresponding point of the point from the angle and intersection point of the two lines of intersection the visual sensor
Claim 3, wherein the benzalkonium be calculated as a parameter or
Or the method of measuring imaging system parameters according to 4 . 6. A process for detecting a line segment , a point, or a graphic which is a graphic to be drawn on an image display means as an input graphic of a visual sensor, by inverting a graphic to be displayed on the image display means or changing brightness. 6. The method according to claim 3, wherein the difference processing is performed on a plurality of images.
3. A method for measuring parameters of an imaging system according to item 1.