JP2021167776A - Calibration device for three-dimensional shape measurement - Google Patents

Abstract

To provide a calibration device for three-dimensional shape measurement that can simplify the three-dimensional measurement of a measurement object, such as a large forged product, by eliminating the need to install a large calibration board.SOLUTION: A calibration device for three-dimensional shape measurement includes a plurality of stereo cameras 1A and 1B oriented toward an object S1 whose three-dimensional shape is to be measured, a laser light-emitter 2, which irradiates the object S1 with a line laser beam LL and enables the generation of a captured image in which a plurality of line laser beams intersects each other, and a control device 3. The control device detects intersection points P in the captured image obtained by each stereo camera by capturing the line laser images generated on the object S1 by the irradiation of the line laser light LL from each stereo camera 1A and 1B, calculates three-dimensional coordinates of each detected intersection point P, calculates a transformation parameter for each stereo camera 1A and 1B that best matches the position of the intersection point for which three-dimensional coordinates have been calculated for each stereo camera 1A and 1B, and transforms a coordinate system of each stereo camera 1A and 1B into a reference coordinate system common to these stereo cameras 1A and 1B by each transformation parameter.SELECTED DRAWING: Figure 1

Description

The present invention relates to a calibrator for measuring a three-dimensional shape, and more particularly to a calibrator that can be suitably used when measuring a three-dimensional shape using a plurality of stereo cameras.

To measure the three-dimensional shape of an object, a stereo camera equipped with a pair of cameras and measuring the three-dimensional shape based on the parallax of both cameras is used. For example, in Patent Document 1, prior to shape measurement, in order to prevent an error in shape measurement from occurring due to fluctuations in the center of rotation of the rotary table surface on which the object to be measured is placed or fluctuations in the inclination of the rotary table surface. For example, a method is shown in which a calibration board having a constant pattern in which a large number of circular spots are arranged at regular intervals in the vertical and horizontal directions is placed on a rotary table surface to enable calibration of shape measurement values.

JP 2017-3537

By the way, when measuring the three-dimensional shape of a large forged product, it is necessary to use multiple (usually two) stereo cameras because a single stereo camera cannot secure a sufficient field of view. There is a problem that the positional relationship of the stereo cameras fluctuates due to deformation of the stereo camera mount due to radiant heat, which causes a shift in the coordinate system that is matched in advance between the stereo cameras, resulting in an error in shape measurement. rice field. Therefore, conventionally, a calibration board having a size that can cover the imaging range of the forged product is used to eliminate the measurement error caused by the deviation of the relative positions between the stereo cameras. However, there is a problem that the work of installing a large calibration board each time it is calibrated is burdensome.

Therefore, the present invention solves such a problem, and provides a calibration device for three-dimensional shape measurement that can simplify three-dimensional measurement of a measurement object such as a large forged product without the need to install a large calibration board. The purpose is to provide.

In order to achieve the above object, in the first invention, a plurality of stereo cameras (1A, 1B) arranged in the direction of the object (S1) for measuring the three-dimensional shape, and the object (S1). ) Or the line light (LL), and the line light (LL) means of irradiating the line light (LL) to generate an image in which a plurality of line lights intersect each other at a plurality of places. An intersection point (P) in which a line light image generated on the object (S1) or its mounting surface by irradiation is captured from the stereo cameras (1A, 1B) and the intersection (P) in the captured image obtained by each stereo camera is detected. Three-dimensional coordinates for the detection means (3, step 102), the intersection coordinate calculation means (3, step 107) for calculating the three-dimensional coordinates of each detected intersection (P), and each of the stereo cameras (1A, 1B). For the intersection group in which It is provided with a coordinate conversion means (3, step 111) for converting the coordinate system of the above into a reference coordinate system common to these stereo cameras (1A, 1B).

According to the first invention, it is possible to generate an image in which a plurality of line lights intersect each other at a plurality of places by irradiating the object or its mounting surface with the line light by the line light irradiation means. By detecting the intersection of the images, it is possible to obtain an captured image similar to the case where the conventional calibration board is imaged. This eliminates the need to install a large calibration board as in the past, so that it is possible to simplify the three-dimensional measurement of an object such as a large forged product. In particular, since it is line light, it is possible to obtain an image similar to the case where the calibration board is imaged by directly irradiating the surface of the object, and the calibration board removes the object from the table and the temperature is sufficient. The calibration work can be made much more efficient than the case where the installation can be performed only after the temperature is lowered.

In the second invention, the reference coordinate system is any one of the plurality of stereo cameras (1A, 1B).

According to the second invention, since the reference coordinate system is set to the coordinate system of one stereo camera, the conversion is performed as compared with the case where a reference coordinate system different from the coordinate system of each stereo camera is set on the object side. The calculation of parameters becomes easy.

In the third invention, the intersection detecting means (3, step 102) detects the position of the center of gravity of the brightness of each intersecting line light in each intersecting region of the captured image, and detects the position of the center of gravity of the brightness, and a plurality of centers of gravity. The approximate line is calculated by the method of least squares with respect to the position, and the point where the approximate lines intersect is set as the intersection (P).

According to the third invention, the intersection can be obtained with high accuracy.

In the fourth invention, the least squares method is a weighted least squares method in which the weight decreases as the absolute deviation value increases.

According to the fourth invention, the influence of outliers on the calculation of the approximate line can be suppressed to a small extent.

In the fifth invention, the coordinate conversion means (3, step 111) detects the position of the center of inertia of the intersection group of the captured images obtained by the stereo cameras (1A, 1B) and each intersection group around the center of gravity. The singular value decomposition of the moment of inertia is performed to calculate the provisional conversion parameters for each stereo camera (1A, 1B), and each stereo camera (1A) is subjected to nonlinear optimization processing with the provisional conversion parameters and the position of the center of gravity as initial values. Each of the provisional conversion parameters that minimizes the average of the errors between the intersections of 1B) is used as each of the conversion parameters.

According to the fifth invention, the conversion parameters can be obtained with high accuracy.

In the sixth invention, when the object is a red-hot material having an emission color of 590 to 1100 nm (orange to red), the wavelength range of the line light (LL) is set to 400 to 590 nm (purple to yellow). This distinguishes between the red-hot material and the line light (LL).

The reference numerals in parentheses indicate the correspondence with the specific means described in the embodiments described later for reference.

As described above, according to the calibration device for three-dimensional shape measurement of the present invention, it is possible to easily perform three-dimensional measurement of an object such as a large forged product without the trouble of installing a large calibration board. ..

It is a perspective view which shows the whole structure of the calibration apparatus for three-dimensional shape measurement. It is an image of intersecting straight lines extracted from the captured image. It is an image extracted from the pixel data around the intersection of the captured images. It is an image explaining the detection of the position of the center of gravity of the brightness of pixel data. This is an image in which a straight line is rediscovered by the weighted least squares method. It is a graph which shows the relationship between the deviation and the weight in the weighted least squares method. It is an image in which a straight line passing through a region having the highest brightness is detected. It is a processing flowchart in a control device.

The embodiments described below are merely examples, and various design improvements made by those skilled in the art within the scope of the present invention are also included in the scope of the present invention.
FIG. 1 shows the overall configuration of the calibration device for three-dimensional shape measurement. In FIG. 1, a stereo camera 1A is supported at an obliquely upper position on both sides of a circular mount S on which an object S1 to be measured is placed, such as a large columnar forged product, by a mount shown in the drawing. , 1B is installed. Each of the stereo cameras 1A and 1B includes a pair of left and right CCD cameras 11 and 12, and a laser light emitter 13 for irradiating a line laser beam for shape measurement is provided between the cameras 11 and 12.

When measuring the shape of the object S1, the left and right stereo cameras 1A and 1B are swung so that the line laser beams emitted from them cover the entire surface of the object S1 and are scanned. The three-dimensional shape of the object S1 seen from each of the stereo cameras 1A and 1B is measured by a known method from the disparity of the images obtained by the cameras 11 and 12. Both stereo cameras 1A and 1B are set so that their fields of view partially overlap each other.

Below the stereo camera 1A, a laser light emitter 2 that irradiates a line laser beam in order to obtain an intersection for calibration described below is provided. The laser light emitter 2 is supported by a frame (not shown) so as to generate a lattice pattern consisting of a group of straight lines extending at regular intervals in a direction orthogonal to each other on a flat surface of the object S1 with line laser light. Is scanned into. Here, when the object S1 is an object material whose surface becomes red-hot due to heat treatment in rolling, forging, casting, etc. (hereinafter referred to as red-hot material), the sensitivity wavelength of the camera is 400 nm to 1100 nm, and the wavelength of the line laser light. When the range is set to 590 nm to 1100 nm (orange to red), the linear image cannot be identified because it overlaps with the emission color of the red heat material, which is 590 nm to 1100 nm (orange to red). Therefore, by using line laser light in the wavelength range of 400 nm to 590 nm (purple to yellow) that can be distinguished from the emission color of the red hot material, the three-dimensional shape of the red hot material can be accurately determined even in a high-temperature heating environment such as heat treatment. It becomes possible to measure.

A control device 3 equipped with a CPU and a memory is provided to control scanning and light emission of the laser emitter 2 and imaging by the stereo cameras 1A and 1B, and lasers are used at regular intervals on the surface of the object S1. By capturing the line light images captured by the stereo cameras 1A and 1B while scanning the light emitter 2 and synthesizing them, parallel straight lines at regular intervals as shown in FIG. 1 intersect at right angles to form a lattice pattern. Obtain the captured image. In Fig. 1, unlike the actual situation, each intersection of the grid pattern is indicated by a large circle.

In the control device 3, in order to obtain the coordinates of the intersections of the straight lines forming the checkered pattern and acquire the conversion parameters required for the coordinate conversion based on the coordinates, the following processing as shown in each step of the flowchart of FIG. 8 is performed. conduct. That is, the control device 3 extracts straight lines L1 and L2 from the obtained captured image by Hough transform or the like (step 101, FIG. 2), detects the intersection P of each straight line L1 and L2 (step 102), and detects the intersection point. Peripheral pixel data is extracted (step 103, FIG. 3). Subsequently, the position of the center of gravity of the brightness of a predetermined number of pixel data in the normal direction of the straight lines L1 and L2 is detected at regular intervals along the straight lines L1 and L2 (step 104, FIG. 4A, (B)). Subsequently, the straight lines L3 and L4 are rediscovered by the weighted least squares method for each of the detected positions of the center of gravity (step 105, FIG. 5). As shown in FIG. 6, the weighting in this case decreases as the absolute value (distance) of deviation from the straight lines L3 and L4 increases. Steps 104 and 105 are repeated appropriately times to rediscover the straight line, obtain straight lines L5 and L6 passing through the region having the highest brightness, and calculate the two-dimensional coordinates of the intersection Pf of the straight lines L5 and L6 (step 106, FIG. 7).

As described above, the two-dimensional coordinates of each intersection Pf of the lattice pattern are calculated, and the three-dimensional coordinates of each intersection Pf are calculated by a known method from the parallax of the left and right cameras 11 and 12 of the stereo cameras 1A and 1B. (Step 107). Subsequently, the position of the center of gravity of each of the intersections obtained by the stereo cameras 1A and 1B is calculated (step 108). Then, the center of gravity of the intersection group image of one stereo camera 1A is aligned with the center of gravity of the intersection group image of the other stereo camera 1B by parallel movement, and the inertial principal axis of each of the intersection group images is obtained by singular value decomposition of the moment of inertia. Then, the moment of inertia principal axis of the intersection group image of one stereo camera 1A is aligned with the moment of inertia principal axis of the intersection group image of the other stereo camera 1B (step 109). Let Ht be the provisional conversion parameter that matches the inertial spindles at this time.

Subsequently, nonlinear optimization (sequential quadratic programming) processing is performed with the center of gravity and the provisional conversion parameter Ht as initial values, and the conversion parameter Hm that minimizes the average value of the errors between the corresponding points of both intersection group images is calculated. (Step 110). Further, the standard deviation value of the error between the corresponding points is obtained, and all the corresponding points are weighted according to the standard deviation value to eliminate the influence of the outliers, and the above conversion parameter Hm is used as the initial value for nonlinear optimum again. Perform the conversion process. Then, this is repeated to obtain the conversion parameter H that minimizes the error between the corresponding points (step 111).

In the control device 3, by using the conversion parameter H obtained in this way, the coordinate system of the intersection group image of one stereo camera 1A and the intersection group image of the other stereo camera 1B can be accurately calibrated and matched. Can be done. As a result, each image in which the three-dimensional shape of the measurement object is partially imaged so as to complement each other with the stereo cameras 1A and 1B is synthesized in the same coordinate system using the conversion parameter H, so that a large measurement object can be measured. It is possible to obtain an accurate three-dimensional overall shape of an object.

(Other embodiments)
In the above embodiment, the coordinate system of one stereo camera is made to match the coordinate system (reference coordinate system) of the other stereo camera, but a coordinate system different from the coordinate systems of both stereo cameras may be used as the reference coordinate system. good.
In the above embodiment, the case where the present invention is applied to calibration when measuring the three-dimensional shape of an object from images captured by a plurality of stereo cameras has been described, but from images captured by a pair of stereo cameras. The present invention can also be applied to calibration when measuring the three-dimensional shape of an object, and the configuration of the present invention at that time is as follows.

A single stereo camera having a pair of cameras arranged toward an object for measuring a three-dimensional shape, and a line that irradiates a plurality of lines that intersect each other at a plurality of locations on the object or its mounting surface. The light irradiation means, the intersection detection means for detecting the intersection of the line light images obtained by each camera by capturing the line light image generated on the object or its mounting surface by the irradiation of the line light from the cameras, and the detection. For the intersection coordinate calculation means for calculating the three-dimensional coordinates of each of the intersections and the intersection group for which the three-dimensional coordinates have been calculated for each camera, the conversion parameters for each camera in which the positions of the intersections best match each other. A calibrator for three-dimensional shape measurement, comprising a coordinate conversion means for calculating and converting the coordinate system of each camera into a reference coordinate system common to these stereo cameras according to each conversion parameter. The reference coordinate system in this case can be the coordinate system of either camera.

In the above embodiment, a single laser light source that emits line laser light is scanned and image processing in the control device is performed to obtain an image of a lattice pattern in which parallel straight lines intersecting each other are arranged. A light source that generates patterns at the same time may be used.
Further, in the above embodiment, the line laser beam is directly irradiated on the surface of the object, but of course, the surface of the platform on which the object is placed may be irradiated.

1A, 1B ... Stereo camera, 11, 12 ... Camera, 13 ... Laser light emitter, 2 ... Laser light emitter (laser light irradiation means), 3 ... Control device, LL ... Line laser light, P ... Intersection, S ... Stand , S1 ... Object.

Claims (6)

Imaging in which a plurality of stereo cameras arranged in the direction of an object for measuring a three-dimensional shape and a plurality of line lights intersect each other at a plurality of places by irradiating the object or its mounting surface with line light. A line light irradiating means capable of generating an image, and a line light image generated on the object or its mounting surface by irradiating the line light are captured from the stereo cameras and captured in the captured image obtained by the stereo cameras. With respect to the intersection detection means for detecting the intersection, the intersection coordinate calculation means for calculating the three-dimensional coordinates of each detected intersection, and the intersection group for which the three-dimensional coordinates have been calculated for each of the stereo cameras, the positions of these intersection groups are mutually exclusive. A three-dimensional shape provided with a coordinate conversion means for calculating a conversion parameter for each stereo camera that best matches and converting the coordinate system of each stereo camera into a reference coordinate system common to these stereo cameras by each conversion parameter. Calibration device for measurement. The calibrator for three-dimensional shape measurement according to claim 1, wherein the reference coordinate system is any one of the plurality of stereo cameras. The intersection detecting means detects the position of the center of gravity of the brightness at each fixed section of each intersecting line light in each intersecting region of the line light image, and approximates a plurality of positions of the center of gravity by the least squares method. The calibrator for three-dimensional shape measurement according to claim 1 or 2, wherein the point at which the approximate lines intersect is the intersection. The calibration device for three-dimensional shape measurement according to claim 3, wherein the least squares method is a weighted least squares method in which the weight decreases as the absolute deviation value increases. The coordinate conversion means detects the position of the center of gravity of the intersection group of the captured images obtained by the stereo cameras, decomposes the moment of inertia of each intersection group around the center of gravity by a singular value, and performs provisional conversion parameters for each stereo camera. Is calculated, and each of the provisional conversion parameters is converted so as to minimize the average of the errors between the intersections of the stereo cameras by the nonlinear optimization process with the provisional conversion parameters and the position of the center of gravity as initial values. The calibrator for three-dimensional shape measurement according to any one of claims 1 to 4, which is a parameter. When the object is a red-hot material having an emission color of 590 to 1100 nm (orange to red), the wavelength range of the line light is set to 400 to 590 nm (purple to yellow), so that the red-hot material and spot light The calibrator for three-dimensional shape measurement according to any one of claims 1 to 5, which is designed to distinguish between the two.

Images