JP2022152480A - Three-dimensional measuring device, three-dimensional measuring method, program, system, and method for manufacturing article - Google Patents

Abstract

To provide a three-dimensional measuring device that can accurately calculate three-dimensional information on an object even within a relatively large measuring range.SOLUTION: A three-dimensional measuring device has a plurality of imaging units whose magnifications or focus positions can be changed, and the magnification or the focus position when picking up an image of an object is selected from a plurality of predetermined positions. The three-dimensional measuring device has: a selection unit that selects, from a plurality of first parameters related to the plurality of magnifications of the plurality of imaging units or a relative position attitude between the plurality of imaging units at the plurality of focus positions, a first parameter corresponding to a predetermined magnification or a predetermined focus position as a parameter for calculation; and an operation unit that, on the basis of an image of the object and the parameter for calculation, calculates three-dimensional information on the object the image of which is picked up at the predetermined magnification or at the predetermined focus position.SELECTED DRAWING: Figure 7

Description

The present invention relates to a three-dimensional measuring device, a three-dimensional measuring method, a program, a system, and an article manufacturing method.

Conventionally, a three-dimensional measuring device such as a three-dimensional distance measuring device that generates three-dimensional information by performing three-dimensional reconstruction based on a plurality of images with parallax captured by a plurality of cameras (imaging units) has been known. It is In order to achieve highly accurate distance measurement (measurement) in a three-dimensional measurement apparatus, accurate information on internal parameters and external parameters is required for each of a plurality of cameras. Note that the intrinsic parameters are information such as focal length, principal point position, and distortion parameters, and the extrinsic parameters are information about relative positions and orientations of a plurality of cameras.

In order to deal with this, Non-Patent Document 1 discloses measuring the calibration pattern flat plate while changing the distance from the three-dimensional distance measuring device. This document describes that the internal parameters and external parameters of the imaging unit are measured, stored as calibration parameters (internal parameters and external parameters), and the captured image is calibrated using the calibration parameters.

Further, in general, an optical system having a zoom (magnification) and focus adjustment mechanism is used in order to expand the measurement range of the three-dimensional distance measuring device. In such a three-dimensional distance measuring device, as a response to the operation of the zoom and focus adjustment mechanism, in Patent Document 1, an internal parameter interpolated from a state closest to the combination of zoom and focus or a plurality of neighboring states Use parameters. Then, three-dimensional reconstruction is performed using the internal parameters. This enables highly accurate distance measurement.

Japanese Patent No. 4858263 ``A flexible new technique for camera calibration'' Z. Zhang, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, 2000.

However, when the zoom and focus adjustment mechanisms are operated, the lens is driven, so there is a possibility of lens eccentricity and the like, and external parameters that are usually considered to be unchanged also change subtly.

Therefore, if the external parameter is made common regardless of the zoom and focus positions, the accuracy of distance measurement will be degraded. Further, even a slight difference in the zoom and focus conditions when measuring the distance and when obtaining the calibration parameters also causes a decrease in accuracy of distance measurement. As a countermeasure, a method of estimating the calibration parameters by interpolation has been proposed, but the calibration parameters may become non-linear with respect to the zoom and focus conditions, and it is difficult to avoid a decrease in distance measurement accuracy.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a three-dimensional measurement apparatus capable of calculating three-dimensional information of an object with high precision even in a relatively large measurement range.

In order to achieve the above object, a three-dimensional measurement apparatus as one aspect of the present invention is a three-dimensional measurement apparatus having a plurality of imaging units capable of changing magnification or focus position, wherein the magnification or The focus position is selected from a plurality of predetermined positions, and a predetermined position is obtained from a plurality of first parameters relating to the relative position and orientation of the plurality of imaging units at the plurality of magnifications of the plurality of imaging units or the plurality of focus positions. a selection unit that selects a first parameter corresponding to a magnification or a predetermined focus position as a calculation parameter; and a calculation unit for calculating three-dimensional information.

According to the present invention, for example, it is possible to provide a three-dimensional measurement apparatus capable of calculating three-dimensional information of an object with high accuracy even within a relatively large measurement range.

1 is a schematic diagram of a three-dimensional measuring device and a calibration device of Example 1; FIG. 4 is a conceptual diagram of a control calculation unit of the three-dimensional measuring device of Example 1. FIG. FIG. 4 is an image diagram of a calibration pattern of the calibration device of Example 1; 4 is a flowchart illustrating a calibration flow of Example 1; 4 is a diagram illustrating a storage state of calibration parameters in Example 1; FIG. 4A and 4B are diagrams illustrating the measurement time of the three-dimensional measurement apparatus of the first embodiment; FIG. 4 is a flowchart illustrating a measurement flow of Example 1; FIG. 10 is a schematic diagram of a three-dimensional measuring device and a calibration device of Example 2; 10 is a flowchart illustrating a calibration flow of Example 2; 1 is a diagram showing an example of a control system including a gripping device equipped with an information processing device; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention will be described below with reference to examples and accompanying drawings. In each figure, the same members or elements are denoted by the same reference numerals, and overlapping descriptions are omitted or simplified.

<Example 1>
First, the calibration of the three-dimensional measuring apparatus of Example 1 will be described below with reference to FIGS. 1, 2, and 3. FIG. FIG. 1 is a schematic diagram of a three-dimensional measuring device and a calibration device in Example 1. FIG. FIG. 2 is a conceptual diagram of the control calculation section of the three-dimensional measuring device. FIG. 3 is an image diagram of a calibration pattern of the calibration device. Note that the three-dimensional measuring device in this embodiment also includes a three-dimensional distance measuring device.

The three-dimensional measurement apparatus of Example 1 can be configured with a measurement head 10 (broken line portion) and a control operation unit (control operation section) 3 . The measurement head 10 further includes two optical units, an imaging unit 1 and an imaging unit 2, for imaging the object. Also, hereinafter, the direction in which the stage 101 (to be described later) moves is defined as the z-axis direction, the direction perpendicular to the z-axis direction is defined as the x-axis direction, and the direction perpendicular to the z-axis is defined as the y-axis direction.

The imaging unit 1 is configured such that the magnification or focus position can be changed, and can include a sensor element 6 having a light receiving surface and an imaging lens (imaging optical system) 4 that guides light from a subject to the light receiving surface. The imaging unit 2 is configured such that the magnification or focus position can be changed, and can include a sensor element 9 having a light receiving surface and an imaging lens (imaging optical system) 7 that guides light from the subject to the light receiving surface. The sensor element 6 and the sensor element 9 function as an imaging unit that receives light from a test object on its light receiving surface and obtains an image of the test object. Moreover, the sensor element 6 and the sensor element 9 can be composed of photoelectric conversion elements such as CMOS and CCD. Furthermore, the imaging lens 4 and the imaging lens 7 are lenses with variable focus, and the focus can be set to a position selected from among a plurality of positions according to the position of the object to be inspected.

The control calculation unit 3 can include a focus control unit 31 that controls the focus of the imaging unit 1 and the imaging unit 2 as illustrated in FIG. Furthermore, it can include a calibration parameter storage unit (storage unit) 32 that stores calibration parameters, which are parameters for calculating three-dimensional information, which will be described later. Further, a calibration parameter selection unit (selection unit) 33 that selects a calibration parameter from the storage unit 32 based on focus driving information of each imaging unit such as information of the focus control unit 31 can be included. Further, it can include a calculation unit 34 that calculates distance information from the images acquired by the imaging unit 1 and the imaging unit 2 and the calibration parameters selected by the selection unit 33 . The control calculation unit 3 incorporates a CPU as a computer, and controls operations of the three-dimensional measurement device and the calibration device by executing programs stored in a memory (not shown).

The calibration apparatus of Example 1 comprises a calibration pattern plate 100 having a calibration pattern with known two-dimensional grid coordinates on a plane for deriving calibration parameters, and a stage 101 on which the calibration pattern plate 100 is placed. can be Further, the stage 101 has a control unit and a driving unit (not shown), and is configured to be movable in the Z-axis direction by the operation of the driving unit. Note that the control of the drive unit may be performed by the control calculation unit 3 .

The calibration pattern is, for example, a dot pattern as shown in FIG. As the calibration pattern flat plate 100, a substrate on which a calibration pattern with high positional accuracy is applied by printing or the like can be used. Alternatively, a substrate having grid-like openings and uneven portions at the reference positions may be used.

Next, a calibration method according to the first embodiment will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a flow chart diagram illustrating the calibration flow in the first embodiment. FIG. 5 is a diagram illustrating a storage state of calibration parameters. 4 is performed by the control calculation unit 3 executing a program stored in a memory (not shown).

First, in step (process) S<b>101 , the user selects one of a plurality of specific focus positions for each of the imaging unit 1 and imaging unit 2 . After selection, the focus control section 31 controls the focus of the imaging unit 1 and the imaging unit 2 .

Here, the specific focus position will be described below. A specific focus position is a discrete position set within the measurement range (measurement area) of the three-dimensional measurement device. This setting is made in consideration of the decrease in measurement accuracy due to the influence of defocus. Specifically, within the measurement range, at adjacent specific focus positions, the ranges in which the measurement accuracy can achieve the target accuracy overlap, that is, the adjacent areas are set to partially overlap. With the above setting, it becomes possible to select a specific focus position so that the measurement accuracy always achieves the target accuracy according to the position of the object within the measurement range.

Next, in step S102, the controller of the calibration apparatus drives the stage 101 to move the calibration pattern flat plate 100 stepwise to a preset known distance position. Next, the control calculation section 3 controls the imaging unit 1 and the imaging unit 2 at each step position, and the imaging unit 1 and the imaging unit 2 respectively capture images of the calibration pattern plate 100 .

Next, in step S103, the control calculation unit 3 calculates internal parameters ( second parameter). Further, relative positions and orientations (relative position and orientation) of the imaging units 1 and 2 with respect to the calibration pattern plate 100 are derived. As a derivation method thereof, for example, Zhang's method disclosed in Non-Patent Document 1 can be used. Further, the internal parameters include focal length, principal point position, distortion parameter, etc., which are imaging performance of the imaging unit 1 and the imaging unit 2 .

Furthermore, based on the relative positions and orientations of the imaging units 1 and 2 with respect to the calibration pattern plate 100, the control calculation unit 3 calculates the relative position between the imaging units 1 and 2, which is an external parameter (first parameter). derived the position and orientation of the That is, the relative position and orientation between the imaging units are derived.

As described above, it is possible to derive the calibration parameters consisting of the derived intrinsic parameters and the extrinsic parameters. The calibration parameters may be either internal parameters or external parameters, but are preferably calibration parameters composed of internal parameters and external parameters as described above.

The calibration parameters composed of the internal parameters and the external parameters of the imaging units 1 and 2 derived by the above steps are saved (stored) in the storage unit 32 by the control calculation unit 3 .

When deriving the calibration parameters, if the use is limited to the range where the imaging units 1 and 2 are in focus, it may not be possible to obtain a sufficient number of data for estimating the internal parameters. In that case, the derivation of the intrinsic parameters of imaging unit 1 and imaging unit 2 may each be performed with an expanded range and the intrinsic parameters used to predict the extrinsic parameters. The expanded range indicates a range that can be used for estimating the internal parameters of the imaging unit 1 when the imaging unit 2 is out of focus but the imaging unit 1 is in focus.

Next, in step S104, the calculation unit 34 performs stereo image processing on the calibration parameters derived in step S103, and obtains three-dimensional information (three-dimensional distance information) corresponding to the distance moved by the calibration pattern plate 100 and the stage 101. Calculate Stereo image processing includes feature extraction processing such as edge detection in addition to preprocessing such as a low-pass filter for performing parallelization processing and noise removal. Furthermore, there is a stereo matching process for obtaining parallax information by searching for corresponding points between stereo images using a process such as normalized correlation block matching.

The computing unit 34 then compares the derived three-dimensional information with the known distance moved by the stage 101, and calculates a measurement error (ranging error) from the difference.

Next, in step S105, the control calculation unit 3 compares the measurement error calculated in step S104 with the preset target accuracy. As a result of the comparison, if the measurement error is smaller than the target accuracy, that is, if it does not exceed the target accuracy, the calibration parameters calculated in step S104 are adopted as calculation parameters for calculating three-dimensional information, and step S106. proceed to As a result of the comparison, if the measurement error is larger than the target accuracy, that is, if the target accuracy is exceeded, the process returns to step S103, and the calibration parameter calculation (calibration parameter estimation) is performed until the measurement error becomes smaller than the target accuracy. )repeat. A predetermined value or threshold is set in advance for the magnitude of the measurement error.

Next, in step S<b>106 , the control calculation unit 3 stores the calibration parameters adopted in step S<b>104 in the storage unit 32 . When saving, as illustrated in FIG. 5, each calibration parameter is saved in the storage unit 32 while being associated with the selected specific focus position, and the process proceeds to step S107.

Next, in step S107, the control calculation unit 3 determines whether or not the storage of the calibration parameters has been completed at all the specific focus positions. If the calibration parameters have not been stored for all the specific focus positions, the process returns to step S101 to reselect the specific focus positions. After that, the processing from steps S102 to S106 is performed, and this is repeated until the calibration parameters are stored at all the specific focus positions. If the calibration parameters have been stored for all specific focus positions, the process ends.

By repeating steps S101 to S106 as described above, the calibration parameters associated with all the specific focus positions can be stored in the storage unit 32. FIG. In other words, each piece of information stored in the storage unit 32 is information whose calibration parameter is uniquely determined by a combination of the specific focus position of the imaging unit 1 and the specific focus position of the imaging unit 2 . When selecting the calibration parameters, the selection unit 33 can select the calibration parameters based on the information about the specific focus position of the imaging unit 1 and the specific focus position of the imaging unit 2 .

The calibration parameters of Example 1 are derived by combining the internal parameters and the external parameters as described above. By deriving them together, it is possible to calibrate even slightly changing components in external parameters that are normally thought not to change due to the possibility of lens eccentricity, etc., due to lens driving accompanying changes in focus position.

Furthermore, since the intrinsic parameter and the extrinsic parameter are not independent components, there is a possibility that the intrinsic parameter and the extrinsic parameter obtained at a specific focus position cancel out each other's errors. If only the extrinsic parameters are kept constant in that state, errors in the extrinsic parameters may continue to remain. By using a set of intrinsic and extrinsic parameters as described above, the effect can be eliminated.

The above is the description of the calibration method in the first embodiment. Next, the measuring method of the three-dimensional measuring apparatus of Example 1 will be described below with reference to FIG. FIG. 6 is a diagram illustrating the time of measurement (during distance measurement) by the three-dimensional measurement device.

The three-dimensional measurement apparatus of Example 1 can be composed of the measurement head 10 and the control calculation unit 3 as described above. Further, as illustrated in FIG. 6, an object 21 to be measured (distance measurement target) is arranged within the measurement range (distance measurement range) of the three-dimensional measurement device. Furthermore, the three-dimensional measurement apparatus of Example 1 has a user input unit (not shown). The focus position can be set by the user inputting the focus position to the focus control section 31 of the control calculation section 3 from the user input section.

Next, referring to FIG. 7, the measurement method of Example 1 will be described below. FIG. 7 is a diagram illustrating the measurement flow of the first embodiment; 7 is performed by the control calculation unit 3 executing a program stored in a memory (not shown).

First, in step S201, the user selects a specific focus position. When selecting a specific focus position, the user selects one of a plurality of specific focus positions for each of the imaging unit 1 and the imaging unit 2 using a user input unit (not shown) based on the approximate position information of the test object 21. do. Although the specific focus position is directly selected here, it is also possible to adopt a method of automatically selecting the specific focus position by calculating the specific focus position with the control calculation unit 3 by inputting the approximate position information and the measurement range. In addition, by capturing an image of the range including the test object with at least one of the imaging units, displaying the captured image on the screen, and specifying the measurement object, an autofocus mechanism (not shown) or a rough distance measurement mechanism can be used. Measure the approximate position of the object. After the measurement, the specific focus position may be automatically selected from the result.

Next, in step S202, the focus control unit 31 of the control calculation unit 3 controls the focus of the imaging unit 1 and the imaging unit 2 so as to be the selected specific focus position, and the imaging unit 1 and the imaging unit 2 are controlled. drive. This drive is limited to the specific focus position stored in the calibration parameter storage unit. By making this limitation, it becomes possible to match the specific focus position at the time of measurement with the specific focus position at the time of calibration, and it becomes possible to select an accurate calibration parameter.

Next, in step S203, after driving to the specific focus position in step S202, the imaging unit 1 and the imaging unit 2 are controlled by the control arithmetic unit 3, and the subject 21 is imaged. Thereby, an image of the test object 21 is obtained (imaging step).

Next, in step S204, the control calculation unit 3 selects from the storage unit 32 the calibration parameter corresponding to the specific focus position selected in step S201 from among a plurality of calibration parameters (selection step).

Next, in step S205, using the calibration parameters selected in step S204, stereo image processing is performed by the calculation unit 34 to calculate three-dimensional information of the test object 21 (calculation step). Stereo image processing includes feature extraction processing such as edge detection in addition to preprocessing such as low-pass filtering for parallelization processing and noise removal. Furthermore, there is a stereo matching process for obtaining parallax information by searching for corresponding points between stereo images using a process such as normalized correlation block matching.

The above is the description of the measurement method in the first embodiment. At the time of calibration in Example 1, calibration parameters were calculated at a specific focus position and stored. Then, during measurement, driving was performed only to a specific focus position. However, not only focus but also zoom (magnification) can be considered in the same way. By calculating and storing calibration parameters at a specific zoom position and driving only to a specific zoom position during measurement, zoom drive can be reduced. It is possible to deal with the case of a necessary specimen. For devices with both focus and zoom, it is preferable to have calibration parameters for each focus and zoom combination.

In addition, in Example 1, the internal parameters of the imaging units 1 and 2 are different for all combinations of all focus positions of the imaging units 1 and 2 . However, if different parameters are used for all combinations, the storage capacity will increase and the number of man-hours required for calibration will increase. Therefore, some or all of the internal parameters of the imaging unit 1 and the imaging unit 2 may be made common.

For example, when the imaging unit 1 illustrated in FIG. 5 is at the specific focus position 1, there are a plurality of combinations with the imaging unit 2 at the specific focus position. In this combination, since the imaging unit 1 has a common focus position, the internal parameters related to the imaging unit 1 can be shared among the internal parameters (1, 1), the internal parameters (2, 1), . . . . Also in this case, when the specific focus position of the imaging unit 2 is changed, the corresponding external parameters are calculated.

That is, the internal parameters of the imaging unit 1 may be used in common, and the internal parameters and external parameters of the imaging unit 2 may be used according to the combination with the focus position of the imaging unit 2 . In other words, the focus positions of the imaging unit 1 and the imaging unit 2 may each have their own internal parameters independently, and the external parameters may be used based on the combination at each focus position.

Further, in Example 1, the imaging unit 1 and the imaging unit 2 are arranged so that their optical axes intersect outside the measurement head 10 . Therefore, normally, when the range in which the imaging units 1 and 2 are simultaneously focused is spatially divided, the number of divisions increases, and calibration parameters must be stored for the number of divisions, which is not efficient. I can not say. Therefore, the number of divisions can be minimized by making the focus planes of the imaging units 1 and 2 approximately parallel to each other. Specifically, the optical axes of the imaging unit 1 and the imaging unit 2 are made parallel. Alternatively, the sensor elements 6 and 9 of the image pickup unit 1 and the image pickup unit 2 are arranged with a predetermined amount of inclination with respect to a plane perpendicular to the respective optical axes, which is a so-called Scheimpflug configuration. can do. Either the imaging unit 1 or the imaging unit 2 may have the Scheimpflug configuration.

When the focus planes of the image pickup unit 1 and the image pickup unit 2 are substantially parallel, once the focus position of the image pickup unit 1 is determined, the focus driving position of the image pickup unit 2 can be uniquely determined. As a result, only the calibration parameters corresponding to the diagonals of the specific focus positions of the imaging units 1 and 2 in FIG. 5 can be used, and the complexity of the calibration method can be significantly streamlined. .

Moreover, the three-dimensional measurement apparatus of the first embodiment may include a display device (display unit). The display device may display a plurality of external parameters, internal parameters, or calibration parameters selected by the selection unit 33 . Furthermore, the content displayed by an input unit (not shown) may be switched to, for example, information regarding the focus of each imaging unit, measurement results, or the like.

As described above, in the first embodiment, external parameters and internal parameters corresponding to a plurality of focus positions are selected as calibration parameters when calculating three-dimensional information. Then, it is possible to provide a three-dimensional measuring apparatus capable of calculating three-dimensional information of an object based on the selected calibration parameters and the image of the object. As a result, it is possible to achieve both a wide measurement range and high-precision measurement (distance measurement) by combining the calibration parameters with the external parameters that are normally considered to remain unchanged.

<Example 2>
A three-dimensional measurement apparatus of Example 2 will be described below with reference to FIG. Note that the three-dimensional measurement apparatus of the second embodiment is the same as the three-dimensional measurement apparatus of the second embodiment except that a projection unit (projection section) 11 for projecting a pattern is added. are omitted, and only different points will be described below.

The calibration of the three-dimensional measuring apparatus of Example 2 will be described using FIG. FIG. 8 is a schematic diagram of a three-dimensional measuring device and a calibration device of Example 2. FIG. The three-dimensional measurement apparatus of the second embodiment is composed of a measurement head 102 (broken line portion) and a control operation section 3. The measurement head 102 further includes an imaging unit 1 and an imaging unit 2 for imaging the object to be inspected, and a projection unit 11. It may contain three optical units.

The projection unit 11 has a function of projecting a preset known light pattern (light intensity distribution) 16 . In FIG. 8, the y-axis is parallel to the line of the light pattern 16, and the xz-plane spanned by the x-axis and z-axis is the object side principal point of the projection optical system 12 and the image side of the imaging lens 7 and imaging lens 4. It is the plane containing the principal point and the object point. The projection unit 11 has a projection pattern setting element (pattern forming section) 13 and a projection optical system 12 .

The projection unit 11 irradiates a projection pattern setting element 13 that forms a light pattern with light emitted from a light source (not shown), and the projection optical system 12 projects the set pattern. A liquid crystal element, a DMD (digital mirror device), or the like is used for the projection pattern setting element 13, and it is possible to set an arbitrary light pattern. Also, instead of an element with a variable light pattern, a glass plate on which a fixed pattern is drawn and which forms a fixed light pattern may be used.

Next, with reference to FIG. 9, the calibration method in Example 2 will be described. FIG. 9 is a flow chart diagram illustrating a calibration flow in the second embodiment. It should be noted that the description of the same as the processing flow of FIG. 4 will be omitted. The processing of each step in FIG. 9 is performed by the control calculation unit 3 executing a program stored in a memory (not shown).

First, in step S<b>301 , the user selects one of each of the plurality of specific focus positions of the imaging unit 1 , the imaging unit 2 and the projection unit 11 . After selection, the focus control section 31 controls the focus of the imaging unit 1 , the imaging unit 2 and the projection unit 11 .

Next, in step S302, the controller of the calibration apparatus drives the stage 101 to move the calibration pattern flat plate 100 stepwise to a known distance position set in advance by the stage 101. FIG. Next, the control calculation section 3 controls the imaging unit 1 and the imaging unit 2 at each step position, and the imaging unit 1 and the imaging unit 2 respectively capture images of the calibration pattern plate 100 . Next, the controller of the calibration apparatus drives the stage 101 to move the calibration pattern flat plate 100 stepwise to a known distance position. After the movement, the image pickup unit 1 and the image pickup unit 2 each pick up an image of the calibration pattern plate 100 in a state in which the projection pattern setting element 13 irradiates stripe patterns in both horizontal and vertical directions at each step position.

Next, in step S303, the control calculation unit 3 calculates the internal parameters of the imaging unit 1 and the imaging unit 2 from the coordinates of each calibration pattern formed on the calibration pattern plate 100 on the captured image and the known two-dimensional grid coordinates. derive Furthermore, the relative positions and orientations of the imaging units 1 and 2 with respect to the calibration pattern plate 100 are derived. The derivation method is the same as in the first embodiment. Further, the internal parameters of the projection unit 11 and the relative position and orientation of the projection unit 11 with respect to the calibration pattern plate 100 are similarly derived from the coordinates of the striped pattern on the captured image and the known two-dimensional grid coordinates. The internal parameters of the projection unit 11 include focal length, principal point position, distortion parameters, and the like.

In addition, the control calculation unit 3 derives the relative positions and orientations between the projection unit 11 and the imaging unit 1 and between the projection unit 11 and the imaging unit 2, which are external parameters. In deriving these relative positions and orientations, the control calculation unit 3 derives them based on the positions and orientations of the imaging unit 1 , the imaging unit 2 and the projection unit 11 with respect to the calibration pattern plate 100 . As described above, it is possible to derive calibration parameters consisting of derived intrinsic parameters and extrinsic parameters.

The calibration parameters composed of the internal parameters and external parameters of the imaging unit 1, the imaging unit 2, and the projection unit 11 derived by the above steps are saved (stored) in the storage unit 32 by the control calculation unit 3.

Next, in step S304, a measurement method using the three-dimensional measurement apparatus according to the second embodiment will be described. The light pattern 16 projected by the projection unit 11 in the second embodiment consists of binary stripes of white and black. While the projection unit 11 projects a plurality of types of light patterns, the control calculation unit 3 controls the imaging unit 1 and the imaging unit 2 to capture images for each pattern, and signals (data) of the captured images in each pattern. ) is stored in the calculation unit 34 .

Next, the control calculation unit 3 divides the measurement space from the set values of the fringes of the light pattern 16 and the imaging result data for each set value. Then, the pixel position of the projection pattern setting element 13 of the projection unit 11, the pixel position of the sensor element 6 of the imaging unit 1, and the sensor of the imaging unit 2 are The pixel positions of the elements 9 are associated with each other. Furthermore, the calculation unit 34 calculates the distance information of the test object based on triangulation using the calibration parameters calculated in step S303.

At the time of calibration, the calibration pattern plate 100 is treated as an object to be measured, and the imaging unit 1 and the projection unit 11 are used for measurement. Calculate Furthermore, the imaging unit 2 and the projection unit 11 are used for measurement, and the calculation unit 34 calculates the measurement error from the difference from the known distance moved by the stage 101 .

Next, in step S305, the control calculation unit 3 compares each measurement error calculated in step S304 with a preset target accuracy. As a result of the comparison, if each measurement error is smaller than the target accuracy, that is, if the target accuracy is not exceeded, the calibration parameter calculated in step S304 is adopted as a calculation parameter for calculating three-dimensional information, Proceed to step S306. As a result of the comparison, if any of the measurement errors is larger than the target accuracy, that is, if the target accuracy is exceeded, the process returns to step S303, and calculation of the calibration parameters (calibration parameter estimation) is repeated. A predetermined value or threshold is set in advance for the magnitude of the measurement error.

Next, in step S<b>306 , the control calculation unit 3 stores the calibration parameters employed in step S<b>304 in the storage unit 32 . When saving, the control calculation unit 3 saves each calibration parameter in the storage unit 32 while being associated with the selected specific focus position, and proceeds to step S307.

Next, in step S307, the control calculation unit 3 determines whether or not the storage of the calibration parameters has been completed at all the specific focus positions. If the calibration parameters have not been stored for all the specific focus positions, the process returns to step S301 to reselect the specific focus positions. After that, the processing from steps S302 to S306 is performed, and this is repeated until the calibration parameters are stored at all the specific focus positions. If the calibration parameters have been stored for all specific focus positions, the process ends.

By repeating steps S301 to S306, the storage of the calibration parameters associated with all the specific focus positions in the storage unit 32 is completed. When selecting the calibration parameters, the selection unit 33 can select the calibration parameters based on the information about the specific focus position of the imaging unit 1 and the specific focus position of the imaging unit 2 .

The above is the description of the calibration method using the three-dimensional measurement apparatus of the second embodiment. Regarding the measurement of the three-dimensional measuring apparatus of the second embodiment, the difference from the first embodiment is that the distance information of the object to be inspected derived from the relationship between the imaging unit 1 and the projection unit 11 and the relationship between the imaging unit 2 and the projection unit 11 are This is where the distance information is calculated by synthesizing the derived measurement information of the subject. As a result, it is possible to achieve both a wide measurement range and high-precision measurement (distance measurement) by combining the calibration parameters with the external parameters that are normally considered to remain unchanged.

Note that the synthesis method may be a simple average method, or a method of calculating the reliability from surrounding distance information and adopting the highly reliable distance information. The above is the description of the measurement method in the second embodiment.

Furthermore, in the second embodiment, a method of obtaining distance information for each combination of the imaging unit 1, the imaging unit 2, and the projection unit 11 is employed. However, it is not limited to this, and it is also possible to calculate the distance information of the object from the relationship between the imaging unit 1 and the imaging unit 2 by simply using the pattern irradiated by the projection unit 11 as a characteristic point of the object. In this case, a dot pattern or the like is used as the irradiation pattern, and the calibration parameters (internal parameters and external parameters) related to the projection unit 11 can be eliminated, and the calculation load related to the calculation of distance information can be reduced.

<Example related to article manufacturing method>
The three-dimensional measuring device in each of the above-described embodiments can be used while being supported by a certain supporting member. In this embodiment, as an example, a control system installed in a robot arm 800 (grasping device) and used as shown in FIG. 10 will be described. The control calculation unit 3 of the three-dimensional measuring device or the arm control unit 810 that acquires the information output from the control calculation unit 3 of the three-dimensional measuring device determines the position and orientation of the object W. FIG. The arm control unit 810 controls the robot arm 800 by sending a drive command to the robot arm 800 based on the position and orientation information (measurement result). In this embodiment, based on the three-dimensional information of the object W calculated by the three-dimensional measuring apparatus in each of the above embodiments, the robot arm 800 holds the object W with a robot hand (grasping portion) or the like at the tip, and moves in translation. or rotate.

Furthermore, by assembling (assembling) the object W with other parts by the robot arm 800, it is possible to manufacture a predetermined article composed of a plurality of parts, such as an electronic circuit board or a machine. Further, by processing (processing) the moved object W, an article can be manufactured. Note that the arm control unit 810 has an arithmetic device such as a CPU as a computer and a storage device such as a memory that stores a computer program. Note that a control unit that controls the robot may be provided outside the arm control unit 810 . In addition, the measurement data measured by the three-dimensional measurement device and the obtained image may be displayed on the display unit 820 such as a display. Also, the object W may be gripped and moved for alignment with other parts. The object W has the same configuration as the test object 21 shown in FIG.

<Other Examples>
The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention. They are not excluded from the scope of the invention.

Moreover, a computer program that implements the functions of the above embodiments may be supplied to the three-dimensional measuring apparatus via a network or various storage media for part or all of the control in the above embodiments. Then, the computer (or CPU, MPU, etc.) in the three-dimensional measurement device may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention.

1 imaging unit 2 imaging unit 3 control calculation unit 31 focus control unit 32 calibration parameter storage unit 33 calibration parameter selection unit 34 calculation unit

Claims (16)

In a three-dimensional measuring device having a plurality of imaging units capable of changing magnification or focus position,
The magnification or the focus position when capturing an image of the object is selected from among a plurality of predetermined positions, and a plurality of magnifications of the plurality of imaging units or between the plurality of imaging units at a plurality of focus positions. a selection unit that selects, as a calculation parameter, a first parameter corresponding to a predetermined magnification or a predetermined focus position from a plurality of first parameters relating to the relative position and orientation;
a calculation unit that calculates three-dimensional information of the object imaged at the predetermined magnification or the predetermined focus position based on the image of the object and the calculation parameters;
A three-dimensional measuring device characterized by: The selection unit selects a second parameter corresponding to the predetermined magnification or the predetermined focus position from a plurality of second parameters relating to imaging performance of each of the plurality of imaging units at the plurality of magnifications or the plurality of focus positions. 2. The three-dimensional measuring apparatus according to claim 1, wherein the parameters are selected as the calculation parameters. 3. The three-dimensional structure according to claim 2, wherein the selection unit selects a combination of the first parameter and the second parameter corresponding to the predetermined magnification or the predetermined focus position as the calculation parameter. measuring device. 4. The three-dimensional measuring apparatus according to claim 2, wherein the plurality of first parameters or the plurality of second parameters are measured in advance at the plurality of magnifications or the plurality of focus positions. 5. The three-dimensional measuring apparatus according to claim 2, further comprising a storage unit for storing said plurality of first parameters or said plurality of second parameters. 6. The three-dimensional measurement apparatus according to any one of claims 1 to 5, further comprising a projection section for projecting light having a predetermined pattern. 7. The method according to claim 6, wherein the plurality of first parameters include parameters relating to the relative position and orientation between the projection unit and the plurality of imaging units at the plurality of magnifications or the plurality of focus positions. Dimensional measuring device. 8. The three-dimensional measurement apparatus according to claim 6, wherein the plurality of second parameters include parameters relating to imaging performance of the projection unit at the plurality of magnifications or the plurality of focus positions. The three-dimensional measuring apparatus according to any one of claims 1 to 8, wherein each of said plurality of magnifications or said plurality of focus positions has a predetermined area, and adjacent areas partially overlap each other. . The plurality of imaging units each include a sensor and an optical system,
9. The optical axis of the sensor and the optical axis of the optical system in any one of the plurality of imaging units are arranged to be inclined relative to each other. The three-dimensional measuring device according to the item. a display unit capable of displaying either the plurality of first parameters or the calculation parameter selected by the selection unit;
The three-dimensional measuring device according to any one of claims 1 to 10, characterized in that: 12. The third method according to claim 11, wherein the display unit is capable of displaying a plurality of second parameters that are information relating to imaging performance of each of the plurality of imaging units at the plurality of magnifications or at the plurality of focus positions. Dimensional measuring device. A three-dimensional measurement method using a three-dimensional measurement device having a plurality of imaging units capable of changing magnification or focus position,
The magnification or the focus position when capturing an image of the object is selected from among a plurality of predetermined positions, and a plurality of magnifications of the plurality of imaging units or between the plurality of imaging units at a plurality of focus positions. a selection step of selecting, as a calculation parameter, a first parameter corresponding to a predetermined magnification or a predetermined focus position from a plurality of first parameters relating to the relative position and orientation;
a calculation step of calculating three-dimensional information of the object imaged at the predetermined magnification or the predetermined focus position based on the image of the object and the calculation parameters;
A three-dimensional measurement method characterized by: A program for controlling each part of the three-dimensional measuring device according to any one of claims 1 to 12 by a computer. The three-dimensional measurement device according to any one of claims 1 to 12;
a robot that grips and moves the object based on the three-dimensional information of the object calculated by the three-dimensional measuring device;
A system characterized by comprising: Based on the three-dimensional information of the object calculated by the three-dimensional measuring device according to any one of claims 1 to 12, by gripping the object and processing the object, a predetermined A method for manufacturing an article characterized by having a step of manufacturing an article of

Images