JP2014059261A - Distance measuring device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance measuring device which accurately measures distance to a multiple reflection area of a measurement target object.SOLUTION: A distance measuring device comprises: at least one projection means for projecting pattern light on a measurement target object 4; at least one imaging means for imaging the measurement target object on which the pattern light is projected; measurement means for measuring distance to the measurement target object from the projection means or the imaging means on the basis of an image imaged by the imaging means; determination means for determining whether or not the measured distance is effective; and control means for extinguishing or reducing the luminance of the pattern light projected on an area whose distance is determined to be effective by the determination means among the projected pattern light.

Description

  The present invention relates to a technique for performing three-dimensional measurement of a target object.

  Various techniques have been proposed for 3D measurement of objects such as parts of industrial products, and one of them is to project a predetermined projection pattern (striped pattern or grid pattern) onto the measurement target object. Then, the measurement target object is imaged, the correspondence between each image position (pixel) and the position of the stripe in the projection pattern is obtained from the captured image, the height of the measurement target object is calculated based on triangulation, and the measurement target What measures the three-dimensional shape of an object is a highly accurate and stable measurement method.

  In such a three-dimensional measurement method, when the measurement target object has a gloss such as a metal part, depending on the shape of the measurement target object, the pattern light projected on the other surface may be reflected multiple times between the measurement target objects, That is, multiple reflection occurs. Light reflected once on the surface of the measurement target object is primary reflected, light reflected once on the surface of the measurement target object is reflected again by the target object, is reflected secondarily, and is reflected again by the target object. The secondary reflection or the like is called secondary reflection, and the reflection of the second or higher order is called multiple reflection. Since multiple reflection occurs mutually between objects to be measured, it is also called mutual reflection. When multiple reflection occurs, a reflection that should not originally exist in the measurement target object occurs. If a three-dimensional measurement process is applied based on such a captured image, the reflection may act as noise, which may reduce the measurement accuracy.

  As a countermeasure against reflection, as disclosed in Patent Document 1, a method of dimming or extinguishing a part of a projection pattern imaged by multiple reflection on the surface of a test object has been proposed. Specifically, two different pattern lights are projected, and a three-dimensional shape is obtained from each. A region where there is a difference between the two three-dimensional measurement results is regarded as a region where multiple reflections have occurred. Based on the captured image obtained by finding the surface causing the multiple reflection by ray tracing, extinguishing the projection pattern corresponding to the surface, and projecting the newly extinguished pattern onto the measurement target object The 3D measurement result is corrected.

JP 2008-309551 A

The method according to Patent Document 1 has a problem that the processing is complicated and time-consuming because ray tracing is performed in order to find a part of the projection pattern that causes multiple reflection. Moreover, since the projection pattern corresponding to the mutually reflected area is extinguished, there is also a problem that the mutually reflected area cannot be measured.
The present invention has been made in view of the above problems, and an object thereof is to reduce the influence of multiple reflections and improve measurement accuracy by repeating simple processing.

  In order to achieve the above object, a distance measuring device according to the present invention has the following configuration, for example. That is, based on at least one projection unit that projects pattern light onto a measurement target object, at least one imaging unit that images the measurement target object onto which the pattern light is projected, and an image captured by the imaging unit A measuring unit that measures a distance from the projection unit or the imaging unit to the measurement target object, a determination unit that determines whether the measured distance is valid, and the projected pattern light And control means for extinguishing or dimming the brightness of the pattern light projected on the area where the distance is determined to be effective by the determining means.

  According to the present invention, it is possible to accurately measure the distance of the measurement target object to the multiple reflection region.

The figure which shows the structural example of the distance measuring device of 1st Embodiment. Explanatory drawing of the principle of distance measurement by the triangulation method. The figure which shows the example in which the measurement object was influenced by the multiple reflection. Explanatory drawing of the measurement error generation mechanism by multiple reflected light. The figure which shows the example which light-projected and projected a part of pattern. The flowchart which shows the flow of a process of 1st Embodiment. The figure which shows the structural example of the distance measuring device of 2nd Embodiment. The figure which shows the structural example of the distance measuring device of 3rd Embodiment. The figure which shows the example of the pattern for multiple reflection detection.

[First Embodiment]
In the first embodiment, a pattern projection unit (projector) that projects pattern light onto a measurement target object and an imaging unit (camera) that captures the measurement target object onto which the pattern light is projected are used to three-dimensionally measure the measurement target object. Measure. In the present embodiment, the three-dimensional measurement of the measurement target object is to measure the distance from the imaging unit or the projection unit to the measurement target object. In the present embodiment, three-dimensional measurement (distance measurement to a measurement target object) is performed using two imaging units and one projection unit, and it is determined whether the measured distance is valid. Here, the distance measurement value is valid means that the measured distance value is less affected by multiple reflections and image noise and is acceptable as a distance measurement value, that is, a more accurate distance measurement value can be measured. Indicates. Although details will be described later, distance measurement is performed for each pixel of the projection unit. Therefore, the determination of whether or not it is valid is performed for each pixel. In other words, distance measurement is performed for each area of the measurement target object included in one pixel of the display element of the projection unit, and it is determined whether or not it is valid. Then, the area determined to be effective is the third order using a pattern in which the pixels of the display element of the projection unit corresponding to the area are extinguished or dimmed and a part of the display element of the projection unit is extinguished or dimmed. Repeat the original measurement. In addition, the number of repetition end determination conditions is arbitrarily specified.

  In this embodiment, the effectiveness of the distance measurement value of the measurement target object included is determined for each pixel of the display element of the projection unit, but the present invention is not limited to this. For example, the effectiveness of the distance measurement value of the measurement target object included may be determined for each of the plurality of pixels of the display element of the projection unit. Moreover, you may judge the effectiveness of the distance measurement value of the measurement target object contained for every 1 pixel or several pixel of the display element of an imaging part.

  In the present embodiment, extinction or dimming is performed for each pixel of the display element of the projection unit corresponding to the area determined to be effective, but the present invention is not limited to this. For example, extinction or dimming may be performed for each of the plurality of pixels of the display element of the projection unit.

  FIG. 1 shows the configuration of a distance measuring apparatus according to this embodiment. As shown in the figure, the configuration in the present embodiment includes a projection unit 1, imaging units 2 </ b> A and 2 </ b> B, and a control / calculation processing unit 3.

  The projection unit 1 projects pattern light, for example, line pattern light, onto the measurement target object 4. The imaging unit 2 images the measurement target object 4 onto which the pattern light is projected. The control / calculation processing unit 3 controls the projection unit 1 and the imaging unit 2, calculates the captured image data, and measures the distance to the measurement target object 4.

  The projection unit 1 includes a light source 11, an illumination optical system 12, a display element 13, and a projection optical system 14. The light source 11 is various light emitting elements such as a halogen lamp and an LED. The illumination optical system 12 has a function of guiding the light emitted from the light source 11 to the display element 13. When the light emitted from the light source enters the display element 13, it is better that the incident luminance is uniform. Therefore, for the illumination optical system 12, for example, an optical system suitable for uniform luminance such as Koehler illumination or a diffusion plate is used. The display element 13 is, for example, a transmissive LCD or a reflective LCOS / DMD. The display element 13 has a function of spatially controlling the transmittance or the reflectance when the light from the illumination optical system 12 is guided to the projection optical system 14. The projection optical system 14 is an optical system configured to form an image of the display element 13 at a specific position of the measurement target object 4. As an example satisfying such a configuration, a DLP projector or a diffraction type projector may be used. In the present embodiment, the configuration of a projection apparatus including a display element and a projection optical system is shown, but a projection apparatus including a spot light and a two-dimensional scanning optical system can also be used.

  Each of the imaging units 2 </ b> A and 2 </ b> B includes an imaging lens 21 and an imaging element 22. The imaging lens 21 is an optical system configured to image a specific position of the measurement target object 4 on the imaging element 22. The imaging element 22 is various photoelectric conversion elements such as a CMOS sensor and a CCD sensor.

  The control / calculation processing unit 3 includes a projection pattern control unit 31, an image acquisition unit 32, a distance calculation unit 33, an effectiveness determination unit 34, a repetition control unit 35, and a parameter storage unit 36. The hardware of the control / calculation processing unit 3 includes a general-purpose computer (hardware) including a CPU, a storage device such as a memory and a hard disk, and various interfaces for input / output. The software of the control / calculation processing unit 3 includes a distance measurement program that causes a computer to execute the distance measurement method according to the present invention.

  When the CPU executes the distance measurement program, the projection pattern control unit 31, the image acquisition unit 32, the distance calculation unit 33, the validity determination unit 34, the repetition control unit 35, and the parameter storage unit 36 are realized.

  The projection pattern control unit 31 generates a projection pattern described later and stores it in the storage device in advance. Further, the stored projection pattern data and the validity determination result described later are read out as necessary, and the projection pattern data is transmitted to the projection unit 1 via a general-purpose display interface such as DVI. Further, it has a function of controlling the operation of the projection unit 1 via a general-purpose communication interface such as RS232C or IEEE488. The projection unit 1 displays the projection pattern on the display element 13 based on the transmitted projection pattern data. Examples of the measurement pattern to be projected include a gray code by a spatial encoding method, a sine wave pattern by a phase shift method, and the like.

  The image acquisition unit 32 takes in the digital image signal sampled and quantized by the imaging unit 2. Furthermore, it has a function of acquiring image data represented by luminance (density value) of each pixel from the captured image signal and storing it in the memory. The image acquisition unit 32 has a function of controlling the operation of the imaging unit 2 (imaging timing, etc.) via a general-purpose communication interface such as RS232C or IEEE488.

  The image acquisition unit 32 and the projection pattern control unit 31 operate in cooperation with each other. When the pattern display on the display element 13 is completed, the projection pattern control unit 31 transmits a signal to the image acquisition unit 32. When the image acquisition unit 32 receives a signal from the projection pattern control unit 31, the image acquisition unit 32 operates the imaging unit 2 to perform image capturing. When the image capturing is completed, the image acquisition unit 32 transmits a signal to the projection pattern control unit 31. When the projection pattern control unit 31 receives a signal from the image acquisition unit 32, the projection pattern control unit 31 switches the projection pattern to be displayed on the display element 13 to the next projection pattern. By repeating this in sequence, all the projection patterns are imaged.

  The distance calculation unit 33 calculates the distance from the projection unit 1 or the imaging unit 2 to the measurement target object 4 using the captured image of the distance measurement pattern and the parameters stored in the parameter storage unit 36.

  The validity determination unit 34 determines the validity of whether the measured distance value to the measured area is correct and reliable for each point or area. In addition, the validity determination result is sent to the parameter storage unit 36, and the projection pattern is updated using the result.

  The iterative control unit 35 projects a pattern on the measurement target object, calculates the distance from the result of imaging the pattern, determines the validity of the distance calculation result, and terminates a series of processes for updating the projection pattern based on the result. Control is performed repeatedly until the determination condition is satisfied. In the first embodiment, it is assumed that the process is terminated when an arbitrary number of times designated by the user is repeated.

  The parameter storage unit 36 stores parameters necessary for calculating a three-dimensional distance. As parameters, device parameters of the projection unit 1 and the imaging units 2A and 2B, internal parameters of the projection unit 1 and the imaging units 2A and 2B, external parameters of the projection unit 1 and the imaging units 2A and 2B, and projection pattern parameters There is. The parameter storage unit 36 stores the validity determination result obtained by the validity determination unit 34 and is used when the projection pattern control unit 31 updates the projection pattern.

  Device parameters of the projection unit 1 and the imaging units 2A and 2B are the number of pixels of the display element and the number of pixels of the imaging element. Internal parameters of the projection unit 1 and the imaging units 2A and 2B are a focal length, an image center, a coefficient of image distortion due to distortion, and the like. External parameters between the projection unit 1 and the imaging units 2A and 2B are a parallel progression and a rotation matrix that represent the relative positional relationship between the projection unit 1 and the imaging units 2A and 2B. The parameters of the projection pattern are information related to the distance measurement pattern and the extinction / dimming mask pattern. The validity judgment result is data that stores whether the pixel is valid or invalid for the display element.

  In the present embodiment, description will be made assuming that both the projection unit 1 and the imaging units 2A and 2B are approximated by a pinhole camera model. However, the camera model to which the present invention can be applied is not limited to the pinhole camera.

  With reference to FIG. 2, the principle of distance measurement by the triangulation method will be described. If the lens center of the imaging unit is Cc, the image plane is Ic, the focal length is fc, the pixel coordinates of the image center cc are (uco, vco), and the pixel size of the imaging device is Psc, the internal matrix Ac of the imaging unit is It is described as equation (1).

  When the lens center of the projection unit is Cp, the image plane is Ip, the focal length is fp, the pixel coordinates of the image center cp are (upo, vpo), and the pixel size of the display element is Psp, the internal matrix Ap of the projection unit is It is described as the following formula (2).

  The internal matrix Ac of the imaging unit and the internal matrix Ap of the projection unit are calculated by using an internal parameter calibration method that is a known technique.

  External parameters representing the relative positional relationship between the camera coordinate system XYZ of the imaging unit and the camera coordinate system XpYpZp of the projection unit are a rotation matrix R and a parallel progression T. The rotation matrix R is a 3 × 3 matrix, and the parallel progression T is a 3 × 1 matrix. The rotation matrix R and the parallel progression T are calculated by using an external parameter calibration method which is a known technique.

  Let (X, Y, Z) be the coordinates of a point M in the three-dimensional space with the camera coordinate system of the imaging unit as the origin. The pixel coordinate of the point mc obtained by projecting the point M onto the image plane Ic of the imaging unit is (uc, vc). The corresponding relationship is described by the following equation (3).

  s is a scalar. Further, the pixel coordinates of the point mp obtained by projecting the same point M onto the image plane Ip of the projection unit are assumed to be (up, vp). The corresponding relationship is described by the following equation (4).

  s' is a scalar. When the above expressions (3) and (4) are expanded, the following four simultaneous equations are obtained by the following expression (5).

  The pixel coordinates (uc, vc) of the point mc and the pixel coordinates (up, vp) of the point mp are obtained by using various pattern projection methods described in the background art. Since Cij (i = 1 to 3, j = 1 to 4) and Pij (i = 1 to 3, j = 1 to 4) are calculated from the internal matrix and the external parameters, they are obtained in advance by calibration. It has been. In the equation (5), only the coordinate value (X, Y, Z) of the point M is an unknown, and can be obtained by solving simultaneous equations. Since the coordinate value of the point M, which is an unknown, is (X, Y, Z), the coordinate value of the point M can be obtained by calculating one of the pixel coordinate values (up, vp) of the projection unit. Can be calculated. The above is the principle of distance measurement based on the triangulation method.

  In the first embodiment, it is assumed that there are two cameras as imaging units, but distance measurement based on the triangulation method is performed on each of the projection unit 1 and the imaging units 2A and 2B. Moreover, based on the triangulation method between the imaging units 2A and 2B by considering the correspondence relationship with the pixel coordinates of the other imaging unit instead of the pixel coordinates (up, vp) of the point mp corresponding to the projection unit. Therefore, a total of three distance measurement results of the projection unit 1 and the imaging unit 2A, the projection unit 1 and the imaging unit 2B, and the imaging unit 2A and the imaging unit 2B can be obtained. However, the correspondence of the pixels between the two imaging units can be taken, for example, through the correspondence with the projection unit. However, the present invention is not limited to this, and the correspondence between the imaging units 2A and 2B may be obtained using various matching methods in binocular stereo as shown in the following document.

D. Scharstein and R.M. Szeliski, “A taxonomy and evaluation of dense two-frame stereocorrespondence algorithms,” IJCV, vol. 47, no. 1, 2002
When the multiple reflected light does not occur or the influence does not become apparent even if it occurs, the distance measurement of the measurement target object can be performed by the triangulation method described above. However, if multiple reflected light is generated and the influence becomes obvious, a measurement error occurs. For example, as shown in FIG. 3, when pattern light Pt1 is projected onto the measurement target object 4, it is observed as O1 in the imaging unit, and the influence of multiple reflection is seen in the R1 portion.

  With reference to FIG. 4, the mechanism of occurrence of measurement errors due to multiple reflected light will be described. For the sake of simplicity, FIG. 4 will be described in the form of a two-dimensional cross-sectional view of the X axis and the Z axis.

  A light beam L1 emitted from the lens center Cp of the projection unit 1 in a direction passing through the point up1 on the image plane Ip is incident on the point P1 on the measurement target object 4. The light reflected at the point P1 intersects with the image coordinates of the imaging unit 2 at the point uc1 and enters the lens center Cc.

  A light beam L2 emitted from the lens center Cp of the projection unit 1 in a direction passing through the point up2 on the image plane Ip is incident on the point P2 on the measurement target object 4. The light reflected at the point P2 is secondarily reflected at the point P1. Thereafter, they intersect at a point uc1 on the image coordinates of the imaging unit 2 and enter the lens center Cc of the imaging unit 1. When secondary reflection occurs, the light originally emitted from the lens center Cp of the projection unit 1 behaves as if it is emitted from the point P2.

  When the measurement target object has low gloss and is less affected by multiple reflections, the luminance observed by the imaging unit 2 is dominantly reflected by L1. Since the pixel coordinate value is correctly detected as up1, the distance measurement result is also correct Zt.

  On the other hand, when the object to be measured has high glossiness and the reflected light is observed in the regular reflection direction where the incident angle and the reflection angle are equal, and the influence of multiple reflection is large, the reflected light by L2 is dominant. . At this time, since the pixel coordinate value is erroneously detected as up2, the measurement point is measured as being at Pe. The distance measurement result is Ze, and an error of ΔZ = Zt−Ze is generated.

  Actually, not only the secondary reflected light shown in FIG. 4 but also the third or higher order reflected light exists, and a more complicated measurement error may occur. The above is the explanation of the mechanism of occurrence of measurement error due to multiple reflected light.

  On the other hand, according to the present invention, the projection pattern portion corresponding to the area that is determined to be effective without being affected by the multiple reflection by the validity determination unit is extinguished or dimmed, and the projected / imaging / distance is updated with the updated pattern. By repeating measurement and judgment, the area that can be measured is expanded while reducing the occurrence of multiple reflections.

  A specific example is shown in FIG. V1 represents a region that is three-dimensionally measured using the observation image O1 shown in FIG. 3 and the like and is determined to be effective by the validity determination unit. On the other hand, V2 represents an area determined to be invalid due to the influence of multiple reflection or the like. The update of the projection pattern is to extinguish or diminish the area of V1 determined to be effective of the measurement target object with respect to Pt1 of FIG. 3, and to change the pattern to Pt2 of FIG. By projecting and imaging again using an updated projection pattern group such as Pt2, an observation image such as O2 can be obtained, and as a result, three-dimensional measurement that is not affected by multiple reflection is possible. .

  With reference to the flowchart of FIG. 6, the overall flow up to the measurement of the multiple reflection area according to the first embodiment will be described.

(Process S1)
In step S1, a pattern group for measurement is projected from the projection unit 1 onto the measurement target object 4, and the measurement target object 4 is imaged by the imaging unit 2A and the imaging unit 2B. When imaging is completed in each imaging unit, the next pattern for measurement is projected and imaged. It repeats until all the pattern groups for measurement are projected and imaged. The measurement pattern includes, for example, a gray code, but is not limited thereto. The imaged data is sent to the image acquisition unit 32.

(Process S2)
In step S2, using the above-described triangulation method, a total of three distance measurements of the projection unit 1 and the imaging unit 2A, the projection unit 1 and the imaging unit 2B, and the imaging unit 2A and the imaging unit 2B are performed for each region based on the captured image. The result is calculated.

(Process S3)
In step S3, it is determined based on the three distance measurement results whether the distance measurement results in each region are valid. If the three distance measurement values in each region match, it can be determined that there is no influence of multiple reflections and the result is an effective distance measurement result. This is because the influence of the multiple reflection of the measurement target object with high glossiness differs in the direction of the viewpoint to be observed, and therefore the presence or absence of the influence of the multiple reflection can be determined by observing with different imaging units (different viewpoints). Further, if the three distance measurement values do not match due to influences other than multiple reflection, such as randomly generated image noise, the distance measurement result is regarded as invalid.

  However, even when there is no influence of multiple reflections or image noise, the distance measurement values may not completely match in a plurality of distance measurement systems. This is because, in the case of not being in an ideal state (simulation), the internal and external parameters of the projection unit and the imaging unit obtained in advance by calibration may include a slight error. As described in the principle of triangulation described above, the distance value is calculated using the internal / external parameters of the projection unit and the imaging unit, and thus the error of the internal / external parameter appears as the error of the distance measurement value.

  It is difficult to remove or correct the calibration error other than re-calibration. Therefore, as a measure against mismatch of distance measurement values in a plurality of distance measurement systems due to calibration errors, the position and orientation of a certain projection unit and imaging unit (for example, projection unit 1 and imaging unit 2A) are determined as calibration standards, This can be solved by setting an allowable range from there. However, although this standard may be different from the true value, calibration error cannot be removed or corrected, so it is considered to be the best method for determining the coincidence / disagreement of distance measurement values. Specifically, using the calculated internal and external parameters, re-projection is performed on the calibration data, and the error of the distance measurement result by calibration is calculated. Establish tolerances from standards that can be considered In this case, since the distance is not fixed to one measurement value, the reference system value is used as a representative value. However, the present invention is not limited to this, and the representative value may be determined by an average value or a median value.

(Process S4)
In step S4, based on the validity determination result, the projection pattern is updated so that the portion of the projection pattern corresponding to the area where the distance measurement value is deemed valid is extinguished or dimmed. Since the desired distance measurement value is obtained for the area where the distance measurement value is valid, it is not necessary to perform the process in the subsequent iterative processes. Then, the pattern light illuminating the area is quenched or dimmed. If the pattern light reflected in the area causes multiple reflections that illuminate other surfaces in the measurement target object, the projection pattern updated in the next iteration is projected to The effect can be removed. However, the method of updating the projection pattern is not limited to extinction or dimming, but may be a method that relatively weakens the influence of light that illuminates a region deemed effective. For example, the projection pattern portion corresponding to the area where the distance measurement value is determined to be invalid may be increased.

(Process S5)
In step S5, it is determined whether or not the end determination condition is satisfied. If satisfied, the process proceeds to step S6. If not satisfied, the process returns to step S1, and the process is repeated using the projection pattern updated in step S4. In the first embodiment, it is assumed that the end determination condition is repeated an arbitrary designated number N times. Therefore, if the number of repetitions is N-1 or less, the process returns to step S1, and if it is N times, the process proceeds to step S6. By performing the process repeatedly a plurality of times, it is possible to eliminate the influence of the third and higher order multiple reflections or to make it less susceptible to the influence of image noise and the like. In addition, since it can be specified any number of times (including one repetition), processing can be performed in accordance with the on-site tact time.

  When returning to step S1 repeatedly, the projection pattern group updated in step S4 is projected and imaged, each step of steps S1 to S4 is executed once, and whether or not the end determination condition of step S5 is satisfied The determination is repeated until the end determination condition is satisfied.

(Step S6)
In step S6, the distance measurement results are integrated. While the iterative process is being performed, since the distance measurement value is newly obtained in the area in which the distance measurement value is considered to be valid, the value is retained. Once a region is considered valid, it is associated with the distance measurement value retained, and no processing is performed between iterations. Therefore, if the distance measurement value to the area where the distance measurement value is newly considered valid at the i-th iteration is held, in step S6, the data of N distance measurement values can be integrated. A distance measurement value of the measurement target object is obtained. However, the present invention is not limited to this, and the distance to the area for which the distance measurement value is newly determined to be valid during the repeated processing (for example, when determining the end determination condition of step S5). The measurement value may be added to the distance measurement result data. In that case, the distance measurement results are already integrated when the process proceeds to step S6.

  The process ends here.

  According to the first embodiment, the influence of the multiple reflection is reduced by extinguishing or dimming the pixels of the projection pattern corresponding to the area where the effective distance measurement value is obtained without being affected by the multiple reflection. Thus, it is possible to increase an area where an effective distance measurement value can be obtained by repeated processing.

  In the first embodiment, an example in which two imaging units are used has been introduced. The present invention is not limited to this example, and an arbitrary plurality of imaging units may be used. In that case, three or more distance measurement results can be obtained, but it is not necessary to calculate the distance measurement result by all combinations of the projection unit and the imaging unit, and the distance measurement by triangulation is performed in any M combinations. May be. Further, in the determination of the effectiveness of step S3, it is not necessary that all M distance measurement values match, and if they match at a certain number or ratio, they may be regarded as effective distance measurement values.

  The above is the description of the distance measuring device according to the first embodiment of the present invention.

[Second Embodiment]
In the second embodiment, three-dimensional measurement is performed using two projection units and one imaging unit as validity determination means, and it is determined whether the distance to the measured region is valid. FIG. 7 is a schematic configuration of a distance measuring apparatus according to the second embodiment. 1 is substantially the same as FIG. 1 described in the first embodiment, except that this embodiment uses two projection units (projection units 1A and 1B) and one imaging unit (imaging unit 2). . Since the flowchart of the second embodiment is substantially the same as that of FIG. 6 described in the first embodiment, the description of the same processing will be omitted, and the step S1 having a difference will be described.

  In step S <b> 1 in the second embodiment, a pattern group for measurement is projected onto the measurement target object 4 from the projection units 1 </ b> A and 1 </ b> B, and the measurement target object 4 is imaged by the imaging unit 2. At this time, images are projected and imaged one by one in order so that the patterns projected by the projection units 1A and 1B are not confused by the imaging unit. However, the present invention is not limited to this, and light sources may be multiplexed so as not to confuse the patterns, for example, the pattern light projected from the projection units 1A and 1B may have a different color. Since the flow after the completion of imaging by the imaging unit is the same as in the first embodiment, a description thereof will be omitted.

  The supplement about description of the range which applies a triangulation method is demonstrated.

  In the second embodiment, it is assumed that there are two projection units. However, distance measurement based on the triangulation method is performed on each of the projection units 1A and 1B and the imaging unit 2. Further, instead of the pixel coordinates (uc, vc) of the point mc corresponding to the image pickup unit, based on the triangulation method between the projection units 1A and 1B by considering the correspondence relationship with the pixel coordinates of the other projection unit. Distance measurement. Therefore, a total of three distance measurement results of the projection unit 1A and the imaging unit 2, the projection unit 1B and the imaging unit 2, and the projection unit 1A and the projection unit 1B can be obtained. However, the correspondence of the pixels between the two projection units can be taken through the correspondence with the imaging unit, for example.

  By obtaining three distance measurement results in this way, the processes in steps S2 to S6 can be performed in substantially the same manner as in the first embodiment, and thus description thereof is omitted.

  In the second embodiment, an example in which two projection units are used has been introduced. The present invention is not limited to this example, and any number of projection units may be used in the same manner as described in the first embodiment.

  The above is the description of the distance measuring device according to the second embodiment of the present invention.

[Third Embodiment]
In the third embodiment, as the effectiveness determining means, a projection unit and an imaging unit are used to project a plurality of different projection pattern groups to perform three-dimensional measurement, and the distance to the measured region is effective. Determine if there is. FIG. 8 is a schematic configuration of a distance measuring apparatus according to the third embodiment. Although it is substantially the same as FIG. 1 demonstrated in 1st Embodiment, and FIG. 6 demonstrated in 2nd Embodiment, in this embodiment, one projection part (projection part 1) and one imaging part ( The difference is that only the imaging unit 2) is used. Since the flowchart of the third embodiment is substantially the same as that of FIG. 6 described in the first embodiment, the description of the same processing is omitted, and the steps S1 and S3 having differences will be described.

In step S1 in the third embodiment, a plurality of different measurement pattern groups are sequentially projected from the projection unit 1 onto the measurement target object 4, and the measurement target object 4 is sequentially imaged by the imaging unit 2. At this time, the different measurement patterns include, for example, a gray code, XOR-04 code, Gray codes with maximum min-SW, or the like as shown in the following document, or a phase shift pattern with a different period. May be. If the measurement pattern group changes, the influence of multiple reflection also changes, and therefore the distance measurement value differs depending on the projection pattern group in an area where multiple reflection occurs. Then, the effectiveness determination unit in step S3 can determine whether the distance measurement results for the region of interest are valid by checking whether the distance measurement values match each other. Gupta, A.D. Agrawal, A.A. Veeraraghavan, and S.M. G. Narasiman, “Structured light 3D scanning in the presence of global illumination,” CVPR2011, pp 713-720, 2011.
In step S3, the first embodiment has described the case where the three distance measurement values do not completely match due to an error during calibration. In the third embodiment, since there is only one projection unit and one imaging unit, a case where they do not match due to calibration errors cannot occur. Therefore, the validity can be determined using the calculated distance measurement value itself.

  The supplement about description of the range which applies a triangulation method is demonstrated.

  In the third embodiment, it is assumed that there are one projection unit and one imaging unit, and a plurality of different projection pattern groups are projected. However, the triangulation is performed for each of the plurality of different projection pattern groups. Perform distance measurement based on the law. Therefore, distance measurement results can be obtained for the number of different projection pattern groups. By obtaining a plurality of distance measurement results in this way, the processes in steps S2 to S6 can be performed in substantially the same manner as in the first embodiment, and thus description thereof is omitted.

  The above is the description of the distance measuring device according to the third embodiment of the present invention.

[Fourth Embodiment]
In the fourth embodiment, as the effectiveness determination means, one projection unit, one imaging unit, and a set of projection pattern groups are projected to perform three-dimensional measurement, and measurement is performed using information on the shape of the measurement target object. It is determined whether the distance measurement value to the specified area is valid. Since the schematic configuration of the distance measuring apparatus according to the fourth embodiment is the same as that of FIG. 8 described in the third embodiment, the description thereof is omitted. Since the flowchart of the fourth embodiment is almost the same as that of FIG. 6 described in the third embodiment, the description of the same processing will be omitted, and the steps S1 and S3 having differences will be described.

  The process S1 in the fourth embodiment has a configuration of one projection unit and one imaging unit as in the third embodiment, but only one pattern group for measurement is projected instead of a plurality.

  In step S3, in the first to third embodiments, effectiveness is determined based on whether or not a plurality of distance measurement values match. In the fourth embodiment, shape information about the measurement target object is used. Determine effectiveness. For example, assuming that the CAD model of the object to be measured is known, alignment with the distance measurement value is performed, and it is determined that the distance measurement value to the area where the error from the CAD model is equal to or less than the threshold value is valid. However, the above-described validity determination method can be used only when there is no CAD model position / posture shift with respect to the distance measurement result and there is an influence of a shape measurement error. If the position / orientation deviation of the CAD model has occurred, the CAD model is re-aligned using a point with high reliability of the distance measurement result, and the position / orientation deviation is reduced as much as possible. It is desirable to determine the effectiveness. The method of selecting a point with high reliability is not limited to a specific method. For example, the user may arbitrarily specify the point, or may select the point based on information on image luminance at which the point is observed.

  In the fourth embodiment, an example has been introduced in which effectiveness is determined using a CAD model of a measurement target object. The present invention is not limited to this example, and if the object to be measured is composed only of a flat surface, the effectiveness may be determined by examining the flatness for each region with respect to the distance measurement result. Further, when information such as the normal line of the measurement target object, the curvature distribution, and the edge position is obtained, the effectiveness may be determined based on the information.

  The above is the description of the distance measuring device according to the fourth embodiment of the present invention.

[Fifth Embodiment]
In the fifth embodiment, as the effectiveness determination means, after projecting a measurement projection pattern using one projection unit and one imaging unit to perform three-dimensional measurement, projecting a multiple reflection determination pattern, Determine whether multiple reflections are occurring. And it is judged whether the distance measurement value to the measured area | region is effective. The schematic configuration of the distance measuring apparatus according to the fifth embodiment is the same as that of FIG. 8 described in the third embodiment. Since the flowchart of the fifth embodiment is substantially the same as that of FIG. 6 described in the third embodiment, the description of the same processing will be omitted, and the steps S1 and S3 having differences will be described.

  In step S1 in the fifth embodiment, one measurement pattern group is projected, and further, a multiple reflection detection pattern is projected. Here, the pattern for detecting multiple reflections is, for example, a stripe pattern substantially parallel to the epipolar line on the image plane of the projection unit obtained from the geometric arrangement of the projection unit and the imaging unit. As shown in FIG. 9, when the measurement target object obtained by projecting a stripe pattern Pt3 substantially parallel to the epipolar line in the projection unit as a multiple reflection detection pattern with respect to one measurement pattern Pt1 is observed by the imaging unit. As described above, the primary reflected light has a characteristic of being observed as the same stripe pattern regardless of the depth. On the other hand, second-order or higher-order reflected light that is observed after the projected fringe pattern is once reflected within the object surface, such as R3 of O3, is imaged as a line having an angle different from that of the fringe pattern. Utilizing such epipolar line characteristics, in step S3, a portion where a line having an angle different from that of the fringe pattern is observed is regarded as invalid for the corresponding distance measurement value, assuming that multiple reflection is observed. On the contrary, it is assumed that the corresponding distance measurement result is valid for a portion where a line having the same angle as the stripe pattern is observed.

  The above is the description of the distance measuring device according to the fifth embodiment of the present invention.

[Sixth Embodiment]
In the sixth embodiment, it is assumed that an arbitrary total measurement time T has elapsed as the end determination condition in step S5. Therefore, if the measurement time T has not been exceeded, the process returns to step S1, the process is repeated using the projection pattern updated in step S4, and if it exceeds T, the process proceeds to step S6. The schematic configuration of the distance measuring apparatus according to the sixth embodiment is the same as that of FIG. 1 described in the first embodiment, and the flowchart is also the same as that of FIG. 6 described in the first embodiment, and thus the description thereof is omitted. To do.

  The above is the description of the distance measuring device according to the sixth embodiment of the present invention.

[Seventh Embodiment]
In the seventh embodiment, it is assumed that a designated distance measurement result is obtained as the end determination condition in step S5. For example, if the number of distance measurement points in the designated area does not exceed a certain number, the process returns to step S1, and the process is repeated using the projection pattern updated in step S4. If the number exceeds a certain number, the process proceeds to step S6. move on. The schematic configuration of the distance measuring apparatus according to the seventh embodiment is the same as that of FIG. 1 described in the first embodiment, and the flowchart is also the same as that of FIG. 6 described in the first embodiment, so the description is omitted. To do.

  In the seventh embodiment, an example has been introduced in which the number of distance measurement points in the designated area is used as the determination criterion for the end determination condition. The present invention is not limited to this example. The distance measurement result may be presented to the user to interactively determine whether or not to repeat, and the distance measurement result is appropriate as an input to another system. It may be determined whether or not.

  The above is the description of the distance measuring device according to the seventh embodiment of the present invention.

[Eighth Embodiment]
In the eighth embodiment, as an end determination condition in step S5, determination is performed based on a comparison between a past distance measurement result and a distance measurement result newly measured this time. For example, in the distance measurement result obtained repeatedly N times, if the number or ratio of newly measured points exceeds a certain standard as compared with the N-1th distance measurement result, the process proceeds to step S1. Returning, the process is repeated using the projection pattern updated in step S4. If it is considered that the laser beam has converged below a certain standard, the process proceeds to step S6. The schematic configuration of the distance measuring apparatus according to the eighth embodiment is the same as that of FIG. 1 described in the first embodiment, and the flowchart is also the same as that of FIG. 6 described in the first embodiment. .

  In the eighth embodiment, an example has been introduced in which the distance measurement result obtained by repeating N times is compared with the N-1th distance measurement result. The present invention is not limited to this example, and may be compared with any Kth distance measurement result in the past, or may be determined based on a plurality of past distance measurement results.

  The above is the description of the distance measuring device according to the eighth embodiment of the present invention.

[Other Embodiments]
The second to fifth embodiments described in the first half are all modifications related to the method for determining validity. On the other hand, the sixth to eighth embodiments described in the latter half are modified examples regarding the end determination condition in the repeated measurement. Since these validity judgment and end judgment conditions can be changed independently of each other, as an example in which the second to fifth embodiments and the sixth to eighth embodiments are freely combined with the first embodiment. You may implement.

  In addition, a combination of the first embodiment and the second embodiment may include a plurality of projection units and imaging units, or a combination of the third embodiment and a plurality of projection units, imaging units, and projections. A pattern may be used. As described above, the validity determination may be performed by combining some of the first embodiment and the second to fifth embodiments, the sixth to eighth embodiments may be combined, an AND condition OR condition, or the like may be used. Then, the end determination condition may be used.

  In the first to eighth embodiments of the present invention, the projection pattern for measurement includes, for example, a gray code, but is not limited to this, as already described. Here, a measurement pattern which is the best mode from the viewpoint of countermeasures against various multiple reflections will be described. In a measurement target object with high glossiness, brightness differs depending on the reflection direction even if it is called primary reflection or secondary reflection. For example, light that is reflected in the regular reflection direction with the same incident angle and reflection angle of light is called a specular reflection component, which is very bright compared to light reflected in other directions (diffuse reflection component), and is affected by multiple reflections. large. The first and second embodiments of the present invention are particularly excellent in dealing with the influence of the specular reflection component of this multiple reflection compared to the other embodiments. This is because in the first and second embodiments, specular reflection components are avoided by performing observations other than the regular reflection direction using a plurality of distance measurement systems.

  On the other hand, the influence of multiple reflection due to the diffuse reflection component is not so great, but the accuracy of the three-dimensional measurement result may be deteriorated depending on the shape of the measurement target object and the position and orientation relationship of the projection unit and the imaging unit. As a countermeasure against the diffuse reflection component of multiple reflection, as shown in the following document, a high-frequency stripe pattern group (thin stripes are thin) such as the XOR-04 code is used as a measurement pattern. Thus, the influence can be reduced.

M.M. Gupta, A.D. Agrawal, A.A. Veeraraghavan, and S.M. G. Narasiman, “Structured light 3D scanning in the presence of global illumination,” CVPR2011, pp 713-720, 2011.
Therefore, in the first and second embodiments, the best mode of the measurement pattern group is to set the measurement pattern to be projected as a high-frequency fringe pattern group. That is, since the area that can be measured can be expanded while reducing the influence of both the specular reflection component and the diffuse reflection component of multiple reflection, the compatibility with the present invention is very good.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed. The program may be provided by being recorded on a computer-readable recording medium.

<Effect of embodiment>
In the first embodiment, distance measurement of a multiple reflection region is performed using a projection unit and a plurality of imaging units. The effectiveness of the distance measurement result is determined by obtaining a plurality of distance measurement results from the results measured by a plurality of imaging units. By extinguishing or dimming the projection pattern corresponding to the area for which the distance measurement value is determined to be effective, the area where effective distance measurement can be performed is reduced during the next measurement by reducing the influence of multiple reflections. Can do. In particular, in the first embodiment, since a plurality of imaging units are used, many input images can be obtained, and the measurement accuracy can be improved robustly.

  In the second embodiment, distance measurement of a multiple reflection region is performed using an imaging unit and a plurality of projection units. By projecting with a plurality of projection units by multiplexing light sources, the object to be measured can be imaged brighter, less affected by current noise, etc., and more accurate measurement can be performed.

  In the third embodiment, different measurement pattern groups are projected from the projection unit, and the distance of the multiple reflection region is measured. In this way, measurement can be performed with a small hardware configuration of one projection unit and one imaging unit.

  In the fourth embodiment, the effectiveness of the distance measurement result is determined based on information on the shape of the measurement target object, and the distance of the multiple reflection region is measured. In this way, measurement can be performed with a small hardware configuration of one projection unit and one imaging unit, and only the number of measurement pattern groups is required for the number of projection imaging required for one measurement.

  In the fifth embodiment, the effectiveness of the distance measurement result is determined by projecting a pattern for multiple reflection detection, and the distance of the multiple reflection region is measured. In this way, measurement can be performed with a small hardware configuration of one projection unit and one imaging unit, and the number of projection imaging required for one measurement is the same as the number of measurement pattern groups and multiple reflection detection patterns. It's okay. In addition, it is possible to easily visually grasp a region where multiple reflections are detected.

  In the sixth embodiment, the repetition is terminated according to the total time required for measurement, and the distance measurement of the multiple reflection region is performed. In this way, since the time required for measurement can be easily estimated, the system can be flexibly introduced in accordance with the on-site tact time.

  In the seventh embodiment, the repetition is ended based on the distance measurement result, and the distance measurement of the multiple reflection region is performed. In this way, the repetition can be terminated as soon as the measurement of the necessary part is completed, so that the measurement can be efficiently completed for the measurement purpose.

  In the eighth embodiment, the repetition is terminated based on the past measurement result, and the distance measurement of the multiple reflection region is performed. In this way, it is possible to know the limit of the measurable area when the measurement target object is measured using this system in a certain measurement environment.

DESCRIPTION OF SYMBOLS 1 Projection part 11 Light source 12 Illumination optical system 13 Display element 14 Projection optical system 2A Imaging part 21A Imaging lens 22A Imaging element 2B Imaging part 21B Imaging lens 22B Imaging element 3 Control and calculation process part 31 Projection pattern control part 32 Image acquisition part 33 Distance calculation unit 34 Effectiveness determination unit 35 Repeat control unit 36 Parameter storage unit 4 Object to be measured

Claims (12)

At least one projection means for projecting pattern light onto the measurement target object;
At least one imaging means for imaging the measurement target object onto which the pattern light is projected;
A measuring unit that measures a distance from the projection unit or the imaging unit to the measurement target object based on an image captured by the imaging unit;
Determining means for determining whether the measured distance is valid;
The distance measurement comprising: a control means for quenching or dimming the brightness of the pattern light projected on the area of the projected pattern light, the distance of which is judged to be effective by the judging means apparatus. A plurality of either or both of the projection unit and the imaging unit are provided,
The measuring unit measures a distance from the projecting unit or the imaging unit to the measurement target object,
The determination means determines whether the measured distance is effective based on whether or not the measured distances for each of the images picked up by using one or both of the projection means and the imaging means match. The distance measuring device according to claim 1, wherein it is determined whether or not. The projection means sequentially projects a plurality of pattern lights onto the measurement target object,
The imaging means sequentially images the measurement target object onto which the plurality of pattern lights are sequentially projected for each of the plurality of pattern lights,
The measurement unit measures the distance from the projection unit or the imaging unit to the measurement target object for each image based on the captured images.
2. The determination unit according to claim 1, wherein the determination unit determines whether the measured distance is valid based on whether or not the measured distances match for each of the captured images. 2. The distance measuring device according to 2.   The determination unit determines whether the distance from the imaging unit or the projection unit to the measurement target object, which is measured by the measurement unit, is valid based on information on the shape of the measurement target object. The distance measuring device according to claim 1, wherein   Furthermore, after each means according to claim 1 is executed once, it is provided with an end determination means for determining whether or not a predetermined end determination condition is satisfied, and the end determination means determines that the end determination condition is not satisfied. 5. If any of the above is performed, a series of processes of the projection unit, the imaging unit, the measurement unit, the determination unit, and the control unit are repeatedly performed at least once. The distance measuring device according to item 1.   6. The termination determination unit performs determination based on the number of repetitions of a series of processes performed by the projection unit, the imaging unit, the measurement unit, the determination unit, and the control unit. The distance measuring device described in 1.   6. The termination determination unit performs determination based on a time required for a series of processing performed by the projection unit, the imaging unit, the measurement unit, the determination unit, and the control unit. The distance measuring device described in 1.   The said end determination means performs determination based on the result of the process performed by the said projection means, the said imaging means, the said measurement means, the said determination means, and the said control means, The said determination means is characterized by the above-mentioned. Distance measuring device.   The end determination means performs a determination based on the past measurement result and the current measurement result performed by the projection means, the imaging means, the measurement means, the determination means, and the control means. 6. The distance measuring device according to claim 5, wherein A projection step of projecting pattern light onto the measurement object;
An imaging step of imaging the measurement target object onto which the pattern light is projected;
A measurement step of measuring a distance to the measurement target object based on the image captured in the imaging step;
A determination step of determining whether the measured distance is valid;
A control step of extinguishing or dimming the brightness of the pattern light projected on the region of the projected pattern light that is determined to be effective in the measured distance in the determination step. Characteristic distance measurement method.   A program for realizing the distance measurement method according to claim 10.   The recording medium which recorded the program for implement | achieving the distance measuring method of Claim 10.

Images