JP2014134426A - Three-dimensional measuring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional measuring device that can measure an object partially in three dimensions, greatly shortens a measurement time, and achieves template matching and three-dimensional measurement through one-time image capturing.SOLUTION: An image registration part 26 stores, as a registration image, an image obtained by imaging a part of a workpiece 5 while a texture is not projected. A filter processing part 25 performs processing for lowering a spatial frequency on original images 23, 24 obtained by imaging the workpiece 5 while the texture is projected. A template matching processing part 27 detects a position corresponding to the registration image in the original images 23, 24 having been processed by the filter processing part 25. A three-dimensional processing part 28 calculates three-dimensional information on the workpiece 5 based upon a detection result detected by the template matching processing part 27.

Description

  The present invention relates to a three-dimensional measuring apparatus using a stereo camera, for example.

  Conventionally, as a three-dimensional measuring apparatus, there is one described in JP 2001-147110 A (Patent Document 1). This three-dimensional measuring device projects a pattern onto an object and performs three-dimensional measurement. More specifically, the three-dimensional measuring device calculates a pattern suitability index value indicating the suitability as a pattern of the light projection pattern generated by the light projection pattern generation means based on the generated light projection pattern, By comparing the calculated pattern suitability index value with a predetermined threshold value, it is determined whether or not the generated projection pattern has suitability as a projection pattern for three-dimensional shape measurement.

JP 2001-147110 A

  By the way, in the conventional three-dimensional measuring apparatus, when a target is three-dimensionally measured with a stereo camera, a pattern is projected to make it easy to obtain corresponding points in the left and right images, but a pattern or texture (pattern) is projected. When performing three-dimensional measurement, there are the following problems.

  When it is desired to measure only a desired part of the object, the measurement time can be greatly reduced by specifying the desired part and measuring only that part rather than measuring the entire object. There is a method called template matching as a method for specifying a desired part. Template matching using a desired part as a registered image is performed to specify the position of the part in the image, and only that part may be three-dimensionally processed.

  In this case, a case where an image projected with a pattern or texture is prepared as a registered image and a case where an image not projected is prepared are considered. Light must be projected so that the position of the pattern or texture projected on a desired part at the time of measurement matches the position of the pattern or texture on the registered image, and a very complicated process is required for the adjustment.

  On the other hand, if an image that is not texture-projected is prepared as a registered image, the texture that is projected onto a desired part during three-dimensional measurement interferes with template matching, and an accurate matching result cannot be obtained. In order to avoid this, at the time of three-dimensional measurement, it is necessary to pick up an image that does not project texture once and perform template matching, and then to pick up the image that has been texture-projected again and perform three-dimensional measurement. That is, it is necessary to shoot twice.

  Accordingly, an object of the present invention is to provide a three-dimensional measurement apparatus that enables partial three-dimensional measurement of an object, greatly reduces the measurement time, and enables template matching and three-dimensional measurement with a single photographing. It is to provide.

In order to solve the above problems, the three-dimensional measurement apparatus of the present invention
An imaging means for imaging an object;
Illumination means for projecting a texture onto the object;
Image registration means for storing, as a registered image, an image captured by the imaging means in a state in which at least a part including the measurement point of the object is not projected from the illumination means and the texture;
Pre-processing means for performing a process of reducing the spatial frequency on the original image captured by the imaging means in a state where the texture is projected from the illumination means on the object;
Image position detecting means for detecting a position corresponding to the registered image from the original image processed by the preprocessing means;
3D processing means for calculating 3D information of the object based on the original image picked up by the image pickup means and a detection result detected by the image position detection means. .

  According to the three-dimensional measurement apparatus of the present invention, the image registration means stores a registered image of an object on which no texture is projected. The preprocessing means performs a process of reducing the spatial frequency on the original image of the target object on which the texture is projected. The image position detecting means detects a position corresponding to the registered image from the original image processed by the preprocessing means. The three-dimensional processing means calculates three-dimensional information of the object based on the original image picked up by the image pickup means and the detection result detected by the image position detecting means.

  As described above, an image on which no texture is projected is prepared as the registered image. However, since the original image is in a state in which the texture is projected, the texture component is reduced by the process of reducing the spatial frequency of the original image. Thus, template matching between the processed original image and the registered image can be performed.

  Therefore, partial three-dimensional measurement of the object is possible, and the measurement time is greatly shortened. Also, template matching and three-dimensional measurement can be performed with a single shooting.

  In the three-dimensional measurement apparatus according to one embodiment, the three-dimensional processing unit is configured to apply the object only to a position corresponding to the measurement point of the object detected by the image position detection unit and a peripheral portion of the position. Calculate three-dimensional information of an object.

  According to the three-dimensional measurement apparatus of this embodiment, the three-dimensional processing unit is configured to detect only the position corresponding to the measurement point of the target detected by the image position detection unit and the peripheral portion of the target. Three-dimensional information is calculated. Thereby, a target object can be partially measured three-dimensionally and measurement time can be further shortened.

  In the three-dimensional measurement apparatus according to an embodiment, the three-dimensional processing unit performs three-dimensional measurement in a measurement area including a position corresponding to the measurement point of the object detected by the image position detection unit. Then, from the three-dimensional measurement value in this measurement area, the three-dimensional measurement value of the position corresponding to the measurement point of the object detected by the image position detection means is calculated.

  According to the three-dimensional measurement apparatus of this embodiment, the three-dimensional processing means performs three-dimensional measurement in a measurement area including a position corresponding to the measurement point of the object detected by the image position detection means. A three-dimensional measurement value at a position corresponding to the measurement point detected by the image position detection means is calculated from the three-dimensional measurement value in the measurement area.

  Thereby, even if noise or the like exists at a position corresponding to the measurement point detected by the image position detection means, only the position corresponding to the measurement point detected by the image position detection means is not three-dimensionally measured. The influence of noise and the like can be reduced.

  In the three-dimensional measurement apparatus according to an embodiment, the process of reducing the spatial frequency of the preprocessing unit is a spatial filter process or a resolution reduction process.

  According to the three-dimensional measurement apparatus of this embodiment, the process of reducing the spatial frequency of the preprocessing means is a spatial filter process or a resolution reduction process. In the case of low resolution processing, the processing time can be shortened.

In the three-dimensional measurement apparatus according to the embodiment,
The imaging means is a stereo camera including two two-dimensional cameras,
The preprocessing means performs a process of reducing the spatial frequency on the right image or the left image of the original image of the object from the stereo camera,
The image position detecting means detects a position corresponding to the registered image from the right image or the left image processed by the preprocessing means.

  According to the three-dimensional measurement apparatus of this embodiment, the imaging unit is a stereo camera, and the preprocessing unit calculates a spatial frequency with respect to the right image or the left image of the original image of the object from the stereo camera. Apply processing to reduce. The image position detecting means detects a position corresponding to the registered image from the right image or the left image processed by the preprocessing means. Thereby, based on the right image or the left image from a stereo camera, the partial three-dimensional measurement of a target object is attained.

  According to the three-dimensional measurement apparatus of the present invention, the image registration means stores a registered image of an object on which no texture is projected. The preprocessing means performs a process of reducing the spatial frequency on the original image of the target object on which the texture is projected. The image position detecting means detects a position corresponding to the registered image from the original image processed by the preprocessing means. The three-dimensional processing means calculates three-dimensional information of the object based on the original image picked up by the image pickup means and the detection result detected by the image position detecting means. As a result, partial three-dimensional measurement of the target object is possible, the measurement time is greatly shortened, and template matching and three-dimensional measurement can be performed by one shooting.

It is a block diagram which shows the three-dimensional measuring apparatus of 1st Embodiment of this invention. It is explanatory drawing explaining arrangement | positioning of L camera, R camera, a projector, and a workpiece | work. It is explanatory drawing explaining a texture. It is a block diagram which shows a three-dimensional process part. It is explanatory drawing explaining a registration image. It is explanatory drawing explaining the filter process by a filter process part. It is explanatory drawing explaining the template matching process by a template matching process part. It is a block diagram which shows the three-dimensional measuring apparatus of 2nd Embodiment of this invention. It is explanatory drawing explaining the resolution reduction process by a resolution reduction process part. It is explanatory drawing explaining the three-dimensional process of the three-dimensional measuring apparatus of 3rd Embodiment of this invention.

  Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

(First embodiment)
FIG. 1 is a block diagram showing a three-dimensional measuring apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the three-dimensional measuring apparatus includes a stereo camera 10 as an imaging unit, a projector 3 as an illumination unit, an image registration unit 26 as an image registration unit, and a filter processing unit as a preprocessing unit. 25, a template matching processing unit 27 as an image position detecting unit, and a three-dimensional processing unit 28 as a three-dimensional processing unit.

  The stereo camera 10 has an L camera 1 for a left image and an R camera 2 for a right image. The L camera 1 and the R camera 2 are two-dimensional cameras and pick up an image of the workpiece 5 as an object. A CCD or CMOS sensor may be used as the image sensor of the cameras 1 and 2, and an image is formed on the image sensor using an appropriate optical system such as a lens.

  The projector 3 projects illumination light onto an area captured by the L camera 1 and the R camera 2. Each of the left and right images captured by the L camera 1 and the R camera 2 is input to the gray scale conversion unit 21 and the image is converted to a gray scale. Gray scale makes it possible to reduce the amount of information compared to processing with a color image, so that the processing load can be reduced and the cost of the processing unit can be reduced and the processing time can be shortened. If there is no problem with the processing capability, the gray scale unit 21 may be omitted and the subsequent processing may be performed with a color image.

  Next, the left and right grayscale images are corrected to a parallelized image that is easily subjected to lens distortion and stereo matching by the parallelization processing unit 22. The left parallelized image 23 obtained here is subjected to low-pass filter processing by the filter processing unit 25.

  The image registration unit 26 stores an image for template matching. In the template matching processing unit 27, the image registered in the image registration unit 26 is the most similar to the image processed by the filter processing unit 25. Extract the location of a close image.

  The three-dimensional processing unit 28 executes stereo three-dimensional measurement processing from the left parallelized image 23 and the right parallelized image 24. However, the three-dimensional measurement process execution area is executed only in the area matched by template matching.

  The measurement result output unit 29 is a part that outputs three-dimensional measurement result data to the overall control unit 30. The robot arm 6 is for moving and fixing the L camera 1, the R camera 2, and the projector 3 attached to the tip of the arm to a predetermined position with respect to the work 5. The overall control unit 30 performs appropriate execution control of each member. In addition, although the image which performs the filter process part 25 was made into the left parallelization image, it is good also as a right parallelization image.

  FIG. 2 shows an arrangement example of the L camera 1, the R camera 2, the projector 3, and the work 5. As shown in FIG. 2, the L camera 1, the R camera 2, and the projector 3 constitute a camera unit 4. Since the L camera 1 and the R camera 2 constitute a stereo camera, the imaging ranges of the cameras 1 and 2 are optically arranged so that a common imaging portion exists. Further, the projection range by the projector 3 is projected so as to include the imaging range of the L camera 1 and the R camera 2 or the common imaging part of the cameras 1 and 2. The illumination light from the projector 3 projects a figure including a texture (pattern). FIG. 3 shows an example of the texture. This is a texture in which dots are arranged at random positions in a plane.

  Here, the effect of the texture projected by the projector 3 will be described. When stereo measurement is performed from the left and right camera images, feature points on the left and right images are measured, but when there are few feature points in the image, the number of measurement points is reduced. That is, the planar resolution of the three-dimensional measurement point becomes coarse. When the planar resolution becomes coarse, the shape of the workpiece 5 is not sufficiently measured. Therefore, a texture that can be a feature point from the outside is projected onto the subject so that a sufficient number of feature points can be obtained. The projector 3 projects this. The projection range of the projector 3 needs to be projected so as to include the imaging range of the L camera 1 and the R camera 2 or the common imaging part of the cameras 1 and 2.

  FIG. 4 shows a block diagram of the three-dimensional processing unit 28. As illustrated in FIG. 4, the three-dimensional processing unit 28 includes a differentiation processing unit 32, a corresponding point search processing unit 33, and a distance image generation processing unit 34. The corresponding point search processing unit 33 searches for the same feature point from the right and left parallelized images 23 and 24 as the original image using the texture projected by the projector 3 as a feature point (referred to as a corresponding point search process). The corresponding point search process is a process for specifying the coordinates of the same feature point on the left and right images, and there is a correlation block matching method or the like as the method. The differential processing unit 32 can emphasize edge information that tends to be a feature point by executing the differential processing before the corresponding point search processing, and is effective in ensuring the accuracy of the corresponding point search processing.

  Once the coordinates of each feature point on the left and right images are specified, the position of each feature point from the camera (generally, the optical center of either the left or right camera is the origin based on the optical specifications of the camera and the principle of stereo measurement. 3D coordinates) is calculated by the distance image generation processing unit 34, and three-dimensional data, that is, a distance image is obtained by the distance image generation processing unit 34.

  The image registration unit 26 stores a registered image. This registered image will be described with reference to FIG. A workpiece having a shape as shown in the three views of FIG. 5A is a workpiece 5, and a three-dimensional position from the cameras 1 and 2 is measured at a point indicated by a measurement point P in the workpiece 5. . Assume that the cameras 1 and 2 are arranged so as to photograph the upper surface of the work 5 as shown in FIG. An example of a registration image generation procedure in this case will be described.

  First, an upper surface of an actual work or a sample work is photographed to obtain an image shown in FIG. At this time, photographing under texture illumination is not performed, and normal illumination is performed. Second, edge enhancement processing (differentiation processing) is performed to enhance the feature of the shape, and the image shown in FIG. 5C is obtained. 3rdly, the image of FIG.5 (d) which cut out the part containing the measurement point P from the obtained image is stored in the image registration part 26 as a registration image. Here, in the case where sufficient features cannot be obtained by cutting out a part of the image, the registered image may be used as a whole instead of the part.

  A method for measuring the three-dimensional position of the measurement point P of the workpiece 5 using the registered image will be described with reference to FIG. As shown in FIG. 2, it is assumed that the workpiece 5 is imaged in a state in which a texture is projected from the projector 3, and the left parallelized image 23 (as an original image) obtained thereby is as shown in FIG. A low-pass filter is executed on the image 23 by the filter processing unit 25 to obtain an image with reduced texture contrast as shown in FIG. At this point, the sharpness of the contour of the work 5 is somewhat reduced. If necessary, the sharpness of the contour of the work 5 is restored by applying a sharpening process as shown in FIG. Also good. However, it is necessary not to increase the texture contrast too much.

  Next, as shown in FIG. 7, a template matching process is performed with an image registered in the image registration unit 26 from the image obtained by the filter processing by the filter processing unit 25. Thereby, the coordinates corresponding to the measurement point P can be detected from the image obtained by the filtering process, that is, the left parallelized image 23. When the coordinates in the image of the measurement point P can be detected, the coordinates in the three-dimensional space of the measurement point P can be measured by executing the three-dimensional measurement already described for the measurement point P.

  The reason for using the normal illumination instead of taking the image under the texture illumination when creating the registered image will be described. Actually, when performing three-dimensional measurement, texture illumination is projected. However, if the texture position projected on the workpiece and the texture position of the registered image are different, template matching will not be performed well, and matching position detection will not be performed. Inaccurate or undetectable. Alternatively, if the texture position projected on the workpiece can be made the same position as the registered image, the template matching will be successful. And an automatic mechanism for determining the degree of coincidence of texture positions is required, resulting in a very complicated device. Therefore, in the present invention, the registered image is an image without a texture, and the template matching process can be performed by adopting a method of reducing the texture component from the image used for measurement.

  According to the three-dimensional measurement apparatus having the above configuration, the image registration unit 26 registers an image captured by the stereo camera 10 in a state in which at least a part including the measurement point P of the workpiece 5 is not projected from the projector 3. Store as an image. The filter processing unit 25 performs a process of reducing the spatial frequency on the original images (left and right parallelized images 23 and 24) captured by the stereo camera 10 in a state where the texture of the workpiece 5 is projected from the projector 3. The template matching processing unit 27 detects a position corresponding to the registered image from the original images 23 and 24 processed by the filter processing unit 25. The three-dimensional processing unit 28 calculates three-dimensional information of the work 5 based on the original images 23 and 24 captured by the stereo camera 10 and the detection result detected by the template matching processing unit 27.

  As described above, an image on which a texture is not projected is prepared as a registered image. However, since the original images 23 and 24 are in a state in which the texture is projected, the processing of reducing the spatial frequency of the original images 23 and 24 is performed. By reducing the texture component, template matching between the processed original images 23 and 24 and the registered image can be performed.

  Therefore, partial three-dimensional measurement of the workpiece 5 is possible, and the measurement time is greatly shortened. Also, template matching and three-dimensional measurement can be performed with a single shooting.

  Further, the three-dimensional processing unit 28 calculates the three-dimensional information of the workpiece 5 only for the position corresponding to the measurement point P of the workpiece 5 detected by the template matching processing unit 27 and the peripheral portion of this position. Thereby, the workpiece | work 5 can be partially measured three-dimensionally and measurement time can be shortened further.

(Second Embodiment)
FIG. 8 is a block diagram showing a three-dimensional measuring apparatus according to the second embodiment of the present invention. The second embodiment is different from the first embodiment only in the configuration of the image processing unit. Only this different configuration will be described below. In the second embodiment, the same reference numerals as those in the first embodiment are the same as those in the first embodiment, and the description thereof is omitted.

  As shown in FIG. 8, the image processing unit 20A includes a resolution reduction processing unit 35 instead of the filter processing unit 25 of the image processing unit 20 of the first embodiment.

  The resolution reduction processing unit 35 performs resolution reduction processing on the left parallelized image 23. When the image change is described with reference to FIG. 9, when there is a texture in units of one pixel in 8 × 10 pixels as shown in FIG. 9A, four adjacent pixels in the image are regarded as one pixel, and the pixel When the value is converted into an image obtained by averaging the density values of the original four pixels, it is as shown in FIG. The number of pixels in FIG. 9B is reduced to 4 × 5 pixels, and since the unit is 4 pixels of the original image, the spatial frequency is reduced and the texture contrast is lowered. Thereby, the same effect as a low-pass filter is acquired. In addition, since the reduction in resolution can reduce the number of pixels, an effect of shortening the processing time can be obtained.

(Third embodiment)
FIG. 10 is an explanatory diagram for explaining the three-dimensional processing of the three-dimensional measuring apparatus according to the third embodiment of the present invention. The third embodiment is different from the first embodiment in three-dimensional processing. Only this different configuration will be described below.

  In the first embodiment, when the in-image coordinates of the measurement point P are detected by template matching, three-dimensional measurement is performed for the point P. However, if noise or the like exists on the point P for some reason, the three-dimensional measurement is accurate. The measured value cannot be obtained.

  In order to prevent this, the pixels in the surrounding measurement area including the coordinates of the detected measurement point P in the image are also three-dimensionally measured, and the average value is set as the three-dimensional measurement value of the measurement point P, thereby affecting the influence of noise and the like. Can be reduced.

  As shown in FIG. 10, a predetermined measurement area Z centering on the measurement point P is set, and the inside of the measurement area Z is measured three-dimensionally. An invalid pixel MG is extracted from the obtained three-dimensional coordinates of each pixel. The invalid pixel MG is, for example, a pixel that is clearly measured to be out of the position where the workpiece 5 exists as a distance value. The average value of the three-dimensional measurement values of the remaining effective pixels YG is calculated and used as the measurement result of the measurement point P. As the measurement area Z, it is desirable to keep the measurement area P at a site where the difference in distance is small.

  In short, the three-dimensional processing unit of the third embodiment performs three-dimensional measurement in the measurement area Z including the position corresponding to the measurement point P of the workpiece 5 detected by the template matching processing unit 27, and this measurement area A three-dimensional measurement value at a position corresponding to the measurement point P detected by the template matching processing unit 27 is calculated from the three-dimensional measurement value in Z.

  As a result, even if noise or the like is present at the position corresponding to the measurement point P detected by the template matching processing unit 27, only the position corresponding to the measurement point P detected by the template matching processing unit 27 is three-dimensionally displayed. Since measurement is not performed, the influence of noise or the like can be reduced.

  In addition, this invention is not limited to the above-mentioned embodiment. For example, the feature points of the first to third embodiments may be variously combined.

1 L camera 2 R camera 3 Projector (illumination means)
5 Workpiece (object)
10 Stereo camera (imaging means)
20, 20A Image processing unit 23 Left parallelized image (original image)
24 Right parallelized image (original image)
25 Filter processing section (preprocessing means)
26 Image registration unit (image registration means)
27 Template matching processing unit (image position detection means)
28 3D processing unit (3D processing means)
35 Low resolution processing unit (pre-processing means)
P Measurement point Z Measurement area

Claims (5)

An imaging means for imaging an object;
Illumination means for projecting a texture onto the object;
Image registration means for storing, as a registered image, an image captured by the imaging means in a state in which at least a part including the measurement point of the object is not projected from the illumination means and the texture;
Pre-processing means for performing a process of reducing the spatial frequency on the original image captured by the imaging means in a state where the texture is projected from the illumination means on the object;
Image position detecting means for detecting a position corresponding to the registered image from the original image processed by the preprocessing means;
And a three-dimensional processing unit that calculates three-dimensional information of the object based on the original image captured by the imaging unit and a detection result detected by the image position detection unit. Three-dimensional measuring device. The three-dimensional measurement apparatus according to claim 1,
The three-dimensional processing means calculates the three-dimensional information of the object only for a position corresponding to the measurement point of the object detected by the image position detection means and a peripheral portion of the position. 3D measuring device. The three-dimensional measuring apparatus according to claim 1 or 2,
The three-dimensional processing means performs three-dimensional measurement in a measurement area including a position corresponding to the measurement point of the object detected by the image position detection means, and from the three-dimensional measurement value in the measurement area, A three-dimensional measurement apparatus, wherein a three-dimensional measurement value of a position corresponding to the measurement point of the object detected by the image position detection means is calculated. In the three-dimensional measuring apparatus as described in any one of Claim 1 to 3,
3. The three-dimensional measuring apparatus according to claim 1, wherein the process of reducing the spatial frequency of the preprocessing means is a spatial filter process or a resolution reduction process. In the three-dimensional measuring apparatus as described in any one of Claim 1 to 4,
The imaging means is a stereo camera including two two-dimensional cameras,
The preprocessing means performs a process of reducing the spatial frequency on the right image or the left image of the original image of the object from the stereo camera,
The three-dimensional measuring apparatus, wherein the image position detecting means detects a position corresponding to the registered image from the right image or the left image processed by the preprocessing means.

Images