JP2024055569A - Three-dimensional measurement device, three-dimensional measurement method, program, system, and article manufacturing method - Google Patents

Abstract

[Problem] To provide a three-dimensional measuring device capable of calculating a distance value to an object with high accuracy. [Solution] The three-dimensional measuring device has a plurality of image capturing units that capture images of the object from different viewpoints, and a processing unit that calculates a distance value to the object by performing correspondence using a first image group obtained by the plurality of image capturing units under a first condition and a second image group obtained by the object under a second condition different from the first condition, the processing unit performs correspondence between the first and second image groups based on at least one of information of image characteristics based on the wavelength of the illumination light in each of the first and second image groups, image characteristics caused by the light source of the illumination light, and size of features caused by the pattern of the illumination light, or performs correspondence between the first and second image groups after performing processing on the first and second image groups before the correspondence based on the information, and calculates the distance value using a parallax value obtained by the correspondence. [Selected Figure] Figure 4

Description

The present invention relates to a three-dimensional measuring device, a three-dimensional measuring method, a program, a system, and a method for manufacturing an article.

Traditionally, in factory production lines, robots equipped with 3D measurement devices have been used to assemble and inspect products. A conventional 3D measurement device is known to use stereo measurement, which searches for corresponding points between images taken by two cameras with different viewpoints, and measures distance points based on the principles of triangulation using the parallax information of the corresponding points.

Patent document 1 discloses a method that uses two cameras in different positions and a projector that projects one pattern, calculates a first distance map and a second distance map, and performs weighted voting based on the reliability of the maps to calculate a final distance map.

The distance map is measured by the principle of triangulation based on the correspondence between the dot pattern on the image captured by one camera and the dot pattern on the projector, after projecting a dot pattern on the object to be measured. The first distance map is also called the active stereo method, since it is a method that actively creates matching clues. The second distance map is a distance map measured in stereo from images captured by two different cameras at different distances, without projecting a dot pattern on the object to be measured. The second distance map is also called the passive stereo method, since it is a method that passively performs matching.

By combining active and passive stereo in this way, it becomes possible to measure depth maps for a variety of objects, including those with little texture, achieving more accurate measurements.

U.S. Pat. No. 1,063,6155

However, conventional methods use a method in which matching is performed separately for both the active stereo method and the passive stereo method, and then the distance is calculated once. As a result, when the correspondence cannot be determined correctly for each method, even if the information is combined in post-processing, the accuracy is insufficient, and there is room for improvement.

Therefore, the present invention aims to provide a three-dimensional measuring device that can calculate the distance to an object with high accuracy.

In order to achieve the above object, a three-dimensional measuring device according to one aspect of the present invention has a plurality of imaging units that capture images of an object from different viewpoints, and a processing unit that calculates a distance value to the object by performing correspondence between a first image group obtained by the plurality of imaging units under a first condition and a second image group obtained by the object under a second condition different from the first condition, and the processing unit performs correspondence between the first and second image groups based on at least one piece of information among image characteristics based on the wavelength of the illumination light in each of the images of the first and second image groups, image characteristics caused by the light source of the illumination light, and size of features caused by the pattern of the illumination light, or performs correspondence between the first and second image groups after performing processing on the first and second image groups before the correspondence based on the information, and calculates the distance value using the parallax value obtained by the correspondence.

The present invention provides a three-dimensional measuring device that can calculate the distance to an object with high accuracy.

1 is a diagram showing a three-dimensional measuring apparatus according to a first embodiment; 4 is a flowchart showing an operation process of the three-dimensional measuring apparatus according to the first embodiment. 3A and 3B are diagrams illustrating examples of an active image and a passive image acquired by a first imaging unit and a second imaging unit according to the first embodiment. 11 is a flowchart showing a distance value calculation process in a processing unit according to the first embodiment. 10 is a flowchart showing an operation process of the three-dimensional measuring apparatus according to the second embodiment. FIG. 11 is a diagram showing an example of an image synthesized using an active image and a passive image according to the second embodiment. 13 is a flowchart showing a distance value calculation process in a processing unit according to the third embodiment. 13A and 13B are diagrams illustrating an example of an active image acquired by a first imaging unit and a second imaging unit according to a third embodiment. FIG. 1 is a diagram showing an example of a control system including a gripping device equipped with a three-dimensional measuring device.

Below, the mode for carrying out the present invention will be described in detail with reference to the attached drawings. The embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiment.

Example 1
In the first embodiment, a method for calculating a distance value from an active image and a passive image captured by one of a plurality of image capturing units (the same image capturing unit) by integrating and using them together in a matching evaluation value so that both images work effectively will be described. Specifically, after performing preprocessing such as aberration correction and noise removal processing in consideration of the characteristics of the active image and the passive image, an integrated evaluation value is calculated using an area based on the size of the features of each image. This makes it possible to calculate the distance with higher accuracy than before.

An active image is an image taken by shining a pattern light onto the surface of the object to be measured, in which the pattern (illumination light pattern) projected onto the surface of the object to be measured is observed.A passive image is an image taken without shining a pattern light onto the surface of the object to be measured, in which no pattern (illumination light pattern) is projected onto the surface of the object to be measured, in which the three-dimensional structure of the object to be measured and the texture on its surface are observed.Hereinafter, stereo measurement from images taken with two different cameras while shining a pattern light will be referred to as active measurement, and hereinafter, stereo measurement from images taken with two different cameras without shining a pattern light will also be referred to as passive measurement.

FIG. 1 is a diagram illustrating a three-dimensional measuring device 100 according to the first embodiment. The three-dimensional measuring device 100 according to the first embodiment is composed of a control unit 101, a projection unit 102, a first imaging unit 103, a second imaging unit 104, and a processing unit 105. FIG. 1 also illustrates an example of a measurement object 106 measured by the three-dimensional measuring device 100 according to the first embodiment.

The control unit 101 includes a CPU, memory (storage unit), etc., and is composed of at least one computer. The control unit 101 controls various operations and the start and end of operations of the projection unit 102, first image capture unit 103, second image capture unit 104, and processing unit 105 so that the three-dimensional measurement device 100 outputs the distance value of the measurement object 106. Specifically, the control unit 101 sends a pattern light projection start command and a pattern light projection end command to the projection unit 102, and sends an image capture start command and an image capture end command to the first image capture unit 103 and the second image capture unit 104. A command to start distance value calculation, etc. is sent to the processing unit 105.

When the projection unit 102 receives a pattern light projection start command from the control unit 101, it irradiates the pattern light as illumination light having a pattern until it receives a pattern light projection end command from the control unit 101, and projects the pattern light onto the surface of the measurement object 106. It is desirable that the pattern light irradiated at this time has a non-periodic intensity pattern based on uniform random numbers or normal random numbers. Examples of light sources for the pattern light include, but are not limited to, lasers and LEDs. Note that it is sufficient that the wavelength of the projected pattern light is included in a wavelength band to which the first imaging unit 103 and the second imaging unit 104 are sensitive, and that the pattern on the surface of the measurement object 106 can be observed in the active image.

The first imaging unit 103 and the second imaging unit 104 are arranged at least at different positions or angles relative to the measurement object 106, and the first imaging unit 103 and the second imaging unit 104 image the measurement object 106 from different viewpoints. When the first imaging unit 103 and the second imaging unit 104 receive an imaging start command from the control unit 101, they start imaging the measurement object 106. Then, they capture images in each line of sight direction until they receive an imaging end command from the control unit 101, and output the acquired image data to the processing unit 105. At this time, the timing of capturing the images captured by the first imaging unit 103 and the second imaging unit 104 is controlled so as to be synchronized with each other.

When the pattern light projected from the projection unit 102 is near-infrared light, the wavelength bands received as reflected light from the measurement object 106 differ between the active image and the passive image. As a result, a position shift occurs between the images observed in the active image and the passive image due to chromatic aberration. As a means for reducing the effects of chromatic aberration, the processing unit 105 corrects the chromatic aberration using calibration parameters for active measurement and passive measurement, respectively. The sensitivity of the first imaging unit 103 and the second imaging unit 104 needs to be sensitive to the wavelength band of the pattern light projected from the projection unit 102 and the wavelength band of visible light.

When the processing unit 105 receives each image data as a group of images of the measurement object 106 from the first imaging unit 103 and the second imaging unit 104 and receives a distance calculation start command from the control unit 101, the processing unit 105 performs a distance value calculation process based on each received image data. Specifically, the processing unit 105 performs a process of matching the image data of the measurement object 106 captured by the first imaging unit 103 and the second imaging unit 104, and calculates a distance value to the measurement object 106 using the matched image data. The process of calculating the distance value to the measurement object 106 in the processing unit 105 will be described in detail later.

The processing unit 105 also holds calibration parameters for the first imaging unit 103 and the second imaging unit 104 as data necessary for performing distance value calculation processing. Here, the calibration parameters include parameters representing the relative position and orientation of the first imaging unit 103 and the second imaging unit 104, parameters representing the image center and focal length of each imaging unit, and distortion aberration. Note that when the wavelength of the pattern light projected from the projection unit 102 during active image capture is near-infrared light, calibration parameters for active measurement and calibration parameters for passive measurement are respectively held. Alternatively, only the magnification difference based on the wavelength difference between the two may be held.

FIG. 2 is a flowchart showing the operation (processing) up to calculating the distance value of the measurement object 106 by the three-dimensional measuring device 100 according to the first embodiment. Note that each operation (processing) shown in the flowchart in FIG. 2 is controlled by the control unit 101 executing a computer program.

First, in step S1101, the control unit 101 sends a pattern light projection start command to the projection unit 102, and the projection unit 102 irradiates the pattern light. This causes the pattern to be projected onto the surface of the measurement object 106.

Next, in step S1102, the control unit 101 sends an imaging start command to the first imaging unit 103 and the second imaging unit 104. When the imaging start command is sent to each imaging unit that performs imaging, the first imaging unit 103 and the second imaging unit 104 capture a scene under a first condition in which a pattern of pattern light is projected onto the surface of the measurement object 106 from each viewpoint (imaging process). This imaging process is performed until an imaging end command is sent from the control unit 101. This step makes it possible to obtain a group of active images (first image group) captured by each imaging unit under the first condition in which a pattern of pattern light is projected onto the surface of the measurement object 106. In this step, the image data of each active image (first image group) captured and acquired by the first imaging unit 103 and the second imaging unit 104 is sent to the processing unit 105.

Figure 3 shows an example of a conceptual diagram of an active image and a passive image acquired by the first imaging unit 103 and the second imaging unit 104 according to the first embodiment. Figure 3 (A) is an example of a conceptual diagram of an active image acquired by the first imaging unit 103 and the second imaging unit 104. Figure 3 (B) is an example of a conceptual diagram of a passive image acquired by the first imaging unit 103 and the second imaging unit 104.

The first active image 110 shown in FIG. 3(A) is a conceptual diagram of an active image captured by the first imaging unit 103, and the second active image 111 is a conceptual diagram of an active image captured by the second imaging unit 104. In the first active image 110 and the second active image 111 shown in FIG. 3(A), the pattern observed on the surface of the measurement object 106 is used as a clue for calculating a parallax value in the process of calculating the distance value to the measurement object 106, which will be described later.

Next, in step S1103, the control unit 101 sends a pattern light projection end command to the projection unit 102, and the projection unit 102 stops emitting the pattern light and ends the projection of the pattern light onto the measurement object 106. Here, in this step, the control unit 101 switches the irradiation condition (projection condition) from a first condition in which the pattern light is irradiated to a second condition in which the pattern light is not irradiated.

Next, in step S1104, an image capture start command is sent from the control unit 101 to the first image capture unit 103 and the second image capture unit 104. When the image capture start command is sent to each image capture unit that performs image capture, the first image capture unit 103 and the second image capture unit 104 capture a scene from each viewpoint under a second condition in which the pattern of the pattern light is not projected onto the surface of the measurement object 106 (image capture process). This image capture process is performed until an image capture end command is sent from the control unit 101. At this time, the first image capture unit 103 and the second image capture unit 104 receive visible light reflected from the measurement object 106. This step makes it possible to obtain a group of passive images (a second group of images) captured under the second condition in which each image capture unit does not project the pattern light onto the measurement object 106. In this step, the first imaging unit 103 and the second imaging unit 104 capture images, and the image data of each acquired passive image (second image group) is sent to the processing unit 105.

The first passive image 112 shown in FIG. 3B is a conceptual diagram of a passive image captured by the first imaging unit 103, and the second passive image 113 is a conceptual diagram of a passive image captured by the second imaging unit 104. Features such as steps, edges, and surface textures resulting from the three-dimensional structure of the measurement object 106 observed in the first passive image 112 and the second passive image 113 are used as clues for calculating a parallax value in the process of calculating the distance value to the measurement object 106.

The operational flow from step S1101 to step S1104 described so far is a flow for capturing images in the order of active image and passive image, but this order may be changed as desired. For example, when capturing images in the order of passive image and active image, various processes are executed in the order of steps S1104, S1102, and S1103.

Next, in step S1105, the control unit 101 sends a distance calculation start command to the processing unit 105, and the processing unit 105 calculates a distance value using each image data of the active image and each image data of the passive image, as well as the calibration parameters (processing step).

Fig. 4 is a flowchart showing the specific operations of the process for calculating the distance value of the measurement object 106, which is performed by the processing unit 105. That is, Fig. 4 is a flowchart explaining the details of the process of step S1105 shown in Fig. 2. Note that each operation (process) shown in the flowchart in Fig. 4 is controlled by the control unit 101 executing a computer program.

First, in step S1111, noise removal processing is performed on the first active image 110 and the second active image 111. For example, when capturing an active image, if pattern light using a near-infrared laser as a light source is irradiated, an irregular spotted pattern (speckle noise) occurs due to interference between scattered lights, which is an image characteristic caused by the light source of the illumination light. Speckle noise is observed as a pattern that differs depending on the position, and therefore causes a decrease in the accuracy of stereo matching. As a countermeasure, it is effective to use a smoothing filter such as a Gaussian filter or a median filter.

By optimizing the size of the smoothing filter, for example by setting it to approximately the size of the speckles observed in the first active image 110 and the second active image 111, a higher noise removal effect can be achieved. On the other hand, since speckle noise does not occur in passive images and noise removal processing is not necessary, noise removal processing is performed on at least the active images in the first image group. Alternatively, a smoothing filter with a kernel size of about several pixels can be applied so that shot noise can be removed.

Next, in step S1112, aberration (distortion) occurring in the first passive image 112, the second passive image 113, and the first active image 110 and the second active image 111 after noise removal is corrected (distortion correction is performed). The distortion correction is performed on at least one of the active image and the passive image. Furthermore, parallelization processing is performed on the first active image 110 and the second active image 111, the first passive image 112, and the second passive image 113. Calibration parameters of the first imaging unit 103 and the second imaging unit 104 are used to correct the aberration.

When the wavelength of the illumination light received by the first imaging unit 103 and the second imaging unit 104 differs between active and passive image capture, it is preferable to use different calibration parameters to reduce the effects of chromatic aberration. Specifically, it is preferable to use different calibration parameters for active and passive measurement. For example, when capturing an active image by irradiating a near-infrared pattern light, the calibration parameters for the active image are calculated in advance using a captured image irradiated with near-infrared illumination.

When capturing a passive image using visible light, calibration parameters for the passive image are calculated using the image captured under visible light illumination. The calculated calibration parameters for active measurement are then used for aberration correction in the active image, and the calculated calibration parameters for passive measurement are used for aberration correction in the passive image. This makes it possible to reduce (or essentially eliminate) the effects of aberration, which is an image characteristic based on the wavelength of the illumination light, in the collimated image. By performing the above steps S1111 and S1112, image correction can be performed as pre-processing prior to the matching described below.

Next, in step S1113, an evaluation value is calculated using the active image and the passive image to evaluate the similarity for each combination of pixels of interest between each image captured by the first imaging unit 103 and each image captured by the second imaging unit 104.

Here, a method of using SAD (Sum of Absolute Difference) will be described as a method of calculating the evaluation value. The luminance values of the pixel of interest (x, y) in the first active image and the second active image are _ILA (x, y) and _IRA (x, y), respectively, and the luminance values of the pixel of interest (x, y) in the first passive image and the second passive image are _ILP (x, y) and _IRP (x, y), respectively. In this case, the evaluation value SAD _A in which the similarity of the active image is the evaluation value, and the evaluation value SAD _P in which the similarity of the passive image is the evaluation value are expressed by the following formulas (1) and (2), respectively.

The smaller the evaluation value SAD _A in the active image and the evaluation value SAD _P in the passive image, the more similar the periphery of the region of interest is. The window size for calculating the similarity is set so that the features of each image are appropriately included according to the density of the projection pattern (illumination light pattern) in the active image and the size of the features of the measurement target object in the passive image, thereby improving the accuracy of the correspondence.

For example, as shown in Fig. 3, the width wa and height ha of a window 107 for calculating an evaluation value SAD _A in an active image are set so that at least one dot pattern is included within the window. Furthermore, the width wp and height wp of a window 108 for calculating an evaluation value SAD _P in a passive image are set as small as possible within a range where matching is possible so that correspondence can be performed making use of the fine structure of the measurement object. In this way, the accuracy of correspondence is improved.

The size of the window used for matching strongly affects the horizontal resolution of the distance map obtained as a result of matching, so it is best to make it as small as possible while still allowing stable matching. Note that in addition to the size of the region, the number of pixels used within the region may also be changed.

尚、アクティブ画像とパッシブ画像の重要度に応じて、上記の式（３）の代わりに、以下の式（４）で示すように、両者の比率を変えて統合してもよい。

ここで、ｋはアクティブ画像の情報とパッシブ画像の情報を統合する比率であり、０≦ｋ≦１の範囲で選択される。例えば、アクティブ画像の情報を重視したい場合には、ｋの値を０．５より大きく設定すればよい。 Specifically, the pixels used to calculate the evaluation value may be appropriately thinned out within the region so that the necessary number of valid features are obtained. After the two evaluation values SAD _A and SAD _P are calculated, an integrated evaluation value SAD is calculated. When integrating the two, it is preferable to prevent the evaluation value of the larger window size from contributing excessively. The numbers of pixels used within the window for calculating the two evaluation values SAD _A and S _P are respectively S _A and S _P , and are calculated according to the following formula (3).

Depending on the importance of the active image and the passive image, the ratio of the two may be changed and integrated as shown in the following equation (4) instead of the above equation (3).

Here, k is the ratio at which active image information and passive image information are integrated, and is selected in the range of 0≦k≦1. For example, if it is desired to place importance on active image information, the value of k should be set to be greater than 0.5.

Next, in step S1114, the disparity value is calculated using the evaluation value SAD calculated in step S1113. For example, when calculating the disparity value based on the image captured by the first imaging unit, a corresponding point exists on a line at the same height as the pixel position of interest Pi on the image captured by the second imaging unit. In step S1113, evaluation values (Eij1, Eij2, Eij3, ..., Eijn) representing the similarity between each pixel position (Pj1, Pj2, Pj3, ..., Pjn) on the corresponding line of the image captured by the second imaging unit and the pixel of interest Pi have already been calculated. In this step, the combination with the highest similarity among these evaluation values is selected, and the corresponding disparity value is calculated as the disparity value for the pixel position of interest Pi.

The same process is performed while changing the pixel position of interest. Although SAD is used as an example of a method for calculating an evaluation value representing similarity, other methods may be used. For example, methods such as SSD (Sum of Squared Difference) and NCC (Normalized Cross Correlation) may be used. Furthermore, methods such as ZNCC (Zero Means Normalized Cross Correlation) may be used.

The evaluation formula used in this step is not limited to these, and any formula can be used as long as it can evaluate the similarity between the features in the vicinity of the pixel position of interest in the first integrated image 114 and the features in the vicinity of the pixel position of interest in the second integrated image 115.

Next, in step S1115, the parallax value is converted into a distance value. The distance value can be calculated from the parallax value using a calibration parameter including the relative positional relationship between the first image capture unit 103 and the second image capture unit 104.

The above describes a method for calculating the distance value to the measurement object 106 in Example 1 using the combined image group after matching, but Example 1 is not limited to the above-mentioned method, and various modifications are possible as described below.

(Variation 1)
When near-infrared pattern light is used in capturing an active image in step S1102, if the amount of ambient light is relatively large, the image will be affected by the visible light component contained in the ambient light, so the ambient light component may be removed in advance. One method for removing the ambient light component is to use the same imaging unit to capture an ambient light image separately and subtract it. The ambient light image is used to remove ambient light noise from the active image as part of the noise removal process.

For the luminance values of the same pixel position in an active image and an ambient light image captured by the same imaging unit, the ambient light component can be removed by subtracting the luminance value derived from the ambient light image from the luminance value derived from the active image. Note that if the exposure time for capturing the passive image in step S1104 and the exposure time for capturing the active image in step S1102 are the same, the ambient light component can be removed by not capturing the ambient light image and instead subtracting the passive image.

(Variation 2)
Also, in step S1113, instead of performing the calculation using the luminance values as they are, a predetermined conversion process, specifically a census conversion, may be performed to calculate the evaluation value using the Hamming distance of the bit string obtained. The census conversion is a conversion in which the luminance values of a pixel of interest and each pixel in the surrounding area are compared to determine which is larger, and if the pixel of interest is larger, the luminance value of the compared pixel is replaced with 0, and if it is smaller, it is replaced with 1. In this case, a census conversion is performed on the first active image 110 and the second active image 111 for an area of width wa and height ha, respectively, and the Hamming distance is calculated using the converted bit strings between bit strings at the same position within the window.

The evaluation value of the active image calculated here is designated as evaluation value CH _A. Similarly, census conversion is performed on the passive image for an area of width wp and height hp, and the Hamming distance using the converted bit string is calculated as evaluation value CH _P of the passive image. Then, the evaluation value SAD _A and evaluation value SAD _{P in the above formula (3) or (4) are replaced with the calculated evaluation values CH A} and _{CH P ,} respectively, to calculate evaluation value SAD as an evaluation value that combines both. Alternatively, the Hamming distance may be calculated using a bit string in which bit strings obtained by performing census conversion on areas of different sizes for the active image and the passive image are arranged.

For example, when census transformation is performed on an active image in a region of 49 pixels with a window size of 7x7, a bit string of 48 (=49-1) bits is obtained. Similarly, when census transformation is performed on a passive image in a region of 25 pixels with a window size of 5x5, a bit string of 24 (=25-1) bits is obtained. In this case, each bit string can be lined up, and the Hamming distance can be calculated using a bit string of 72 (=48+24) bits, and this value can be used as the evaluation value. In this case, it is possible to change the weight of both depending on the number of pixels used in the census transformation from each image. Note that, as in Example 1, processing such as appropriately thinning out the number of pixels used within the region can be performed.

(Variation 3)
In steps S1101 to S1104, the active image and the passive image are captured at different times. However, it is also possible to capture the active image with a specific wavelength of pattern light and separate the corresponding component from the image, thereby acquiring the active image and the passive image in a single capture.

For example, assume that the pattern light for capturing an active image is projected in a specific color such as red and captured with an RGB sensor. In this case, it is possible to use (replace) an image from which the relevant color components are extracted as the active image, and an image from which other components (or even just green) are extracted as the passive image, and both the active image and the passive image can be captured in a single capture. Even in this case, the characteristics of each image differ depending on the capture method, as in Example 1, so pre-processing such as noise removal and aberration correction appropriate for each image as described above is performed, and then the evaluation value calculation process is performed. Here, the pre-processing and the evaluation value calculation method themselves are the same as those described in Example 1, so a detailed description is omitted.

As described above, according to the three-dimensional measuring device 100 in the first embodiment, an evaluation value used for matching is calculated from each active image and each passive image based on information on various image characteristics and features of each image. Then, the calculated evaluation value can be used to calculate a distance value to the measurement object 106. This makes it possible to provide a three-dimensional measuring device that can robustly and accurately calculate a distance value to the measurement object 106.

Example 2
In the second embodiment, an active image and a passive image are integrated with the same device configuration as in the first embodiment to generate (create) a new image (integrated image, composite image) to be used for searching for corresponding points in stereo measurement. When generating a new integrated image (third image), the integrated image is generated so that features observed in each of the active image and the passive image that serve as a clue for matching remain, and the distance value is calculated using the integrated image. This allows the distance value to be calculated with higher accuracy than when active measurement and passive measurement are performed separately and then combined. As described above, the second embodiment has the same device configuration as the first embodiment, and therefore a detailed description of the device configuration will be omitted.

In Example 2, the active image and passive image are subjected to appropriate correction processing respectively, and then evaluation values for the active image and passive image are calculated, as was done in Example 1, and the calculated evaluation values for the active image and passive image are not integrated. Instead, after performing correction processing appropriate to the image characteristics and features of each image, the images are integrated to form a pair of stereo images, and the integrated image (integrated image) is used to calculate the distance value. The three-dimensional measuring device of Example 2 will be described below with reference to FIG. 5. Here, in Example 2, the differences from Example 1 will be described, and overlapping parts will be omitted as appropriate.

Figure 5 is a diagram showing the distance value calculation flow in Example 2. Here, the processing contents of steps S2111, S2112, S2115, and S2116 in Figure 5 are the same as the processing contents of S1111, S1112, S1114, and S1115 in Figure 4, respectively. Therefore, the explanation will be omitted, and the processing of steps S2113 and S2114, which include processing different from the distance value calculation processing in Example 1, will be explained below.

ここで、ｋは、アクティブ画像の情報とパッシブ画像の情報を統合する比率であり、０≦ｋ≦１の範囲で選択される。たとえば、アクティブ画像の情報を重視したい場合にはｋの値を０．５より大きく設定すればよい。 In step S2113, the first active image 110 and the second active image 111, and the first passive image 112 and the second passive image 113 are used to generate an integrated image (third image) as an image for matching. When generating the integrated image, the integrated image is generated based on an active image (first image) captured by one imaging unit from each active image (first image group) and a passive image (second image) captured by the one imaging unit from each passive image (second image group). Specifically, the ratio between the two is set and the integrated image is generated by a simple sum of the luminance values. The luminance values of the same pixel in the active image and the passive image are I _A and I _P , respectively, and the luminance value Is of the same pixel in the composite image is calculated as shown in the following formula (5).

Here, k is the ratio at which active image information and passive image information are integrated, and is selected in the range of 0≦k≦1. For example, when emphasis is placed on active image information, the value of k should be set to a value greater than 0.5.

Note that other methods can also be used to generate images for matching. Areas in an image with a large luminance gradient can be useful as a clue for matching, so a method of combining images that increases the ratio of areas with a large luminance gradient in each of the active and passive images is also effective. In this case, a Sobel filter or the like is applied to the active and passive images to detect the ratio of the luminance gradient of each pixel, and the active and passive images are weighted and added together. This makes it possible to combine images with weighting according to the strength of the luminance gradient for each pixel position.

Figure 6 shows a conceptual diagram of a first integrated image 114 newly generated from the first active image 110 and the first passive image 112 in Figure 3 (A). In addition, a conceptual diagram of a second integrated image 115 newly generated from the second active image 111 and the second passive image 113 in Figure 3 (B) is shown.

Here, in the active image shown in FIG. 3(A), the pattern on the surface of the measurement object 106 can be clearly observed, but the step, which is a feature derived from the measurement object 106, is unclear, and it is difficult to correctly match the area near the step. Furthermore, in the passive image shown in FIG. 3(B), the step derived from the measurement object 106 can be clearly observed, but there are no features on the surface of the measurement object 106, so it is difficult to correctly match this area. On the other hand, in the integrated image shown in FIG. 6 (first integrated image 114, second integrated image 115), both the features derived from the active image and the features derived from the passive image remain, so the features in both images can be clearly observed. In other words, the newly generated integrated image is an image in which the pattern projected on the surface of the measurement object 106 and the three-dimensional structure and texture on the surface of the measurement object 106 can be clearly observed.

In step S2114, an evaluation value is calculated for each combination of pixels of interest in the first integrated image 114 and the second integrated image 115. The evaluation value is calculated by replacing the first active image and the second active image in the first embodiment with the first integrated image 114 and the second integrated image 115, and calculating the evaluation value shown in the above formula (1). Note that in the second embodiment, since an integrated image is generated and the evaluation value is calculated, it is not necessary to calculate the evaluation values shown in the above formulas (2) and (3), and the evaluation value calculated in the above formula (1) can be used as is in the subsequent step S2115.

As described above, according to the three-dimensional measuring device 100 in the second embodiment, the images are integrated so that the features derived from the active image and the features derived from the passive image remain, and the integrated image is used to calculate an evaluation value and calculate a distance value to the measurement object 106. This makes it possible to calculate a distance value with higher accuracy than if active measurement and passive measurement were performed separately and then combined.

Example 3
In the third embodiment, instead of capturing passive images as in the first embodiment, the measurement target 106 is irradiated with pattern light having different properties, another set of active images is captured, and the distance value is calculated using the two sets of active images. This makes it possible to measure the distance value robustly and with high accuracy by reinforcing information with the pattern light having different properties.

The device configuration of Example 3 is the same as that of Example 1 except for the projection unit 102, so detailed description of the device configuration other than the projection unit 102 will be omitted. The projection unit 102 of Example 3 is configured to be capable of using multiple pattern lights that can be switched for irradiation. In other words, the projection unit 102 of Example 3 is configured to be capable of projecting pattern lights each having different properties onto the measurement object 106.

Here, as an example, the projection unit 102 of Example 3 can switch between irradiating a large sparse dot pattern using a near-infrared laser as a light source and a small dense dot pattern using a white LED as a light source. In this way, the projection unit 102 of Example 3 can irradiate the measurement target 106 with pattern light of different properties by using different light sources and different pattern light. Note that the light source of the irradiated pattern light and the combination of pattern light can be used, and various pattern combinations such as those described in Example 1 can be used. Furthermore, the projection unit 102 is configured to be able to irradiate pattern light with a changed wavelength. Below, the processing flow of Example 3 is explained using FIG. 7.

First, in step S3101, the control unit 101 sends a pattern light projection start command to the projection unit 102, and the projection unit 102 irradiates a first pattern light based on a first condition. This causes a pattern of the first pattern light (a pattern of the first illumination light) to be projected onto the surface of the measurement object 106. Here, the first pattern light irradiated under the first condition is pattern light using a near-infrared laser as a light source.

Next, in step S3102, the control unit 101 sends an image capture start command to the first image capture unit 103 and the second image capture unit 104. When the image capture start command is sent to each image capture unit that performs image capture, the first image capture unit 103 and the second image capture unit 104 capture a scene under the first condition in which the first pattern light pattern is projected onto the surface of the measurement object 106 from each viewpoint (image capture process). This image capture process is performed until an image capture end command is sent from the control unit 101. This step allows each image capture unit to acquire a group of active images (first image group) in which the first pattern light pattern is projected onto the surface of the measurement object 106. Note that the image data of each active image (first image group) captured and acquired by the first image capture unit 103 and the second image capture unit 104 in this step is sent to the processing unit 105.

Next, in step S3103, the control unit 101 sends a pattern light projection end command to the projection unit 102, and the projection unit 102 stops emitting the first pattern light and ends the projection of the first pattern light onto the measurement object 106. Here, in this step, the control unit 101 switches the irradiation conditions (projection conditions) from the first conditions for emitting the first pattern light to the second conditions for emitting the second pattern light.

Next, in step S3104, the control unit 101 sends a command to the projection unit 102 to start projection of the second pattern light, and the projection unit 102 irradiates the measurement object 106 with the second pattern light based on the second condition. This causes the second pattern light pattern (second illumination light pattern) to be projected onto the surface of the measurement object 106. The second pattern light irradiated in this step is a pattern light with different properties from the first pattern light projected in step S3101, and here is assumed to be a pattern light with a white LED as the light source.

Next, in step S3105, the control unit 101 sends an image capture start command to the first image capture unit 103 and the second image capture unit 104. The first image capture unit 103 and the second image capture unit 104 capture a scene under the second condition in which the second pattern light pattern is projected on the surface of the measurement object 106 from each viewpoint (image capture process). This image capture process is performed until an image capture end command is sent from the control unit 101. At this time, the first image capture unit 103 and the second image capture unit 104 receive visible light reflected from the measurement object 106. This step allows each image capture unit to acquire a group of active images (second image group) in which the second pattern light pattern is projected on the surface of the measurement object 106. Note that the image data of each active image (second image group) captured and acquired by the first image capture unit 103 and the second image capture unit 104 in this step is sent to the processing unit 105.

Figure 8 shows an example of a conceptual diagram of an active image acquired by the first imaging unit 103 and the second imaging unit 104 in Example 3, and a conceptual diagram of an active image irradiated with pattern light having a different property from the active image.

Fig. 8(A) is an example of a conceptual diagram of an active image acquired by each imaging unit when a first pattern of light is projected onto the surface of the measurement object 106. Note that Fig. 8(A) is a conceptual diagram similar to the conceptual diagram of the active image shown in Fig. 3(A). Fig. 8(B) is an example of a conceptual diagram of an active image acquired by each imaging unit when a second pattern of light is projected onto the surface of the measurement object 106. Taking the processing flow of Fig. 7 as an example, the active image in Fig. 8(A) is an image captured in step S3102, and the active image in Fig. 8(B) is an image captured in step S3104. The pattern observed on the surface of the measurement object 106 is used as a clue for calculating a parallax value in the distance value calculation process at a later stage.

Next, in step S3106, the control unit 101 sends a pattern light projection end command to the projection unit 102, and the projection unit 102 stops irradiating the second pattern light and ends the projection of the second pattern light onto the measurement object 106.

Next, in step S3107, the control unit 101 sends an image capture start command to the processing unit 105. When the image capture start command is sent to each imaging unit that will capture the image, the processing unit 105 calculates a distance value using the image data of each of the active images 110, 111, 116, and 117 and the calibration parameters.

Here, the specific processing flow of the process for calculating the distance value of the measurement object 106 performed by the processing unit 105 is basically the same as that shown in FIG. 4 of the first embodiment if the passive image is replaced with the active image acquired in step S3104 of FIG. 7, so a description thereof will be omitted. Below, steps S1111, S1112, and S1113, which include processes different from those in the first embodiment, will be described.

In step S1111, noise removal processing is performed according to the characteristics of each pattern light irradiated in steps S3101 and S3104. For example, to reduce the effect of speckle noise, a smoothing filter of about the size of the observed speckle is applied to active images 110 and 111 captured by irradiating the first pattern light using a near-infrared laser as a light source, as in Example 1. Since no speckle occurs in active images 116 and 117 captured by irradiating the second pattern light using a white LED as a light source, speckle noise removal processing is not performed.

In step S1112, aberration (distortion) occurring in active images 110, 111 and active images 116, 117 is corrected, and parallelization processing is performed on active images 110, 111 and active images 116, 117. Calibration parameters of the first imaging unit 103 and the second imaging unit 104 are used to correct the aberration. Note that since the wavelengths of light received by the first imaging unit 103 and the second imaging unit 104 differ when capturing an active image in step S3102 and when capturing an active image in step S3105, it is preferable to use different calibration parameters.

In step S1113, active images 110, 111 and active images 116, 117 are used to calculate an evaluation value that evaluates the similarity for each combination of pixels of interest between the image captured by the first imaging unit 103 and the image captured by the second imaging unit 104. The method of calculating the evaluation value is the same if passive images 112, 113 in Example 1 are replaced with active images 116, 117, so a description thereof will be omitted.

The window size for calculating the similarity is set so that the characteristics of each image are appropriately included, depending on the density of the projection pattern (illumination light pattern) in each active image, improving the accuracy of the correspondence. For this reason, it is preferable to do so. For example, as shown in FIG. 7, it is preferable to set the size so that at least one dot in the dot pattern is included, such as window 107 for active images 110 and 111, and window 118 for active images 116 and 117.

As described above, according to the three-dimensional measuring device 100 of the third embodiment, active images are captured by irradiating each active image with a pattern light having a different property. Then, an evaluation value is calculated using each image group (first image group and second image group) of each captured active image, and the distance value to the measurement object 106 can be calculated using the calculated evaluation value. This makes it possible to robustly and highly accurately calculate the distance value to the measurement object 106 by reinforcing information with the pattern light having a different property.

The above describes the method for calculating the distance value to the measurement target object 106 in Example 3, but Example 3 is not limited to the above-mentioned method, and various modifications are possible as described below.

For example, in the above, pattern light of different properties is irradiated onto the measurement object 106, and distance values are calculated using two sets of different active images. Here, in this example, the number of active images used for distance value calculation may be increased to three sets, and pattern light of different properties may be irradiated onto each set. In this case, as in the third embodiment, in step S1111, noise removal is performed for each image according to the light source characteristics of the irradiated pattern light. In addition, in step S1112, aberration correction and parallelization processing are performed using calibration parameters according to the wavelength of the pattern.

Then, in step S1113, a window size is set based on the size of features resulting from the illumination light pattern, for example, the size of dots in a dot pattern. After that, the three sets of active images are used to calculate an evaluation value for evaluating the similarity of each. As a result, similar to the above-mentioned third embodiment, information can be reinforced with an image projected with a pattern of pattern light having different properties, i.e., at least one of the light source, wavelength, pattern, etc. of the illumination light is changed, making it possible to calculate the distance value to the measurement object 106 robustly and with high accuracy. Note that in this embodiment, three sets are used, but the above is also true when three or more sets are used. Also, in this embodiment, it is not limited to only active images, and three or more sets of images may be obtained by combining active images and passive images, and each of the obtained images may be used when calculating the distance value.

Moreover, the modified example 2 shown in the first embodiment can also be applied to the third embodiment. That is, in step S1113 in the third embodiment, instead of performing the calculation using the luminance value as it is, a predetermined conversion process, specifically, a census conversion, can be performed to calculate the evaluation value using the Hamming distance of the bit string obtained.

<Example of article manufacturing method>
The above-mentioned three-dimensional measuring device 100 can be used in a state where it is supported by a certain support member. In this embodiment, as an example, a control system that is attached to a robot arm 200 (gripping device) and used as shown in FIG. 9 will be described. Although not shown in FIG. 9, the three-dimensional measuring device 100 has a first imaging unit 103 and a second imaging unit 104, and uses the first imaging unit 103 and the second imaging unit 104 to capture an image (image data) of an object W placed on a support table T in the same manner as in each of the above-mentioned embodiments. Then, the control unit 101 of the three-dimensional measuring device 100, or the arm control unit 210 that has acquired the image output from the control unit 101 of the three-dimensional measuring device 100, determines the position and orientation of the object W. The arm control unit 210 sends a drive command to the robot arm 200 based on the information on the position and orientation (measurement result) and the information on the distance value of the object W calculated by the three-dimensional measuring device 100 of each of the above-mentioned embodiments to control the robot arm 200.

Based on the distance value of the object W calculated by the 3D measuring device 100 of each of the above-mentioned embodiments, the robot arm 200 holds the object W with a robot hand (gripping part) at the tip, and moves it in a translational or rotational manner (movement process). Furthermore, by assembling the object W to other parts using the robot arm 200 (performing an assembly process), a predetermined item composed of multiple parts, such as an electronic circuit board or a machine, can be manufactured. Furthermore, by processing (treating) the moved object W, an item can be manufactured. The arm control unit 210 has a calculation device such as a CPU as a computer, and a storage device such as a memory that stores a computer program. A control unit that controls the robot may be provided outside the arm control unit 210.

The measurement data and images obtained by the three-dimensional measuring device 100 may be displayed on a display unit 220 such as a display. The object W may be grasped and moved in order to align it with other parts. The object W has a similar configuration to the measurement target 106 illustrated in FIG. 1. The three-dimensional measuring device 100 may be configured to include a robot arm 200.

The disclosure of this embodiment includes the following configurations, methods, programs, and systems.

(Configuration 1)
A plurality of imaging units that capture images of an object from different viewpoints;
a processing unit that calculates a distance value to the object by associating a first image group obtained by capturing an object under a first condition with a plurality of image capturing units and a second image group obtained by capturing an object under a second condition different from the first condition,
The processing unit includes:
Corresponding the first and second image groups based on at least one of information on image characteristics based on the wavelength of the illumination light in each of the images of the first and second image groups, image characteristics caused by the light source of the illumination light, and size of features caused by the pattern of the illumination light, or correlating the first and second image groups after performing processing on the first and second image groups before the correspondence based on the information;
A three-dimensional measuring device that calculates a distance value using a parallax value obtained by matching.

(Configuration 2)
The processing unit calculates an evaluation value to be used for matching from the images of the first and second image groups based on the information, integrates the calculated evaluation values, and calculates a distance value using the integrated evaluation value.

(Configuration 3)
3. The three-dimensional measuring apparatus according to configuration 2, wherein the processing unit calculates an evaluation value based on luminance values in each of the first and second image groups.

(Configuration 4)
3. The three-dimensional measuring device according to configuration 2, wherein the processing unit calculates an evaluation value using a Hamming distance of a bit string obtained by performing a predetermined conversion process on each of the first and second image groups.

(Configuration 5)
5. The three-dimensional measuring device according to any one of configurations 1 to 4, wherein the processing unit performs image distortion correction on at least one of the first and second image groups based on the wavelength of the illumination light.

(Configuration 6)
6. The three-dimensional measuring apparatus according to any one of configurations 1 to 5, wherein the processing unit performs a predetermined noise removal process on at least one of the images in the first and second image groups.

(Configuration 7)
The three-dimensional measuring device described in any one of configurations 2 to 4, characterized in that the processing unit sets areas to be used for matching in the images of the first and second image groups based on the size of features caused by the illumination light pattern, and calculates an evaluation value to be used for matching.

(Configuration 8)
The three-dimensional measuring device described in configuration 1, characterized in that the processing unit generates a third image as an integrated image based on the first image included in the first image group and the second image included in the second image group.

(Configuration 9)
9. The three-dimensional measuring apparatus according to configuration 8, wherein the processing unit calculates the distance value by using the third image.

(Configuration 10)
A projection unit projects a pattern of illumination light onto an object,
A three-dimensional measuring device described in any one of configurations 1 to 9, characterized in that the multiple imaging units acquire a first group of images including images in which an illumination light pattern is projected onto the object by a projection unit, and a second group of images including images in which an illumination light pattern is not projected onto the object.

(Configuration 11)
a projection unit capable of projecting at least a first illumination light pattern and a second illumination light pattern, the second illumination light pattern being different from the first illumination light pattern in at least one of a light source, a wavelength, and an illumination light pattern, onto an object;
A three-dimensional measuring device described in any one of configurations 1 to 7, characterized in that the multiple imaging units acquire a first image group including images of a first illumination light pattern projected onto an object, and a second image group including images of a second illumination light pattern projected onto an object.

(Configuration 12)
A three-dimensional measuring device described in any one of configurations 1 to 7, characterized in that the imaging unit acquires a first image included in a first image group and a second image included in a second image group from images acquired in a single imaging operation by irradiating illumination light of a specific wavelength.

(Configuration 13)
An imaging step of imaging an object from different viewpoints;
a processing step of calculating a distance value to the object by associating a first image group obtained by the imaging step under a first condition with a second image group obtained by the imaging step under a second condition different from the first condition,
In the processing step,
Corresponding the first and second image groups based on at least one of information on image characteristics based on the wavelength of the illumination light in each of the images of the first and second image groups, image characteristics caused by the light source of the illumination light, and size of features caused by the pattern of the illumination light, or correlating the first and second image groups after performing processing on the first and second image groups before the correspondence based on the information;
A three-dimensional measuring method, comprising the steps of: calculating a distance value using a disparity value obtained by matching;

(Configuration 14)
A program for causing a computer to execute a processing method for calculating a distance value to an object, the processing method comprising:
a processing step of calculating a distance value to the object by matching a first image group, which is obtained by capturing the object from different viewpoints under a first condition, with a second image group, which is obtained by capturing the object from different viewpoints under a second condition different from the first condition;
In the processing step,
Corresponding the first and second image groups based on at least one of information on image characteristics based on the wavelength of the illumination light in each of the images of the first and second image groups, image characteristics caused by the light source of the illumination light, and size of features caused by the pattern of the illumination light, or correlating the first and second image groups after performing processing on the first and second image groups before the correspondence based on the information;
A program for calculating a distance value using a disparity value obtained by matching.

(Configuration 15)
A three-dimensional measurement apparatus according to any one of configurations 1 to 12,
and a robot that grasps and moves an object based on a distance value of the object calculated by the three-dimensional measuring device.
A system characterized by:

(Configuration 16)
a moving step of gripping and moving the object based on the distance value of the object calculated by the three-dimensional measuring device according to any one of configurations 1 to 12;
A method for manufacturing an article, comprising the steps of: manufacturing a specified article by processing an object.

The above describes preferred embodiments of the present invention, but the present invention is not limited to these embodiments, and various modifications and variations are possible within the scope of the gist of the invention.

The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. The device is, for example, a three-dimensional measurement device 100. In this case, the program and the storage medium on which the program is stored constitute the present invention. It can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

REFERENCE SIGNS LIST 100 Three-dimensional measuring device 101 Control unit 102 Projection unit 103 First imaging unit 104 Second imaging unit 105 Processing unit 106 Measurement object

Claims (16)

A plurality of imaging units that capture images of an object from different viewpoints;
a processing unit that calculates a distance value to the object by associating a first image group obtained by capturing the object under a first condition with a second image group obtained by capturing the object under a second condition different from the first condition,
The processing unit includes:
performing the association between the first and second image groups based on at least one piece of information regarding image characteristics based on the wavelength of illumination light in each of the first and second image groups, image characteristics due to a light source of the illumination light, and a size of a feature due to a pattern of the illumination light, or performing the association after performing a process prior to the association on the first and second image groups based on the information;
a three-dimensional measuring device that calculates the distance value using the parallax value obtained by the correspondence. The three-dimensional measuring device according to claim 1, characterized in that the processing unit calculates evaluation values to be used in the matching from the images of the first and second image groups based on the information, integrates the calculated evaluation values, and calculates the distance value using the integrated evaluation value. The three-dimensional measuring device according to claim 2, characterized in that the processing unit calculates the evaluation value based on the brightness values in each of the first and second image groups. The three-dimensional measuring device according to claim 2, characterized in that the processing unit calculates the evaluation value using the Hamming distance of the bit string obtained by performing a predetermined conversion process on each of the images of the first and second image groups. The three-dimensional measuring device according to claim 1, characterized in that the processing unit performs image distortion correction for at least one of the first and second image groups based on the wavelength of the illumination light. The three-dimensional measuring device according to claim 1, characterized in that the processing unit performs a predetermined noise removal process on at least one of the images in the first and second image groups. The three-dimensional measuring device according to claim 2, characterized in that the processing unit sets areas to be used for the matching in the images of the first and second image groups based on the size of features caused by the illumination light pattern, and calculates an evaluation value to be used for the matching. The three-dimensional measuring device according to claim 1, characterized in that the processing unit generates a third image as an integrated image based on a first image included in the first image group and a second image included in the second image group. The three-dimensional measuring device according to claim 8, characterized in that the processing unit calculates the distance value using the third image. a projection unit that projects the pattern of the illumination light onto the object;
2. The three-dimensional measuring device according to claim 1, wherein the plurality of imaging units acquire a first group of images including images in which the illumination light pattern is projected onto the object by the projection unit, and a second group of images including images in which the illumination light pattern is not projected onto the object. a projection unit capable of projecting at least a first illumination light pattern and a second illumination light pattern, the second illumination light pattern being different from the first illumination light pattern in at least one of a light source, a wavelength, and an illumination light pattern, onto the target;
2. The three-dimensional measuring device according to claim 1, wherein the plurality of imaging units acquire a first group of images including images obtained by projecting the first illumination light pattern onto the object, and a second group of images including images obtained by projecting the second illumination light pattern onto the object. The three-dimensional measuring device according to claim 1, characterized in that the imaging unit acquires a first image included in the first image group and a second image included in the second image group from images acquired in one imaging operation by irradiating illumination light of a specific wavelength. An imaging step of imaging an object from different viewpoints;
a processing step of calculating a distance value to the object by performing correspondence between a first image group obtained by the imaging step under a first condition and a second image group obtained by the imaging step under a second condition different from the first condition,
In the processing step,
performing the association between the first and second image groups based on at least one of information regarding image characteristics based on the wavelength of illumination light in each of the first and second image groups, image characteristics due to a light source of the illumination light, and a size of a feature due to a pattern of the illumination light, or performing the association after performing a process prior to the association on the first and second image groups based on the information;
a three-dimensional measuring method, comprising: calculating the distance value using the parallax value obtained by the correspondence; A program for causing a computer to execute a processing method for calculating a distance value to an object, the processing method comprising:
a processing step of calculating a distance value to the object by matching a first image group, which is obtained by capturing the object from different viewpoints under a first condition, with a second image group, which is obtained by capturing the object from different viewpoints under a second condition different from the first condition;
In the processing step,
performing the association between the first and second image groups based on at least one piece of information regarding image characteristics based on the wavelength of illumination light in each of the first and second image groups, image characteristics due to a light source of the illumination light, and a size of a feature due to a pattern of the illumination light, or performing the association after performing a process prior to the association on the first and second image groups based on the information;
A program for calculating the distance value using the disparity value obtained by the association. The three-dimensional measuring device according to claim 1 ;
a robot that grasps and moves the object based on the distance value of the object calculated by the three-dimensional measuring device.
A system characterized by: a moving step of gripping and moving the object based on the distance value of the object calculated by the three-dimensional measuring device according to claim 1;
and manufacturing a predetermined article by processing the object.

Images