JP5633058B1 - 3D measuring apparatus and 3D measuring method - Google Patents

Abstract

A three-dimensional measuring apparatus and a three-dimensional measuring method capable of appropriately performing three-dimensional measurement even in the case of a measurement object having a strong specular reflection component. A projector 3 that projects pattern light onto a measurement object 2, a plurality of imaging means 4 that images the measurement object 2 onto which the pattern light has been projected from the projector 3 from different directions, and a projector 3 Calibrating means 81 for calibrating between the imaging means 4 and each imaging means 4, three-dimensional point group measuring means 83 for measuring a three-dimensional point group based on images captured by the imaging means 4, 3 Based on the calibration results obtained by the calibration means, the respective three-dimensional point groups measured by the dimension point group measurement means are projected onto the coordinate system of the pattern projecting means, and the respective reliability is compared by projecting them. A three-dimensional measurement apparatus 1 comprising: an integration unit 84 that integrates measurement results of a three-dimensional point group. [Selection] Figure 1

Description

  The present invention relates to a three-dimensional measurement apparatus and a three-dimensional measurement method for performing three-dimensional measurement by projecting pattern light onto a measurement object.

  Conventionally, as a method for measuring the three-dimensional shape of an object, for example, a plurality of images obtained by photographing a measurement object with two or more cameras, that is, a reference camera and a reference camera provided at different positions are used. By identifying the corresponding pixel between them and applying the principle of triangulation to the difference in position (parallax) between the pixel on the associated reference image and the pixel on the reference image. There is what is called a stereo method for measuring a distance to a point on a corresponding measurement object (for example, see Patent Document 1). However, in the conventional stereo method, when the measurement object is an object having almost no feature such as a white wall, it is difficult to perform association between images.

  Further, in order to measure the three-dimensional shape of the measurement object, for example, a method of measuring the distance to each point by projecting a laser beam onto the measurement object and measuring the reflected light or the measurement object Conventionally, an active method such as a light cutting method for measuring the three-dimensional shape of a measurement object by projecting pattern light onto the object and measuring the shape of the pattern projected onto the measurement object is known ( For example, see Patent Document 2). Further, in order to measure the shape of the measurement object, a stripe pattern is projected onto the object, and the stripe pattern is imaged a plurality of times while being phase-shifted. There is a so-called phase shift method for measuring (see, for example, Patent Document 3).

JP 2001-091245 A JP 2008-292434 A JP-A-11-014327

  However, in a measurement object having a strong specular reflection component such as a metallic product having a glossy surface, as shown in FIG. 1, very strong light (the length of the arrow) is near the reflection angle θout equal to the incident angle θin of the pattern light. Is reflected), so that depending on the direction of the measurement object, a so-called whitening phenomenon may occur in which a photographed image is partially whitened. On the contrary, if the distance is far from the reflection angle equal to the incident angle of the pattern light, the light is hardly reflected, so that a so-called black crushing phenomenon may occur in which the photographed image is partially black. When such whiteout or blackout occurs, pattern light cannot be read, and there is a problem that three-dimensional measurement cannot be performed properly.

  The present invention has been made in view of the above problems, and a three-dimensional measurement apparatus and a three-dimensional measurement capable of appropriately performing three-dimensional measurement even in the case of a measurement object having a strong specular reflection component. It aims to provide a method.

In order to achieve the above object, the three-dimensional measuring apparatus according to claim 1, wherein the pattern light is projected by a pattern light projecting unit that projects pattern light onto a measurement object, and the pattern light projecting unit. Three or more imaging means for imaging the measurement object from different directions, a rotation matrix, a translation vector, a focal length, a principal point, showing the relationship between the pattern light projecting means and the respective imaging means , Using a nonlinear mathematical model composed of a distortion coefficient, by applying a nonlinear least square method to the mathematical model, a rotation matrix used for calibration between the imaging unit and the pattern projecting unit, a translation vector, a focal length, principal point, and calibration means for calibrating by calculating the optimal solution of the distortion factor, on the basis of the images captured by the respective imaging means, 3D point group meter for measuring the three-dimensional point group respectively And the respective three-dimensional point groups measured by the three-dimensional point group measuring means are projected onto a predetermined coordinate system based on the calibration result obtained by the calibration means, and the respective reliability is compared. And integrating means for integrating the measurement results of the respective three-dimensional point groups.

  The three-dimensional measuring apparatus according to claim 2, wherein the pattern light is a fringe pattern light, and the pattern projecting unit has a fringe direction of the fringe pattern light between the pattern projecting unit and each imaging unit. The stripe pattern light is projected onto the measurement object so as not to be parallel to each epipolar line.

  The three-dimensional measurement apparatus according to claim 3, wherein the pattern light projected onto the measurement object by the pattern light projecting unit is reflected directly on the surface of the measurement object and directly enters the imaging unit. The reflection component and the pattern light projected on the measurement object by the pattern light projecting means are reflected by the surface of the measurement object, and then reflected by a portion different from the surface and enter the imaging means. Indirect reflection component separation means for separating the indirect reflection component is provided.

The three-dimensional measuring method according to claim 4, wherein a pattern projecting unit that projects pattern light onto a measurement object and the measurement object onto which the pattern light is projected by the pattern projecting unit are viewed from different directions. Using a nonlinear mathematical model composed of a rotation matrix, translation vector, focal length, principal point, and distortion coefficient indicating the relationship between three or more imaging means for imaging, and applying a nonlinear least square method to the mathematical model A calibration step for performing calibration by calculating an optimum solution of a rotation matrix, a translation vector, a focal length, a principal point, and a distortion coefficient used for calibration between the imaging unit and the pattern projecting unit, and A three-dimensional point group measuring step for measuring a three-dimensional point group based on an image picked up by the image pickup means, and the respective third order measured by the three-dimensional point group measuring step. Based on the calibration result obtained in the calibration step, the point cloud is projected onto the coordinate system of the pattern projecting means, and the reliability of the respective three-dimensional point cloud is integrated by comparing the reliability. And an integration step.

  The three-dimensional measurement method according to claim 5, wherein the pattern light is a fringe pattern light, and the pattern projecting unit has a fringe direction of the fringe pattern light between the pattern projecting unit and each imaging unit. The stripe pattern light is projected onto the measurement object so as not to be parallel to each epipolar line.

  The three-dimensional measurement method according to claim 6, wherein the pattern light projected onto the measurement object by the pattern projecting unit is performed on the surface of the measurement object before performing the three-dimensional point cloud measurement step. The direct reflection component that is reflected and directly incident on the imaging unit and the pattern light that is projected onto the measurement object by the pattern projecting unit are reflected from the surface of the measurement object and then differ from the surface. An indirect reflection component separating step for separating the indirect reflection component reflected from the portion and incident on the imaging means is provided.

According to the first and second aspects of the present invention, the image pickup device includes three or more image pickup units that pick up images of the measurement object projected with the pattern light by the pattern light projecting unit from different directions. Based on the image picked up by the above, each measured three-dimensional point cloud is projected onto a predetermined coordinate system based on the calibration result obtained by the calibration means, and the respective three-dimensional points are compared by comparing the reliability. Integrate point cloud measurement results. Therefore, the specular reflection component of the object to be measured is strong, so even if whiteout occurs in any of the imaging means or the reflected light is weak and blackout occurs, Since the measurement result of each three-dimensional point group with high reliability measured based on the image obtained by the imaging means can be obtained, appropriate three-dimensional measurement can be performed.

  According to the second and fifth aspects of the present invention, since the epipolar line of the image obtained by each imaging unit and the fringe direction of the fringe pattern light projected by the pattern projecting unit are not parallel, Measurement can be performed appropriately.

  According to the third and sixth aspects of the invention, by separating the direct reflection component and the mutual reflection component, the pattern light is read from a portion where the pattern light is not actually projected, thereby enabling three-dimensional measurement. Since it is possible to prevent the deviation from occurring, the accuracy of the three-dimensional measurement can be further improved.

It is a schematic diagram which shows an example of a structure of the three-dimensional measuring apparatus which concerns on this invention. It is a flowchart which shows an example of the flow of a process by the three-dimensional measuring apparatus which concerns on this invention. It is a schematic explanatory drawing for demonstrating the method of calibration using a calibration board. It is a flowchart which shows the flow of the calibration process using a calibration board. It is the schematic which shows an example of the pattern light of the structured light projection method by a Gray code. It is the schematic which shows an example of the pattern light of the phase analysis method by a phase shift. It is the schematic which shows an example of the pattern light used for isolation | separation of a direct reflection component and an indirect reflection component.

  Hereinafter, a three-dimensional measurement apparatus 1 according to an embodiment of the present invention will be described with reference to the drawings. The three-dimensional measuring apparatus 1 performs the three-dimensional measurement by imaging the measurement object 2 irradiated with the pattern light, and projects the pattern light onto the measurement object 2 as shown in FIG. Obtained by the projector (pattern light projecting means) 3, a plurality of image capturing means 4 (4a to 4f) for capturing the measurement object 2 on which the pattern light is projected from different directions, and the respective image capturing means 4a to 4f. And a computer 5 that performs arithmetic processing and the like for three-dimensional measurement of the measurement object 2 using the obtained image.

  The projector 3 is for projecting various pattern lights conventionally used for three-dimensional measurement onto the measurement object 2, for example, projecting pattern lights as shown in FIGS. It is. By projecting such pattern light onto the measurement object 2, three-dimensional measurement can be performed even if the measurement object 2 is an object having almost no white feature. The pattern light projected from the projector 3 is not limited to these, and various pattern lights conventionally used for three-dimensional measurement can be used.

  Each of the imaging units 4a to 4f includes an imaging element and a lens that forms an image of the measurement object 2 on the imaging surface of the imaging element. As the image sensor, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) is used. The same lenses are used for the imaging means 4a to 4f, and the viewing angle and the resolution are set to substantially match. In addition, as the imaging means 4a to 4f, those having substantially the same sensitivity are used. Further, when performing the three-dimensional measurement process of the measurement object 2, the camera parameters and the like are adjusted in advance by calibrating the imaging units 4a to 4f. Although FIG. 1 shows an example in which six image pickup means 4 are arranged, the number of image pickup means 4 and the manner of arrangement are not limited to this, and a plurality of image pickup means 4 are arranged to face at least different directions. However, in order to perform more appropriate three-dimensional measurement, it is preferable that three or more imaging units 4 are arranged.

  The respective image data acquired by these imaging means 4 a to 4 f are input to the computer 5. The computer 5 performs operations for obtaining three-dimensional information of the measurement object 2 using the respective image data acquired by the imaging means 4a to 4f, and includes a CPU (Central Processing Unit) 6 and a hard disk 7, an image processing unit 8, a RAM (Random Access Memory) 9, a display unit 10, an operation unit 11, and the like. Further, as shown in FIG. 1, these units are mutually connected to a system bus 12, and various data and the like are input / output via the system bus 12, and various processes are executed under the control of the CPU 6. The

  In the hard disk 7, three-dimensional measurement of the measurement object 2 is performed based on a processing program for performing calibration between the projector 3 and each of the imaging units 4a to 4f and image data input from each of the imaging units 4a to 4f. A processing program or the like is stored. In this embodiment, an example in which a program for performing these processes is stored in the hard disk 7 is used. Instead, the program is stored in a computer-readable storage medium (not shown). It is also possible to read the processing program from this recording medium.

  The image processing unit 8 performs calibration between the projector 3 and each of the imaging units 4a to 4f and three-dimensional measurement of the measurement object 2 under the control of the CPU 6 based on a processing program stored in the hard disk 7. The arithmetic processing is performed. The RAM 9 temporarily stores a processing program read from the hard disk 7 and is used as a work area for the CPU 5.

  The display unit 10 is composed of a liquid crystal display or the like, and displays images and the like acquired by the imaging units 4a to 4f. The operation unit 11 includes a mouse, a keyboard, and the like, and is used by an operator to input various data and operation commands.

  Hereinafter, the flow of processing for performing the three-dimensional measurement of the measurement object 2 by the three-dimensional measurement apparatus 1 will be described with reference to the flowchart of FIG. 2 is executed by the image processing unit 8 or the like under the control of the CPU 6 in accordance with a processing program stored in the hard disk 7.

  First, in performing the three-dimensional measurement process of the measurement object 2, calibration between the projector 3 and each of the imaging units 4a to 4f is performed (S101). Here, using the calibration board 13 as shown in FIG. 3, rotation and translation between the camera coordinate system set for each of the imaging units 4 a to 4 f and the projector coordinate system set for the projector 3 by the calibration unit 81. Calibrate. As shown in FIG. 3, the calibration board 13 used for this calibration is formed in a planar shape and has a plurality of circular dot patterns 13a indicating the reference position.

  As shown in the flowchart of FIG. 4, when performing the calibration process, first, each imaging unit 4a is set in a state where the pattern board is not projected or a black image is projected on the calibration board 13 arranged at a predetermined position. Photograph by ~ 4f (S201).

  Next, the calibration board 13 on which the pattern light is projected from the projector 3 is photographed by each of the imaging means 4a to 4f (S202). Note that the light projection timing of the pattern light projected from the projector 3 and the photographing timing by each of the imaging units 4a to 4f are controlled to be synchronized by a timing synchronization unit (not shown) of the computer 5. In S202, first, as the pattern light from the projector 3, for example, a vertical gray code pattern which is a kind of stripe pattern light as shown in FIG. To do. At this time, the projector 3 projects the gray code pattern in an oblique direction so that the fringe direction is not parallel to the epipolar lines of the projector 3 and the imaging units 4a to 4f. Thereby, it can prevent that the epipolar line of the image obtained by each imaging means 4a-4f and the stripe direction of a gray code pattern become parallel. Next, as the pattern light from the projector 3, for example, a vertical phase shift pattern, which is a kind of stripe pattern light as shown in FIG. . At this time, similarly to the projection of the gray code pattern, the projector 3 tilts the phase shift pattern so that the fringe direction is not parallel to the epipolar lines of the projector 3 and the imaging units 4a to 4f. Flood light. That is, the phase shift pattern is projected so that the same phase direction of the epipolar line and the stripe pattern is not parallel. Then, the image pickup means 4 a to 4 f shoots with the horizontal gray code pattern projected onto the calibration board 13 as pattern light from the projector 3 this time. Further, the image pickup means 4 a to 4 f shoots with the phase shift pattern in the horizontal direction projected onto the calibration board 13 as pattern light from the projector 3. Such photographing in S201 and S202 is repeated a plurality of times (about 15 to 30 times) while changing the posture of the calibration board 13.

In the structured light projection method using the gray code pattern projected in S202, a measurement is performed by projecting a special binary code fringe pattern called a gray code as shown in FIG. It is. In this structured light projection method, as shown in FIG. 5, the pattern light is projected in the order of (a) to (c), so that the space can be divided into regions represented by gray codes. Here, the black part of the pattern to be imaged corresponds to 0, and the white part corresponds to 1. In FIG. 5A, the whole is divided into two and the two areas are encoded into 0 and 1. In FIG. 5B, the four areas of the bright part and the dark part are encoded into 0, 1, 1, 0. In FIG. 5C, the eight areas are encoded into 0, 1, 1, 0, 0, 1, 1, 0. As a result, an encoded region number is assigned to each region. In FIG. 5, each region is assigned (0, 0, 0), (0, 0, 1), (0, 1, 1), (0, 1, 0), (1, 1, 0), (1, 1, 1), (1, 0, 1), (1, 0, 0). Thus, when three gray code patterns are used, the space can be divided into eight. Then, the three-dimensional coordinates can be calculated by using the three-point surveying principle based on the space code which is a number indicating the divided area and the line of sight of the imaging means 4. In the case of measuring the shape in more detail, the irradiation area of the projector 3 is divided into 2 ^m areas by projecting m gray code patterns while sequentially reducing the bright and dark areas. be able to.

Further, in the phase analysis method using the phase shift pattern projected in S202, stripe-shaped pattern light in which luminance values are distributed in a cosine wave as shown in FIG. The light is projected onto the measurement object 2 while being shifted by (2π), and each of the imaging units 4a to 4f is photographed a plurality of times to obtain the phase distribution. Further, phase connection is performed as necessary to uniquely specify the emission direction of the pattern light. Since the phase value can be obtained as a continuous real value, each imaging means 4 and the projector 3 can be associated with each other by sub-pixels. In addition, since the phase can be obtained independently for every pixel with respect to all the pixels, three-dimensional information can be obtained with high density. The luminance distribution of the phase shift pattern is expressed as the following formula (1).

In addition, the luminance at one point (x, y) when the phase shift amount α is changed every π / 2 is I ₀ (x, y), I ₁ (x, y), I ₂ (x, y), respectively. , I ₃ (x, y), these luminances are expressed by the following formula (2).

As a result, the phase Φ at the position (x, y) is expressed as Equation (3), and the phase shift amount α is expressed as Equation (4). The phase analysis method using the phase shift pattern has a problem of phase connection (unwrapping) because of its periodicity. Here, in order to solve this problem, a gray code pattern is added to the phase shift method. By combining the structured light projection methods used and assigning independent gray codes to each phase period, robust and high-density association is possible.

  Next, the calibration unit 81 performs calibration between the imaging units 4a to 4f using the respective image data obtained in S201 (S203). In this embodiment, since six imaging means 4a to 4f are used, between the imaging means 4a and 4b, between the imaging means 4a and 4c, between the imaging means 4a and 4d, between the imaging means 4a and 4e, and between the imaging means 4a. 4f, between the imaging means 4b and 4c, between the imaging means 4b and 4d, between the imaging means 4b and 4e, between the imaging means 4b and 4f, between the imaging means 4c and 4d, between the imaging means 4c and 4e, and between the imaging means 4c and Calibration is performed by simultaneously optimizing all the relationships between the imaging units 4d and 4e, between the imaging units 4d and 4f, and between the imaging units 4e and 4f.

  Further, the calibration unit 81 uses the calibration result of each of the imaging units 4a to 4f obtained in S203, and sets 3 of the center points of the respective dot patterns 13a of the calibration board 13 for each image data acquired by photographing in S201. Dimensional coordinates are calculated (S204).

  Then, using the image data obtained by the photographing in S202, the horizontal phase and the vertical phase are calculated with respect to the two-dimensional coordinates of the center point of each dot pattern 13a of the calibration board 13, and the horizontal direction is calculated. From the phase value in the direction and the vertical direction, sub-pixel coordinates of the corresponding pixel on the projector image are calculated (S205). The projector image is an image itself in which a pattern projected by the projector 3 is configured, and this image is defined in advance when the pattern is formed.

Next, the calibration unit 81 associates the three-dimensional coordinates of the dot pattern 13a of the calibration board 13 calculated in S204 with the two-dimensional coordinates of the projector image calculated in S205, thereby projecting the projection matrix ^P of the projector 3. seeking P, by decomposing the projection matrix ^P P, the rotation matrix ^P R of the projector 3, the translation vector ^P t, and the internal parameter matrix ^P K (focal length _^P f ^{_μ, P} f ^ν and principal point ^P mu _c , ^P ν _c are included) (S206).

Then, as calibration of the M imaging means 4 (six in this embodiment), the external parameters and internal parameters of each imaging means 4 and the projector 3 are minimized by minimizing the back projection error of the following formula (5). Are simultaneously optimized (S207). The external parameters to be optimized here are a rotation matrix ^j R and a translation vector ^j t between the camera coordinate system of each imaging means 4 and the projector coordinate system of the projector 3, and the internal parameters are focal lengths ^j f = [ ^j f μ, ^j f _ν], principal point _{^{j u c = [j μ c}} , j ν c], and the distortion coefficient _{^{j k = [j k 1,}} j k 2, j k 3, j p 1, j p 2 ]. [ ^J ^ μ _i , ^j ^ ν _i ] in Expression (5) represents the coordinates of the projected point with the assumed parameters, and is calculated in the following Expression (6). Further, for example, a standard Levenberg-Marquardt method can be used to minimize the backprojection error. The calibration between the projector 3 and each of the imaging units 4a to 4f is not limited to the method of this embodiment, and other conventionally known calibration methods may be used.

Thus, when performing the three-dimensional measurement process of the measurement object 2, the calibration between the projector 3 and each of the imaging units 4a to 4f is performed in advance using the calibration board 13 (S101). Then, the pattern light projected onto the measurement object 2 by the projector 3 by the indirect reflection component separation means 82 is reflected from the surface of the measurement object 2 and directly enters each imaging means 4. The direct reflection component and the pattern light projected on the measurement object 2 by the projector 3 are reflected by the surface of the measurement object 2, then reflected by a portion different from the surface, and enter each imaging unit 4. An analysis for separating each indirect reflection component is performed (S102). In S <b> 102, a black and white chessboard pattern as shown in FIG. 7 is projected from the projector 3 as pattern light onto the measurement object 2 and photographed by each imaging unit 4. As shown in FIG. 7, this chessboard pattern has square black-and-white patterns arranged regularly, has a sufficiently high frequency two-dimensionally, and has a white portion ratio β = 0. 5. Here, the light L observed by the imaging means 4 is expressed as the following mathematical formula (7).

At this time, when the two-dimensional frequency of the projected pattern light is sufficiently high, a portion projected on the surface of the measurement object 2 (a white portion of the pattern image) and a portion not projected ( The brightness of the black portion of the pattern image can be calculated by the following formula (8). As a result, the direct reflection component and the indirect reflection component can be obtained from two images captured by projecting a high-frequency pattern with a known white portion ratio β in the pattern and an inverted pattern thereof. In practice, since the luminance at the black and white boundary portion of the pattern cannot be measured accurately, in step S102, the projector 3 first projects the black and white chessboard pattern as it is, and then sequentially checks the chessboard. There are four patterns: a pattern in which the board pattern is shifted in the vertical direction by half the window width, a pattern in which the horizontal pattern is shifted by half the window width, and a pattern in which the board pattern is shifted in the diagonal direction by half the window diagonal width. The maximum luminance of each pixel is set to Lon and the minimum from each of a plurality of images obtained by photographing the measurement object 2 on which a total of eight patterns of black and white are projected by each imaging means 4 The luminance is calculated as Loff.

  Next, in the three-dimensional measuring apparatus 1, the three-dimensional point group of the measuring object 2 is measured based on the images photographed by the imaging units 4a to 4f by the three-dimensional point group measuring unit 83 (S103). In S103, based on the image data photographed by each of the imaging units 4a to 4f by a method in which the structured light projection method using the gray code pattern and the phase analysis method using the phase shift pattern are combined as in S202 of the calibration process. Then, each 3D point group is measured. Thereby, the measurement result of the three-dimensional point group for a total of six of each imaging means 4a-4f can be obtained. In this case as well, similarly to S202, the projector 3 is separately provided so that the stripe directions of the gray code pattern and the phase shift pattern are not parallel to the epipolar lines of the projector 3 and the imaging units 4a to 4f. Flood light. Further, in the structured light projection method using the gray code pattern, when the pattern light image photographed by the imaging unit 4 is binarized and decoded, the luminance distribution in the pattern light image is the surface reflection of the measurement object 2. Since it varies depending on the rate and also includes an indirect reflection component, there is a case where a stable binarization process cannot be performed by a simple threshold process. Therefore, here, the maximum value of the indirect reflection component Lg of the target pixel obtained by the analysis of the indirect reflection component in S102 is used as the binarization threshold. Specifically, a pixel brighter than the indirect reflection component Lg is determined as a white pixel, and a pixel darker than the direct reflection component Ld is determined as a black pixel. Thereby, the direct reflection component and the indirect reflection component can be separated. Note that the method of measuring the three-dimensional point group of the measurement object 2 is not limited to this, and may be a method using a conventionally known pattern light.

  Then, in the three-dimensional measuring apparatus 1, each three-dimensional point group measured in S103 by the integrating unit 84 is projected onto the coordinate system of the projector 3 based on the calibration result obtained by the processing in S101, and the respective reliability is obtained. By comparing the degrees, the measurement results of the respective three-dimensional point groups are integrated (S104). In S104, for example, a three-dimensional point group measured based on an image photographed by the imaging unit 4a is projected onto the coordinate system of the projector 3, and the coordinates are converted into integers. If three-dimensional coordinates have already been assigned to the projected pixel, the reliability is compared, the higher reliability is left, and the lower reliability is deleted from the candidates. In the present embodiment, the luminance amplitude a obtained when performing the phase analysis method using the phase shift pattern of S103 is compared for reliability comparison, and the higher amplitude a is selected as having higher reliability. To do. Such processing is repeatedly performed for each three-dimensional point group measured based on the image photographed by each imaging unit 4, and the remaining three-dimensional point group is also integrated in the projector coordinate system. As a result, since the specular reflection component of the measurement object 2 is strong, even if whiteout occurs in any of the imaging means 4 or the reflected light is weak, even if black crushing occurs, images are taken from different directions. Since it is possible to obtain the measurement results of the respective three-dimensional point groups with high reliability measured based on the images obtained by the other imaging means 4, appropriate three-dimensional measurement can be performed. Note that the value used for comparison of the reliability is not limited to the luminance amplitude a, and the reliability is compared using other values depending on the measurement method of the three-dimensional point group. Also good. In S104, the reliability is compared by projecting the three-dimensional point group obtained in S103 to the projector coordinate system, but the reliability is compared by projecting to the other predetermined coordinate system. Also good.

  In the present embodiment, one projector 3 is arranged and a plurality of imaging units 4 are arranged in different directions, but the relationship between the projector 3 and the plurality of imaging units 4 may be switched. Specifically, a plurality of projectors 3 may be arranged in different directions, and the measurement object 2 on which pattern light is projected from each projector 3 may be photographed a plurality of times by a single imaging unit 4. . Also in this case, the relationship between each projector 3 and the image pickup means 4 is calibrated in advance, and the projector 3 and the image pickup means 4 respectively measure the three-dimensional point group of the measurement object 2, whereby the projector 3 As a result, it is possible to obtain the measurement results of the three-dimensional point group corresponding to the number. Then, by integrating the measurement results of the three-dimensional point group, highly reliable three-dimensional measurement can be performed.

  In addition, embodiment of this invention is not restricted to the above-mentioned form, In the range which does not deviate from the range of the thought of this invention, it can change suitably.

  The three-dimensional measurement apparatus and the three-dimensional measurement method according to the present invention can be effectively used as a technique for performing three-dimensional measurement of a static or dynamic measurement object at high speed and with high accuracy.

DESCRIPTION OF SYMBOLS 1 3D measuring device 2 Measurement object 3 Projector (pattern light projection means)
4, 4a to 4f Image pickup means 81 Calibration means 82 Indirect reflection component separation means 83 Three-dimensional point group measurement means 84 Integration means

Claims (6)

Pattern light projecting means for projecting pattern light onto a measurement object;
Three or more imaging means for imaging the measurement object projected with the pattern light by the pattern light projecting means from different directions;
Using a nonlinear mathematical model comprising a rotation matrix, a translation vector, a focal length, a principal point, and a distortion coefficient indicating the relationship between the pattern light projecting means and the respective imaging means, a nonlinear least square method is applied to the mathematical model. By applying a calibration means for calculating and correcting the rotation matrix, translation vector, focal length, principal point, distortion coefficient used for calibration between the imaging means and the pattern light projecting means, and
Three-dimensional point cloud measuring means for measuring a three-dimensional point cloud based on images captured by the respective imaging means;
By projecting each of the three-dimensional point groups measured by the three-dimensional point group measuring unit onto a predetermined coordinate system based on the calibration result obtained by the calibration unit, and comparing the respective reliability levels. A three-dimensional measurement apparatus comprising: an integration unit that integrates measurement results of the respective three-dimensional point groups. The pattern light is a stripe pattern light,
The pattern projecting unit projects the fringe pattern light onto the measurement object so that the fringe direction of the fringe pattern light is not parallel to the epipolar lines of the pattern projecting unit and each imaging unit. The three-dimensional measuring apparatus according to claim 1, wherein:   The pattern light projected on the measurement object by the pattern light projecting means is reflected by the surface of the measurement object and directly incident on the imaging means, and the measurement by the pattern light projecting means. Indirect reflection component separation means for separating the indirect reflection component incident on the imaging means after the pattern light projected on the object is reflected from the surface of the measurement object and then reflected by a portion different from the surface. The three-dimensional measuring apparatus according to claim 1, comprising: Between pattern light projecting means for projecting pattern light onto a measurement object and three or more image capturing means for capturing images of the measurement object on which the pattern light is projected by the pattern light projecting means from different directions. By using a nonlinear mathematical model composed of a rotation matrix, a translation vector, a focal length, a principal point, and a distortion coefficient indicating the relationship, and applying a nonlinear least square method to the mathematical model, the imaging unit and the pattern projecting unit A calibration step for calculating and calibrating the optimal solution of rotation matrix, translation vector, focal length, principal point, distortion coefficient used for calibration with
A three-dimensional point cloud measuring step for measuring a three-dimensional point cloud based on the images picked up by the respective image pickup means;
Based on the calibration result obtained by the calibration step, the respective three-dimensional point group measured by the three-dimensional point group measurement step is projected onto a predetermined coordinate system, and the reliability is compared with each other. An integration step of integrating the measurement results of the respective three-dimensional point groups. The pattern light is a stripe pattern light,
The pattern projecting unit projects the fringe pattern light onto the measurement object so that the fringe direction of the fringe pattern light is not parallel to the epipolar lines of the pattern projecting unit and each imaging unit. The three-dimensional measurement method according to claim 4, wherein:   Before performing the three-dimensional point cloud measurement step, the pattern light projected onto the measurement object by the pattern light projecting means is reflected directly on the surface of the measurement object and directly enters the imaging means. The reflection component and the pattern light projected on the measurement object by the pattern light projecting means are reflected by the surface of the measurement object, and then reflected by a portion different from the surface and enter the imaging means. 6. The three-dimensional measuring method according to claim 4, further comprising an indirect reflection component separating step for separating the indirect reflection component.

Images