JP6583674B2 - Three-dimensional measuring apparatus, pattern generating apparatus, and method - Google Patents

Description

  The present invention relates to a three-dimensional measurement technique for measuring a three-dimensional position of a measurement object by projecting pattern light, and more particularly to determination of a line pattern in a three-dimensional measurement technique for projecting a line pattern having a random cut.

  Active stereo surveying is known as a technique for measuring the three-dimensional shape of a measurement object. In active stereo surveying, an image is taken with projected pattern light (structured light) projected onto the object to be measured, and the object to be measured using the principle of triangulation from the position of the pattern on the photographed image. The three-dimensional shape (three-dimensional position of each pixel) is obtained. In order to obtain an accurate three-dimensional position, it is necessary to accurately obtain the code (index) of the pattern light encoded from the captured image.

  It is known to use a line pattern having a random cut as an effective pattern even when there is a three-dimensional undulation on the object surface (Patent Document 1, Non-Patent Document 1). The line pattern having a random cut is a pattern in which straight lines (line segments) arranged in parallel in a plurality of columns are cut at random positions and intervals and divided into a plurality of line segments. In the present specification, a line pattern having a random cut is simply referred to as a “line pattern” for the sake of simplicity. When the epipolar line is horizontal (lateral direction, X direction), each straight line of the line pattern is generally parallel to the vertical direction (vertical direction, Y direction). In the line pattern, encoding is performed using the relationship between the relative position (coordinate) of the Y coordinate of a certain line segment and the surrounding line segments. Since the relative position of the Y coordinate of the projection pattern does not depend on the distance or shape with respect to the object, the code information can be measured without loss of accuracy.

  Since the code information in the line pattern is determined by the positional relationship of the Y coordinates of the surrounding line segments, it is necessary to determine a line segment adjacent to the target line segment for which the code information is obtained (hereinafter referred to as a target line segment). is there. In the prior art, six adjacent line segments are obtained on the upper side, lower side, upper right side, upper left side, lower right side, and lower left side of the target line segment. The upper and lower adjacent line segments of the target line segment are obtained as line segments in the direction in which the target line segment is extended. The adjacent line segment on the upper right side is located on the right side of the target line segment, and is specified as the line segment that first intersects with the horizontal line (more precisely, the epipolar line) passing through the upper end position of the target line segment on the right side. The same applies to the upper left, lower right, and lower left side adjacent line segments.

  Since the adjacent portion is searched as described above, when the upper and lower end positions of the target line segment coincide with the cut portion of the adjacent column (the portion without the line segment) in the line pattern, the line segment of two or more adjacent columns is A situation occurs in which a target line segment is obtained as an adjacent line segment or an adjacent line segment is not found. That is, the adjacent line segment of the target line segment is erroneously detected, and correct code information cannot be obtained, resulting in erroneous detection of the three-dimensional position.

JP-A-1-274007

Minoru Maruyama et.al., Range Sensing by Projecting Multiple Slits with Random Cuts, IEEE Transactions on pattern analysis and machine intelligence, Vol. 15, No., 6, June 1993.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of accurate three-dimensional position measurement in active stereo survey using a line pattern.

  In order to solve the above problems, the first aspect of the present invention is to measure pattern light having a pattern in which straight lines arranged in parallel in a plurality of rows are cut at random positions and intervals and divided into a plurality of line segments. Illuminating means for projecting onto the object, imaging means for photographing the measurement object onto which the pattern light is projected, positions of the upper and lower ends of the line segment in the photographed image, and the upper end of the line segment around the line segment and A three-dimensional measurement apparatus comprising: a calculation unit that calculates a three-dimensional position of the measurement object by obtaining a correspondence between the line segment in the captured image and a line segment in the pattern using a lower end position. . Here, when a straight line is drawn along the epipolar line from the upper end and the lower end position of the line segment in the pattern, these lines intersect with the line segments of the adjacent columns on either side of the line segment. It is characterized by satisfying the constraint.

  When a line pattern satisfying such a restriction is adopted, when searching for an adjacent line segment along an epipolar line passing through the upper end and the lower end of each line segment in the line pattern, the line segment of the adjacent column is always found. That is, it is possible to suppress erroneous detection when obtaining an adjacent line segment in the captured image. Note that erroneous detection includes both detecting a line segment that is not actually adjacent as an adjacent line segment and not being able to find an adjacent line segment.

  In addition, it is preferable that the line pattern in the present invention satisfies the above-described restrictions for all the line segments that are constituent elements (however, the line segments in the pattern end row need only satisfy the restrictions between adjacent line segments. ). This is because adjacent line segments can be obtained correctly for all line segments. However, even if some of the line segments do not satisfy the above-mentioned restrictions, overall accurate measurement is possible. Therefore, some of the line segments need not satisfy the above constraints. In the present invention, it is preferable to satisfy the above-mentioned restrictions for 90% or more of all line segments constituting the line pattern, and it is more preferable to satisfy the above-mentioned restrictions for 95% or more of all line segments. Most preferably, the above constraints are satisfied for all line segments.

  In the above description, the projection light pattern is described as a pattern in which straight lines arranged in parallel in a plurality of columns are each cut at random positions and intervals and divided into a plurality of line segments. It does not specify the generation method. This description specifies the pattern shape itself. Therefore, the above pattern may be generated by an arbitrary method.

  In this aspect, it is preferable that the optical axes of the illumination unit and the imaging unit are substantially parallel, and the line segment in the pattern is perpendicular to the epipolar line. In this case, the above constraint is equal to the constraint that the perpendicular line of the line segment passing through the upper and lower positions of the line segment intersects the line segment of the adjacent column on either side of the line segment.

Moreover, in this aspect, it is preferable that the line segments adjacent in the left-right direction have an overlap of a predetermined length or more in the line segment direction. That is, when the pattern draws a straight line along the epipolar line from the upper end position of the line segment in the pattern, the straight line is more predetermined than the lower end position of the line segment of the adjacent column on either side of the line segment. When the straight line is drawn along the epipolar line from the lower end position of the line segment, the straight line is located at the upper end position of the adjacent line segment on either side of the line segment. It is preferable to cross at a lower position more than a predetermined length. The predetermined length here is preferably a value that is greater than or equal to the amount of positional deviation that can occur due to distortion or noise in the captured image. In this way, the adjacent line segment can be obtained correctly even when distortion or noise occurs during image capture.

A second aspect of the present invention is a pattern generation device that generates a pattern of projection light projected by an illumination unit in a three-dimensional measurement apparatus having an illumination unit and an imaging unit. The pattern generated by the pattern generation apparatus according to this aspect is a pattern in which straight lines arranged in parallel in a plurality of columns are each cut at random positions and intervals and divided into a plurality of line segments, and the line segments in the pattern are When straight lines are drawn along the epipolar line from the upper end and lower end positions, these straight lines satisfy the restriction that they intersect with the line segments of the adjacent columns on either side of the line segment.

  The pattern generation apparatus according to this aspect temporarily generates one or a plurality of line segment patterns, regenerates the pattern if the constraint is not satisfied, and adopts the pattern as the pattern if the constraint is satisfied. That is, the pattern generation apparatus according to this aspect includes a generation unit that randomly determines the length of a line segment and the interval (vertical interval) between the line segments and generates one or more line segments, and a newly generated 1 Confirmation means for confirming whether or not the above-mentioned constraint is satisfied between one or a plurality of line segments and a line segment already determined as a pattern. A plurality of line segments are determined as a part of a pattern, and when the constraint is not satisfied, generation by the generation unit and confirmation by the confirmation unit can be repeated until the constraint is satisfied. In this way, a pattern that satisfies the above constraints can be generated. In addition, what is necessary is just to use a fixed value for the space | interval of the thickness of a line segment, and the line segment of an adjacent column, respectively.

  In the above configuration, it is possible to check whether random generation and constraints are satisfied for each column. Alternatively, it is possible to confirm whether or not random generation and constraints are satisfied for each of a plurality of columns.

The pattern generation apparatus according to this aspect generates a half of the total number of columns using the generation unit and the confirmation unit, and generates the remaining half using the generation unit and the confirmation unit. The generated pattern is preferably generated as a pattern rotated by 180 degrees. A pattern that satisfies the constraint satisfies the constraint even if it is rotated by 180 degrees. Therefore, the constraint is always satisfied except for the boundary between the randomly generated pattern and the rotated pattern. If the constraint is not satisfied at the boundary, the pattern of either or both columns of the boundary may be regenerated to satisfy the constraint. In this way, it is possible to generate a pattern that satisfies the constraints as a whole while reducing the number of randomly generated columns to half of the entire number, so that the processing speed is increased.

  The pattern generation apparatus according to this aspect regards a pattern obtained by adding a predetermined deformation process to the generated pattern as an image captured by an imaging unit in the three-dimensional measurement apparatus, and a correspondence relationship between the generated pattern and a line segment And evaluating means for evaluating the accuracy of the obtained correspondence, generating a plurality of patterns satisfying the constraints, and determining a pattern having the highest evaluation by the evaluation means as a final pattern It is preferable to do. Here, it is preferable that the predetermined deformation process simulates the deformation at the time of line segment extraction caused by distortion or noise generated when actual three-dimensional measurement is performed. As described above, in the three-dimensional measurement using the line pattern, since the correspondence of the line segment is obtained using the vertical position of each line segment, the vertical position of each line segment in the pattern regarded as the photographed image is random. It is preferable to provide an error or an offset error. This random error may be determined by a random number according to a normal distribution according to a predetermined standard deviation with an average of zero, for example. Also, this offset error may be a predetermined constant value, for example. In this way, it is possible to evaluate the accuracy of the correspondence relationship when the three-dimensional measurement is performed using the generated pattern, and it is possible to select the pattern that provides the most accurate correspondence relationship. Note that the accuracy of the correspondence can be obtained as the number or ratio of the line segments in which the correct correspondence is obtained by using the same logic as that used in the three-dimensional measurement.

  The pattern generation apparatus according to this aspect generates N patterns (N is an integer equal to or greater than 2) that satisfy the constraint, and among the N patterns, the top M (M Is an integer from 1 to N−1), and NM patterns are newly generated, and the top M patterns and the newly generated NM patterns are selected by the evaluation unit. It is also preferable to determine a pattern having the highest evaluation as a final pattern. In this way, by generating a new pattern by deforming a highly evaluated pattern, there is a possibility that a more highly evaluated pattern can be obtained.

  Here, it is also preferable to repeatedly execute a process of newly generating NM patterns based on the top M pieces having a high evaluation by the evaluation means from among the N patterns. At the time of repetition, it is preferable to select a top M pattern having a high evaluation anew from a total of N patterns of a top M pattern having a high evaluation and a newly generated NM pattern. Depending on the repetition, the possibility that a pattern with higher evaluation is generated increases. Here, the criterion for ending the repetition is, for example, when the repetition process is performed a predetermined number of times, and when the pattern having the highest evaluation value does not change after the predetermined number of repetition processes, the highest evaluation value is the threshold value. This can be the case.

  Further, generation of a new pattern based on the top M pieces (M is an integer of 1 to N-1) that is highly evaluated by the evaluation means from among the N patterns is 1 out of the top M patterns. This can be done by selecting one and regenerating a portion of the selected pattern. Here, the process of selecting one pattern from the top M patterns may be randomly selected with an equal probability, may be randomly selected with a probability according to the evaluation value, You may select M patterns in order. The selected part may be regenerated at random or may be replaced by a part of a pattern other than the selected pattern among the M patterns with high evaluation.

Note that the present invention can be understood as a three-dimensional measuring apparatus and a pattern generating apparatus including at least a part of the above means. The present invention can also be understood as a three-dimensional measurement method and pattern generation method including at least a part of the processing performed by the above means. Moreover, it can also be grasped as a computer program for causing a computer to execute each step of these methods, or a computer-readable storage medium storing the program in a non-temporary manner. Each of the above configurations and processes can be combined with each other as long as no technical contradiction occurs.

  According to the present invention, accurate three-dimensional position measurement is possible in active stereo survey using a line pattern.

It is a figure which shows the structure of the three-dimensional measuring apparatus which concerns on 1st Embodiment. It is a figure explaining the pattern (line pattern) of the projection light which a projection part projects in 1st Embodiment. It is a flowchart which shows the whole flow of the three-dimensional measurement process which concerns on 1st Embodiment. It is a figure which shows the example of the pattern of the projection light and picked-up image in 1st Embodiment. It is a flowchart which shows the detailed flow of the line segment extraction process in a three-dimensional measurement process. It is a flowchart which shows the detailed flow of the matching process in a three-dimensional measurement process. It is a figure explaining the identification method of an adjacent line segment in matching processing. It is a figure explaining the extraction method of the correspondence candidate of a line segment in a matching process. It is a figure explaining the evaluation value (cost) which evaluates the correctness of the correspondence of a line segment in matching processing. It is a figure explaining the miscorrespondence removal method in a matching process. It is a figure explaining the epipolar line in case a projection part and the optical axis of a camera are not parallel, and the restrictions which a line pattern should satisfy | fill. It is a figure which shows the structure of the pattern generation apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the flow of the pattern generation method which concerns on 2nd Embodiment. It is a figure explaining the design parameter in 2nd Embodiment. It is a figure explaining the pattern generation method in 2nd Embodiment. It is a figure which shows the structure of the pattern generation apparatus which concerns on 3rd Embodiment. It is a flowchart which shows the flow of the pattern generation method which concerns on 3rd Embodiment. It is a flowchart which shows the flow of the pattern evaluation method in 3rd Embodiment. It is a figure explaining an example of the pattern regeneration process in 3rd Embodiment. It is a figure explaining another example of the pattern regeneration process in 3rd Embodiment.

(First embodiment)
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. The three-dimensional measurement apparatus according to the present embodiment performs imaging by projecting pattern light onto a measurement object, and measures the three-dimensional shape of the measurement object using the principle of triangulation.

[Constitution]
FIG. 1 is a diagram illustrating a configuration of a three-dimensional measurement apparatus 100 according to the present embodiment. The three-dimensional measuring apparatus 100 is generally composed of a projection unit 110, a camera 120, and an arithmetic unit 130.

(Projection part)
The projection unit 110 is for projecting pattern light (structured light) onto the measurement object 150, and forms a pattern on a light source such as a halogen lamp or a xenon lamp, or light emitted from the light source. A pattern generating element such as a liquid crystal element and an optical system such as a microlens. In addition, when a semiconductor laser is used as the light source, the projection unit 110 can be configured using an optical diffraction element or a MEMS scanner as the pattern generation element. The light projected by the projection unit 110 may be invisible light such as near infrared light. In the present embodiment, the light pattern is stored in the pattern storage unit 131, and the control unit 132 controls the projection unit 110 based on data output from the pattern storage unit 131, whereby the pattern from the projection unit 110 is stored. Light is projected.

  In the present embodiment, the optical axis of the projection unit 110 is set parallel to the optical axis of the camera 120. When the direction in which the projection unit 110 and the camera 120 are arranged is the horizontal direction (X direction), the epipolar line in the projection pattern is the horizontal direction. In the following description, it is assumed that the epipolar line is horizontal.

  The light pattern projected by the projection unit 110 will be described. In the present embodiment, the light pattern projected by the projection unit 110 is a pattern in which straight lines arranged in parallel in a plurality of rows are cut at random (irregular) positions and intervals and divided into a plurality of line segments. Hereinafter, such a pattern used in the present embodiment is referred to as a “line pattern”. An example of a line pattern used in this embodiment is shown in FIG. In the line pattern, the upper end position and the lower end position of each line segment, and the upper end position of each adjacent line segment (6 in total) adjacent to the line segment in each of the upper, lower, upper right, upper left, lower right, and lower left directions. Encoding is performed by the lower end position (a combination of 14 values). That is, by obtaining adjacent line segments in the captured image and calculating their upper end position and lower end position, the corresponding line segment can be determined in the projection pattern.

The line pattern in this embodiment is as shown in FIG.
(Constraint 1) When a straight line is drawn in the horizontal direction from the upper end position and the lower end position of each line segment (more precisely, along the epipolar line), these straight lines intersect with the line segment portions of the adjacent columns.
Satisfies the constraint. In FIG. 2B, straight lines 201 to 204 drawn in the horizontal direction from the upper end position and the lower end position of the line segment 200 all intersect with the line segments of the adjacent columns.

The line pattern in this embodiment is further
(Restriction 2) There is an overlap of a predetermined length or more in the direction of the line segment of the adjacent line segment.
This constraint is further satisfied. In FIG. 2B, the overlaps 205 to 208 of the line segment 200 and the adjacent line segments are all longer than a predetermined length. This restriction 2 is such that when a straight line is drawn in the horizontal direction from the upper end position of the line segment (more precisely, along the epipolar line), it intersects at least a predetermined length from the lower end position of the line segment of the adjacent column. It can also be expressed that, when a straight line is drawn in the horizontal direction from the lower end position of the line segment, it intersects at a predetermined length or more below the upper end position of the line segment of the adjacent column. Here, it is preferable that the predetermined length is equal to or larger than the amount of positional deviation that may occur due to distortion or noise of the captured image.

  A pattern that does not satisfy the above constraints is shown in FIG. In the pattern shown in FIG. 2C, when a straight line 209 is drawn in the horizontal right direction from the upper end position of the line segment 200, the straight line 209 does not intersect with the line segment of the adjacent column. Therefore, the pattern shown in FIG. 2C does not satisfy the above constraints.

As will be described later, when finding adjacent line segments in the captured image, a search is performed along the horizontal direction from the upper end position and lower end position of the line segment, and the first intersecting line segment is determined as the adjacent line segment. To do. The shape of the pattern in the captured image changes depending on the shape of the measurement object 150, but the positional relationship in the vertical direction (Y direction) does not change. Therefore, by using a line pattern that satisfies the above constraints, adjacent line segments in the captured image can be obtained correctly.

(camera)
The camera 120 takes an image of the measurement object 150 onto which the pattern light is projected. The camera 120 performs a photographing process in synchronization with the projection of the pattern light from the projection unit 110 based on a command from the control unit 132. Image data captured by the camera 120 is input to the image input unit 133.

(Arithmetic unit)
The arithmetic device 130 is a computer including a processor such as a CPU, a memory (storage device) such as a RAM and a ROM, an interface with an external device, an input device such as a keyboard and a mouse, and an output device such as a display and a speaker. The arithmetic device 130 provides functions such as a pattern storage unit 131, a control unit 132, an image input unit 133, and a three-dimensional position calculation unit 137 when the CPU executes a program stored in the memory. Note that the three-dimensional position calculation unit 137 includes functions of a line segment extraction unit 134, an association processing unit 135, and a triangulation unit 136. Each functional unit of the arithmetic device 130 will be described with the following flowchart.

[processing]
FIG. 3 is a flowchart showing the flow of the three-dimensional measurement process performed by the three-dimensional measurement apparatus 100 according to the present embodiment. The flowchart shown here is an example of processing, and the processing content and processing order may be changed as appropriate. It is assumed that the calibration (calibration processing) of the projection unit 110 and the camera 120 has been completed, and the points on the pattern and the corresponding captured image points are adjusted to have the same Y coordinate.

  First, in step S <b> 301, pattern light of a line pattern is projected from the projection unit 110 onto the measurement target 150, and the camera 120 captures the measurement target 150 in accordance with the projection of the pattern light. In this process, the control unit 132 reads pattern data from the pattern storage unit 131 and outputs the pattern data to the projection unit 110. The projection unit 110 controls the liquid crystal element according to the pattern data, and projects the light of the designated pattern onto the measurement object 150. Image data photographed by the camera 120 is input to the image input unit 133 and temporarily stored.

  FIG. 4 is a diagram illustrating an example (B) of a captured image obtained by capturing the pattern (A) of the projection light projected in step S301 and the measurement object 150 onto which the pattern light is projected. The pattern light composed of straight lines is photographed as a distorted shape according to the shape of the measurement object 150. Here, each point in the pattern of projection light is located on the epipolar line in the captured image. In this embodiment, it is assumed that the epipolar line is in the horizontal direction in the captured image. Therefore, the Y coordinate position (vertical position) of each point in the projection light pattern does not change in the captured image.

In step S302, the line segment extraction unit 134 extracts a line segment from the input image. A detailed flow of the line segment extraction processing S302 is shown in FIG. The line segment extraction unit 134 first binarizes the input image (S501). More specifically, the line segment extraction unit 134 reduces the size of the input image by half, extracts a portion having a high edge strength by applying a horizontal differential filter, and applies an adaptive second to the extracted region. Apply value processing. In the adaptive binarization process, for example, binarization is performed using an average luminance value in a window having a size of 15 pixels on one side as a threshold value. The line segment extraction unit 134 doubles the size of the image after the adaptive binarization processing and returns it to the original size. Binarization is performed after halving the image size because if the width of the line segment is thick, the edge strength at the center in the width direction of the line segment is detected weakly and the center part is lost during binarization processing. This is to avoid it.

  Next, the line segment extraction unit 134 extracts a line segment region from the binarized image (S502). Specifically, the line segment extraction unit 134 labels the connected areas of the high-luminance portion in the image, removes the erroneous detection areas, and then performs a thinning process. The erroneous detection area can be detected as a connected area having a predetermined width or more, for example. In the thinning process, the center pixel in the horizontal direction (X direction) of each connected region is used as the coordinates of the line segment. With this process, the coordinates of the line segment can be obtained in units of pixels.

  The line segment extraction unit 134 determines the coordinates of the line segment in units of subpixels. First, sub-pixel estimation (S503) of the X coordinate will be described. The line segment extraction unit 134 acquires a luminance profile in the X direction in a predetermined range centered on the line segment position obtained in step S502. This luminance profile is performed on an image before binarization processing. The line segment extraction unit 134 obtains the barycentric position in the luminance profile in units of subpixels. After obtaining the X-direction barycentric position for each Y coordinate in the line segment, the line segment extracting unit 134 further performs a smoothing process on the line formed of the X-direction barycentric positions of a plurality of Y coordinates. The line segment extraction unit 134 sets the sub-pixel accuracy barycentric position thus obtained as the X coordinate of the line segment.

  Next, subpixel estimation (S504) of the Y coordinate of the upper end and lower end positions of the line segment will be described. The line segment extraction unit 134 obtains the slope of the line segment using a point that is a predetermined pixel inside the end point of the line segment. The reason why the inner point is used instead of the end point is that the end point is noisy and accurate slope calculation is difficult. The line segment extraction unit 134 obtains the luminance profile of the image before binarization along the obtained inclination, and obtains the Y coordinate of the point taking the intermediate value between the maximum value and the minimum value in the profile in units of subpixels. This coordinate is determined as the Y coordinate of the line segment end point.

  Through the above processing, the position (coordinates) of the line segment in the captured image can be obtained in units of subpixels.

  When the line segment extraction process S302 ends, in step S303 (FIG. 3), the association processing unit 135 obtains a correspondence relationship between the line segment in the captured image and the line segment in the pattern. A detailed flow of the association processing S303 is shown in FIG. The association process S303 identifies adjacent line segments in six directions for each line segment in the captured image in the process of the loop L1.

In step S601, the association processing unit 135 obtains line segments that are adjacent to the target line segment LS1 in the vertical direction. Here, a method for obtaining an upper adjacent line segment will be described with reference to FIG. The association processing unit 135 searches for a line segment LS2 that satisfies the following conditions.
(Condition 1) The Y coordinate position of the lower end B2 of the line segment LS2 is above the upper end T1 of the target line segment LS1, and
(Condition 2) The angle θ formed by the straight line connecting the lower end B2 of the line segment LS2 and the upper end T1 of the target line segment LS1 and the inclination of the target line segment LS1 (the straight line connecting the upper end T1 and the lower end B1) is within 45 degrees. .
When there are a plurality of line segments LS2 that satisfy these conditions, the association processing unit 135 selects the line segment having the smallest weighted distance between the upper end T1 and the lower end B2 as the upper adjacent line of the target line segment LS1. Specify as minutes. Note that the weighted distance is, for example, [4Lx ² when the displacement along the inclination of the line segment LS1 is Ly and the displacement perpendicular to the inclination of the line segment LS1 is Lx among the displacements of the upper end T1 and the lower end B2. + Ly ² ] ^1/2 . That is, line segment L
A distance obtained by weighting the displacement in the direction perpendicular to the inclination of S1 larger than the displacement in the direction along the inclination is defined as a weighted distance. Here, the method of obtaining the adjacent line segment in the upper direction has been described, but the adjacent line segment in the downward direction can be obtained in the same manner.

  Next, in step S602, the association processing unit 135 obtains adjacent line segments in the four directions of upper left, upper right, lower left, and lower right. Here, how to determine the adjacent line segment in the upper left direction will be described with reference to FIG. Specifically, the association processing unit 135 extends a straight line from the upper end T1 of the target line segment LS1 in the horizontal left direction, and sets a line segment that first intersects with the straight line as an adjacent line in the upper left direction of the target line segment LS1. Specify as minutes. The adjacent line segments in the upper right, lower left, and lower right directions can be obtained in the same manner.

  When an adjacent line segment is identified for each line segment in the captured image, the association processing unit 135 identifies a line segment in the captured image corresponding to each line segment in the pattern in the process of loop L2. In the following loop L2, the process is repeated with each line segment in the pattern as a target line segment.

  The association processing unit 135 first extracts line segment candidates corresponding to the target line segment from the captured image (S603). Specifically, as shown in FIG. 8, at least one of the upper end position and the lower end position of the line segment coincides with the upper end position and the lower end position of the target line segment 801 within the allowable error range, and the parallax ( A line segment that satisfies the condition that the position in the captured image in the X direction) is within a predetermined search range is extracted as a corresponding candidate. In FIG. 8, the upper and lower ends of the line segment 802 are the upper and lower ends of the line segment 801, the upper end of the line segment 803 is the upper end of the line segment 801, and the lower end of the line segment 804 is the lower end of the line segment 801. Match within tolerance. Further, the parallax of the line segments 802 to 804 is within the search range. Therefore, line segments 802 to 804 are extracted as line segment candidates corresponding to the target line segment 801 (a circle next to the line segment indicates a corresponding candidate). For the line segment 805, the upper end and the lower end coincide with the upper end and the lower end of the line segment 801 within the allowable error range, but are not extracted as correspondence candidates because the parallax is larger than the search range. In this case, it is a condition that at least one of the upper end position and the lower end position matches within the allowable error range. However, both the upper end position and the lower end position may match within the error range.

The association processing unit 135 calculates an evaluation value indicating the accuracy of the correspondence with the target line segment for each extracted correspondence candidate (S604). Here, an evaluation value is used that is more likely to be correctly associated as the value is smaller, and is more likely to be incorrect as the value is greater. Therefore, the evaluation value is also referred to as cost. As shown in FIG. 9, the cost (evaluation value) is the difference between the upper end Y coordinate difference (absolute value) and the lower end Y coordinate difference (absolute value) for each of the target line segment and the adjacent line segment in 6 directions. Defined as the sum. The upper and lower end Y coordinates of the target line segment in the pattern are y _p0 ^U and y _p0 ^D, and the upper and lower end Y coordinates of the adjacent line segment are y _pi ^U and y _pi ^D (i = 1 to 6). If the upper and lower end Y coordinates of the target line segment are y _c0 ^U and y _c0 ^D, and the upper and lower end Y coordinates of the adjacent line segment are y _ci ^U and y _ci ^D (i = 1 to 6), the cost is Is defined as follows.

  Here, the Y coordinate of each point in the pattern and the captured image does not change, and the above cost is ideally zero if the correspondence between the line segments is correct. Therefore, the association processing unit 135 determines the line segment with the lowest cost as the line segment in the captured image corresponding to the target line segment in the pattern (S605). Line segments corresponding to each other are stored as corresponding pairs.

  The association processing unit 135 confirms whether or not the correspondence obtained in step S605 is an incorrect correspondence, and removes it from the correspondence pair if the correspondence is incorrect (S606). When a correct corresponding line segment is obtained, as shown in FIG. 10A, the adjacent line segment LS2 ′ of the line segment LS1 ′ in the captured image corresponding to the target line segment LS1 in the pattern is a pattern. This is equal to the line segment LS2 ′ in the captured image corresponding to the adjacent line segment LS2 of the target line segment LS1 in the middle. On the other hand, when the correct corresponding line segment is not obtained, as shown in FIG. 10B, the line segment LS3 ′ adjacent to the line segment LS1 ′ in the captured image corresponding to the target line segment LS1 in the pattern is obtained. The line segment LS2 ′ in the captured image corresponding to the adjacent line segment LS2 of the target line segment LS1 in the pattern is not equal. Therefore, it can be determined whether or not the correspondence relationship is correct by determining whether or not the adjacent line segment of the corresponding line segment is equal to the corresponding line segment of the adjacent line segment. In the present embodiment, when the adjacent line segment of the target line segment and the corresponding line segment of the adjacent line segment are different in any of the six directions, the association processing unit 135 determines that the correspondence obtained in step S605 is incorrect. To do. However, when the adjacent line segment of the target line segment and the corresponding line segment of the target line segment are different for a predetermined number (1 to 6) or more of the six directions, it may be determined that there is an incorrect response. The association processing unit 135 removes from the corresponding pair the line segment pair that has been determined to be incorrect in step S606. In this example, the correctness of the association is confirmed using the target line segment in the pattern as a starting point, but the correctness of the association is confirmed using the line segment in the captured image determined to correspond to the target line segment as the starting point. Also good.

  Note that the association processing unit 135 may also determine that the correspondence is incorrect even when the depth of the adjacent line segment differs by a predetermined distance or more in the captured image. The difference in depth may be evaluated as a difference between intermediate values of the depth at each position of each line segment.

  Through the above processing, line segments in the captured image corresponding to each line segment in the pattern are obtained.

  Next, in step S304 (FIG. 3), the triangulation unit 136 obtains a three-dimensional position for each pixel of the captured image. Specifically, the triangulation unit 136 obtains the depth for each point in the line segment from the parallax between the corresponding line segments. Next, the triangulation unit 136 interpolates the depth in the horizontal direction to obtain the depth in all the pixels including the pixel where the line segment does not exist, and finally the three-dimensional position of the subject shown in each pixel A point cloud representing is obtained. Thus, the measurement of the three-dimensional shape of the measurement object 150 is completed.

[Advantageous effects of this embodiment]
In this embodiment, a line pattern is used, and the position is encoded using the relationship between the Y coordinate positions of the upper and lower end points of each line segment and the surrounding line segments. Therefore, it is necessary to appropriately determine the line segments (adjacent line segments) around each line segment. When the line pattern projected by the projection unit 110 in the present embodiment draws a straight line in the horizontal direction from the upper end position and the lower end position of each line segment of the line pattern, these straight lines intersect with the line segment portions of the adjacent columns. Satisfies the first constraint. In the present embodiment, since the epipolar line is horizontal in the captured image, the Y coordinate of each point (end point of the line segment) in the pattern does not change in the captured image. Therefore, even in the captured image, when straight lines are drawn in the horizontal direction from the upper end position and the lower end position of the line segments, these straight lines intersect with the line segment portions of the adjacent columns. That is, when the adjacent line segments in the captured image are obtained, the adjacent line segments can be reliably detected. If a line pattern that does not satisfy the above conditions is used, there is a possibility that a line segment in two or more columns is detected as an adjacent line segment, but in the present embodiment, such erroneous detection can be avoided. By correctly detecting the adjacent line segment, it is possible to avoid erroneous detection in the calculation of the three-dimensional position.

  Furthermore, since the overlap in the Y direction between adjacent line segments is a predetermined distance or more, even if the calibration is not complete and deformation occurs or noise is added to the captured image, the adjacent line segments are erroneously detected. The possibility of doing can be suppressed.

[Modification]
The above description has been made on the assumption that the optical axes of the projection unit 110 and the camera 120 are parallel, and therefore the epipolar line is in the horizontal direction. However, the optical axes of the projection unit 110 and the camera 120 are not necessarily parallel. When the optical axes of the projection unit 110 and the camera 120 are not parallel, the epipolar line is not parallel and changes depending on the position. FIG. 11A is a diagram showing (a part of) an epipolar line in a pattern. As described above, since the pattern is formed on the light emitted from the light source using the pattern generation element, the relationship between the pattern and the pattern generation element is clear, and the epipolar line in the pattern is obtained from the epipolar line in the pattern generation element. be able to.

  In general, adjacent line segments in the left and right (upper left, upper right, lower left, lower right) directions are obtained as a line segment that first intersects by extending a straight line in the direction along the epipolar line from the upper end position and the lower end position of the target line segment. . Therefore, as shown in FIG. 11B, the line pattern projected by the projection unit 110 is obtained by drawing straight lines 1101 and 1102 along the epipolar line from the upper end position and the lower end position of each line segment. It is necessary to set so as to satisfy the constraint that it intersects with a line segment portion of an adjacent column.

  Note that the direction of the epipolar line at each position of the pattern is determined by the arrangement (position and orientation) of the projection unit 110 and the camera 120. Therefore, a line pattern suitable for the arrangement of the projection unit 110 and the camera 120 may be generated in advance. Alternatively, a line pattern when the epipolar line is horizontal may be generated, and the pattern may be deformed according to the arrangement of the projection unit 110 and the camera 120. This modification is an inverse conversion process of the rectification process.

(Second Embodiment)
The second embodiment of the present invention is a pattern generation device for generating a pattern of projection light used in the first embodiment. That is, the pattern generation device generates a line pattern that satisfies the following constraints.
(Constraint 1) When straight lines are drawn in the horizontal direction from the upper end position and the lower end position of each line segment (more precisely, along the epipolar line), these straight lines intersect with the line segment portions of the adjacent columns. (Restriction 2) There is an overlap of a predetermined length or more in the line segment direction of adjacent line segments.

[Constitution]
The pattern generation apparatus 1200 according to the present embodiment is configured as a normal computer in hardware. That is, the pattern generation apparatus according to the present embodiment includes a processor such as a CPU, a memory (storage device) such as a RAM and a ROM, an interface with an external device, an input device such as a keyboard and a mouse, and an output device such as a display and a speaker. It is comprised as a computer provided with. As shown in FIG. 12, the pattern generation apparatus 1200 according to the present embodiment executes a program stored in a memory by a CPU, so that a design parameter acquisition unit 1201, a line generation unit 1202, a constraint confirmation unit 1203, an inversion pattern, and the like. It functions as the generation unit 1204. Each of these functional units will be described together with the following flowcharts.

[processing]
FIG. 13 is a flowchart showing a flow of pattern generation processing performed by the pattern generation apparatus according to the present embodiment. The flowchart shown here is an example of processing, and the processing content and processing order may be changed as appropriate. Here, a case where the illumination apparatus (projection unit) and the optical axis of the camera are parallel and the epipolar line is horizontal will be described as an example.

  In step S1301, the design parameter acquisition unit 1201 acquires line pattern design parameters. The design parameters will be described with reference to FIG. In the present embodiment, the design parameters include a line length range (maximum value and minimum value), a vertical interval range between line segments (maximum value and minimum value), and an overlapping range (minimum value) of adjacent line segments. ), Number of columns (fixed value), line width (fixed value), and spacing between adjacent columns (fixed value). The design parameter acquisition unit 1201 acquires these design parameters by reading a setting file or receiving input from the user.

  Hereinafter, the pattern generation device 1200 generates patterns for each column at random using random numbers. Here, it is assumed that the pattern is sequentially generated from the leftmost column to the rightmost column. Hereinafter, the k-th column represents the k-th column from the left end. Also, the left half of the entire pattern is generated using random numbers, and the right half is determined as a pattern obtained by rotating (inverting) the left half pattern 180 degrees. Details will be described below.

  First, the line generation unit 1202 randomly generates a pattern in the first column in step S1302. The length of each line segment and the vertical interval between line segments are determined by uniform random numbers within a range specified by design parameters. A value designated by a design parameter is used for the thickness (line width) of the line segment.

  Next, from the second column to the L / 2 column are generated by the processing of the loop L3. Here, L is the number of columns specified by the design parameter and is an even number. The pattern generation apparatus repeatedly executes the following steps S1303 to S1307 while incrementing the variable k in the range of 2 to L / 2. In step S1303, the line generation unit 1202 randomly generates a pattern in the k-th column. This process is the same as the process of step S1302. In step S1304, the constraint checking unit 1203 checks whether the constraints 1 and 2 are satisfied between the pattern in the kth column and the pattern in the k−1th column. If either of the constraints 1 and 2 is not satisfied, the process returns to step S1303 to regenerate the kth column pattern. If the variable k is not L / 2 (S1305-NO), the pattern generated in step S1303 is determined as the kth column pattern. On the other hand, if the variable k is L / 2 (S1305-YES), in step S1306, the constraint checking unit 1203 performs the above-described processing even between the generated pattern for one column and the pattern obtained by vertically inverting this pattern. It is further confirmed whether or not the constraints 1 and 2 are satisfied. If not, the process returns to step S1303 to regenerate the pattern in the k (= L / 2) column.

  When the processing of the loop L3 ends, the pattern of the first half to the L / 2th column, that is, the left half column of the entire pattern is determined. In step S1308, the reverse pattern generation unit 1204 rotates the pattern on the left half (from the first column to the L / 2 column) by 180 degrees, and converts the pattern on the right half (from the L / 2 + 1 column to the L column). Determine as the pattern. This will be described with reference to FIG. FIG. 15A shows the left half pattern 1501 generated by the process of the loop L3. The reverse pattern generation unit 1204 determines a pattern 1502 obtained by rotating the pattern 1501 by 180 degrees as the right half pattern. That is, the pattern of the L / 2 + 1 column is a pattern obtained by inverting the pattern of the L / 2 column, and the pattern of the L / 2 + 2 column is a pattern obtained by inverting the pattern of the L / 2-1 column. And so on. As a result, the finally generated pattern is a point-symmetric pattern with respect to the center.

  Data representing the pattern generated as described above is stored in the pattern storage unit 1205. The pattern data stored in the pattern storage unit 1205 is read into the three-dimensional measuring apparatus 100 and used for three-dimensional measurement.

[Effect of this embodiment]
According to the pattern generation apparatus according to the present embodiment, a pattern that satisfies the above constraints 1 and 2 can be generated. When the constraints 1 and 2 are satisfied, the adjacent line segment can be reliably obtained in the three-dimensional measurement, and the measurement with high accuracy can be performed.

  At the time of pattern generation, only a pattern in a half area of the entire area is generated using random numbers, and the remaining half is determined as a pattern obtained by rotating the pattern by 180 degrees. Even if rotated 180 degrees, the constraints 1 and 2 are satisfied, so that a pattern that satisfies the constraints 1 and 2 as a whole can be generated at high speed.

[Modification]
In the above description, in the process of the loop L3, the pattern is generated one column at a time from the end of the region toward the center. However, the pattern may be generated one column at a time from the center toward the end. That is, the processing may be performed while the variable k is decreased by 1 from k = L / 2 to k = 1 in the loop L3. In this case, the pattern in the L / 2 column needs to satisfy the constraints 1 and 2 only with the pattern obtained by inverting itself up and down, so that the constraint becomes relatively loose.

  The number L of line pattern columns may be an odd number. In this case, in order to make a pattern obtained by rotating the half pattern generated previously by 180 degrees as a pattern of the remaining area, the (L + 1) / 2nd column (the central column of the entire pattern) is It may be generated so as to be vertically symmetrical. Alternatively, the (L−1) / 2 column is generated so as to satisfy the constraints 1 and 2 between both the (L + 1) / 2 column and its upside down, and the (L−1) / A pattern obtained by vertically inverting the pattern in the second column may be generated as a pattern in the Lth column from the (L + 1) / 2 + 1th column. When the number of columns L is an odd number, as described above, the pattern of the (L + 1) / 2 columns is first generated vertically and then the pattern of each column is generated randomly from the center column to the end. This is preferable because the constraints are relatively relaxed.

  In the above description, random generation and confirmation are performed for each column, but confirmation may be performed each time one line segment or one gap (vertical interval) is randomly generated. In this case, every time one or more line segments are randomly generated, it is checked whether or not the constraints 1 and 2 are satisfied with the line segments generated (determined) so far. Determine one or more line segments as a pattern, otherwise regenerate randomly. In addition, generation and confirmation may be performed for each of a plurality of columns instead of for each column. When a plurality of columns are generated collectively, the constraint checking unit 1203 satisfies the constraints 1 and 2 between the plurality of columns and generates the constraints 1 and 2 at the boundary with the pattern generated (determined) so far. Check if it meets. If these constraints are satisfied, the generated pattern of the plurality of columns is determined, otherwise it is regenerated randomly. As described above, it is arbitrary in what unit the random generation and confirmation of whether or not the constraint is satisfied are performed.

  In the above description, a half area pattern is generated using random numbers, and a pattern obtained by rotating the pattern by 180 degrees is used as the remaining half area pattern. However, all the areas may be generated using random numbers.

  In the above description, the length of the line segment and the vertical interval of the line segment are determined randomly within the range of the design parameter, and then it is determined whether or not the constraint is satisfied. However, the range of the upper end position and the lower end position of the line segment necessary for satisfying the constraint is determined from the pattern of adjacent columns determined for all. That is, the length of the line segment and the range of the vertical interval of the line segment to satisfy the constraints 1 and 2 are determined from the already determined adjacent column pattern. Therefore, the length of the line segment and the vertical interval of the line segment may be determined using a random number from the range where the range given as the design parameter and the range for satisfying the constraints 1 and 2 overlap.

  In the above description, the description has been made on the assumption that the optical axes of the illumination device and the camera are parallel and the epipolar line is horizontal. However, when the optical axes of the illumination device and the camera are not parallel, the epipolar line differs depending on the position. In such a case, the above restriction 1 is read as that when straight lines are drawn along the epipolar line from the upper end position and the lower end position of each line segment, these straight lines intersect with the line segments of the adjacent columns. The same processing may be performed. Note that the direction of the epipolar line in each pixel can be grasped in advance based on the arrangement of the illumination device and the camera. The pattern generation device may generate a line pattern when the epipolar line is horizontal, and generate a final line pattern by deforming the line pattern according to the arrangement of the illumination device and the camera. . This modification is an inverse conversion process of the rectification process.

(Third embodiment)
The third embodiment of the present invention is a pattern generation apparatus for generating a pattern of projection light used in the first embodiment, as in the second embodiment. The difference from the second embodiment is that the present embodiment includes a process for generating a line pattern capable of performing three-dimensional measurement with higher accuracy.

  The pattern generation apparatus according to the present embodiment is also configured as a normal computer in hardware as in the second embodiment. That is, the pattern generation apparatus according to the present embodiment includes a processor such as a CPU, a memory (storage device) such as a RAM and a ROM, an interface with an external device, an input device such as a keyboard and a mouse, and an output device such as a display and a speaker. It is comprised as a computer provided with. As shown in FIG. 16, the pattern generation apparatus according to the present embodiment executes a program stored in a memory as a pattern generation unit 1601, an evaluation unit 1602, a pattern selection unit 1603, and a pattern deformation unit 1604. Function. Here, the pattern generation unit 1601 has the same function as the pattern generation apparatus 1200 (FIG. 12) of the second embodiment, and generates a line pattern that satisfies the above constraints. The detailed description of the pattern generation unit 1601 will not be repeated, and the other functional units will be described together with the following flowcharts.

  In step S1701, the pattern generation unit 1601 generates N patterns that satisfy the constraints 1 and 2. Since the pattern generation processing by the pattern generation unit 1601 has been described in the second embodiment, a repeated description is omitted. Here, the number N of patterns to be generated may be appropriately determined as long as it is an integer of 2 or more.

  In step S1702, the evaluation unit 1602 calculates, for each of the generated N patterns, an evaluation value that represents the accuracy when the three-dimensional measurement is performed using the pattern. Details of the pattern evaluation processing S1602 will be described with reference to FIG. The evaluation unit 1602 regards the pattern to be evaluated as a pattern projected from the illumination device of the three-dimensional measurement apparatus and is a captured image captured by the camera (S1801). Next, the evaluation unit 1602 performs a predetermined deformation process on the photographed image (pattern data that is actually regarded as a photographed image; hereinafter, such description is omitted) (S1802). If the epipolar line is horizontal, the Y coordinate does not change ideally between the pattern and the captured image, but in actual three-dimensional measurement, the image is captured due to imperfect calibration, image distortion, noise, and other effects. The Y coordinate position of the line segment extracted from the image changes. The deformation process in step S1802 is for simulating this effect. Specifically, the evaluation unit 1602 adds a random number according to a normal distribution having an average of zero and a predetermined standard deviation to the Y coordinates of the upper and lower ends of each line segment of the captured image. Alternatively, a predetermined constant value (offset value) may be added to the Y coordinate, or both a random number and an offset value may be added to the Y coordinate. The reason why an error is given only to the Y coordinate is that the Y coordinate position of the line segment is important when associating the line segment as described above.

  The evaluation unit 1602 performs line segment association processing between the pattern and the deformed captured image (S1803). Since this association processing is the same as the processing described in the first embodiment (the flowchart in FIG. 6), description thereof is omitted. The evaluation unit 1602 calculates, as an evaluation value, the number (or ratio) of line segments that can be correctly associated by this association process (S1804). The case where correct association cannot be performed includes a case where an incorrect line segment is associated and a case where a corresponding line segment cannot be obtained.

  It can be said that the evaluation value calculated in this way is a value representing the accuracy of correspondence between line segments in actual three-dimensional measurement. Since the accuracy of the line segment association is related to the accuracy of the three-dimensional measurement, the evaluation value can be said to be a value representing the accuracy of the three-dimensional measurement.

  Next, in step S1703, the pattern selection unit 1603 selects the top M patterns having a high evaluation value from the N patterns. The value of M may be 1 or more and N-1. Hereinafter, the top M patterns selected here are referred to as elite patterns.

  In step S1704, the pattern transformation unit 1604 regenerates NM child patterns using the M elite patterns selected by the pattern selection unit 1603 as parent patterns. Specifically, a new pattern is generated by changing a part of the elite pattern. Hereinafter, the pattern deformation process S1704 will be described in detail.

  FIG. 19A is a flowchart showing a detailed flow of the pattern deformation process S1704. The pattern deforming unit 1604 randomly selects one pattern from the M elite patterns (S1901). The selection here may select all the elite patterns with the same probability, or may select with a higher probability as the evaluation value is larger. However, each elite pattern may be selected in order without performing a stochastic selection.

  The pattern deforming unit 1604 randomly selects one of a plurality of columns constituting the line pattern (S1902). As shown in FIG. 19B, it is assumed that the column selected here is the k-th column from the left. Then, as shown in FIG. 19C, the pattern deforming unit 1604 regenerates a column on the right side of the k-th column, that is, the column after the (k + 1) -th column using a random number (S1903). Since the pattern generation processing here is the same as the method described in the second embodiment, detailed description thereof is omitted. Note that, here, from the (k + 1) th column to the rightmost column is regenerated, only the middle range of the entire column may be reproduced.

  As described above, one pattern can be regenerated. By repeating this process NM times, NM patterns based on the M elite patterns are regenerated.

FIG. 20A is a flowchart showing another example of the pattern deformation process S1704. Here, a new pattern is generated by combining two elite patterns. The pattern deforming unit 1604 randomly selects two patterns P1 and P2 from the M elite patterns (S2001). Similarly to the above, this selection may be performed with equal probability, or may be performed with probability according to the evaluation value. Next, as shown in FIG. 20 (B), the pattern deforming unit 1604 randomly selects a partial range of the plurality of columns constituting the line pattern (S2002), and selects the selected range of the pattern P1. A pattern in which the column is replaced with a column in the same range of the pattern P2 is generated (S2003). In FIG. 20B, the intermediate range in the entire column is selected and replaced, but the selected range may be a range from a certain column to the end. It is conceivable that the pattern generated by combining the two patterns in this way does not satisfy the constraints 1 and 2 at the boundary of the selection range. Therefore, the pattern deforming unit 1604 performs adjustment (regeneration of the column) so that the column at the boundary of the selection range satisfies the constraints 1 and 2 (S2004).

  As described above, one pattern can be regenerated. By repeating this process NM times, NM patterns based on the M elite patterns are regenerated.

  When NM new patterns are obtained based on the M elite patterns, it is determined whether or not the iterative process is determined (S1705). If it is determined that the process is to be repeated, the processes in steps S1702 to S1704 are performed again for a total of N patterns of M elite patterns and NM new patterns. On the other hand, if it is determined that the repetition is to be terminated, the evaluation unit 1602 calculates evaluation values for a total of N patterns of M elite patterns and NM new patterns (S1706), and a pattern selection unit 1603 Selects the pattern having the highest evaluation value (S1707). Data representing the pattern generated as described above is stored in the pattern storage unit 1605. The pattern data stored in the pattern storage unit 1605 is used when the three-dimensional measurement apparatus 100 is read and three-dimensional measurement is performed.

  The repeated end determination in step S1705 can be performed as follows. For example, it is possible to determine whether or not to end the repetition depending on whether or not the number of repetition processes has been set in advance and the number of repetitions has reached this number. Alternatively, when the pattern having the highest evaluation value has not changed the predetermined number of repetitions, it can be determined that the repetition process is to be ended. In addition, when the maximum value or average value of the evaluation values of each pattern is equal to or greater than a predetermined value, it can be determined that the repetitive processing is ended.

[Advantageous effects of this embodiment]
In this embodiment, the measurement accuracy in actual three-dimensional measurement is evaluated using the evaluation value. Since a new pattern is regenerated based on a pattern with a high evaluation value, it can be expected that a better pattern can be obtained. Therefore, the line pattern generated by the pattern generation apparatus according to the present embodiment is less likely to cause an incorrect correspondence of the line segment in actual measurement, and can perform three-dimensional measurement with high accuracy.

[Modification]
In the above description, the pattern regeneration process (steps S1702 to S1704 in FIG. 17) is repeatedly executed. However, this process may be repeated only once. Even if the regeneration process is performed only once, it is possible to generate a better pattern than simply generating N line patterns and selecting a pattern with good evaluation.

  Further, N line patterns may be generated and the pattern having the best evaluation may be selected without performing the pattern regeneration process. In this way, a better pattern can be generated than when only one pattern is generated.

DESCRIPTION OF SYMBOLS 100: Three-dimensional measuring apparatus 110: Projection part 120: Camera 130: Arithmetic device 131: Pattern light generation part 132: Control part 133: Image input part 134: Line segment extraction part 135: Association processing part 136: Three-dimensional value calculation Part
1200: Pattern generation device 1201: Design parameter acquisition unit 1202: Line generation unit 1203: Restriction confirmation unit 1204: Inversion pattern generation unit 1205: Pattern storage unit 1600: Pattern generation device 1601: Pattern generation unit 1602: Evaluation unit 1603: Pattern selection Unit 1604: Pattern deformation unit 1605: Pattern storage unit

Claims (13)

Illumination means for projecting pattern light having a pattern obtained by cutting straight lines parallel to a plurality of rows at random positions and intervals into a plurality of line segments onto a measurement object;
Imaging means for photographing the measurement object onto which the pattern light is projected;
Using the positions of the upper and lower ends of the line segment in the captured image and the positions of the upper and lower ends of the line segments around the line segment, the correspondence between the line segment in the captured image and the line segment in the pattern is obtained. Calculating means for calculating a three-dimensional position of the measurement object;
With
The pattern, when a straight line is drawn along the epipolar lines from the top and bottom positions of the line segment in the pattern, Ri also Majiwa a segment of the adjacent rows of these straight lines on either side of the line segment,
The calculating means includes
Extracting line segment candidates corresponding to the target line segment in the pattern based on the positions of the upper and lower ends of the line segment in the captured image,
Obtain an evaluation value representing the corresponding accuracy based on the positions of the upper and lower ends of the six adjacent line segments above, below, upper left, upper right, lower left, and lower right of the candidate line segment,
Determining a line segment in the captured image corresponding to a target line segment in the pattern based on the evaluation value;
A three-dimensional measuring apparatus. The optical axes of the illumination means and the imaging means are substantially parallel,
The line segment in the pattern is perpendicular to the epipolar line;
The perpendicular of the line segment passing through the upper and lower end positions of the line segment in the pattern intersects with the line segment of the adjacent column on either side of the line segment,
The three-dimensional measuring apparatus according to claim 1. When the pattern draws a straight line along the epipolar line from the upper end position of the line segment in the pattern, the straight line has a predetermined length on either side of the line segment than the lower end position of the line segment of the adjacent column. More than this, when intersecting at the upper position and drawing a straight line from the lower end position of the line segment along the epipolar line, the straight line is on either side of the line segment than the upper end position of the line segment of the adjacent row Intersect at a lower position for a predetermined length or more,
The three-dimensional measuring apparatus according to claim 1 or 2. A pattern generation device that generates a pattern of projection light projected by the illumination unit in a three-dimensional measurement apparatus having an illumination unit and an imaging unit,
The pattern is a pattern in which straight lines arranged in parallel in a plurality of rows are each cut at random positions and intervals and divided into a plurality of line segments, and
When a straight line is drawn along the epipolar lines from the top and bottom positions of the line segment in the pattern, these lines are also meet the constraint that intersects the line segment of the adjacent columns on either side of the line segment,
The pattern generation device includes:
Generating means for randomly determining the length of the line segment and the interval between the line segments to generate one or more line segments;
Confirmation means for confirming whether or not the above-mentioned constraints are satisfied between the newly generated line segment or lines and the line segment already determined as a pattern;
With
If the constraint is satisfied, one or more newly generated line segments are determined as a part of the pattern. If the constraint is not satisfied, generation by the generation unit and confirmation by the confirmation unit are performed until the constraint is satisfied. repeat,
A pattern generation apparatus characterized by that. The generating means generates a pattern for one column,
When the pattern generated by the generation unit satisfies the constraint, the newly generated pattern for one column is determined as a new column pattern. When the pattern does not satisfy the constraint, the generation unit generates the pattern until the constraint is satisfied. Repeat the confirmation by the confirmation means,
The pattern generation apparatus according to claim 4 . The generating means generates a pattern for a plurality of columns,
When the pattern generated by the generation unit satisfies the constraint, a newly generated pattern for a plurality of columns is determined as a new column pattern. When the pattern does not satisfy the constraint, the generation unit generates the pattern until the constraint is satisfied. Repeat the confirmation by the confirmation means,
The pattern generation apparatus according to claim 4 . The pattern generation device includes:
The half of the total number of columns is generated using the generation unit and the confirmation unit,
For the other half, the pattern generated by using the generation unit and the confirmation unit is generated as a pattern rotated by 180 degrees.
Pattern generating apparatus according to any one of claims 4 to 6. The pattern generation device includes:
A pattern obtained by applying a predetermined deformation process to the generated pattern, and a corresponding line segment between the generated pattern and an evaluation unit that evaluates the accuracy of the calculated correspondence;
A plurality of patterns satisfying the constraints are generated, and a pattern having the highest evaluation by the evaluation unit is determined as a final pattern.
The pattern generation apparatus of any one of Claim 4 to 7 . N patterns (N is an integer of 2 or more) are generated that satisfy the constraints,
NM patterns are newly generated based on the top M pieces (M is an integer of 1 to N-1) that are highly evaluated by the evaluation means from among the N patterns,
A pattern having the highest evaluation by the evaluation unit is determined as a final pattern among the top M patterns and NM newly generated patterns.
The pattern generation apparatus according to claim 8 . Repetitively executing a process of newly generating NM patterns based on the top M highest evaluated by the evaluation means from among the N patterns;
The pattern generation apparatus according to claim 9 . The generation of a new pattern based on the top M pieces (M is an integer of 1 to N-1) that is highly evaluated by the evaluation means from among the N patterns is performed by selecting one of the top M patterns. Done by selecting and regenerating a portion of the selected pattern,
The pattern generation apparatus according to claim 9 or 10 . An illumination step of projecting pattern light having a pattern obtained by cutting straight lines arranged in parallel in a plurality of rows at random positions and intervals into a plurality of line segments from an illumination unit onto a measurement object;
An imaging step of taking an image of the measurement object on which the pattern light is projected, by an imaging means;
The calculation means obtains a correspondence between the line segment in the captured image and the line segment in the pattern using the position of the cut point of the line segment in the captured image and the position of the cut point of the line segment around the line segment. Calculating a three-dimensional position of the measurement object;
Including
The pattern, when a straight line is drawn along the epipolar lines from the top and bottom positions of the line segment in the pattern, Ri also Majiwa a segment of the adjacent rows of these straight lines on either side of the line segment,
In the calculating step,
Extracting line segment candidates corresponding to the target line segment in the pattern based on the positions of the upper and lower ends of the line segment in the captured image,
Obtain an evaluation value representing the corresponding accuracy based on the positions of the upper and lower ends of the six adjacent line segments above, below, upper left, upper right, lower left, and lower right of the candidate line segment,
Determining a line segment in the captured image corresponding to a target line segment in the pattern based on the evaluation value;
A three-dimensional measuring method characterized by the above. A pattern generation method for generating a pattern of projection light projected by the illumination unit in a three-dimensional measuring apparatus having an illumination unit and an imaging unit,
The pattern is a pattern in which straight lines arranged in parallel in a plurality of rows are each cut at random positions and intervals and divided into a plurality of line segments, and
When a straight line is drawn along the epipolar line from the upper end and lower end position of the line segment in the pattern, these straight lines satisfy the restriction that the line segment intersects with the line segment of the adjacent column on either side of the line segment,
Computer
A step of generating a pattern for one column by randomly determining the length of the line segment and the interval between the line segments;
A confirmation step for confirming whether or not the above-mentioned constraints are satisfied between the newly generated pattern for one column and the pattern for one column determined immediately before;
Run
If the constraint is satisfied, the newly generated pattern for one column is confirmed as a new column pattern.If the constraint is not satisfied, generation by the generation step and confirmation by the confirmation step are repeated until the constraint is satisfied.
A pattern generation method characterized by that.

Images