JP6166539B2 - Three-dimensional shape measuring apparatus, control method for three-dimensional shape measuring apparatus, and program - Google Patents

Description

  The present invention relates to a three-dimensional shape measuring apparatus, a three-dimensional shape measuring apparatus control method, and a program for acquiring a three-dimensional shape data of a test object by projecting and imaging a pattern on the test object. The present invention relates to a three-dimensional shape measuring apparatus, a method for controlling a three-dimensional shape measuring apparatus, and a program that use a method of projecting and imaging a plurality of patterns on a test object and calculating the position of a light / dark boundary.

  A three-dimensional shape measurement technique for acquiring three-dimensional shape data of a test object by projecting a pattern onto the test object and imaging the pattern is widely known.

  An important element in this technology is that the projected pattern is imaged, the feature points of the pattern are recognized as markers, the position of this marker is accurately calculated, and an identification code is used to identify multiple markers individually. Giving and specifying the three-dimensional coordinates of the marker from the marker position on the captured image and this identification code.

  In general, a striped binary light-dark distribution is used as a projection pattern, and the light-dark boundary line is used as a marker.

  Patent Document 1 and Non-Patent Document 1 describe that a marker position is calculated by a method called complementary pattern projection. In the complementary pattern projection method, two types of patterns having alternating shades are projected and imaged on a surface to be examined, and a pattern intersection on the image sensor is used as a marker. The marker is expressed as “bright / dark boundary” in Patent Document 1, and is expressed as “striped pattern edge position” in Non-Patent Document 1.

  As described above, a stripe-shaped light / dark distribution is often used as a pattern, and as a result, the intersection locus of light and dark is distributed in a linear shape. Therefore, in the description of the present invention, the marker is hereinafter referred to as a measurement line.

  After the position of the measurement line on the image sensor is measured by such a method, a pattern for identifying each measurement line is projected. This pattern is composed of a plurality of patterns, and a brightness distribution of brightness and darkness is formed for each pattern at the same position as the measurement line on the image sensor, and this is regarded as a binary signal column as an identification number. Such a method is a technique called so-called spatial coding or spatial coding.

JP 2009-042015 A

IEICE Transactions D Vol. J71-D No. 7 pp. 1249-1257

  However, in the techniques of Patent Document 1 and Non-Patent Document 1, when this method is used, when the light / dark boundary of the spatial coding pattern coincides with the measurement line position, the light / dark value is not fixed, and the identification code is not attached. There is a problem of becoming stable.

  In view of the above problems, an object of the present invention is to reduce the instability of spatial coding pattern determination and improve the accuracy of spatial coding.

The three-dimensional shape measuring apparatus according to the present invention that achieves the above object is as follows.
Projecting means for projecting a plurality of patterns onto the test object,
Imaging means for capturing images of the test objects onto which the plurality of patterns are respectively projected, and acquiring the plurality of pattern image data;
Determining means for determining the contrast of the region of interest of the specific pattern image data among the plurality of pattern image data based on the contrast of the corresponding region of the other pattern image data;
Calculation means for calculating the three-dimensional coordinates of the test object based on the brightness of the specific pattern image data determined by the determination means and the brightness of the other pattern image data;
Equipped with a,
Wherein the plurality of patterns, characterized by a gray code pattern der Rukoto.

  According to the present invention, the instability of spatial coding pattern determination can be reduced and the accuracy of spatial coding can be improved.

The figure which shows the structural example of the three-dimensional shape measuring apparatus which concerns on 1st Embodiment. The figure which shows the measurement line setting pattern group A which concerns on 1st Embodiment. FIG. 3 is a captured image diagram of a projection image according to the first embodiment. The figure which shows the space coding pattern group which concerns on 1st Embodiment. The figure which shows the imaging data of the measurement line setting pattern group A which concerns on 1st Embodiment. The figure which shows the complementary pattern difference signal Sa which concerns on 1st Embodiment. The figure which shows the gray code pattern corresponding to the even-numbered measurement line which concerns on 1st Embodiment. The figure which shows the gray code pattern corresponding to the odd-numbered measurement line which concerns on 1st Embodiment. The figure which shows the measurement line setting pattern groups B and C which concern on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

(First embodiment)
With reference to FIG. 1, the structural example of the three-dimensional shape measuring apparatus which concerns on 1st Embodiment is demonstrated. In FIG. 1, a projection unit 1 projects a liquid crystal panel 3, an illumination unit 2 that illuminates the liquid crystal panel 3, and a projection optical system that projects an image of the liquid crystal panel 3 onto a test object 7 disposed near the test surface 6. 4. The projection unit 1 projects a predetermined pattern having a stripe structure onto the test object 7 according to a command from the projection imaging control unit 20 described later.

In addition, the imaging unit 8 includes an imaging element 10 and an imaging optical system 9 that forms an image of a pattern projected on the test object 7 as a luminance distribution on the imaging element 10 and images the image. The imaging unit 8 performs an imaging operation in accordance with a command from a projection imaging control unit 20 described later, and calculates a gradation intersection described later as a gradation distribution discretely sampled by the imaging device 10. To the unit 21.
The projection imaging control unit 20 controls the projection unit 1 to project a predetermined pattern onto the test object 7 at a predetermined timing, and controls the imaging unit 8 to image the pattern of the test object 7. The gradation intersection calculation unit 21 calculates the three-dimensional shape by calculating the three-dimensional coordinates of the test object 7 by the spatial encoding method based on the information output from the imaging unit 8.

  FIG. 2 shows a stripe structure of a measurement line setting pattern group A that is a projection pattern for defining a measurement line. The projection imaging control unit 20 commands the projection structure 1 to project it onto the test object 7. Part of the predetermined pattern.

  In FIG. 2, white indicates a bright portion and black indicates a dark portion. In the patterns A_ref and A_inv, a bright portion and a dark portion are alternately repeated, and the light and darkness are reversed in both patterns. A complementary illumination distribution can be formed. The numbers in FIG. 2 are pattern numbers, and are set so that A_ref is bright when the number is even and A_inv is bright when the number is odd.

  FIG. 3 schematically shows a two-dimensional captured image when the imaging unit 1 captures a pattern projected onto the test object 7 when the test object 7 has a planar shape.

  Here, the direction in which the pattern alternates on the image sensor 10 is the X direction, and the direction with little change is the Y direction. Further, hereinafter, the pixel number in the X direction is denoted by i, and the pixel number in the Y direction is denoted by j.

  FIG. 4 is a spatial coding pattern using a Gray code for setting a number to a measurement line. The pattern is a predetermined pattern that is projected by the projection imaging control unit 20 onto the test object 7 as instructed by the projection imaging control unit 20. It is a part.

  In FIG. 4, n corresponds to the bit number of the Gray code, n = 1 indicates the least significant bit, n = 2 indicates the second bit, and so on. In the spatial coding method, these patterns are sequentially projected and imaged respectively. Hereinafter, a pattern corresponding to bit number n is referred to as Gn, and a captured image corresponding to this is expressed as IGn.

  Each processing according to the present embodiment described below is mainly performed by the gradation intersection calculation unit 21 included in the three-dimensional shape measurement apparatus.

<Acquisition of measurement line>
The projection imaging control unit 20 projects the two types of patterns A_ref and A_inv of the measurement line setting pattern group A shown in FIG. The captured images IA_ref (i, j) and IA_inv (i, j) for each are sequentially acquired using the unit 8.

  FIG. 5 shows luminance data when the imaging line setting pattern group A projected onto the test object 7 is imaged by the imaging unit. The solid line corresponds to IA_ref, and the broken line corresponds to IA_inv.

  The horizontal axis in FIG. 5 is the imaging data pixel number in the coordinate X direction. The vertical axis represents the luminance at each pixel and is normalized by the maximum value of the pattern imaging luminance. Also, the brightness is set so that the dark part is 0 and the bright part is positive. In the complementary pattern method, the X-coordinate of the intersection of the solid line and the broken line in FIG. 5 is calculated as the measurement line position where the luminances of IA_ref and IA_inv are equal, but this takes the difference between the two signals and the value is 0. May be calculated as the measurement line position.

  FIG. 6 shows a complementary pattern difference signal Sa obtained by subtracting IA_inv from IA_ref. In FIG. 6, there are positions where the value is 0 between pixel numbers 4 and 5, between 9 and 10, between 14 and 15, and between 19 and 20, and these positions are taken as measurement line positions.

  Two pixels sandwiching a measurement line on the image sensor 10 are referred to as “adjacent pixels” in the present invention, and among these, a pixel having a smaller pixel number is referred to as a “left adjacent pixel” for convenience and a pixel having a larger pixel number is referred to as a “right adjacent pixel”. To do. When the measurement line coincides with the pixel position, this pixel number is referred to as “left adjacent pixel”, and the pixel whose pixel number is one larger is referred to as “right adjacent pixel”.

  Since the A_ref pattern is turned on at the even number and turned off at the odd number, and the A_inv pattern is the opposite, the complementary pattern difference signal Sa obtained by subtracting the IA_inv luminance value from the IA_ref luminance value has an even number in the positive part and a negative part in the complementary pattern difference signal Sa. Each corresponds to an odd number of patterns. The measurement line position exists at the boundary between these pattern bright and dark, and if the number given to each pattern is defined as the smaller absolute value of the adjacent pattern number, the measurement line A is an even number (4) in FIG. It turns out that B is an odd number (9). That is, if the difference data changes from positive to negative at the measurement line position, it may be determined as an even measurement line, and if the difference data changes from negative to positive, it may be determined as an odd measurement line.

<Numbering measurement lines>
Next, the projection imaging control unit 20 sequentially projects the spatial coding pattern group Gn (n = 1 to 5) shown in FIG. 4 onto the test object 7 using the projection unit 1, and the imaging unit 8 is The captured images IGn (i, j) for each are sequentially acquired. Conventionally, the numbering of the measurement line is performed by examining how the value of the adjacent pixel of the measurement line changes in each IGn, and this is regarded as a well-known gray code. This is done by converting to hex.

  For example, the adjacent pixel corresponding to the measurement line number 7 exists in the vicinity of the boundary between the numbers 7 and 8 in the pattern group of FIG. 4 and the gray code is 0, n because IG1 where n = 1 is a dark part. IG2 with = 2 is 0 because it is a dark part, IG3 with n = 3 is 1 because it is a bright part, and IG5 with n = 5 is 0 because it is a dark part. In IG4 where n = 4, the brightness changes at the boundary portion, but the adjacent projection pixel with a small pattern number is a dark portion, so that the gray code is set to 0. In this case, it becomes “00100” in order from the upper bits, and the number 7 can be obtained by using the Gray code conversion technique.

  However, in IG4, since the brightness changes in the adjacent pixel portion, it is difficult to specify the value, and erroneous determination may occur. If it is determined that IG4, which should be 0, is 1, the gray code is “01100”, and if this is converted, the number 8 is obtained. In this invention, it is expressed as “indeterminate” that the value of the spatial coding pattern changes at adjacent pixel positions and thus it becomes difficult to obtain a value.

  Therefore, in the present embodiment, when encoding a measurement line, an encoding pattern number in which the value of an adjacent pixel of the measurement line is indefinite is predicted, and another image is not determined without determining the bit corresponding to this image. The value is determined from the information.

<Signaling of measurement line with even code>
FIG. 7 shows how the gray code pattern changes for even measurement line numbers. In the measurement line to which an even number should be given, only the pattern corresponding to the least significant bit (n = 1) has the light and darkness reversed in the vicinity of the adjacent pixel of the measurement line, and the higher order bits (n = 2 to 5). Then there is no inversion of light and dark between adjacent pixels.

  In the first place, the Gray code has a characteristic that only one bit changes when the code changes, 0 and 1 change, and the least significant bit (n = 1) switches between light and dark every two pixels. This is because there are characteristics.

  In the present embodiment, the measurement accuracy is improved by omitting the evaluation of the pattern with respect to the pattern in which the brightness inversion occurs in the pixels adjacent to the measurement line. That is, for the even-numbered measurement line, the evaluation of the pattern of the least significant bit (n = 1) is ignored, and the pattern of the other bits is measured. For example, as for the value of the adjacent pixel corresponding to the measurement line code 6, n = 1 is indefinite, and the upper part from n = 2 to n = 5 is dark, bright, dark, dark, so the gray code is an indefinite bit. X is “0010X” in order from the upper bit.

  The gray code has a characteristic that the even number of the numerical value to which it corresponds and the parity of the code (the number of 1s in the code) match. The parity known by “0010X” is 1, which is an odd number, but the numerical value to be represented should be an even number, so X must be 1. Therefore, the corresponding gray code is “00101”, which represents the number 6.

  For the measurement line code 8, similarly, the indefinite bit is X and the gray code is “0110X”. To make the parity even, X = 0, that is, “01100” may be used.

  In this way, among the plurality of pattern image data (IGn), the specific pattern image data (IG1) whose light and darkness should be determined is determined according to the position of interest (measurement line position having an even code) on the pattern image data. Identify. The brightness of the specific pattern image data (IG1) is determined based on the brightness of the other pattern image data (IG2 to IG5) and the parity value (even number) corresponding to the position of interest (measurement line position having an even code). To decide.

  In addition, since even numbers are indeterminate only when n = 1, instead of predicting X, a numerical value is obtained with a Gray code excluding n = 1 and finally doubled. By doing so, the correct value can be obtained. For example, for “0010X”, the gray code “0010” excluding X is number 3 and is doubled to become number 6, and for “0110X”, the gray code “0110” excluding X is number 4. This is doubled to become number 8.

<Signaling of measurement line having odd number code>
FIG. 8 shows how the gray code pattern changes for odd measurement line numbers. For a measurement line to which an odd number is to be given, brightness is inverted at a pixel adjacent to the measurement line with a high-order bit of n = 2 or more, but the position changes depending on the measurement line code.

  However, there is a characteristic that the bit lower by one of the bits to be inverted is always 1 and that it is n = 1, or the bit lower than that bit is always 0.

  In FIG. 8, when X is an indefinite bit, “000X1” is measured in order from the upper bit in measurement line code 1, “00X10” in measurement line code 3, “001X1” in measurement line code 5, and “0X100” in measurement line code 7. "011X1" for the measurement line code 9, "01X10" for the measurement line code 11, "010X1" for the measurement line code 13, and "X1000" for the measurement line code 15, indicating that the above characteristics are obtained.

  That is, for a measurement line having an odd number of codes, 0 and 1 are evaluated from the lower bits, the upper bits of the portion that becomes 1 for the first time are specified as indefinite, and the remaining bits are evaluated.

  Finally, a parity check excluding the indefinite bits is performed, and the codes are calculated by determining 0 and 1 of the indefinite bits so that the overall parity becomes an odd number.

  For example, at the position of the measurement line code 7, it is “0X100”, and since the parity is 1, X is set to 0 to “00100” in order to make the parity odd, and the code 7 is obtained therefrom. Further, since the position of the measurement line code 13 is “010X1” and the parity is 2, X is set to 1 to “01011” in order to make the parity odd, and the code 13 can be obtained therefrom.

  In this way, among the plurality of pattern image data (IGn), specific pattern image data for which light and darkness should be determined (0 to 1 are evaluated from the lower bits, and IGk corresponding to the upper bits of the portion that becomes 1 for the first time) ) Is specified according to the position of interest (measurement line position having an odd code) on the pattern image data. Further, the light and darkness of the specific pattern image data (IGk), the parity corresponding to the lightness and darkness of the other pattern image data (IG1,..., IGk-1, IGk + 1,...) And the position of interest (measurement line position having an odd code). It is determined based on the value (odd number).

  As described above, in the present embodiment, among a plurality of pattern image data (IGn), a region of interest of specific pattern image data (for example, IG1 (n = 1) in the example of a measurement line having an even code), an odd number In the example of the measurement line having a code, 0 and 1 are evaluated in order from the lower bit of n = 1, and the lightness of IGk + 1), which is the upper bit of n = k corresponding to the bit that has become 1 first, It is determined based on the contrast of the pattern image data (for example, IG2 to IG5 (n = 2 to 5) in the example of the measurement line having an even code and IGn other than IGk + 1 in the example of the measurement line having an odd code). . Then, based on the determined brightness of the specific pattern image data and the brightness of the other pattern image data, the three-dimensional coordinates of the test object are calculated by a spatial encoding method.

  According to the present embodiment, it is possible to improve the accuracy of spatial coding by reducing instability of spatial coding pattern determination, and to shorten processing time by omitting unnecessary pattern determination. .

(Second Embodiment)
In the first embodiment, a pattern that repeats light and dark for each code is used as the complementary pattern, but other patterns may be used. In the second embodiment, a case where four patterns that repeat light and dark for every two pixels are used will be described.

  FIG. 9 shows measurement line setting pattern groups B and C according to the second embodiment. The pattern according to the second embodiment has a merit that it is less affected by the resolution of the projection system and the imaging system because the pattern is coarser and the spatial frequency is lower than that of the first embodiment.

  Similar to the first embodiment, the measurement line is set at the boundary of the gray code, and the smaller one of the adjacent gray codes is associated with the code of the measurement line. The position of the measurement line is specified as a position where the difference between the complementary patterns is obtained as in the first embodiment and becomes zero.

  When the difference data of the complementary pattern B is Sb = B_ref−Binv, the 0 point position is configured to correspond to an even number of measurement line codes. When the difference data of the complementary pattern C is Sc = C_ref−C_inv, the 0 point position is configured to correspond to the odd number of the measurement line code.

  Further, the measurement line at the position where Sc changes from positive to negative corresponds to 1, 5, 9,..., That is, {4 × (k−1) +1; k = 1, 2,. The measurement line at the positive change position is 3, 7, 11,..., That is, {4 × (k−1) +3}.

  A measurement line defined by the complementary pattern B is referred to as a B column, and is defined by the complementary pattern C. Further, as described above, 1, 5, 9,..., {4 × (k−1) +1; The measurement line group that becomes 2, ...} is referred to as column C1, and 3, 7, 11, ..., that is, the measurement line group that becomes {4 × (k-1) +3}, is referred to as column C3.

  Since all measurement lines are even in column B, the encoding method for even measurement lines described in the first embodiment may be used. In the C1 column, all the measurement lines are odd numbers, and the pattern of n = 1 is always 1 as shown in FIG. 8, and there is an indefinite bit at n = 2. In the C3 column, all the measurement lines are odd numbers, the pattern of n = 1 is always 0, and there are undefined bits at n = 3 or higher.

  Therefore, in the measurement line determined to be the C1 column, the parity is measured by determining the brightness of the pattern of n = 3 or more in the adjacent pixels, and the value of the indefinite bit is determined so that the parity of all bits is an odd number. For the measurement line determined to be the column C3, the indefinite bit search and determination similar to the odd measurement line described in the first embodiment may be performed.

  In this way, it is not necessary to search for the position of the indefinite bit in the B column and the C1 column, and it is not necessary to perform the pattern determination of n = 1 for all the columns, so the corresponding pattern image is referred to. There is no need.

  For this reason, according to the present embodiment, it is possible to reduce the time required for image brightness determination and the indefinite bit search time, and there are advantages such as brightness determination, reduction in search error probability, and the like.

  As described above, according to the present invention, a pattern in which a light / dark boundary exists at a corresponding position in a spatially encoded pattern group for each measurement line is predicted, and a value corresponding to this pattern is set to another pattern group. Therefore, it is possible to accurately specify a code without identifying an unstable pattern for light / dark determination.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (5)

Projecting means for projecting a plurality of patterns onto the test object,
Imaging means for capturing images of the test objects onto which the plurality of patterns are respectively projected, and acquiring the plurality of pattern image data;
Determining means for determining the contrast of the region of interest of the specific pattern image data among the plurality of pattern image data based on the contrast of the corresponding region of the other pattern image data;
Calculation means for calculating the three-dimensional coordinates of the test object based on the brightness of the specific pattern image data determined by the determination means and the brightness of the other pattern image data;
Equipped with a,
Wherein the plurality of patterns, three-dimensional shape measuring device according to claim gray code pattern der Rukoto.   The said determination means specifies the said specific pattern image data which should determine light and dark among these pattern image data according to the focus position on pattern image data. 3D shape measuring device.   The said determination means determines the brightness of the said specific pattern image data based on the brightness of the said other pattern image data, and the value of the parity corresponding to the said attention position. 3D shape measuring device. A control method for a three-dimensional shape measuring apparatus comprising a projecting means, an imaging means, a determining means, and a calculating means,
A projecting step in which the projecting means projects each of the plurality of patterns onto the test object;
An imaging step in which the imaging means images each of the test objects onto which the plurality of patterns are projected, and acquires the plurality of pattern image data;
A determining step in which the determining means determines the contrast of the region of interest of the specific pattern image data among the plurality of pattern image data based on the contrast of the corresponding region of the other pattern image data;
A calculating step for calculating the three-dimensional coordinates of the test object based on the brightness of the specific pattern image data determined in the determining step and the brightness of the other pattern image data;
I have a,
Wherein the plurality of patterns, the control method of the three-dimensional shape measuring apparatus, wherein a gray code pattern der Rukoto. The program for making a computer perform each process of the control method of the three-dimensional shape measuring apparatus of Claim 4 .

Images