JP5032943B2 - 3D shape measuring apparatus and 3D shape measuring method - Google Patents

Description

  The present invention relates to a three-dimensional shape measuring apparatus and a three-dimensional shape measuring method for measuring a three-dimensional shape of an object to be measured.

  Conventionally, the phase shift method has been widely used as a method for measuring the three-dimensional shape of an object. The phase shift method is a kind of light cutting method, in which a projection device projects a specific pattern onto the object to be measured, and the projection image irregularly reflected on the surface of the object to be measured is taken from a direction different from the projection optical axis of the projection device. The three-dimensional shape of the object to be measured is measured by image processing with an apparatus and image processing of a grayscale image (hereinafter simply referred to as “image”) captured by the imaging apparatus. Hereinafter, the principle of the phase shift method will be described in more detail.

  First, the projection device projects a fringe pattern whose luminance changes in a sine wave shape onto the object to be measured, and the phase of the fringe pattern is shifted by, for example, π / 2 and imaged by the imaging device. Repeat several times (minimum 3 times, usually 4 times or more) until moving for one cycle. Here, the luminance (density) at the same position on the four images captured by the imaging device is the absolute brightness, even if the absolute brightness changes depending on the surface property or color of the measurement object at that position, Since the relative luminance difference always shows a change of only the phase difference of the projected pattern, the relative phase value of the projected pattern at that position is obtained.

  Since the relative phase φ is a value for each period of the projected fringe pattern, that is, a value between −π and π, in order to obtain the absolute phase φ of the fringes projected for a plurality of periods, the fringe order n (from one end to the other) It is necessary to perform processing for estimating where the fringes (values representing the fringes in the nth cycle counted toward the end) are located on each captured image.

  Then, the absolute phase Φ (= φ + 2nπ) is obtained using the relative phase value and the fringe order at each point of the four captured images. This process is hereinafter referred to as “phase connection process”. A line (equal phase line) obtained by connecting points having the same absolute phase Φ obtained by the phase connection processing represents the shape of a cross section obtained by cutting the object to be measured along a certain plane in the same manner as the cutting line in the optical cutting method. Based on this absolute phase Φ, the three-dimensional shape of the object to be measured (height information at each point in the image) can be measured by the principle of triangulation.

  Thus, since the absolute phase value of the sine wave fringe, that is, the position on the actual lattice used for the projection of the fringe pattern and the imaging point position (coordinates on the image) on the image pickup device of the image pickup apparatus can be specified. Based on the principle of triangulation based on the optical arrangement of the imaging device, the absolute coordinate value (X, Y, Z) in the three-dimensional space of the projection point on the object to be measured corresponding to the point on the image is obtained. If it calculates | requires, the three-dimensional shape of a to-be-measured object can be obtained.

  As described above, in the phase shift method, three-dimensional coordinates corresponding to each point of the obtained image are obtained by a simple calculation process, and measurement data of a high-density three-dimensional shape is obtained by imaging at least three times. When evaluated by the measurement time per point, the measurement can be performed at a speed several to several tens of thousands times faster than other measurement methods.

  However, when there is a step or a hole on the surface of the object to be measured and the boundary (phase boundary) between the stripes cannot be specified, the phase connection processing becomes difficult because the stripe order n cannot be obtained correctly, There is a disadvantage that a problem that a correct distance cannot be obtained (phase jump problem) occurs. In addition, the phase shift method projects as many fringes as possible and takes an image so that the contrast of the projected fringes is as high as possible on the captured image. There is a problem that the phase jump problem is likely to occur and it is difficult to obtain an absolute phase value necessary for height measurement.

Therefore, various three-dimensional shape measuring apparatuses and three-dimensional shape measuring methods for solving the phase skip problem have been proposed (see Patent Documents 1 and 2). In the method described in Patent Document 1, a reference sine wave and a sine wave having an integral multiple of the reference sine wave are superimposed and projected, and information on the relative phase obtained at each wavelength is integrated to determine the phase boundary position. Get uniquely. Further, in the method described in Patent Document 2, a so-called spatial code method and a phase shift method are combined, and a spatial code pattern for determining the fringe order n is obtained by light having a wavelength different from the wavelength of the light that projects the fringe pattern. Projecting simultaneously with the fringe pattern, and separating the captured image in which the fringe pattern and the spatial code pattern are simultaneously projected into an image on which the fringe pattern is projected based on the wavelength of light and an image on which the spatial code pattern is projected, The stripe order n is determined from a plurality of images onto which the spatial code pattern is projected. The space code pattern is for coding a space. For example, for a stripe pattern in the T period, a dark pattern if the bit value is 0, corresponding to the binary representation of T, If the bit value is 1, it is a light / dark pattern configured as a light pattern, and is projected in order from the light / dark pattern for the least significant bit.
Japanese Patent No. 399041 JP 2006-177771 A

  However, in the conventional example described in Patent Document 1, the number of images required for one measurement (the number of times of image pickup by the image pickup apparatus) increases, so that the time required for measurement becomes long (for example, In Patent Document 1, eight times of imaging are required). On the other hand, in the conventional example described in Patent Document 2, it is possible to measure with a smaller number of images (number of times of imaging) than in the conventional example described in Patent Document 1, but the light that projects the fringe pattern is used. In order to simultaneously project a spatial code pattern with light of a wavelength different from the wavelength, a projection device capable of projecting a color pattern and an imaging device capable of capturing a color image are required, and costs are lower than when measuring with a monochrome image. There is a problem of rising.

  The present invention has been made in view of the above points, and provides a three-dimensional shape measuring apparatus and a three-dimensional shape measuring method capable of measuring a three-dimensional shape of an object to be measured in a relatively short time while suppressing an increase in cost. The purpose is to do.

  In order to achieve the above object, the invention of claim 1 projects a projection pattern that projects an arbitrary pattern of light onto the object to be measured and a projection pattern comprising a predetermined light pattern by controlling the projection means on the object to be measured. A projection pattern control means for imaging, an imaging means for imaging the measurement object, an image acquisition means for acquiring a grayscale image of the measurement object on which the projection pattern is projected from a captured image of the imaging means, and the density acquired by the image acquisition means Relative phase calculation means for calculating the relative phase of the projection pattern at an arbitrary position in the image, absolute phase calculation means for calculating the absolute phase of the projection pattern by connecting the relative phases of the arbitrary positions, and measurement from the absolute phase A three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the object, and the projection pattern control means includes a plurality of types of fringe patterns in which the intensity of light is changed by shifting the intensity of the sine wave and the known phase. In FIG. 5, a projection pattern in which each amplitude is changed for each period is generated and projected onto the measurement object by the projection means, and the relative phase calculation means projects the projection pattern on the grayscale image of the measurement object imaged. The relative phase is calculated based on the density value, and the absolute phase calculation means calculates the amplitude of the projection pattern based on the density value in the grayscale image of the measured object projected by projecting the projection pattern, and from the amplitude It is characterized in that the period is estimated and the relative phase is connected in phase to calculate the absolute phase.

  According to a second aspect of the present invention, in order to achieve the above object, a projection unit that projects an arbitrary pattern of light onto the object to be measured, and a projection pattern composed of a predetermined light pattern is projected onto the object to be measured by controlling the projection unit. A projection pattern control means for imaging, an imaging means for imaging the measurement object, an image acquisition means for acquiring a grayscale image of the measurement object on which the projection pattern is projected from a captured image of the imaging means, and the density acquired by the image acquisition means Relative phase calculation means for calculating the relative phase of the projection pattern at an arbitrary position in the image, absolute phase calculation means for calculating the absolute phase of the projection pattern by connecting the relative phases of the arbitrary positions, and measurement from the absolute phase A three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the object, and the projection pattern control means includes a plurality of types of fringe patterns in which the intensity of light is changed by shifting the intensity of the sine wave and the known phase. Among them, a projection pattern in which the amplitude is changed according to the period in at least one type of stripe pattern, and a projection pattern in which the amplitude is changed in a manner different from the projection pattern according to the period in the remaining stripe pattern, A uniform projection pattern consisting of the center value of the amplitude of the fringe pattern is generated and projected onto the object to be measured by the projecting means, and the relative phase calculating means projects the projection pattern onto the grayscale image of the object to be imaged. The relative phase is calculated based on the value, and the absolute phase calculating means calculates the amplitude of the projection pattern based on the density value in the grayscale image of the object to be measured that is projected by projecting the projection pattern, and the period from the amplitude. And calculating the absolute phase by connecting the relative phases in phase.

  According to a third aspect of the present invention, in the first or second aspect of the invention, the image acquisition means acquires a grayscale image of the object to be measured in a state where the projection pattern is not projected from the captured image of the imaging means. It is provided with.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the projection pattern control means is a plurality of types of stripes in which the intensity of light is changed by shifting the phase in a sine wave shape and at equal intervals. A projection pattern is generated from the pattern.

  In order to achieve the above object, the invention according to claim 5 projects a projection pattern composed of a predetermined light pattern onto the measurement object, obtains a grayscale image of the measurement object on which the projection pattern is projected, and projects the projection pattern. The relative phase of the projection pattern at an arbitrary position in the grayscale image is calculated, the absolute phase of the projection pattern is calculated by connecting the relative phases of the arbitrary positions in phase, and the three-dimensional coordinates of the object to be measured are calculated from the absolute phase. A three-dimensional shape measuring method for measuring a three-dimensional shape of an object to be measured by calculating, wherein each amplitude is measured in a plurality of types of fringe patterns in which the intensity of light is changed by shifting the phase of a sine wave and a known amount of phase. A process of generating a projection pattern changed every cycle and projecting it onto the measurement object, and an excessive calculation of the relative phase based on the density value in the grayscale image of the measurement object imaged by projecting the projection pattern. And calculating the amplitude of the projection pattern based on the density value in the grayscale image of the object measured by projecting the projection pattern, estimating the period from the amplitude, and connecting the relative phase in phase to obtain the absolute phase. And a process of calculating.

  In order to achieve the above object, the invention of claim 6 projects a projection pattern composed of a predetermined light pattern onto the measurement object, obtains a grayscale image of the measurement object on which the projection pattern is projected, and projects the projection pattern. The relative phase of the projection pattern at an arbitrary position in the grayscale image is calculated, the absolute phase of the projection pattern is calculated by connecting the relative phases of the arbitrary positions in phase, and the three-dimensional coordinates of the object to be measured are calculated from the absolute phase. A three-dimensional shape measurement method for measuring a three-dimensional shape of an object to be measured by calculating, wherein at least one of a plurality of types of fringe patterns in which the intensity of light is changed by shifting the intensity of a sine wave and a known amount phase. A projection pattern in which the amplitude is changed according to the period in the stripe pattern of more than one type, and a projection pattern in which the amplitude is changed in a different manner from the projection pattern in accordance with the period in the remaining stripe pattern. Based on the density value in the grayscale image of the object measured by projecting the projection pattern and generating a uniform projection pattern consisting of the center of amplitude of the fringe pattern and projecting it onto the object. Calculating the relative phase, and calculating the amplitude of the projection pattern based on the density value in the grayscale image of the measured object projected by projecting the projection pattern, estimating the period from the amplitude, and calculating the relative phase. And a step of calculating an absolute phase by connecting the phases.

  The invention according to claim 7 is the invention according to claim 5 or 6, wherein in the process of obtaining the gray image of the measurement object on which the projection pattern is projected, the gray image of the measurement object in a state where the projection pattern is not projected is obtained. It has the process to perform.

  The invention according to claim 8 is the invention according to any one of claims 5 to 7, wherein the projection pattern control means is a plurality of types of stripes in which the intensity of light is changed by shifting the phase in a sine wave shape and at equal intervals. A projection pattern is generated from the pattern.

  According to the first and fifth aspects of the present invention, in each of the sinusoidal fringe patterns essential in the three-dimensional shape measurement using the phase shift method, a projection pattern with the amplitude changed for each period is generated to be covered. Since it is projected onto the measurement object, the period can be estimated from the difference in amplitude for each period and the relative phase can be connected in phase, so there is no need to use an expensive projection device or imaging device as in the conventional example, and as a result, The three-dimensional shape of an object having a step or a hole on the surface can be measured with a small number of imaging operations while suppressing an increase in cost.

  According to the second and sixth aspects of the present invention, in each of the sinusoidal fringe patterns essential for the three-dimensional shape measurement using the phase shift method, the amplitude is changed in accordance with the period in at least one kind of fringe pattern. Projecting means for generating a projected pattern, a projected pattern in which the amplitude of the remaining fringe pattern is changed in a manner different from that of the projected pattern according to the period, and a uniform projected pattern composed of the amplitude center value of the fringe pattern Since the relative phase can be phase-connected by estimating the period from the difference in amplitude for each period in order to project onto the object to be measured more, there is no need to use an expensive projection apparatus or imaging apparatus as in the conventional example, and as a result, The three-dimensional shape of an object having a step or a hole on the surface can be measured with a small number of imaging operations while suppressing an increase in cost. In addition, since the period is estimated by combining projection patterns having different amplitudes, it is possible to estimate many periods even if the number of types of amplitude is small, thus avoiding phase detection with a small amplitude, It can prevent that detection accuracy falls.

  According to the third and seventh aspects of the present invention, it is possible to measure a three-dimensional shape in consideration of light other than light irradiated as a projection pattern from the projection device irradiated to the object to be measured. Measurements can be made without preparing a place that blocks ambient light.

  According to the fourth and eighth aspects of the invention, since the projection pattern is generated from a plurality of types of fringe patterns whose phases are shifted at equal intervals, the calculation necessary for estimating the period is shifted by a known amount phase. This can be done more easily than

  Hereinafter, the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
As shown in FIG. 1, the three-dimensional shape measurement apparatus of the present embodiment includes a projection apparatus 1 that projects light of an arbitrary pattern onto the measurement object M, an imaging apparatus 2 that images the measurement object M, and the projection apparatus 1. And an image processing device 3 that controls the imaging device 2 and performs image processing on a captured image captured by the imaging device 2.

  The projection device 1 includes a light source 10, a transmissive liquid crystal panel 11 disposed in front of the light source 10, and a so-called lens 12 that condenses the light that has passed through the liquid crystal panel 11 on the surface of the measurement object M. It consists of a liquid crystal projector. In addition, the imaging device 2 is a solid-state imaging device such as a CCD or CMOS, an optical system that forms an image of a subject on the imaging surface of the solid-state imaging device, and performs signal processing on the output of the solid-state imaging device to obtain a luminance (density value) ) Is a so-called monochrome type still camera. The image processing apparatus 3 executes a general-purpose computer (hardware) including a CPU, a memory, a display, a storage device such as a hard disk, various input / output interfaces, and the three-dimensional shape measurement method according to the present invention. 3D shape measurement program (software) to be executed, and by executing the 3D shape measurement program on the CPU, the projection pattern control means 30, the image acquisition means 31, the non-pattern image acquisition means 32, the relative phase calculation Means 33, absolute phase calculation means 34, and three-dimensional coordinate calculation means 35 are realized.

  The projection pattern control means 30 generates a projection pattern, which will be described later, and stores it in a storage device in advance and reads out the stored projection pattern data as necessary, for example, for a general-purpose display such as DVI (Digital Visual Interface). A function of transmitting projection pattern data to the projection apparatus 1 via the interface and controlling the operation of the projection apparatus 1 (turning on / off the light source 10, adjusting the light amount, etc.) via a general-purpose communication interface such as RS232C or IEEE488. have. The projection apparatus 1 displays a projection pattern on the liquid crystal panel 11 based on the projection pattern data.

  The image acquisition means 31 acquires a digital image signal sampled and quantized by the imaging device 2 and acquires image data represented by luminance (density value) of each pixel from the acquired image signal. It has a function of storing in a memory. Note that the image acquisition unit 31 functions to control the operation of the imaging device 2 (imaging timing, etc.) via a general-purpose communication interface such as RS232C or IEEE488, and projects the projection pattern control unit 30 onto the projection device 1. It also has a function of instructing to project a pattern.

  The non-pattern image acquisition means 32 has a function of acquiring an image data by acquiring an image signal captured by the imaging device 2 in a state where a projection pattern is not projected from the projection device 1 and storing the image data in a memory. That is, the measurement object M is also irradiated with light other than the light irradiated as a projection pattern from the projection apparatus 1 (hereinafter, such light is referred to as “environment light”). The image data of the measurement object M when only ambient light is irradiated is acquired.

  The relative phase calculation means 33 has a function of calculating the relative phase φ described in the prior art by calculation. The absolute phase calculation means 34 has a function of determining the fringe order n and calculating the absolute phase Φ (= φ + 2nπ) by using the determined fringe order n and the relative phase φ as will be described later. . Further, the three-dimensional coordinate calculation means 35 has a function of calculating the three-dimensional coordinates of the projection point P (X, Y, Z) on the measurement object M corresponding to the point p (u, v) on the image by calculation. Thus, the three-dimensional shape of the measurement object M can be obtained (measured) from the three-dimensional coordinates obtained by the three-dimensional coordinate calculation means 35.

Here, in the present embodiment, in each of four types of stripe patterns (0, π / 2, π, 3π / 2) in which the intensity of light is changed sinusoidally and the phase is shifted by π / 2, the following four types of projection patterns obtained by changing the amplitude a _n of the sinusoidal wave in each cycle as shown in Table 1 were produced by a projection pattern control unit 30, to project the projection pattern from the projection apparatus 1 to the measured object M .

The four types of projection patterns P _k (k = 0, 1, 2, 3) and the stripe order n are expressed by the following equations. Where _An is the amplitude of the sine wave, Φ is the absolute phase, C is the center value of the sine wave, and [m] is the maximum integer not exceeding m.

Here, when the projection-pattern in the projection apparatus 1 is to consist of 8-bit image data, the center value C = 125 sine wave, when the maximum value = 125 of the amplitude A _n of the sinusoidal, projection pattern since the density value of P _k can be expressed in a range of 0 to 250, it can be configured as an 8-bit image data. The waveforms of the projection patterns P ₀ , P ₁ , P ₂ , and P ₃ at this time are shown in FIGS.

Next, the operation of the three-dimensional shape measurement apparatus of this embodiment, that is, the three-dimensional shape measurement method according to the present invention will be described. First, when the three-dimensional shape measurement apparatus starts operating by executing a three-dimensional shape measurement program on the CPU of the image processing apparatus 3, the image acquisition means 31 gives an instruction to the projection pattern control means 30, and projection is performed according to the instruction. The pattern control means 30 reads the image data of the first projection pattern P ₀ stored in the storage device into the work area of the memory, and further transmits it to the projection device 1 via the display interface to control the projection device 1. The projection pattern P ₀ is projected onto the measurement object M. After the image data is transmitted to the projection apparatus 1, the image acquisition unit 31 controls the imaging apparatus 2 to image the measurement object M on which the projection pattern P ₀ is projected, and outputs the image signal output from the imaging apparatus 2. The image data of the measurement object M on which the projection pattern P ₀ is captured and projected is acquired and stored in the memory. In addition, the image acquisition means 31 projects the remaining three types of projection patterns P ₁ , P ₂ , and P ₃ onto the projection device 1 in order, and the projection target P ₁ , P ₂ , and P ₃ are projected to be measured. Image data of the object M is acquired and stored in the memory.

On the other hand, the non-pattern image acquisition unit 32 causes the imaging device 2 to image the measurement object M in a state where the projection apparatus 1 is stopped, that is, in a state where only the environmental light is irradiated to the measurement object M. Image data of the measurement object M in a state where _Pk is not projected is acquired and stored in the memory.

The luminances (density values) I _{0 to} I ₃ at arbitrary pixels (u, v) on the image captured by the imaging device 2 in a state in which the projection pattern P _k is projected are expressed by the following formulas including the influence of ambient light. It is represented by However, E is the brightness | luminance by environmental light, (rho) is the surface reflectance of the to-be-measured object M, (beta) is a photoelectric conversion coefficient in the imaging device 2. FIG. I ₄ represents the luminance (density value) at an arbitrary pixel (u, v) on the image captured by the imaging device 2 in a state where the projection pattern P _k is not projected.

Here, when a _n = βρA _n , c = βρC, and e = βρE, each luminance I _k (k = 0, 1, 2, 3, 4) is expressed by the following equation.

Relative phase calculating means 33, using the respective difference values of _{I 0} and _{I 2} and _{I 1} and _{I 3,} the estimated value of the relative phase phi by the following equation <phi> sine wave amplitude _{a n} (= _βρA n the estimated value of) obtaining the <a _n>.

Then, the absolute phase calculation means 34 obtains an estimated value <c> of the center value c (= βρC) of the sine wave using the addition value of I _{0 to} I ₄ by the following equation.

Here, the absolute phase calculating means 34, the sine wave center value C, and the estimated value <c>, estimate of the amplitude a _n sine wave using <a _n>, the estimation of the amplitude A _n of the sinusoidal obtains the value <A _n> by the following equation, the estimated value of the fringe order n corresponding to the estimated value <A _n> of the amplitude a _n of the sine wave with reference to Table 1 Request <n>.

  Further, the absolute phase calculation means 34 obtains the estimated value <Φ> of the absolute phase Φ from the estimated value <φ> of the relative phase φ and the estimated value <n> of the fringe order n by the following equation.

  Then, the three-dimensional coordinate calculation means 35 obtains height information (coordinate values of each pixel in the three-dimensional space) at each pixel (u, v) from the absolute phase estimated value <Φ> by the principle of triangulation. If the absolute coordinate value in the three-dimensional space of the projection point on the measurement object M corresponding to each pixel (u, v) is obtained, the three-dimensional shape of the measurement object M can be obtained.

As described above, according to this embodiment, the projection at each of the sinusoidal fringe pattern is essential in the three-dimensional shape measurement using the phase shift method, which changes the amplitude A _n of the sine wave in each cycle since the generated pattern is projected on the measured object M, since the relative phase can be phase connection by estimating the period from the difference of the amplitude a _n of the sine wave of each cycle, conventionally as described in Patent Document 2 There is no need to use an expensive projection device or imaging device as in the example, and as a result, the three-dimensional shape of an object having a step or a hole on the surface is reduced in the number of times of imaging (5 times in this embodiment) while suppressing an increase in cost. It can be measured.

By the way, although the three-dimensional shape was measured using four types of projection patterns in the above, the three types (0, 2π / 3, 3) in which the intensity of light is changed in a sine wave shape and the phase is shifted by 2π / 3. Measurement may be performed using a 4π / 3) stripe pattern. In this case, the number of times of imaging can be reduced by one compared to the case of using four types of projection patterns. Hereinafter, a case where three types of stripe patterns are used will be described. Three types of projection patterns P _k (k = 0, 1, 2) are expressed by the following equations.

Luminances (density values) I _{0 to} I ₂ at arbitrary pixels (u, v) on the image captured by the imaging device 2 in a state in which the projection pattern P _k is projected are expressed by the following equation including the influence of ambient light. It is represented by Note that I ₃ represents the luminance (density value) at an arbitrary pixel (u, v) on the image captured by the imaging device 2 in a state where the projection pattern P _k is not projected.

Here, when a _n = βρA _n , c = βρC, and e = βρE, each luminance I _k (k = 0, 1, 2, 3) is expressed by the following equation.

Relative phase calculating means _33, using I 0 ~I _2, the estimated value of the relative phase phi by the following equation: <phi> an estimate of the amplitude _{a n} of the sinusoidal seek <a _n>.

Then, the absolute phase calculation means 34 obtains an estimated value <c> of the center value c of the sine wave by the following formula using the added value of I _{0 to} I ₃ .

Here, the absolute phase calculating means 34, the sine wave center value C, and the estimated value <c>, estimate of the amplitude a _n sine wave using <a _n>, the estimation of the amplitude A _n of the sinusoidal value <A _n> determined by the above equation (2), the estimated value of the fringe order n corresponding to the estimated value <A _n> of the amplitude _{a n} of the sine wave with reference to Table 1 Request <n>. Further, the absolute phase calculation means 34 obtains the estimated value <Φ> of the absolute phase Φ from the estimated value <φ> of the relative phase φ and the estimated value <n> of the fringe order n by the above equation (3). Then, the three-dimensional coordinate calculation means 35 obtains height information at each pixel (u, v) from the absolute phase estimated value <Φ> by the principle of triangulation, and the object corresponding to each pixel (u, v). If the absolute coordinate value in the three-dimensional space of the projection point on the measurement object M is obtained, the three-dimensional shape of the measurement object M can be obtained.

When there is no ambient light, that is, it can be approximated to E = 0, as in measurement in a dark room, the patternless image acquisition unit 32 is not required and the above four types of projection patterns are used. In any case of using different types of projection patterns, the number of times of imaging can be reduced by one. When there is no ambient light, in the measurement using the four types of projection patterns, each calculation is performed with E = 0, e = 0, and I ₄ = 0. In the measurement using the three types of projection patterns, Each calculation may be performed with E = 0, e = 0, and I ₃ = 0.

The surface reflectance of the object to be measured M [rho, sine wave from long photoelectric conversion coefficient β is known in the imaging apparatus 2, if the estimated value of the amplitude a _n sine wave <a _n> Motomare (1) it is possible to the obtain an amplitude a _n estimate <A n _of>, in any case of using a case and three projection pattern using four types of projection patterns of the well, the estimation of the center value c of the sine wave Since the process for obtaining the value <c> is not required, and thus the non-pattern image acquisition means 32 is not required, the number of times of imaging can be reduced by one in the presence of ambient light.

Incidentally, in the above, but the relationship between the amplitude A _n and fringe order n sine wave was defined as shown in Table 1, line order n changes in every period of the amplitude A _n of the sine wave, the amplitude change coefficient α using May be defined as:

  In this case, the estimated value <n> of the stripe order n is expressed by the following equation from the above equation and the equation (2). In addition, を m を indicates an integer closest to m.

And Thus, by changing the amplitude change coefficient α appropriately, can be measured using the projection pattern was varied for each cycle of the amplitude A _n of the sine wave.

(Embodiment 2)
The three-dimensional shape measurement apparatus of the present embodiment is different in that the projection pattern control unit 30 generates a projection pattern different from that of the first embodiment, but the configuration of the apparatus is the same as that of the first embodiment. Constituent elements common to the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Further, the three-dimensional shape measurement method is also common to the first embodiment, and thus the description of the points common to the first embodiment is omitted.

In the present embodiment, each of four types of stripe patterns (0, π / 2, π, 3π / 2) in which the intensity of light is changed sinusoidally and shifted in phase by π / 2, according to the period. two of changing the amplitude a _n of the sinusoidal Te (0, π / 2) projection pattern, and the remaining two of changing the amplitude B _n of the sine wave in accordance with the period of the (π, 3π / 2) The projection pattern control means 30 generates a uniform projection pattern composed of the projection pattern 1 and the center value C of the sine wave, and projects the projection pattern from the projection apparatus 1 onto the measurement object M. As shown in Table 2 below, the amplitudes A _n and B _n of the sine wave are defined to change differently according to the period.

The four types of projection patterns P _k (k = 0, 1, 2, 3) and the stripe order n are expressed by the following equations. Where A _n and B _n are amplitudes of the sine wave, Φ is an absolute phase, C is a center value of the sine wave, and [m] is a maximum integer not exceeding m.

Here, assuming that the pattern that can be projected by the projection apparatus 1 is composed of 8-bit image data, assuming that the sine wave center value C = 125 and the maximum values of the sine wave amplitudes A _n and B _n = 125. Since the density value of the projection pattern _Pk can be expressed in the range of 0 to 250, it can be configured as 8-bit image data. The waveforms of the projection patterns P ₀ , P ₁ , P ₂ , P ₃ and P ₄ at this time are shown in FIGS. 3 (a) to 3 (e), respectively.

The luminances (density values) I _{0 to} I ₄ at arbitrary pixels (u, v) on the image captured by the imaging device 2 in the state in which the projection pattern P _k is projected are expressed by the following equations including the influence of ambient light. It is represented by However, E is the brightness | luminance by environmental light, (rho) is the surface reflectance of the to-be-measured object M, (beta) is a photoelectric conversion coefficient in the imaging device 2. FIG. Note that I ₅ represents the luminance (density value) at an arbitrary pixel (u, v) on the image captured by the imaging device 2 in a state where the projection pattern P _k is not projected.

Here, if a _n = βρA _n , b _n = βρB _n , c = βρC, e = βρE, each luminance I _k (k = 0, 1, 2, 3, 4, 5) is expressed by the following equation. Is done.

The relative phase calculation means 33 obtains an estimated value <φ> of the relative phase φ by the following equation using the difference values of I ₀ and I ₂ and I ₁ and I ₃ .

Then, the absolute phase calculating means 34, the sum of _{I 0} and _{I 2} and _{I 1} and _{I 3,} and _I 4, with _{I 5,} the sine wave amplitude _a n, the estimated value of _{b n <a} n> , <B _n >, an estimated value <c> of the center value c of the sine wave is obtained by the following equation.

Here, the absolute phase calculating means 34, the center value C of the sine wave, and its estimate <c>, the amplitude _a n of the sine wave, the estimate of _{b n <a} n>, using _{<b n>,} amplitude _a n of the sine wave, the estimate _<A n of _{B n>,} <the _{B n>} calculated by the following equation, the sine wave with reference to Table 2 the amplitude _a n, the estimated value of _{B n <A} n>, An estimated value <n> of the fringe order n corresponding to <B _n > is obtained.

  Further, the absolute phase calculation means 34 obtains the estimated value <Φ> of the absolute phase Φ from the estimated value <φ> of the relative phase φ and the estimated value <n> of the fringe order n by the above equation (3). Then, the three-dimensional coordinate calculation means 35 obtains height information at each pixel (u, v) from the absolute phase estimated value <Φ> by the principle of triangulation, and the object corresponding to each pixel (u, v). If the absolute coordinate value in the three-dimensional space of the projection point on the measurement object M is obtained, the three-dimensional shape of the measurement object M can be obtained.

As described above, according to the present embodiment, the sine wave amplitude _An is changed in accordance with the period in each of the sine wave-like fringe patterns essential in the three-dimensional shape measurement using the phase shift method. The remaining two types (π, 3π) in which the amplitude B _n of the sine wave is changed in a manner different from the change in amplitude of the two types of projection patterns according to the type (0, π / 2) projection pattern and the period. / 2) and a uniform projection pattern composed of the center value C of the sine wave are generated and projected onto the object M, the period is estimated from the difference between the amplitudes A _n and B _n for each period. Since the relative phases can be phase-connected, there is no need to use an expensive projection device or imaging device as in the conventional example described in Patent Document 2, and as a result, a three-dimensional shape of an object having a step or a hole on the surface can be obtained. Less photography while suppressing cost increase Number (in this embodiment 5 times) can be measured by. In addition, since the period is estimated by combining projection patterns having different amplitudes, it is possible to estimate many periods even if the number of types of amplitude is small, thus avoiding phase detection with a small amplitude, It can prevent that detection accuracy falls.

By the way, although the three-dimensional shape was measured using four types of projection patterns in the above, the three types (0, 2π / 3, 3) in which the intensity of light is changed in a sine wave shape and the phase is shifted by 2π / 3. Measurement may be performed using a 4π / 3) stripe pattern. In this case, the number of times of imaging can be reduced by one compared to the case of using four types of projection patterns. Hereinafter, a case where three types of stripe patterns are used will be described. Three types of projection patterns P _k (k = 0, 1, 2) are expressed by the following equations.

The luminances (density values) I _{0 to} I ₃ at arbitrary pixels (u, v) on the image captured by the imaging device 2 in a state in which the projection pattern P _k is projected are expressed by the following formulas including the influence of ambient light. It is represented by I ₄ represents the luminance (density value) at an arbitrary pixel (u, v) on the image captured by the imaging device 2 in a state where the projection pattern P _k is not projected.

Here, when a _n = βρA _n , b _n = βρB _n , c = βρC, e = βρE, each luminance I _k (k = 0, 1, 2, 3, 4) is expressed by the following equation. .

Relative phase calculating means _33, using I 0 ~I _3, estimates of relative phase phi by the following equation: <phi> an estimate of the amplitude _a n, _{b n} of the sinusoidal _<a n>, _<b n> Ask for.

Then, the absolute phase calculation means 34 uses I ₃ and I ₄ to obtain an estimated value <c> of the center value c of the sine wave by the following equation.

Here, the absolute phase calculating means 34, the center value C of the sine wave, and its estimate <c>, the amplitude _a n of the sine wave, the estimate of _{b n <a} n>, using _{<b n>,} amplitude _a n of the sine wave, the estimate of _{B n <A n>, <B} n> the equation (5), (6) determined by the equation, the amplitude _a n of the sine wave with reference to Table 2, _{B n} estimates _<A n>, obtaining an estimate of the fringe order n corresponding to _{<B n><n>.} Further, the absolute phase calculation means 34 obtains the estimated value <Φ> of the absolute phase Φ from the estimated value <φ> of the relative phase φ and the estimated value <n> of the fringe order n by the above equation (3). Then, the three-dimensional coordinate calculation means 35 obtains height information at each pixel (u, v) from the absolute phase estimated value <Φ> by the principle of triangulation, and the object corresponding to each pixel (u, v). If the absolute coordinate value in the three-dimensional space of the projection point on the measurement object M is obtained, the three-dimensional shape of the measurement object M can be obtained.

When there is no ambient light, that is, it can be approximated to E = 0, as in measurement in a dark room, the patternless image acquisition unit 32 is not required and the above four types of projection patterns are used. In any case of using different types of projection patterns, the number of times of imaging can be reduced by one. When there is no ambient light, in the measurement using the four types of projection patterns, each calculation is performed with E = 0, e = 0, and I ₅ = 0. In the measurement using the three types of projection patterns, Each calculation may be performed with E = 0, e = 0, and I ₄ = 0.

The surface reflectance of the object to be measured M [rho, if the photoelectric conversion coefficient β is known in the image pickup apparatus 2, the amplitude a _n of the sine _wave, the estimate of b _n <a _n>, which Motomema the <b _n> If (4) estimates _<A n of the amplitude _a n, _{B n} of the sinusoidal from expression>, <since _{B n>} can be obtained, if and three using four types of projection pattern of the projection pattern In any case, the process of obtaining the estimated value <c> of the center value c of the sine wave is not required, and therefore the non-pattern image acquisition means 32 is not required. Can be reduced once.

(Embodiment 3)
The three-dimensional shape measurement apparatus of the present embodiment is different in that the projection pattern control unit 30 generates a projection pattern different from that of the first embodiment, but the configuration of the apparatus is the same as that of the first embodiment. Constituent elements common to the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Further, the three-dimensional shape measurement method is also common to the first embodiment, and thus the description of the points common to the first embodiment is omitted.

In the present embodiment, in each of the three kinds of fringe patterns (0, S ₁ , S ₂ ) in which the light intensity is changed by shifting the phase by a sine wave and a known amount of S ₁ , S ₂ , three of the projected pattern obtained by changing the amplitude a _n of the sine wave in each cycle as shown in Table 1 were produced by a projection pattern control unit 30, to project the projection pattern from the projection apparatus 1 to the measured object M .

Three types of projection patterns P _k (k = 0, 1, 2) and fringe order n are expressed by the following equations.

Luminances (density values) I _{0 to} I ₂ at arbitrary pixels (u, v) on the image captured by the imaging device 2 in a state in which the projection pattern P _k is projected are expressed by the following equation including the influence of ambient light. It is represented by Note that I ₃ represents the luminance (density value) at an arbitrary pixel (u, v) on the image captured by the imaging device 2 in a state where the projection pattern P _k is not projected.

Here, when a _n = βρA _n , c = βρC, and e = βρE, each luminance I _k (k = 0, 1, 2, 3) is expressed by the following equation.

Relative phase calculating means 33 _calculates an estimate of the relative phase phi as follows <phi>, and estimates <a _n of the amplitude _{a n} of the sinusoidal> using I 0 ~I _2. First, the coefficients of sin (Φ) and cos (Φ) in the above expression are aligned as in the following expression.

Then, the estimated value of the relative phase phi with the above equation obtains the <phi>, and estimates <a n of the amplitude a _{n sinusoidal>.}

  Note that F, G, and H in the above equation are expressed by the following equations.

Then, the absolute phase calculating means 34, I ₀ and I _3, and estimation of the center value c of the sine wave using an estimate of the relative phase phi <phi>, and estimates <a n of the amplitude a _{n sinusoidal>} The value <c> is obtained by the following equation.

Here, the absolute phase calculating means 34, the sine wave center value C, and the estimated value <c>, estimate of the amplitude a _n sine wave using <a _n>, the estimation of the amplitude A _n of the sinusoidal value <A _n> determined by the above equation (2), the estimated value of the fringe order n corresponding to the estimated value <A _n> of the amplitude _{a n} of the sine wave with reference to Table 1 Request <n>. Further, the absolute phase calculation means 34 obtains the estimated value <Φ> of the absolute phase Φ from the estimated value <φ> of the relative phase φ and the estimated value <n> of the fringe order n by the above equation (3). Then, the three-dimensional coordinate calculation means 35 obtains height information at each pixel (u, v) from the absolute phase estimated value <Φ> by the principle of triangulation, and the object corresponding to each pixel (u, v). If the absolute coordinate value in the three-dimensional space of the projection point on the measurement object M is obtained, the three-dimensional shape of the measurement object M can be obtained.

As described above, according to the present embodiment, even when a projection pattern is generated from a plurality of types of fringe patterns whose phases are shifted by known amounts S ₁ and S ₂ instead of at equal intervals, similarly to the first embodiment. because the relative phase can be phase connection by estimating the period from the difference of the amplitude a _n of the sine wave of each cycle, there is no need to use an expensive projection device or an imaging device as in the conventional example described in Patent Document 2 As a result, the three-dimensional shape of an object having a step or a hole on the surface can be measured with a small number of imaging operations (four times in this embodiment) while suppressing an increase in cost.

When there is no ambient light, such as measurement in a dark room, that is, it can be approximated to E = 0, the non-pattern image acquisition means 32 is not required, and the number of imaging can be reduced by one. When there is no ambient light, each calculation may be performed with E = 0, e = 0, and I ₃ = 0.

The surface reflectance of the object to be measured M [rho, sine wave from long photoelectric conversion coefficient β is known in the imaging apparatus 2, if the estimated value of the amplitude a _n sine wave <a _n> Motomare (1) it is possible to in seeking amplitude a estimated value of _n <A _n>, without the need of a process of obtaining an estimate of the central value c of the sine wave <c>, therefore does not require a no-pattern image acquiring means 32 In addition, when there is ambient light, the number of imaging can be reduced by one.

It is a block diagram which shows embodiment of the three-dimensional shape measuring apparatus which concerns on this invention. (A)-(d) is a wave form diagram of the projection pattern in Embodiment 1. FIG. (A)-(e) is a wave form diagram of the projection pattern in Embodiment 2. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Projection apparatus 2 Imaging apparatus 3 Image processing apparatus 30 Projection pattern control means 31 Image acquisition means 32 Unpatterned image acquisition means 33 Relative phase calculation means 34 Absolute phase calculation means 35 Three-dimensional coordinate calculation means M Measurement object

Claims (8)

  Projection means for projecting an arbitrary pattern of light onto the measurement object, projection pattern control means for controlling the projection means to project a projection pattern composed of a predetermined light pattern onto the measurement object, and imaging means for imaging the measurement object And an image acquisition unit that acquires a grayscale image of a measurement object on which a projection pattern is projected from a captured image of the imaging unit, and a relative phase of the projection pattern at an arbitrary position in the grayscale image acquired by the image acquisition unit Relative phase calculation means, absolute phase calculation means for calculating the absolute phase of the projection pattern by connecting the relative phases at arbitrary positions, and three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the measurement object from the absolute phase The projection pattern control means includes a plurality of types of fringe patterns in which the light intensity is changed by changing the phase of a sine wave and a known amount of phase, and the amplitude is changed for each cycle. A pattern is generated and projected from the projection means onto the measurement object, and the relative phase calculation means calculates the relative phase based on the density value in the grayscale image of the measurement object projected by projecting the projection pattern, and the absolute phase The computing means computes the amplitude of the projection pattern based on the density value in the grayscale image of the object measured by projecting the projection pattern, estimates the period from the amplitude, and connects the relative phases in phase A three-dimensional shape measuring apparatus characterized by calculating a phase.   Projection means for projecting an arbitrary pattern of light onto the measurement object, projection pattern control means for controlling the projection means to project a projection pattern composed of a predetermined light pattern onto the measurement object, and imaging means for imaging the measurement object And an image acquisition unit that acquires a grayscale image of a measurement object on which a projection pattern is projected from a captured image of the imaging unit, and a relative phase of the projection pattern at an arbitrary position in the grayscale image acquired by the image acquisition unit Relative phase calculation means, absolute phase calculation means for calculating the absolute phase of the projection pattern by connecting the relative phases at arbitrary positions, and three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the measurement object from the absolute phase Provided, and the projection pattern control means applies at least one kind of fringe pattern among the plural kinds of fringe patterns in which the intensity of light is changed by shifting the phase of the sine wave and the known amount. The projection pattern in which the amplitude is changed according to the period, the projection pattern in which the amplitude is changed in the aspect different from the projection pattern according to the period in the remaining stripe pattern, and the uniform center value of the stripe pattern A projection pattern is generated and projected onto the measurement object from the projection means, and the relative phase calculation means calculates the relative phase based on the density value in the grayscale image of the measurement object that is imaged by projecting the projection pattern, The absolute phase calculation means calculates the amplitude of the projection pattern based on the density value in the grayscale image of the object to be measured that is imaged by projecting the projection pattern, estimates the period from the amplitude, and phase-connects the relative phases. A three-dimensional shape measuring apparatus characterized by calculating an absolute phase.   The said image acquisition means is equipped with the non-pattern image acquisition means which acquires the grayscale image of the to-be-measured object in the state in which the projection pattern is not projected from the captured image of an imaging means. 3D shape measuring device.   4. The projection pattern control unit according to claim 1, wherein the projection pattern control unit generates a projection pattern from a plurality of types of fringe patterns in which the intensity of light is changed by shifting the phase in a sine wave shape and at equal intervals. The three-dimensional shape measuring apparatus according to item.   Projecting a projection pattern consisting of a predetermined light pattern onto an object to be measured, obtaining a grayscale image of the object to be measured on which the projection pattern is projected, and relative to the projection pattern at an arbitrary position in the grayscale image when the projection pattern is projected The phase is calculated, the relative phase of the arbitrary position is connected in phase, the absolute phase of the projection pattern is calculated, and the three-dimensional coordinates of the measurement object are calculated from the absolute phase 3 Dimensional shape measurement method that generates a projection pattern in which each amplitude is changed for each period in a plurality of types of fringe patterns in which the intensity of light is changed sinusoidally and the phase is shifted by a known amount. The process of projecting onto the object, the process of calculating the relative phase based on the density value in the grayscale image of the object to be measured that is projected by projecting the projection pattern, and the object to be measured that is imaged by projecting the projection pattern A three-dimensional shape comprising: calculating an amplitude of a projection pattern based on a density value in a grayscale image, estimating a period from the amplitude, and calculating an absolute phase by phase-connecting relative phases. Measurement method.   Projecting a projection pattern consisting of a predetermined light pattern onto an object to be measured, obtaining a grayscale image of the object to be measured on which the projection pattern is projected, and relative to the projection pattern at an arbitrary position in the grayscale image when the projection pattern is projected The phase is calculated, the relative phase of the arbitrary position is connected in phase, the absolute phase of the projection pattern is calculated, and the three-dimensional coordinates of the measurement object are calculated from the absolute phase 3 This is a three-dimensional shape measurement method, and the amplitude is changed in accordance with the period in at least one or more stripe patterns among a plurality of types of stripe patterns in which the intensity of light is changed sinusoidally and the phase is shifted by a known amount. Uniform comprising the projection pattern, the projection pattern in which the amplitude of the remaining stripe pattern is changed in a manner different from the projection pattern according to the period, and the amplitude center value of the stripe pattern The process of generating a projection pattern and projecting it onto the object to be measured, the process of calculating the relative phase based on the density value in the grayscale image of the object measured by projecting the projection pattern, and projecting the projection pattern And calculating the amplitude of the projection pattern based on the density value in the grayscale image of the object to be measured, estimating the period from the amplitude, and calculating the absolute phase by connecting the relative phases in phase. A three-dimensional shape measuring method characterized by   7. The process of acquiring a grayscale image of a measurement object on which the projection pattern is projected, and acquiring a grayscale image of the measurement object in a state where no projection pattern is projected. 3D shape measurement method.   The projection pattern control means generates a projection pattern from a plurality of types of fringe patterns in which the intensity of light is changed by shifting the phase in a sine wave shape and at equal intervals. The three-dimensional shape measuring method according to item.

Images