JP2006177781A - Three-dimensional shape measurement method, three-dimensional measurement apparatus, and three-dimensional measurement program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To measure the three-dimensional shape of an object, having steps and holes on its surface with suppressed influence due to disturbance light and fewer number of photographing. <P>SOLUTION: An auxiliary pattern PRi for determining a fringe order is projected with light of a wavelength which is different from the wavelength of light for projecting a fringe pattern PBi simultaneously with the fringe pattern PBi. A photographed image, on which the fringe pattern PBi and the auxiliary pattern PRi are simultaneously projected, is divided on the basis of the wavelength of light into an image on which the fringe pattern PBi is projected, and an image on which the auxiliary pattern PRi is projected. From a plurality of images on which the auxiliary pattern is projected, fringe order n is determined. The fringe pattern PBi having a sinusoidal shape and the auxiliary pattern PRi for determining the fringe order of the fringe pattern PBi are allowed to be projected on an object to be measured and to be photographed simultaneously. Furthermore, by making the light for projecting the fringe pattern PBi monochromatic light of a specific frequency, the influence of disturbance light can be suppressed, as compared with white light. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional shape measurement method, a three-dimensional shape measurement apparatus, and a three-dimensional shape measurement program for measuring a three-dimensional shape of a measurement object by a phase shift method.

  Conventionally, the phase shift method has been widely used as a method for measuring the three-dimensional shape of an object. The phase shift method is a kind of light cutting method. As shown in FIG. 8, the projection device 11 projects a specific pattern onto the measurement target 13 and projects a projection image irregularly reflected on the surface of the measurement target 13. The three-dimensional shape of the measurement object 13 is measured by picking up an image from the direction different from the projection optical axis 11 and image processing the image picked up by the image pickup device 12.

  Hereinafter, the principle of the phase shift method will be described in more detail.

  As shown in the flowchart of FIG. 9, first, a fringe pattern whose brightness changes in a sinusoidal shape from the projection device 11 is projected onto the measurement object 13, and the phase of the fringe pattern is shifted by, for example, π / 2, for example, the imaging device 12. As shown in FIG. 10 (a), the procedure (step 1 to step 6) of imaging is repeated a plurality of times (minimum 3 times, usually 4 times or more) until the phase of the fringe pattern moves by one period. Four images M1 to M4 are obtained. As shown in FIG. 10B, the brightness a0 to a3 at the same position p (x, y) on the four captured images M1 to M4 is the surface property at that position. Even if it changes depending on the color or the like, the relative brightness difference always shows a change of only the phase difference of the projection pattern. Therefore, if the sine wave equation is applied by Equation 1, the relative lightness difference at the position p (x, y) The relative phase value θp of the projected pattern is obtained (step 7 in FIG. 9). FIG. 11 shows a phase restoration image generated by performing this process.

Each pixel value (relative phase value) θp of the phase-restored image shown in FIG. 11 is a value for each phase of the projected fringe pattern, that is, −π to π, as is clear from the fact that Equation 1 is composed of an arctangent function. The absolute phase value of the fringes projected for a plurality of phases (absolute phase expressed as -π to 0 to π to 2π to 3π ... with reference to the fringes at the upper end in FIG. 10A). (Value) θa (step 9 in FIG. 9), the fringes of the fringe order n (value indicating the nth fringe counting from the upper end downward) are captured in the respective images M1 to M4. The process of estimating the position at the top (step 8 in FIG. 9) is necessary. FIG. 12 shows an image in which the fringe order n (= 1, 2,...) Obtained by performing this process is applied to the phase restored image.

  Then, at each point p (x, y) of the four captured images M1 to M4, using the relative phase value θp and the fringe order n, as shown in FIG. 13, the absolute phase value θa (= θp + 2nπ) is obtained. This process (step 9 in FIG. 9) is hereinafter referred to as “phase connection process”. A line (equal phase line) obtained by connecting points having the same absolute phase value θa obtained by the phase connection process has a cross-sectional shape obtained by cutting the measurement target 13 along a certain plane in the same manner as the cutting line in the optical cutting method. Therefore, based on the absolute phase value θa, the three-dimensional shape (height information at each point of the image) of the measurement object 13 can be measured by the principle of triangulation. FIG. 14 shows a phase restoration image of the absolute phase value θa generated by performing the phase connection process.

  Thus, as shown in FIG. 15, the absolute phase value of the sine wave fringe, that is, the position δ on the actual lattice A used for projecting the fringe pattern and the position of the imaging point on the image pickup device B of the image pickup device 12 (on the image) ) P (x, y) can be specified, and the point p (x, y) on the image is represented by Equation 2 representing the principle of triangulation based on the optical arrangement of the projection device 11 and the imaging device 12. If the absolute coordinate value (X, Y, Z) in the three-dimensional space of the projection point P (X, Y, Z) on the corresponding measurement object 13 is obtained, the measurement object 13 as shown in FIG. Can be obtained (step 10 in FIG. 9). However, since the measurement principle of triangulation shown in FIG.

As described above, in the phase shift method, three-dimensional coordinates corresponding to each point of the obtained image are obtained by a simple calculation process, and measurement data of a high-density three-dimensional shape is obtained by imaging at least three times. When evaluated by the measurement time per point, there is a feature that it is possible to measure at a speed several to several tens of thousands times faster than other measurement methods.

  However, when there is a step or a hole on the surface of the measurement object 13 and the boundary (phase boundary) between the stripes cannot be specified, the fringe order n cannot be obtained correctly, making phase connection processing difficult. There is a disadvantage that a problem that a correct distance cannot be obtained (phase jump problem) occurs. In addition, the phase shift method projects as many fringes as possible and takes an image so that the contrast of the projected fringes is as high as possible on the captured image. There is a problem that the phase jump problem is likely to occur and it is difficult to obtain an absolute phase value necessary for height measurement.

  Here, the phase skip problem will be described with a specific example. If the surface of the measurement object is an ideal irregular reflection surface and there are no steps or holes, the relative phase value θp changes from π to −π as the fringe order n = 1 in FIG. With the position as a phase boundary, the first phase boundary appearing on the side where the phase increases when viewed from the position is regarded as one fringe and the fringe order n may be obtained. However, in actuality, the relative phase value θp changes from π to −π and from −π to π at the phase boundary position (the boundary between the stripe orders n = 2 and n = 3 in FIG. 13) due to noise of the imaging device 12. In many cases, data that changes several times appears, and phase connection is often not successful in such a simple method. Further, when there are large steps as in the examples shown in FIGS. 11 and 12, for example, the stripe of the stripe order n = 4 shown in FIG. 12 is divided into three by shadows. Since it cannot be recognized that the stripes have the same order, the correct stripe order cannot be obtained.

Thus, various methods for solving the phase skip problem have been proposed (see Patent Documents 1 to 4). In the method described in Patent Document 1, for the purpose of explicitly acquiring the phase boundary position, slit light or black-and-white striped auxiliary patterns in which image features appear at the phase boundary position are projected onto the measurement object. By taking advantage of both the multi-slit optical cutting method, which is easy to apply to measurement objects with steps and holes, and the phase shift method, which is sensitive to steps but requires high-density height information, steps and holes are used. Even in the case of including high-density data, high-density data is acquired by the phase shift method. Patent Document 2 describes a method of superimposing and projecting a reference sine wave and a sine wave having an integer multiple of the reference sine wave. As described above, if a method of using sine waves having different wavelengths and combining information on relative phases obtained at respective wavelengths is used, the phase boundary position can be uniquely obtained.
JP 2001-124534 A Japanese Patent No. 399041 JP 2004-108950 A JP 2003-279329 A

  However, the methods described in Patent Documents 1 and 2 have a problem that the number of times of imaging the measurement object increases. In the phase shift method, if imaging is completed in a short time, the measurement target (human body) moves during shooting, especially when an object that is difficult to fix completely, such as the human body, is the measurement target. Therefore, it is desirable to measure a three-dimensional shape with a smaller number of imaging.

  As a method for coping with such a problem, Patent Document 3 discloses three auxiliary patterns whose initial positions are different from each other with three colors of RGB light and whose brightness changes stepwise, and a reference white light pattern. A method is described in which the fringe order can be easily specified by projecting. However, this method has a problem that the reference white pattern has to be photographed in order to capture the boundary point where the brightness changes stepwise, and the number of photographed images increases by at least one. Patent Document 4 discloses a method of halving the number of projections by projecting two fringe images having different wavelengths. However, since this method uses image sensors with the number of wavelengths of the sine wave pattern to be projected, unlike the case where a striped image is captured with the same image sensor, a process for measuring and correcting sensitivity variations of the image sensors is performed. Otherwise, there is a problem that a correct phase value cannot be obtained. Furthermore, since most of indoor lighting as well as sunlight is white light, there is a problem that when measuring using white light, the contrast decreases due to the influence of disturbance light, and the error (white noise) of the measurement result increases.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to measure the three-dimensional shape of an object having a step or a hole on the surface with a small number of imaging operations while suppressing the influence of ambient light. A three-dimensional shape measurement method, a three-dimensional shape measurement device, and a three-dimensional shape measurement program are provided.

  In order to achieve the above object, the invention of claim 1 projects a fringe pattern whose light intensity changes in a sinusoidal shape onto a measurement object, and images the measurement object from a direction different from the projection direction of the fringe pattern. The process of obtaining the image is executed a plurality of times while shifting the phase of the fringe pattern at regular intervals, and the phase value of the fringe pattern is based on the change in brightness at the same point in the plurality of captured images obtained in the plurality of processes. In the three-dimensional shape measurement method for measuring the three-dimensional shape of the measurement object by obtaining height information at each point on the picked-up image from the phase value, an auxiliary for determining the fringe order in the picked-up image The pattern is projected simultaneously with the fringe pattern with light having a wavelength different from that of the light that projects the fringe pattern, and the captured image on which the fringe pattern and the auxiliary pattern are simultaneously projected is fringed based on the light wavelength. Turn separated into image image and the auxiliary pattern projected is projected, the auxiliary pattern is characterized in that to determine the fringe order from a plurality of images projected.

  According to a second aspect of the present invention, in the first aspect of the invention, light having a wavelength different from that of the light of the fringe pattern and the auxiliary pattern is projected simultaneously with the fringe pattern and the auxiliary pattern with substantially uniform brightness, and is captured. The image including only the wavelength component of the light having the substantially uniform brightness is separated, and the movement of the measurement object is detected from a plurality of the images obtained in a plurality of processes, and the measurement object is detected based on the detected movement. It is characterized by correcting height information.

  The invention of claim 3 is characterized in that, in the invention of claim 1 or 2, the wavelength of the light for projecting the fringe pattern is in the range of 380 nanometers to 590 nanometers.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the auxiliary pattern is multiplied by a different integer value at each time with respect to the period of the fringe pattern every time the phase of the fringe pattern is shifted and projected. The pattern is characterized in that the lightness and darkness are reversed in the cycle.

  According to a fifth aspect of the present invention, in any one of the first to fourth aspects of the invention, the wavelength of the light of the fringe pattern and the wavelength of the light of the auxiliary pattern are white when the wavelengths of all the light projected on the measurement object are combined. Each wavelength is set so as to be light.

  A sixth aspect of the invention is a three-dimensional shape measuring apparatus for executing the three-dimensional shape measuring method according to any one of the first to fifth aspects, in order to achieve the above object, wherein the fringe pattern and the auxiliary pattern are simultaneously applied. A projection unit that projects onto the measurement object, an imaging unit that captures an image of the measurement object on which the fringe pattern and the auxiliary pattern are projected, and a fringe order is determined by performing image processing on the captured image captured by the imaging unit, Image processing means for calculating the phase value of the fringe pattern based on the determined fringe order and obtaining height information at each point on the captured image from the phase value is provided.

  A seventh aspect of the invention is a three-dimensional shape measurement program for causing a computer to execute the three-dimensional shape measurement method according to any one of the first to fifth aspects, in order to achieve the above object. A procedure for simultaneously projecting the pattern and the auxiliary pattern onto the measurement target, a procedure for causing the imaging unit to capture an image of the measurement target on which the fringe pattern and the auxiliary pattern are projected, and image processing the captured image captured by the imaging unit And a step of causing the computer to execute a procedure for calculating a phase value of the fringe pattern based on the determined fringe order and obtaining height information at each point on the captured image from the phase value. And

  According to the present invention, a sinusoidal fringe pattern essential for three-dimensional shape measurement using the phase shift method and an auxiliary pattern for determining the fringe order of the fringe pattern are simultaneously projected onto the measurement object and imaged. In addition, by making the light that projects the fringe pattern a monochromatic light of a specific frequency, the influence of disturbance light can be suppressed compared to white light, and as a result, steps and holes are formed on the surface. 3D shape measurement method, 3D shape measurement device, and 3D shape measurement program capable of measuring the 3D shape of a certain object with a small number of times of imaging while suppressing the influence of disturbance light can be provided. effective.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a schematic configuration diagram of the three-dimensional shape measuring apparatus of the present embodiment. The three-dimensional shape measurement apparatus includes a projection apparatus 1 that projects a fringe pattern and an auxiliary pattern onto a measurement object with light having different wavelengths, and an imaging apparatus that captures an image of the measurement object on which the fringe pattern and the auxiliary pattern are projected. 2 and a control device 3 that controls the projection device 1 and the imaging device 2 and performs image processing on the captured image captured by the imaging device 2.

  The projection apparatus 1 includes a white light source, three liquid crystal panels that split the light of the white light source into R, G, and B monochromatic light, and an optical path of R, G, and B monochromatic light that is split by the three liquid crystal panels. This is a so-called liquid crystal projector composed of an optical system for matching the two. The image pickup apparatus 2 also includes a CCD image pickup element that employs a primary color filter, an optical system that forms an image of a subject on the image pickup surface of the CCD image pickup element, and R, G, and B by performing signal processing on the output of the CCD image pickup element. This is a so-called single-plate CCD camera composed of a signal processing circuit for obtaining image data for each monochromatic light. The control device 3 causes the computer to execute a general-purpose computer (hardware) including a CPU, a storage device such as a memory, a display, and a hard disk, various input / output interfaces, and the three-dimensional shape measurement method according to the present invention. It consists of a three-dimensional shape measurement program (software). The three-dimensional shape measurement program is recorded on a recording medium such as a magnetic disk or an optical disk, or provided to a computer through an electric communication line such as the Internet. For example, a pattern image is transmitted between the projection device 1 and the control device 3 via a general-purpose display interface such as DVI (Digital Visual Interface), and is controlled via a general-purpose communication interface such as RS232C or IEEE488. The apparatus 3 controls the operation of the projection apparatus 1 (lighting on / off of the light source, light amount adjustment, etc.). The imaging device 2 has a function of quantizing the pixel value (brightness) of each pixel and outputting it for each color of R, G, and B. The control device 3 outputs R, G, and R output from the imaging device 2. The digital signal B is taken in via a digital signal input interface, and the operation (imaging timing, etc.) of the image pickup apparatus 2 is controlled via a general-purpose communication interface such as RS232C or IEEE488.

By the way, the fringe pattern and the auxiliary pattern projected onto the measurement object by the projection device 1 are created in advance using general-purpose application software (for example, retouch software for image processing) and stored in the storage device of the control device 3. Has been. The stripe pattern PB is a pattern in which the brightness of blue monochromatic light changes in a sine wave shape in the vertical direction as shown in FIG. 3A. The four types of stripe patterns PB1 to PB4 have a stripe phase of π / sequentially. It is shifted by 4 each. Further, as shown in FIG. 3B, the auxiliary patterns PR1 to PR4 are inverted in brightness with a period in which the brightness of the red monochromatic light is 2 ⁱ times (i = 1, 2, 3, 4) the period of the stripe pattern PBi. Pattern. FIG. 3C shows motion detection patterns PG1 to PG4 projected on the measurement object simultaneously with the stripe pattern PBi and the auxiliary pattern PRi in order to detect the movement of the measurement object, as will be described later. The brightness of monochromatic light is uniform. Such a pattern has the function of separating an image into three primary color channels of RGB, the function of periodically changing the brightness for each channel (gradation adjustment function), and the three processed channels into one image. It can be created using general-purpose retouching software having the function of combining.

  Thus, when a pattern image obtained by combining the stripe pattern PBi, the auxiliary pattern PRi, and the motion detection pattern PGi (i = 1, 2, 3, 4) is transmitted from the control device 3 to the projection device 1 via the interface. From the projection apparatus 1, the stripe pattern PBi, the auxiliary pattern PRi, and the motion detection pattern PGi are simultaneously projected onto the measurement object with light of blue, red, and green wavelengths, respectively.

  Next, the operation of the three-dimensional shape measurement apparatus of this embodiment, that is, the three-dimensional shape measurement method according to the present invention will be described.

  First, when the three-dimensional shape measurement apparatus starts operating by executing a three-dimensional shape measurement program with the CPU of the control device 3, the first pattern image (the stripe pattern PB1 and the auxiliary pattern PR1 stored in the storage device) is stored. The image obtained by synthesizing the motion detection pattern PG1) is read by the CPU into the work area of the memory, and is further transmitted to the projection apparatus 1 via the display interface, and the projection apparatus 1 transmits the stripe pattern PB1, the auxiliary pattern PR1, and the motion. The detection pattern PG1 is simultaneously projected onto the measurement object with light of blue, red, and green wavelengths. After the pattern image is transmitted to the projection apparatus 1, a command for capturing an image of the measurement target is transmitted from the CPU to the imaging apparatus 2 via the communication interface. When the imaging device 2 receives the above command from the CPU of the control device 3, the imaging device 2 captures an image of the measurement target and transmits image data for each R, G, and B monochromatic light to the control device 3. In the control device 3, the R, G, B image data received from the imaging device 2 is stored in the work area of the memory by the CPU, and then the second pattern image (the stripe pattern PB2 and the auxiliary pattern stored in the storage device) is stored. An image obtained by synthesizing the pattern PR2 and the motion detection pattern PG2) is read into the work area of the memory and further transmitted to the projection apparatus 1 via the display interface. Then, the stripe pattern PB2 whose phase is shifted by π / 4 from the first stripe pattern PB1, the auxiliary pattern PR2 whose brightness is inverted with a period four times the period of the stripe pattern PB2, and the motion detection pattern PG2 are blue, The imaging device 2 that has received a command from the CPU of the control device 3 captures an image of the measurement target and receives R, G, R, G, The image data for each monochromatic light of B is transmitted to the control device 3, and the R, G, B image data received from the imaging device 2 is stored in the work area of the memory by the CPU. Similarly, a third pattern image (an image obtained by combining the stripe pattern PB3, the auxiliary pattern PR3, and the motion detection pattern PG3), and a fourth pattern image (an image obtained by combining the stripe pattern PB4, the auxiliary pattern PR4, and the motion detection pattern PG4). The image of the object to be measured is picked up by the image pickup device 2 in a state where the image is projected from the projection device 1, and the R, G, B image data obtained by the image pickup device 2 is stored in the work area of the memory by the CPU. Then, the image acquisition process ends. This image acquisition process corresponds to the process of steps 1 to 6 in the flowchart of FIG.

  4A to 4D assume that the measurement target 13 is a sphere, and the first to fourth R, G, B image data DBi stored in the work area of the memory of the control device 3, DRi and DGi (i = 1, 2, 3, 4) are represented. The CPU of the control device 3 has an arbitrary position p (x, y) with respect to the four image data DB1 to DB4 (images obtained by imaging the measurement object 13 onto which the stripe patterns PB1 to PB4 are projected) stored in the work area. Is calculated, and the obtained relative phase value θp is stored in the work area of the memory. Note that the arithmetic processing for obtaining the relative phase value θp is the same as that described in the prior art, and thus the description thereof is omitted.

Next, the CPU of the control device 3 performs arithmetic processing for determining the stripe order n of each stripe in the image data DBi using a so-called spatial coding technique. FIG. 1A shows a change in brightness of image data DBi imaged by projecting a fringe pattern PBi, and FIGS. 1B and 1E show images obtained by projecting an auxiliary pattern PRi (i = 1, 2, 3, 4). The change in brightness of the image data DRi is shown, and the CPU of the control device 3 is 1 when the brightness at the same position of the i-th image data DRi is bright at any position of the image data DBi, and 0 when dark. The fringe order code is created by substituting the lightness code of the first i-th bit (see FIG. 1F). For example, since the brightness codes of the first to fourth image data DR1 to DR4 are all 0 in the leftmost stripe in FIG. 1, the stripe order code is 0x0000, and the adjacent stripe (second from the left) is the first. Since only the lightness code of the image data DR1 is 1 and the lightness codes of the other image data DR2 to DR4 are all 0, the fringe order code is 0x0001. As a result, the lower 4 bits are 16 to 16 in which the value of the lower 4 bits is 0000-1111. If the values are obtained as street values and the lower 4 bits are converted into decimal numbers, the stripe order n (= 0 to 15) of the 16 stripes can be uniquely determined (see FIG. 1G). ). The lightness at the i-th imaging is 2 ^{i + 4} times the period of the stripe pattern PBi (i = 1, 2, 3, 4) by light of wavelengths other than blue and red, for example, green wavelengths. By simultaneously projecting and imaging patterns in which light and dark are reversed, it is possible to uniquely determine the stripe order of 2 ⁸ = 256 phases, and project a dense stripe pattern using an optical system with high resolution. Even in this case, measurement without phase jump is possible.

  After the fringe order is determined as described above, the CPU performs a calculation process for obtaining the absolute phase value θa from the fringe order and the relative phase value θp, and further, each point of the image by the principle of triangulation based on the absolute phase value θa. Is obtained (coordinate values of each point in the three-dimensional space), and the coordinate values are stored in the storage device as the three-dimensional shape data of the measurement object 13. However, the arithmetic processing for obtaining the absolute phase value θa from the fringe order and the relative phase value θp and the arithmetic processing for obtaining the coordinate value in the three-dimensional space of each point of the image based on the absolute phase value θa are as described in the prior art. The description is omitted because it is well known. As in the conventional example, if the absolute coordinate value in the three-dimensional space of the projection point P (X, Y) on the measurement object 13 corresponding to the point p (x, y) on the image is obtained, A three-dimensional shape of the measurement object 13 as shown in FIG. 5 can be obtained.

  As described above, according to the present embodiment, the sinusoidal fringe pattern essential in the three-dimensional shape measurement using the phase shift method and the auxiliary pattern for determining the fringe order of the fringe pattern are simultaneously applied to the measurement object. Since it is possible to project images by projecting, imaging can be completed in a short time, and even when measuring an object that is difficult to fix completely, such as a human body, Thus, it is possible to suppress the occurrence of an error in the measurement due to the positional deviation. However, as the fringe order n in the fringe pattern PBi is increased, the influence of the position shift due to the movement of the measurement object becomes larger. In this way, the positional deviation between the images is actively detected and corrected.

That is, since the image data DGi obtained by projecting the motion detection pattern PGi is a simple gray image as shown in FIG. 4, the CPU of the control device 3 uses the optical flow of the measurement object between the image data DGi. Is detected to detect the movement (direction and amount of movement) of the measurement object. Here, since it is possible to use a conventionally well-known method such as a gradient method in order to obtain the optical flow, a detailed description thereof will be omitted. However, in order to detect the movement of the measurement object more precisely, the position of the second to fourth image data DG2 to DG4 is set little by little using the first image data DG1 as a template instead of the method of obtaining the optical flow. It is desirable to use a so-called normalized correlation pattern matching method in which the positions where the highest correlation value is obtained by obtaining the cross-correlation values of the pixel values that are overlapped and shifted are detected. When the image data DG2~DG4 measurement object i th the first image data DG1 as a reference (i = 2 to 4) is detected to have moved by d _.theta.i the phase units, the control device 3 In the CPU, when the relative phase value θp at an arbitrary position p (x, y) is obtained with respect to the image data DBi, the positional deviation due to the movement of the measurement object is obtained by using the following Expression 3 instead of Expression 1. It can be corrected.

By the way, in the present embodiment, blue light having a wavelength of about 450 nm is used as light for projecting the stripe pattern PBi. However, the wavelength of this light can be arbitrarily selected within a range of 380 nm to 590 nm, and within this range. Projecting the fringe pattern PBi with light of the wavelength has the following advantages.

  As shown in FIG. 6, the human skin is formed from the surface by the stratum corneum layer 121, the carotene layer 122, the melanin layer 123, and the dermis layer 124. The stratum corneum layer 121 to the melanin layer 123 are extremely thin and have capillaries. It is not distributed and is translucent. The underlying dermis layer 124 is thicker than the stratum corneum layer 121 and the like, and capillaries are distributed. Ultraviolet light and infrared light 125 pass through the dermis layer 124 and pass through to the deep part of the human body. In particular, infrared light is accompanied by radiation due to body temperature. Further, even in visible light, the red light 126 has a characteristic that it is easily scattered by the dermis layer 124 through the stratum corneum layer 121, the carotene layer 122, and the melanin layer 123, which are translucent. For this reason, when a fringe pattern is projected using ultraviolet light, infrared light, or red light, the fringe pattern light is scattered in the deep part of the skin, so that waveform distortion tends to occur in the image. In the phase shift method, the measurement is performed based on the brightness of the projection light irregularly reflected on the surface of the object. Therefore, it is not preferable to project the fringe pattern with light having a wavelength that easily passes through the measurement target. In contrast, most of the visible light 127 and 128 having a short wavelength ranging from yellow to purple is reflected on the skin surface, and most of the components that are partially transmitted are scattered at a shallow position from the stratum corneum 121 to the melanin layer 123. Therefore, the waveform distortion as described above hardly occurs, and highly accurate measurement is possible.

  Further, if a wavelength that reduces the influence of disturbance light is selected as the wavelength of the light for projecting the fringe pattern, the contrast of the captured image can be improved and the accuracy can be further increased. For example, the three-dimensional shape measurement apparatus of the present embodiment is mainly intended for indoor use and is intended for three-dimensional shape measurement of a human body or an object of about several tens of cm to several meters, It is considered that light from fluorescent lamps usually used for indoor lighting often exists as disturbance light. When measuring the spectral distribution of white fluorescent lamps currently on the market, there are about 3 to 4 strong peak wavelengths, especially strong peaks at 435 nm (blue near purple) and 545 nm (green). If a pattern is projected with light having a wavelength excluding these wavelengths, an image with a contrast as high as that obtained when imaged in a dark room can be obtained. Incandescent lamps have a spectrum that is distributed over a wider wavelength range than fluorescent lamps. Among them, the wavelength range from orange to red has the highest intensity, and the spectrum in the short wavelength range such as blue Therefore, by projecting a pattern with blue light, the influence of disturbance light can be removed to some extent even under incandescent lighting.

  The three-dimensional shape measuring apparatus of the present embodiment is assumed to be used for measuring a human body or an industrial product and using it for computer graphics (CG). In such a use, in addition to the three-dimensional shape data, Thus, it is desirable that the color image of the measurement object to be used for texture mapping is simultaneously measured and the correspondence between the color image and the three-dimensional shape data is obtained.

  Therefore, in this embodiment, image data DBi, DRi obtained for each pattern by using light of the three primary colors B, R, and G as light for projecting the stripe pattern PBi, auxiliary pattern PRi, and motion detection pattern PGi. , Synthesizes an image from which the pattern has been erased from DGi, and synthesizes image data of three types of wavelengths from which the pattern has been erased. The same color image as when captured with a camera is obtained.

  Such a color image is obtained by performing the following arithmetic processing by the CPU of the control device 3.

  First, in the B channel image data DBi that is a projection image of the fringe pattern PBi, the phase of the sine wave fringe changes by π / 4 between the image data DBi. If a composite image of the B channel that averages the brightness in the image data DB1 to DB4 for the batch is generated, the same image as that obtained by projecting the blue flat light can be obtained with the sine wave-like stripes disappearing.

Next, the R channel image data DRi, which is the projection image of the auxiliary pattern PRi, is captured by projecting a binary pattern, so that the following four expressions 5 represent the image data DR1 to DR4 for each pixel. If an R channel composite image having the maximum brightness as a pixel value is generated, the rectangular wave pattern disappears and the same image as that obtained by projecting red flat light can be obtained.

Further, since the G channel image data DGi, which is a projection image of the motion detection pattern PGi, is captured by projecting flat light, the image data DGi of any number of times among the image data DG1 to DG4 for the four times is obtained. Alternatively, an image obtained by averaging the pixel values of the image data DG1 to DG4 for four times may be acquired as a composite image of the G channel using the following Expression 6.

Then, if the pixel values R (x, y), G (x, y), and B (x, y) of the RGB three primary color images obtained by Expressions 4 to 6 are combined by additive color mixing, white light is obtained. The same color image as that obtained by projecting illumination onto a measurement object and photographed with a color camera can be acquired. Since this color image is synthesized from the image itself used for measurement, for each point P (X, Y, Z) of the three-dimensional shape data with respect to the point p (x, y) on the photographed image. Since color information of R, G, and B is given, it is not necessary to align the 3D shape data and the image data for texture mapping, and a 3D CG image with a color texture can be easily generated.

  When a color image is synthesized as described above, weight coefficients Wr, Wg, Wb and base luminance Br that are different from each other for the pixel values of the R, B, and G channels as shown in Expressions 4 to 6. , Bg, and Bb, it is possible to adjust the white balance, the overall luminance value, and the contrast to generate a color image having a natural hue and contrast. However, in order to use the above method, it is obvious that in the auxiliary pattern PRi, it is necessary to project a pattern that causes irradiation at the maximum brightness at least once out of four projections at all positions. For example, when performing spatial coding using rectangular waves with different periods as described above, it is necessary to prevent an area where the spatial code is 0x0000 (all dark and light sides) from occurring.

  By the way, in this embodiment, the fringe order is determined by the spatial coding method using a rectangular wave pattern having a different period for each projection as the auxiliary pattern PRi. However, the level difference and the hole are not so large. In the case of a measurement object that is unlikely to cause phase jumps, as described in Patent Document 1, a pattern in which light and dark are simply reversed for each phase of a sine wave fringe, or a slit at a phase boundary position. A pattern in which a shaped light is projected may be projected as an auxiliary pattern.

  Further, in the present embodiment, a liquid crystal projector is used as the projection apparatus 1. However, as shown in FIG. 7, a spectroscopic apparatus 11A such as a dichroic mirror that reflects the wavelength of light to be projected and transmits light of other wavelengths, 11B may be used to synthesize and project the light from a plurality of projection apparatuses 10A, 10B, and 10C provided for each wavelength to be projected into one light beam. In such a configuration, (1) when it is desired to project a high-resolution pattern by using an individual pattern display device for each pattern, (2) instead of a variable display device such as a liquid crystal, a solid lattice (pattern is printed) (3) When a monochromatic light source such as a light emitting diode is used as the projection light source, (4) The object to be excluded from the influence of disturbance light This is particularly effective when it is desired to perform projection using specific monochromatic light other than RGB, such as to match the color of the surface.

  In the case where the light wavelengths of the fringe pattern and the auxiliary pattern are made different, it is sufficient that at least the imaging device 2 has a wavelength difference that allows the two patterns to be separated. If it is lower than the spectral capacity of 2, it is sufficient that the projection apparatus 1 (or the spectral apparatus 11) has a wavelength difference that is equal to or larger than the minimum wavelength difference that can be dispersed.

  By the way, in this embodiment, although it is set as the structure which controls the projection apparatus 1 and the imaging device 2 by the one control apparatus 3, it is not restricted to this, The structure which controls the projection apparatus 1 and the imaging apparatus 2 separately with a control apparatus, respectively. It does not matter. According to such a configuration, while the imaging device 2 is performing imaging and transferring image data under the control of one control device, the next projection pattern output from the other control device to the projection device 1 is read. Therefore, there is an advantage that the measurement time can be shortened by shortening the time interval between projection and imaging. It is also possible to employ a configuration including a control device for capturing an image from the imaging device 2 and another control device for performing image processing. According to this configuration, when three-dimensional shape measurement is performed repeatedly in time series, image processing can be performed by another control device while an image is being captured by one control device. Thus, the time for acquiring the three-dimensional shape data can be shortened. Furthermore, if the configuration is such that image processing is performed in parallel by a plurality of control devices, the fringe order n is calculated by processing (step 7) for calculating the local phase value θp from the fringe pattern and analysis of the auxiliary pattern as shown in FIG. Since the estimation process (step 8) is performed sequentially, the processes of step 7 and step 8 can be executed in parallel by different control devices, and the image processing time can be shortened. Further, since the processing in step 9 and step 10 can be performed independently for each pixel, the image processing time can be shortened by dividing the image and performing image processing by a plurality of control devices. . Here, since it takes a long time to perform image processing of three-dimensional shape data, if the image processing time is shortened by adopting the above-described configuration, it is particularly effective when observing the obtained image data in real time. It becomes.

  Further, if the control device 3 is installed in a place different from the projection device 1 and the imaging device 2 and the image data is transferred to the control device 3 via a communication line or the like, the measurement object can be remotely monitored. Can do. Alternatively, when the measurement object is installed in an environment where the storage device of the control device 3 is destroyed (for example, a place where strong magnetism, electromagnetic waves, or radiation exists), the control device in which the image data is in a safe place Since the processing can be performed by transferring to 3, it is particularly effective. Further, by installing a plurality of imaging devices 2 and the projection device 1 in different places and integrating the three-dimensional shape data obtained by imaging with these imaging devices 2 with the control device 3, a measurement object can be arranged at a wide angle. It becomes possible to display an image by three-dimensional measurement (for example, 180 degrees or more).

It is a wave form diagram for operation explanation of this embodiment. It is a block diagram same as the above. (A)-(c) is explanatory drawing of the fringe pattern, auxiliary pattern, and pattern for motion detection which are used for the same as the above. (A)-(d) is explanatory drawing of each image data of the 1st time-the 4th time imaged by projecting a fringe pattern, an auxiliary pattern, and a pattern for motion detection in the same as the above. It is a figure which shows the three-dimensional shape measurement result of a measurement target object same as the above. It is sectional drawing for demonstrating the structure of the skin of a human body. It is a block diagram of the other projection apparatus same as the above. It is a block diagram of the conventional three-dimensional shape measuring apparatus. It is a flowchart for operation | movement description same as the above. It is explanatory drawing for demonstrating the principle of a phase shift method. It is a figure which shows the phase restoration image by the same as the above. It is explanatory drawing for demonstrating the phase jump problem in the same as the above. It is explanatory drawing for demonstrating the phase jump problem in the same as the above. It is a restored image of the absolute phase value obtained by the same as above. It is explanatory drawing for calculating | requiring the absolute coordinate value in the three-dimensional space of the projection point on a measurement object based on the principle of a triangulation based on an absolute phase value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Projection apparatus 2 Imaging apparatus 3 Control apparatus

Claims (7)

  The process of projecting a fringe pattern whose light intensity changes in a sinusoidal pattern onto a measurement object and obtaining a captured image obtained by imaging the measurement object from a direction different from the projection direction of the fringe pattern is shifted at regular intervals. The phase value of the fringe pattern is calculated based on the brightness change at the same point in the plurality of captured images obtained in the plurality of processes, and at each point on the captured image from the phase value. In a three-dimensional shape measurement method for measuring a three-dimensional shape of a measurement object by obtaining height information, an auxiliary pattern for determining a fringe order in a captured image has a wavelength different from the wavelength of light for projecting the fringe pattern. A captured image that is projected simultaneously with the fringe pattern with light, and the fringe pattern and the auxiliary pattern are projected at the same time, and an image with the fringe pattern projected based on the light wavelength and the auxiliary pattern are projected Three-dimensional shape measurement method characterized by being separated into image, the auxiliary pattern is determined a fringe order from a plurality of images projected.   An image including only the wavelength component of the light having the substantially uniform brightness from the captured image by projecting and imaging light having a wavelength different from that of the light of the stripe pattern and the auxiliary pattern simultaneously with the stripe pattern and the auxiliary pattern. 2. The measurement object is detected from a plurality of images obtained in a plurality of processes, and the height information of the measurement object is corrected based on the detected movement. The three-dimensional shape measurement method described.   The three-dimensional shape measuring method according to claim 1 or 2, wherein the wavelength of the light for projecting the fringe pattern is in the range of 380 nanometers to 590 nanometers.   The auxiliary pattern is a pattern in which light and dark are inverted at a period obtained by multiplying a period of the fringe pattern by a different integer value at each time every time the phase of the fringe pattern is projected and shifted. 3. The three-dimensional shape measuring method according to any one of 3 above.   The wavelength of the light of the stripe pattern and the wavelength of the light of the auxiliary pattern are set such that each wavelength is white light when the wavelengths of all the light projected on the measurement object are combined. The three-dimensional shape measuring method according to any one of?   A three-dimensional shape measuring apparatus for executing the three-dimensional shape measuring method according to any one of claims 1 to 5, wherein a projecting unit that simultaneously projects the fringe pattern and the auxiliary pattern onto the measurement object, the fringe pattern and the auxiliary pattern An imaging unit that captures an image of the measurement object onto which the image is projected, and a fringe order is determined by performing image processing on the captured image captured by the imaging unit, and a phase value of the fringe pattern is calculated based on the determined fringe order And a three-dimensional shape measuring apparatus comprising image processing means for obtaining height information at each point on the captured image from the phase value.   A three-dimensional shape measurement program for causing a computer to execute the three-dimensional shape measurement method according to any one of claims 1 to 5, wherein a projection device simultaneously projects a fringe pattern and an auxiliary pattern onto a measurement object; The procedure for causing the imaging means to image the image of the measurement object onto which the fringe pattern and the auxiliary pattern are projected, and the fringe order are determined by performing image processing on the captured image captured by the imaging means, and the fringe order is determined based on the determined fringe order. A program for measuring a three-dimensional shape, which causes a computer to execute a procedure for calculating a phase value of a pattern and obtaining height information at each point on a captured image from the phase value.