WO1999034301A9 - Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video - Google Patents

Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video

Info

Publication number
WO1999034301A9
WO1999034301A9 PCT/US1998/027915 US9827915W WO9934301A9 WO 1999034301 A9 WO1999034301 A9 WO 1999034301A9 US 9827915 W US9827915 W US 9827915W WO 9934301 A9 WO9934301 A9 WO 9934301A9
Authority
WO
WIPO (PCT)
Prior art keywords
fringe
fringe pattern
phase
patterns
contouring
Prior art date
Application number
PCT/US1998/027915
Other languages
English (en)
Other versions
WO1999034301A1 (fr
Inventor
Peisen S Huang
Fu-Pen Chiang
Original Assignee
Univ New York State Res Found
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Univ New York State Res Found filed Critical Univ New York State Res Found
Priority to EP98965560A priority Critical patent/EP1042718A4/fr
Priority to JP2000526877A priority patent/JP2002500369A/ja
Priority to CA002316838A priority patent/CA2316838A1/fr
Priority to AU21001/99A priority patent/AU2100199A/en
Publication of WO1999034301A1 publication Critical patent/WO1999034301A1/fr
Publication of WO1999034301A9 publication Critical patent/WO1999034301A9/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0007—Image acquisition

Definitions

  • the present invention relates generally to a method and apparatus for three dimensional surface contouring.
  • the present invention uses a digital video projection system for digitally generating fringe patterns in three dimensional surface contouring.
  • Three dimensional surface contouring techniques have numerous applications in design and manufacturing. For example, surface contouring can be used for inspection of industrial parts whose dimensions and geometry need to be checked against their design specifications during or after manufacturing. These techniques can also be used in reverse engineering where construction of a Computer Aided Design (CAD) model from a physical part is required.
  • CAD Computer Aided Design
  • rapid prototyping technology based on a layered manufacturing concept has been established which allows for rapid fabrication of physical concept models, functional parts, and toolings directly from CAD models.
  • Surface contouring techniques can help extend the capabilities of current rapid prototyping systems to include building physical parts and toolings from hand-crafted models or parts for which a CAD model is not available. Surface contouring techniques can also help improve the accuracy of constructed models by introducing in-process or post-process inspection into the rapid prototyping process.
  • the methods can generally be categorized into two groups: scanning and non-scanning imaging techniques.
  • the scanning techniques are represented by point triangulation (Blais, F. and Rioux, M., "BIRIS: a simple 3-D sensor," Proc. SPIE, Vol. 723, 235 (1986)), laser radar (Svetkoff, D.J., Leonard, P.F., and Sampson, R.E., "Techniques for real-time, 3- D feature extraction using range information," Proc. SPIE, Vol.
  • Point triangulation and structured line methods are based on the triangulation principle and the laser radar methods are based on the measurement of the travel time or phase of either a pulsed or modulated laser. All these techniques require either one-dimensional or two-dimensional scanning of the laser to cover the entire surface of the object. This generally makes the systems more sophisticated and the measurement more time consuming.
  • Typical non-scanning techniques include stereo vision and moire interferometry.
  • Stereo vision obtains three-dimensional information of an object by viewing a scene from two different perspectives and then locating common features in both images. (Hobrough, G. and Hobrough, T., "Stereopsis for robots by iterative stereo image matching," Proc. SPIE, Vol. 449, 62 (1983)).
  • the processing of the images is computationally intensive, which makes the technique unsuitable for highspeed 3-D contouring.
  • Moire interferometry is one of the most commonly used techniques for 3-D surface contouring. Compared to other techniques, it has the primary advantage of fast measurement speed due to the fact that it does not require scanning to cover the whole object surface and the image processing for extracting 3-D contour information is relatively simple.
  • Moire contouring techniques can be classified as either shadow moire (Chiang, F. P., "Moire Methods for Contouring, Displacement, Deflection, Slope, and Curvature," Proc. SPIE, Vol. 153, 113-119 (1978)) or projection moire (Khetan, R. P. and F. P. Chiang, “On the theory of two projection moire methods," Univ. Of 111.
  • Shadow moire uses the same grating for both illumination and observation, while projection moire uses separated gratings.
  • Another surface contouring technique is fringe projection which uses only one grating and measures surface height by triangulation.
  • shadow moire is that it is easy to obtain quantitative contour information from the moire pattern because the grating is flat and its period known.
  • contouring of large objects is difficult because a grating with approximately the same size as the object must be used. Large gratings are difficult to make and have limited mobility.
  • Projection moire and fringe projection offer advantages in their ability to contour large objects and the ease with which phase measuring techniques can be implemented to increase the measurement resolution.
  • Their primary limitation is the tedium associated with obtaining quantitative height information. This limitation arises because it is necessary to calibrate both the projection geometry and the magnification factor.
  • phase shifting techniques developed in interferometry have been widely adopted and used in moire and fringe projection methods for 3-D surface contouring.
  • the resolution of the moire and fringe projection contouring methods depends on the density of the fringe projected on the object.
  • higher fringe density means higher resolution.
  • there is a limit to the fringe density that can be applied because overly dense fringes may not be resolvable by the camera.
  • phase shifting techniques have been developed and widely used in optical contouring applications (Halioua, M. and Liu, H. -C, "Optical Three-Dimensional Sensing by Phase Measuring Profilometry," Opt. Lasers Eng., 11(3), 185-215 (1989); Moore, D.T.
  • Phase shifting dramatically increases measurement resolution without the need of using high density fringes.
  • Traditional phase shifting is accomplished by mechanically shifting a grating to create a series of phase shifted fringe patterns.
  • the phase shifted fringe patterns then are processed to extract the phase of each pixel of the image using algorithms well known in the art.
  • Phase shifted images are generally obtained by mechanically translating a grating.
  • the shortcomings are that the system becomes more complicated because of the introduction of moving parts into the system and the phase shifting may not be accurate due to mechanical errors.
  • the Phase Shifting And Logical Moire was proposed to eliminate some of the problems with traditional phase shifting techniques (Asundi, A., "Projection moire using PSALM," Proc. SPIE, Vol. 1554B, 257-265 (1991)).
  • PSALM uses only one grating with the other grating generated by software in a computer.
  • the phase-shifted moire fringes are obtained through logic calculations on the image of the object and the software created grating. Since no moving parts are necessary, this technique greatly simplifies the contouring system.
  • phase-shifting technique Another traditional problem associated with the phase-shifting technique is the modulo 2 ⁇ ambiguity caused by the phase extraction process using the arc-tangent function which has values only between - ⁇ /2 and ⁇ /2. Even though with corrections, the phase calculation range can be extended to 0 to 2 ⁇ , the absolute phase still cannot be recovered. This means that if the object surface has discontinuous features, such as step-like jumps, and the height change causes a phase change exceeding 2 ⁇ , then the phase extraction process cannot provide the correct height information. Accordingly, traditional phase shifting technology usually cannot be applied to measure surfaces with discontinuous geometric features.
  • Field-shift moire shifts the whole projection system including the grating and the light source to capture a series of field-shifted images.
  • both the fringe order and the phase of each pixel can be extracted to yield absolute measurement of the surface contour even for prismatic objects with discontinuous features.
  • the problem is the need to shift the whole projection system in accurate steps, which makes the system even more complicated than the traditional grating shifting technique.
  • Harding proposed a color-encoded moire technique that retrieves the 3-D surface contour of an object from a single snap shot of the object illuminated by a color-encoded fringe pattern. Contouring speed was limited only by the frame rate of the camera. However, since the color-encoded fringe pattern produced on a Polaroid film had a poor contrast ratio, no actual contouring of objects was attempted.
  • European Patent No. EP0076866 discloses the simultaneous projection of three color-coded patterns on an object with the patterns being phase-shifted by 120 degrees relative to each other.
  • the grating patterns deformed by the object surface are recorded corresponding to their color coding by three associated color-selective cameras.
  • the pattern is recorded simultaneously in three different phase relations so that an evaluation based on phase shifting algorithms may be performed without requiring a mechanical shifting.
  • the present invention is a method and apparatus for three dimensional surface contouring.
  • the present invention uses a digital video projector for projecting fringe patterns in surface contouring.
  • the method of three dimensional surface contouring of an object having a surface defining a geometry includes generating a plurality of phase shifted digitally- interpretable fringe pattern signals with each signal being generated at a separate phase angle.
  • the signals are then converted into optical phase shifted fringe patterns which are projected onto the surface of the object.
  • the geometry of the object distorts the fringe patterns.
  • a reflection of each of the distorted fringe patterns is individually retrieved.
  • the distorted fringe patterns are combined to generate a phase-wrapped image.
  • the phase-wrapped image is unwrapped to reconstruct the surface of the object.
  • the apparatus for three dimensional surface contouring of an object includes a fringe pattern generator that generates a plurality of phase shifted digitally- interpretable fringe pattern signals with each of the signals being generated at a separate phase angle.
  • a digital video projector receives the signals from the fringe pattern generator.
  • the digital video projector converts the signals into optical fringe patterns and projects the fringe patterns onto the surface of the object.
  • the fringe patterns are distorted by the geometry of the object and an optical retrieval device retrieves a reflection of the distorted fringe pattern.
  • An image generator combines the distorted fringe patterns and reconstructs the surface of the object.
  • phase shifted fringe patterns are generated separated by 120 degrees.
  • the phase shifted fringe patterns can be projected sequentially.
  • the sequential projection of the phase shifted fringe patterns is synchronized to increase contouring speed. Contouring speed can also be increased by projecting a plurality of phase shifted fringe patterns substantially simultaneously by color encoding the phase shifted fringe patterns.
  • the fringe pattern generator can be a circuit configured to generate the fringe pattern.
  • the fringe pattern generator can be located within the image generator.
  • the fringe pattern generator can also include a mechanical phase shifter for shifting the phase angle.
  • the fringe pattern generator shifts the phase angle digitally.
  • the phase shifted fringe patterns are separated by the quotient of 360 degrees divided by the number of phase shifted fringe patterns.
  • a method and apparatus for three dimensional surface contouring is provided.
  • a particular advantage is that since the fringe patterns are generated digitally and projected by a digital video projector the fiinge patterns have exceptionally high brightness and contrast ratio.
  • the fringe patterns are generated digitally, fringes with any cross-sectional intensity profile and spacing can be produced.
  • the digitally controlled phase shifting technique eliminates the traditional need for physically shifting a grating or other optical components which translates into higher contouring accuracy.
  • fringe patterns can now be easily color encoded for a variety of applications.
  • Figure 1 is a perspective view of the surface contouring device of the present invention with some of the attributes shown schematically;
  • Figure 2 is a flowchart illustrating an algorithm used by the fringe pattern generator;
  • Figure 3 is an elevational view of a Digital Mirror Device
  • Figure 4 is a schematic view illustrating the Digital Mirror Device switching principle
  • Figure 5 is an exploded perspective view showing a Digital Light Processing projection system
  • Figure 6 A is a view showing a fringe pattern being projected on a dummy face having a phase angle of 0 degrees;
  • Figure 6B is a view showing a fringe pattern being projected on a dummy face having a phase angle of 120 degrees;
  • Figure 6C is a view showing a fringe pattern being projected on a dummy face having a phase angle of -120 degrees;
  • Figure 7 is a view showing a mask generated to remove noise in the background of the phase- wrapped image
  • Figure 8 is a view showing a phase-wrapped image generated from the information contained in Figures 6 A, 6B, and 6C;
  • Figure 9 is a view showing a reconstructed surface of the dummy face shown in Figures 6A, 6B, and 6C;
  • Figure 10 is a view showing a color encoded fringe pattern being projected on a plaster head sculpture
  • Figure 11 A is a view showing the red channel of the color encoded fringe pattern shown in Figure 10 having a phase angle of -120 degrees;
  • Figure 1 IB is a view showing the green channel of the color encoded fringe pattern shown in Figure 10 having a phase angle of 0 degrees;
  • Figure 1 IC is a view showing the blue channel of the color encoded fringe pattern shown in Figure 10 having a phase angle of 120 degrees;
  • Figure 12 is a view showing a phase-wrapped image generated from the information contained in Figures 11A, 1 IB, and 1 IC;
  • Figure 13 is a view showing a reconstructed surface of the plaster head sculpture shown in Figures 11 A, 1 IB, and 1 IC.
  • the surface contouring device 10 includes a fringe pattern generator 16, a digital video projector 18, an optical retrieval device 20, and an image generator 22.
  • the fringe pattern generator 16 generates a digitally-interpretable fringe pattern signal that is received by the digital video projector 18 for conversion to an optical fiinge pattern.
  • the fringe pattern generator 16 is capable of generating digitally-interpretable fiinge pattern signals to produce any type of optical fringe pattern including sinusoidal patterns, binary structured-line patterns, and circular patterns as known to those skilled in the art.
  • the fringe pattern generator 16 can generate a plurality of phase shifted digitally-interpretable fringe pattern signals for producing a plurality of optical phase shifted fiinge patterns without the traditional mechanical shifting of a grating by shifting the phase angle digitally.
  • the fringe pattern generator 16 can be, for example, any suitable computer, processor (e.g., digital signal processor, microprocessor, etc.), microcontroller, or circuit configured with the teachings hereof.
  • processor e.g., digital signal processor, microprocessor, etc.
  • microcontroller or circuit configured with the teachings hereof.
  • the inventors developed a Windows based program in Visual Basic and also in Visual C++ for generating the signals with a computer.
  • the inventors used a keyboard and video monitor as an interface for modifying the fringe type, intensity profile, spacing, phase, and color.
  • the fringe pattern generator can also use embedded circuitry tailored to specific industrial applications.
  • the fringe pattern generator 16 assigns the intensity, /, and color for each pixel of the digital video projector 18.
  • the phase angle ⁇ , pitch p, color, and an array of pixels m x n of the digital video projector for projecting the fringe pattern are selected. If color encoding of the fringe pattern is desired, the intensities are determined as shown in block 25 where I r (x), I g (x), and I b (x) are the intensities of red, green, and blue that are assigned to each line or circle of pixels.
  • I r (x), I g (x), andl b (x) are each phase shifted by 2 ⁇ /3 or 120 degrees.
  • a mathematical formula for a sinusoidal fringe pattern is shown in block 25 by way of example only as it is contemplated that fringe patterns having other intensity profiles could be used as well, e.g., triangular. If a single color fringe pattern is to be generated, the intensity profile is determined next. Again by way of example only blocks 27, 29, and 31 include formulations for sinusoidal, triangular and binary intensity profiles respectively that are commonly used by those skilled in the art. Blocks 33, 35 and 37 illustrate examples of generating red, grayscale, and green fringe patterns respectively. Fringe patterns of other colors can be generated in a similar manner. The fringe type is selected next. As shown in blocks 39 and 41 fiinge patterns are generally linear or circular, but could be arranged in any desired form. The process of assigning each pixel an intensity and color is repeated until the array m x n is completed.
  • the apparatus can further comprise a mechanical phase shifter for shifting the phase angle.
  • Traditional mechanical phase shifters generally include motors to either translate a grating or the object being imaged, or rotate a glass plate that refracts the projected fringe pattern. Examples of mechanical phase shifters are found in U.S. Patent Nos. 4,641,972 to Halioua et al, 4,984,893 to Lange, and 5,561,526 to Huber et al., the disclosures of which are incorporated herein by reference.
  • the digital video projector 18 receives the digitally- interpretable fringe pattern signal generated by the fringe pattern generator 16 and converts the signal into an optical fringe pattern and projects it onto the surface 14 of the object 12. Examples of phase shifted fringe patterns are shown in Figures 6A, 6B, and 6C which have phase angles of 0, 120 and -120 degrees respectively. The projected fringe pattern is then distorted by the geometry of the object 12.
  • the digital video projector 18 uses the Digital Light Processing (DLP) with Digital Micromirror Device (DMD) technology recently developed by Texas Instruments Incorporated.
  • DLP Digital Light Processing
  • DMD Digital Micromirror Device
  • the Digital Micromirror Device (DMD) 24 is a digital light switch integrated circuit having an upper surface that comprises an array of tiny square aluminum pixel mirrors 26. Each pixel mirror 26 of the DMD 24 can be digitally controlled to reflect incident light into or out of an aperture of a projection lens 28 as shown in Figure 4.
  • Figure 4 which is adopted from "Digital Light Processing and MEMS: Timely Convergence for a Bright Future," illustrates the optical switching action of the mirror. When the pixel mirror 26 rotates to its on state, + 10 degrees, light 30 from an illuminator 32 is directed into the projection lens 28 and the pixel 34 appears bright on a projection screen 36.
  • Each pixel mirror 26 When the pixel mirror 26 rotates to its off state, - 10 degrees, light 30 from the illuminator 32 is directed away from the projection lens 28 and the pixel 34 appears dark.
  • Each pixel mirror 26 is capable of switching more than 1,400 times a second and yields a potential of 256 gray levels providing for the rapid direction of light into or out of the projection lens 28.
  • the DLP projection system 38 includes an illuminator 32 which is preferably a metal halide lamp.
  • the illuminator 32 produces white light which is passed through a condenser lens 40 for collecting the light and imaging it on a rotating color wheel 42.
  • the color wheel 42 is generally segmented into at least a red, a green, and a blue portion, which can be used to project these components of colored light to produce over 16 million colors.
  • a second lens 44 collects the light that passes through the color wheel 42 and evenly illuminates the surface of the DMD 24.
  • each pixel mirror 26 (+10 or -10 degrees) of the DMD 24 which are controlled by a DLP circuit board 46, the light is directed either into or away from the projection lens 28.
  • the projection lens 28 then projects an enlarged image on to a projection screen 36.
  • a variety of digital video projectors 18 that use the DLP technology are presently commercially available. Some of the manufacturers include Davis North America, Inc., NEC Technologies Inc., CTX Opto Inc., and In Focus Systems. In the experimentation described herein, the inventors used the LitePro 620 as manufactured by In Focus Systems, Wilsonville, Oregon.
  • the optical retrieval device 20 retrieves a reflection of the distorted fringe patterns from the surface of the object 12.
  • the optical retrieval device 20 is focused at a different angle than the digital video projector 18 with respect to the surface 14 of the object 12 for the retrieval of the reflection of the distorted fringe pattern.
  • the optical retrieval device 20 can be any electronic camera, such as a CCD, CMOS, or Vidicon camera or film.
  • the optical retrieval device 20 is a charge coupled device (CCD).
  • a suitable camera for use with the teachings hereof is the Kodak Megaplus Model 4.2i having a charge-coupled device array of 2029(H) x 2048(V) pixels providing for the measuring of over 4 million data points simultaneously.
  • the image generator 22 combines the distorted phase shifted fringe patterns and reconstructs the surface 14 of the object 12.
  • the image generator 22 can be, for example, any suitable computer, processor (e.g., digital signal processor, microprocessor, etc.), microcontroller, or circuit configured with the teachings hereof.
  • the image generator 22 and the fiinge pattern generator 16 can be located within the same housing, e.g., a computer.
  • the reconstruction of the surface of the object is accomplished by using any of the traditional algorithms known from phase shift interferometry to first combine the information from the phase shifted fringe patterns to acquire a phase-wrapped image and then unwrap the phase-wrapped image to reconstruct the surface 14 of the object 12.
  • An example of a phase-wrapped image is shown in Figure 8 which was generated from the phase shifted images shown in Figures 6 A, 6B, and 6C.
  • the image generator 22 extracts the phase of each pixel of the image to be generated.
  • algorithms include the three-step (Gallagher, J. E. and He ⁇ iott, D.R., "Wavefront measurement,” U.S. Patent 3,694,088 (1972); Creath, K. "Phase-Measurement Interferometry Techniques,” in Progress in Optics. Vol XXVI, E. Wolf, Ed., Elsevier Science Publishers, Amsterdam, 1988, pp. 349-393), four-step least-squares (Bruning, J.
  • phase shifted patterns are generated with each phase shifted pattern being generated at a separate phase angle.
  • the phase shifted patterns are separated by 120 degrees (2 ⁇ / 3).
  • ⁇ l5 ⁇ 2 , and ⁇ 3 represent the separate phase angles
  • the intensity of each pixel in the three patterns can be represented as follows:
  • I x (x,y) I'(x,y)+I"(x,y) cost ⁇ f ⁇ + ⁇ ,] (1)
  • I 2 (x,y) I'(x,y)+I"(x,y) cos[ ⁇ ( ⁇ )+ ⁇ 2 ] (2)
  • the Data modulation (x,y) can be used to check the quality of data at each pixel.
  • a data modulation near one is good, whereas a data modulation near zero is bad meaning the fringes are washed out, saturated, out of focus, etc.
  • the solutions for Equations (1) through (3) for the phase, the average intensity, intensity modulation, and data modulation can be determined by Fourier coefficients as suggested in U.S. Patent No. 3,694,088 to Gallagher et al., the disclosure of which has been previously incorporated herein by reference.
  • the image generator 22 extracts the height information from the phase- wrapped image using a standard phase unwrapping algorithm.
  • a standard phase unwrapping algorithm There are a variety of algorithms that are well known in the art for unwrapping the phase-wrapped image. General methods for unwrapping the phase-wrapped image are described in "Phase Shifting Interferometry," in Optical Shop Testing, by J.E. Greivenkamp and J.H. Bruning, 2d Ed., Daniel Malacara, John Wiley & Sons (1992) at pages 551-553.
  • U.S. Patent No. 5,307,152 to Boehnlein et al. discloses an algorithm (Col. 7, line 7 through Col. 8, line 43) for unwrapping a phase-wrapped image that was generated with Equation (4A) with phase shifts of 90 degrees.
  • Equation (8) Equation (8) where k is the number of 2 ⁇ 's that needs to be added in order to remove the discontinuity.
  • the number k can be positive or negative depending on the geometry of the surface.
  • Equation (8) to the phase-wrapped image provides a series of horizontal surface profiles that are not height related. In order to form a continuous surface, the height relations of the horizontal surface profiles needs to be determined.
  • s y ⁇ f y (M/2,N) -f y ( /2,0)]/N, (12) are the slopes of the middle row and the middle column respectively. The slopes are included in the equation to remove the tilt of the surface contour.
  • the above algorithm is premised upon the object occupying the entire image. If the object occupies only part of the image as shown in Figure 6A, 6B, and 6 C, then a mask is generated to define the boundaries of the object removing the noise in the background as shown in Figure 7.
  • the mask in Figure 7 was generated by taking the average of the three reflected phase shifted fringe patterns and binarizing the average at a gray level of 20.
  • the algorithm is modified to do phase unwrapping only within the boundaries of the object.
  • phase-wrapped images are generated.
  • the phase angles of Figures 6 A. 6B, and 6C are separated by 120 degrees.
  • the image generator 22 combines the distorted fringe patterns to generate a phase-wrapped image as shown in Figure 8 by extracting the phase of each pixel of the image as described above.
  • noise in the background was removed by masking the boundaries of the object as shown in Figure 7.
  • the image generator 22 reconstructs the surface 14 of the object 12 by unwrapping the phase-wrapped image.
  • the phase shifted fringe patterns are color encoded and projected substantially simultaneously.
  • the inventors' initial experimentation using color encoding is described in SPIE Proceedings Vol. 3407, Paper No.: 3407-66, pp.477-482 entitled "Color-encoded fringe projection and phase shifting for 3D surface contouring" published on September 29,1998, the disclosure of which is incorporated herein by reference.
  • This embodiment of the invention will be explained with reference to Figures 10 through 13, which show images that were generated in reconstructing the three dimensional surface contour of a plaster sculpture of a head.
  • the fringe pattern generator 16 generates substantially simultaneously a plurality of phase shifted digitally- interpretable fiinge pattern signals that are color encoded.
  • the three step algorithm is used for phase wrapping, preferably three signals are generated.
  • the signals are encoded with information so that the optical fiinge patterns are phase shifted by 120 degrees with each one having a separate color, e.g., red, green, and blue.
  • the fringe pattern generator 16 generates a signal containing the information of three phase shifted patterns superimposed, but color encoded for identifying the separate phase shifts for image reconstruction.
  • the digital video projector 18 receives the superimposed color encoded signal and converts the signal into a color encoded optical fringe pattern and projects it onto the surface 14 of the object as shown in Figure 10.
  • the fringe pattern shown in Figure 10 includes the three phase shifted components shown in Figures 11 A, 1 IB, and 1 IC.
  • color can be added by three different mechanisms. Where a projector has only one DMD chip as described above with reference to Figure 5, the color is added by the use of a rotating RGB color wheel. In projectors that have two DMD chips, one chip is dedicated to red and the other chip to green and blue. A color wheel with yellow and magenta filters is used to provide a continuous red beam and an alternating green and blue beam.
  • a DMD chip In three-chip projectors, a DMD chip is dedicated for red, green, and blue. As expected the three-chip system provides the best light efficiency, but is also the most expensive.
  • the three phase shifted components shown in Figures 11 A, 1 IB, and 1 IC can truly be projected simultaneously.
  • the three phase shifted components are also projected simultaneously with each component being intermittently projected. The intermittent projection is dependent upon the location of the rotating color wheel with respect to the light being passed therethrough.
  • the optical retrieval device 20 is a color camera.
  • the optical retrieval device 20 is a three-CCD color video camera providing high resolution because each CCD can be dedicated to one color channel.
  • the optical retrieval device retrieves the reflection of the optical fringe pattern shown in Figure 10.
  • a digital camera from Kodak Model DC210 which has 1152 x 864 pixels was used.
  • the camera has a color CCD sensor that produces color images based on a filter array technique that has 50% of the pixels filtered for green and 25% each for red and blue. There are more pixels filtered for green than those for red and blue because human eyes are most sensitive to green.
  • the reflections retrieved by the camera were transferred to an image generator 22 using Kodak's Easy Picture software.
  • the image generator 22 first separates the reflection shown in Figure 10 into its RGB components as shown in Figures 11A, 1 IB, and 1 IC. After the RGB components are acquired, the RGB components are combined through the standard phase wrapping and unwrapping algorithms described above, but with modifications for color coupling and color intensity as discussed below. In the inventors' experimentation, Aldus PhotoStyler image processing software was used to separate the RGB components of the reflection. The modifications discussed below relate to the specific experiments conducted by the inventors. As the discussion will highlight, those skilled in the art will understand that similar modifications may be necessary, but will depend upon the specific application of the invention.
  • the color encoding technique requires that the reflection be separated into RGB components, the outcome of contouring is dependent upon the quality of the separation.
  • the spectra of red, green, and blue channels are usually made to have some overlaps so that there will be no color-blind areas on the spectrum. This implies that the complete separation of the three phase shifted fiinge patterns is not likely.
  • the degrees of overlaps are fixed, compensation for the coupling effects can be done after the image has been taken. Experiments were conducted to find the degree of coupling effects between the three color channels.
  • the target used for this experiment was a flat, white-painted foam board. Purely red, green, and blue fiinge patterns were projected in sequence.
  • I r (x,y), I g (x,y) and I b (x,y) are the original intensities and I rc (x,y), I gc (x,y), and I bc (x,y) are the compensated intensities for the red, green, and blue channels, respectively, a and b represent coupling effect between the red and green channels, and c is the offset that is necessary to keep the intensity values between 0 and 255. The coupling effect between the green and blue channels was ignored. Experimental results showed that this compensation scheme reduced the errors caused by the coupling effect significantly.
  • Figure 10 shows the reflection of the color encoded optical fringe pattern wherein the amplitudes for the three colors are adjusted as described above.
  • Figures 11 A, 1 IB, and 11C show the three phase shifted RGB components extracted from Figure 10 before compensation for the coupling effect.
  • Figure 12 shows the phase-wrapped image that was generated after compensation for coupling effects, and
  • Figure 13 shows the reconstructed 3-D image of the object that was only unwrapped in the area enclosed in the central rectangle shown in Figure 12.
  • the sequential projection of the phase shifted fringe patterns are synchronized to increase contouring speed.
  • the apparatus of the invention as shown in Figure 1 further comprises a synchronizing link 48 which connects the fringe pattern generator 16 to the image generator 22.
  • the synchronizing link 48 can simply be a connection that provides for the coordination between the initial generation of the digitally- interpretable fiinge pattern signal and the retrieval of the reflection of each distorted fringe pattern.
  • This connection can be either be software based or be embedded in the circuitry of either the fringe pattern generator 16 or the image generator 22.
  • a unique operating principle of a single DMD chip digital projector 18 as shown in Figure 5 can be used to control the synchronization.
  • a plurality of phase shifted digitally-interpretable fiinge pattern signals that are color encoded are generated by the fringe pattern generator 16.
  • the RGB components of this color encoded fringe pattern signal will be projected sequentially and repeatedly at a frequency of 60 Hz.
  • the color wheel 42 of the projector is disengaged, the RGB components of the fringe pattern will actually be projected in grayscale.
  • three phase- shifted images of the object can be taken in a very short period, thus boosting up the three dimensional imaging speed significantly.
  • Hardware based image processing and graphics display can also be used to increase the speed of three dimensional reconstruction so that three dimensional imaging at the video rate (30 frames/sec) can be realized.
  • the image generator 22 can be connected to or include an output device for the display or further consideration of the imaging results. More specifically, the output device can include a video monitor or printer for simply displaying the results of the imaging.
  • the image can be compared against a mathematical model of the standard part to determine whether there are any defects in the manufactured part.
  • the image generator 22 can be linked to the fringe pattern generator 16 to generate a signal so that the digital video projector 18 projects information concerning defects.
  • the information concerning the defects can include for example the nature, severity, and area of the defects.
  • the information can also be projected in a variety of ways including: projecting the location of the defects on the part that is being imaged; projecting text describing the defects; projecting colors on the surface of the object; and projecting icons that have a meaning defined by the user.
  • a specific example where the invention can be used for inspection purposes is in the sheet metal manufacturing industry.
  • the invention can also be used in the assembling processes of sheet metal parts which are clamped at various positions. Wrong clamping positions can cause large deformations in the sheet metal parts. With the present invention, the deformation of the sheet metal part can be inspected quickly after clamping and the results can be projected on to the sheet metal surface. The clamping positions can then be adjusted until optimal positions are located.
  • the optical projected fringe patterns are optimized to improve surface contouring.
  • the resolution is generally dependent upon fiinge spacing and contrast in the fringe pattern reflected from the object surface. Since fringe spacing and contrast in the reflected fringe pattern depend on the slope and reflectivity of the surface of the object, the spacing and contrast of the projected fringe pattern should be optimized based on the condition of the surface in order to achieve optimized contouring results. In traditional methods, the spacing and contrast are assigned one value for the entire surface that is being imaged. With the present invention, the object being imaged can be initially reconstructed as described above.
  • the image generator 22 determines the optimal fringe spacing and contrast based upon the initial reconstructed image and through a link to the fringe pattern generator 16 adjusts the fiinge pattern signal accordingly. A second reconstructed image is then obtained using the adjusted fringe pattern signal.
  • the image generator 22 first divides the reconstructed image into a plurality of separate areas with each area being characterized in that it requires a similar fringe pattern for optimal surface contouring. The image generator 22 then assigns each area its optimal fringe spacing and contrast. A second reconstructed image is obtained as described above.
  • the image generator 22 first generates a duplicate of the reconstructed image. The reconstructed image is divided into a plurality of separate areas with each area being characterized in that it requires a similar fringe spacing for optimal surface contouring.
  • the duplicate of the reconstructed image is divided into a plurality of separate areas with each area being characterized in that it requires a similar fringe contrast for optimal surface contouring.
  • the image generator 22 then superimposes the results to assign the entire surface both its optimal fringe spacing and contrast.
  • a second reconstructed image is obtained as described above.
  • the optimizing steps in all embodiments can be repeated as required, but it is expected that only one iteration is necessary for most applications.
  • the fringe pattern is ordered to address the problems associated with discontinuities in the surface of the object.
  • a single fringe is digitally projected and scanned through the surface of the object. While scanning, a plurality of reflected images are retrieved to locate the position of areas with discontinuous features.
  • the reconstruction of the surface is then done in accordance with the invention as described above, but with one fringe encoded with a different color or intensity running through each area identified as having a discontinuous feature.
  • the encoded fringe is now used to reference fringes from which the remaining fringes are ordered.
  • the additional information known from the fringe order is used in phase unwrapping to correctly reconstruct any discontinuous features in the surface of the object.
  • all of the fringes of the fringe pattern are color encoded with a plurality of different colors forming a fringe pattern that is similar to a rainbow.
  • the rainbow pattern can be periodically repeated at a sufficient spacing to insure that there is no overlapping of similar colors.
  • Each fringe can now be ordered on the basis of its rainbow color profile so that any discontinuous features in the surface of the object can be correctly reconstructed during phase unwrapping.
  • a first surface contour is initially extracted by triangulation as known in the art from a plurality of reflected images acquired by projecting and scanning a single fringe through the surface of the object. Since only one image exists in each image, there is no fringe order confusion nor modulo 2 ⁇ ambiguity problem. The surface contour obtained through triangulation is correct, but the tradeoff is that it provides low resolution.
  • a second surface contour is now obtained using the phase shifting method which provides a high resolution of the surface contour but cannot extract the correct height information at surface discontinuities. Once both surface contours are obtained the image generator combines the information from both images to provide a surface contour having high resolution and correct height information at discontinuities.
  • the fringe pattern generator 16 is configured to generate a plurality of digitally-interpretable signals so that the digital video projector 18 projects a plurality of spots or cross-hairs at various points on the surface of the object for determining range information.
  • the range information initially at least three spots or cross-hairs are projected onto separate points on the object.
  • the spots or the centers of the cross-hairs are retrieved by the optical retrieval device 20 and using the principles of triangulation the distance to each point is determined by the image generator 22.
  • the distance information of at least three points on the surface combined with the surface shape obtained by the phase shifting method uniquely determines the absolute position, orientation, and shape of the object surface in space.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)

Abstract

L'invention se rapporte à un procédé pour former les contours tridimensionnels d'une surface, fondé sur une technique de projection de franges sur écran complet. On utilise un système vidéo numérique (18) pour projeter sur un objet (12) des motifs de franges, créés par procédé numérique. Le motif de franges, déformé par la géométrie de la surface (14) de l'objet, est ensuite capturé par une caméra CCD haute résolution (20). Afin d'augmenter la résolution de la formation des contours, on utilise une technique numérique de déphasage entièrement basée sur le logiciel, ce qui rend superflus les systèmes de positionnement précis, utilisés dans les techniques de déphasage traditionnelles. On reconstruit ensuite la surface (14) en appliquant des algorithmes d'habillage et de déshabillage de phase.
PCT/US1998/027915 1997-12-31 1998-12-31 Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video WO1999034301A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP98965560A EP1042718A4 (fr) 1997-12-31 1998-12-31 Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video
JP2000526877A JP2002500369A (ja) 1997-12-31 1998-12-31 デジタル・ビデオ投影システムを使用して三次元表面輪郭描画を行う方法および装置
CA002316838A CA2316838A1 (fr) 1997-12-31 1998-12-31 Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video
AU21001/99A AU2100199A (en) 1997-12-31 1998-12-31 Method and apparatus for three-dimensional surface contouring using a digital video projection system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US7013897P 1997-12-31 1997-12-31
US60/070,138 1997-12-31

Publications (2)

Publication Number Publication Date
WO1999034301A1 WO1999034301A1 (fr) 1999-07-08
WO1999034301A9 true WO1999034301A9 (fr) 1999-09-23

Family

ID=22093382

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1998/027915 WO1999034301A1 (fr) 1997-12-31 1998-12-31 Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video

Country Status (5)

Country Link
EP (1) EP1042718A4 (fr)
JP (1) JP2002500369A (fr)
AU (1) AU2100199A (fr)
CA (1) CA2316838A1 (fr)
WO (1) WO1999034301A1 (fr)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU758952B2 (en) * 2000-01-06 2003-04-03 Canon Kabushiki Kaisha Demodulation and phase estimation of two-dimensional patterns
JP2002081924A (ja) * 2000-07-05 2002-03-22 Ckd Corp 三次元計測装置
US7050876B1 (en) * 2000-10-06 2006-05-23 Phonak Ltd. Manufacturing methods and systems for rapid production of hearing-aid shells
JP3731073B2 (ja) 2002-09-17 2006-01-05 独立行政法人理化学研究所 顕微鏡装置
JP4480488B2 (ja) * 2003-08-28 2010-06-16 富士通株式会社 計測装置、コンピュータ数値制御装置及びプログラム
EP1706839B1 (fr) * 2004-01-15 2014-11-12 Technion Research & Development Foundation Limited Dispositif de balayage video tridimensionnel
GB0415916D0 (en) * 2004-07-16 2004-08-18 Pilkington Plc Glazing inspection
US7898651B2 (en) 2005-10-24 2011-03-01 General Electric Company Methods and apparatus for inspecting an object
DE102006048234A1 (de) * 2006-10-11 2008-04-17 Steinbichler Optotechnik Gmbh Verfahren und Vorrichtung zur Bestimmung der 3D-Koordinaten eines Objekts
US7471383B2 (en) 2006-12-19 2008-12-30 Pilkington North America, Inc. Method of automated quantitative analysis of distortion in shaped vehicle glass by reflected optical imaging
US9846689B2 (en) 2008-01-29 2017-12-19 Adobe Systems Incorporated Method and system to provide portable database functionality in an electronic form
US8165351B2 (en) * 2010-07-19 2012-04-24 General Electric Company Method of structured light-based measurement
CN103487441B (zh) * 2013-09-24 2015-09-30 电子科技大学 一种用于硅晶片缺陷检测和面形测量的方法
CN103528543B (zh) * 2013-11-05 2015-12-02 东南大学 一种光栅投影三维测量中的系统标定方法
CN105300319B (zh) * 2015-11-20 2017-11-07 华南理工大学 一种基于彩色光栅的快速三维立体重建方法
CN109297435A (zh) * 2018-10-24 2019-02-01 重庆大学 一种反向抵消非线性误差的彩色数字光栅编码方法
CN114018176A (zh) * 2021-10-27 2022-02-08 华中科技大学 一种投影图像处理模块、三维重构方法及其系统
CN114812437B (zh) * 2022-03-25 2023-07-04 珠海城市职业技术学院 一种基于像素编码的光学三维测量方法及系统
CN117804381B (zh) * 2024-03-01 2024-05-10 成都信息工程大学 一种基于相机阵列聚焦结构光对物体三维重建方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0076866B1 (fr) * 1981-10-09 1985-05-02 Ibm Deutschland Gmbh Procédé interpolant de la coupe optique
US4634278A (en) * 1984-02-06 1987-01-06 Robotic Vision Systems, Inc. Method of three-dimensional measurement with few projected patterns
US5038291A (en) * 1989-04-03 1991-08-06 General Electric Company Computerized ply pattern generation
US5550960A (en) * 1993-08-02 1996-08-27 Sun Microsystems, Inc. Method and apparatus for performing dynamic texture mapping for complex surfaces
JPH08251520A (ja) * 1995-03-08 1996-09-27 Nikon Corp ビデオプロジェクター
AU683803B2 (en) * 1995-10-17 1997-11-20 Aluminum Company Of America Electronic fringe analysis for determining surface contours

Also Published As

Publication number Publication date
JP2002500369A (ja) 2002-01-08
AU2100199A (en) 1999-07-19
WO1999034301A1 (fr) 1999-07-08
CA2316838A1 (fr) 1999-07-08
EP1042718A4 (fr) 2002-10-16
EP1042718A1 (fr) 2000-10-11

Similar Documents

Publication Publication Date Title
US6438272B1 (en) Method and apparatus for three dimensional surface contouring using a digital video projection system
US6788210B1 (en) Method and apparatus for three dimensional surface contouring and ranging using a digital video projection system
WO1999034301A9 (fr) Procede et appareil pour former les contours tridimensionnels d'une surface au moyen d'un systeme numerique de projection video
US6084712A (en) Three dimensional imaging using a refractive optic design
Huang et al. Color-encoded digital fringe projection technique for high-speed 3-D surface contouring
WO2021027719A1 (fr) Procédé d'étalonnage basé sur réflecteur pour système de projection de franges
US5307151A (en) Method and apparatus for three-dimensional optical measurement of object surfaces
JP3417377B2 (ja) 三次元形状計測方法及び装置並びに記録媒体
JP3525964B2 (ja) 物体の三次元形状計測方法
AU2004273957B2 (en) High speed multiple line three-dimensional digitization
US20070115484A1 (en) 3d shape measurement system and method including fast three-step phase shifting, error compensation and calibration
US20030160970A1 (en) Method and apparatus for high resolution 3D scanning
CN111750806A (zh) 一种多视角三维测量系统及方法
US20120176478A1 (en) Forming range maps using periodic illumination patterns
US20120176380A1 (en) Forming 3d models using periodic illumination patterns
US20140247326A1 (en) Method and system for alignment of a pattern on a spatial coded slide image
JP7413372B2 (ja) 対向配置チャネルを有する三次元センサ
JP2714152B2 (ja) 物体形状測定方法
JP3925266B2 (ja) ３次元形状入力装置および位置ずれ検出方法
CN100417976C (zh) 三维摄影技术方法
CN112562007B (zh) 一种基于三目约束的包裹相位非展开快速立体匹配方法
Huang et al. 3-D Optical measurement using phase shifting based methods
Huang et al. Recent advances in fringe projection technique for 3D shape measurement
Fong et al. Sensing deforming and moving objects with commercial off the shelf hardware
Harding et al. High-speed moire contouring methods analysis

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: C2

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW

AL Designated countries for regional patents

Kind code of ref document: C2

Designated state(s): GH GM KE LS MW SD SZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

COP Corrected version of pamphlet

Free format text: PAGES 1/5-5/5, DRAWINGS, REPLACED BY NEW PAGES 1/11-11/11; DUE TO LATE TRANSMITTAL BY THE RECEIVINGOFFICE

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

ENP Entry into the national phase in:

Ref country code: CA

Ref document number: 2316838

Kind code of ref document: A

Format of ref document f/p: F

Ref document number: 2316838

Country of ref document: CA

ENP Entry into the national phase in:

Ref country code: JP

Ref document number: 2000 526877

Kind code of ref document: A

Format of ref document f/p: F

NENP Non-entry into the national phase in:

Ref country code: KR

WWE Wipo information: entry into national phase

Ref document number: 1998965560

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1998965560

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1998965560

Country of ref document: EP