WO1998028593A1 - Apparatus and method for rapid 3d image parametrization - Google Patents

Apparatus and method for rapid 3d image parametrization Download PDF

Info

Publication number
WO1998028593A1
WO1998028593A1 PCT/US1997/023626 US9723626W WO9828593A1 WO 1998028593 A1 WO1998028593 A1 WO 1998028593A1 US 9723626 W US9723626 W US 9723626W WO 9828593 A1 WO9828593 A1 WO 9828593A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
speckle pattern
plurality
surface
digital images
Prior art date
Application number
PCT/US1997/023626
Other languages
French (fr)
Inventor
Ian W. Hunter
Paul G. Charette
Original Assignee
Pacific Title And Mirage, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US3389396P priority Critical
Priority to US60/033,893 priority
Application filed by Pacific Title And Mirage, Inc. filed Critical Pacific Title And Mirage, Inc.
Publication of WO1998028593A1 publication Critical patent/WO1998028593A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical means
    • G01B11/24Measuring arrangements characterised by the use of optical means for measuring contours or curvatures
    • G01B11/25Measuring arrangements characterised by the use of optical means for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2545Measuring arrangements characterised by the use of optical means for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with one projection direction and several detection directions, e.g. stereo

Abstract

A method and apparatus for rapid three-dimensional geometry parametrization of a three-dimensional surface. A random speckle pattern is projected upon the surface and imaged to obtain a plurality of two-dimensional digital images. The two-dimensional images are processed to obtain a three-dimensional characterization of the surface. The illuminated surface may be modeled to obtain a parameter set characterizing the surface based upon the two-dimensional digital images.

Description

Apparatus and Method for Rapid 3D Image Parametrization

Field of the Invention

This invention relates to the measurement and modeling of a three dimensional surface. More particularly, this invention relates to measuring and modeling a three dimensional surface by projecting a speckle pattern upon the three dimensional surface, imaging the speckle pattern with a plurality of cameras to obtain a plurality of two dimensional digital images, and obtaining from the plurality of two dimensional digital images a three dimensional digital image and a model parameter set representing the three dimensional illuminated surface.

Background of the Invention Speckle techniques, both photographic and interferometric, are known to produce images of surfaces containing a rich set of spatial frequencies thereby providing spatial information on a wide range of spatial scales. A survey of speckle techniques is provided in Jones, Robert, Holographic and speckle interferometry: a discussion of the theory, practice, and application of the techniques, 2nd ed., Cambridge University Press (1989), which is herein incorporated by reference.

Summary of the Invention

A preferred embodiment of the present invention provides a method for measuring and modeling a three dimensional surface. This method has the steps of: (i) illuminating the three dimensional surface with a speckle pattern; (ii) imaging the speckle pattern to obtain a plurality of two dimensional digital images; and (iii) processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the illuminated three dimensional surface. A further embodiment includes the step of modeling the illuminated surface, based upon the plurality of two dimensional digital images, to obtain a parameter set characterizing the illuminated surface. In addition, the embodiment includes performing steps (i)-(iii) as outlined above more than once to provide an ensemble of speckle patterns and an ensemble of two dimensional digital images, wherein the ensemble of speckle patterns contains at least two distinct speckle patterns. In accordance with an alternate embodiment of the invention, the method also has the step of modeling the illuminated surface to obtain a parameter set characterizing the illuminated surface, wherein the modeling is based upon the ensemble of two dimensional digital images.

In accordance with a further aspect of the present invention in one of its embodiments, there is provided an apparatus for rapid three dimensional image parametrization of a three dimensional surface. The apparatus has a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; and a processor in communication with the plurality of cameras for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface. The processor may further provides a parameter set characterizing the three dimensional surface. The speckle pattern generator may have a source of optical radiation coupled through an optical fiber, as well as a speckle pattern shifter for varying the speckle pattern projected upon the three dimensional surface as a function of time. The speckle pattern shifter may be a mechanical strain inducer for applying strain to the optical fiber, and the mechanical strain inducer may be a piezoelectric element.

Yet another embodiment of the present invention is an apparatus for rapid three dimensional image parametrization of a three dimensional surface, where the apparatus comprises a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; a memory in communication with the plurality of cameras for storing the plurality of two dimensional digital images; and a processor in communication with the memory for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface. Brief Description of the Drawing

Fig. 1 illustrates a flow diagram for a preferred embodiment of the invention.

Detailed Description of Specific Embodiments An embodiment of a method for measuring and modeling a three dimensional surface of an object is illustrated in Fig. 1. An object 10 is illuminated by a laser speckle generator 20. The techniques of the present invention are broadly applicable to various optical inspection modalities known in the art, and the application of these techniques to any of such modalities is considered within the scope of the invention and of the appended claims.

In a preferred embodiment, laser speckle generator 20 is a laser coupled to one end of an optical fiber, using optical coupling techniques known to persons of ordinary skill in the art. The end of the optical fiber distal to the laser is used to illuminate the object. The laser speckle generator projects a speckle pattern upon object 10. A speckle pattern is a pattern of illumination in which the intensity profile of the illumination appears as a realization of a random illumination pattern. A laser coupled to an inexpensive, low quality optical fiber may provide a speckle pattern, and it is this combination which serves as a speckle generator in a preferred embodiment. The speckle may be diffraction limited thereby providing a random pattern including the highest spatial frequencies attainable. Furthermore, in a preferred embodiment a mechanical strain may be applied to the optical fiber so that an entire ensemble of uncorrelated speckle patterns can be generated by changing the mechanical strain. This can be accomplished by wrapping the optical fiber around a piezoelectric material, so that a voltage applied to the piezoelectric material causes it to apply a mechanical stress to the optical fiber. In a preferred embodiment, object 10 is illuminated with an ensemble of laser speckle patterns, as indicated in Fig. 1 by the index n = 1,2, 3, . . ..

A plurality of cameras 30 is used to image the speckle patterns illuminated on object 10. In a preferred embodiment, cameras 30 provide digital images. Due to parallax, the images obtained from cameras 30 will be different from each other. From the differences in the digital images obtained from cameras 30, and from knowledge of the relative positions of the cameras to each other, three dimensional coordinates describing the three dimensional surface of object 10 may be obtained for each speckle illumination. Use is made of the random nature of the speckle illumination pattern in determining the differences in the digital images due to parallax. If the speckle pattern is sufficiently random, then small portions of the speckle pattern will be sufficiently different from other small portions of the speckle pattern, and determining the relative shifts of the digital images will be facilitated by making comparisons among these uniquely identifiable small portions.

Parameters of a model may be chosen to fit the digital images obtained from cameras 30 according to a model parameter fitting algorithm, as indicated in step 40 of Fig. 1. Finite element modeling of a surface is the subject of P. Charette et al., "Large deformation mechanical testing of biological membranes using speckle interferometry in transmission. H Finite element modeling," Applied Optics, vol. 36(10), pp. 2246-51 (1997), which is incorporated herein by reference. The parameters may be obtained by a least squares fit using the finite element method in which the basis functions are cubic- Hermite functions. The parameters obtained from model parameter fitting step 40 are used in 50 to obtain a model of the three dimensional surface of object 10. These parameters may be used to display the three dimensional surface of object 10, or they may be used to fabricate or synthesize new three dimensional surfaces which characterize the three dimensional surface of object 10.

As indicated by flow control lines 60 and 70, the illumination and imaging process may be repeated a number of times with or without changing the speckle pattern. For example, the surface of object 10 may be changing as a function of time, in which case the process must be repeated to obtain parameter sets indexed by time. This is indicated by temporal bandwidth control line 60. There is also the spatial frequency aspects of the three dimensional surface which must be properly captured. This is indicated by the spatial bandwidth control line 70, in which object 10 is repeatedly illuminated with different speckle patterns generated by laser speckle generator 20 for each illumination step. By using different speckle patterns which are statistically uncorrelated from each other, it is possible to capture features of the surface of object 10 that might otherwise be missed if the same speckle pattern was used throughout the measurement and modeling process. Statistical averaging may be employed over the ensemble of digital images obtained by illuminating object 10 with different speckle patterns, so that a single parameter set is obtained in which the spatial features of the surface of object 10 are properly captured. Numerous modifications may be made to the embodiments described above without departing from the spirit and scope of the invention.

Claims

What is claimed is:
1. A method for measuring and modeling a three dimensional surface, the method comprising: a. illuminating the three dimensional surface with a speckle pattern; b. imaging the speckle pattern to obtain a plurality of two dimensional digital images; c. processing the plurality of two dimensional digital images to obtain a three dimensional characterization of the illuminated three dimensional surface.
2. The method as set forth in claim 1, further comprising: d. modeling the illuminated surface, based upon the plurality of two dimensional digital images, to obtain a parameter set characterizing the illuminated surface.
3. The method as set forth in claim 1, further comprising: performing steps (a)-(c) more than once to provide an ensemble of speckle patterns and an ensemble of two dimensional digital images, wherein the ensemble of speckle patterns contains at least two distinct speckle patterns.
4. The method as set forth in claim 3, further comprising: modeling the illuminated surface to obtain a parameter set characterizing the illuminated surface, wherein the modeling is based upon the ensemble of two dimensional digital images.
5. An apparatus for rapid three dimensional image parametrization of a three dimensional surface, the apparatus comprising: a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; and a processor in communication with the plurality of cameras for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface.
6. The apparatus as set forth in claim 5, wherein the speckle pattern generator comprises a source of optical radiation coupled through an optical fiber.
7. The apparatus as set forth in claim 5, further comprising a speckle pattern shifter for varying the speckle pattern projected upon the three dimensional surface as a function of time.
8. The apparatus as set forth in claim 6, further comprising a speckle pattern shifter for varying the speckle pattern projected upon the three dimensional surface as a function of time.
9. The apparatus as set forth in claim 8, wherein the speckle pattern shifter is a mechanical strain inducer for applying strain to the optical fiber.
10. The apparatus as set forth in claim 9, wherein the mechanical strain inducer is a piezoelectric element.
11. The apparatus as set forth in claim 5, wherein the processor further provides a parameter set characterizing the three dimensional surface.
12. An apparatus for rapid three dimensional image parametrization of a three dimensional surface, the apparatus comprising: a. a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; b. a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; c. a memory in communication with the plurality of cameras for storing the plurality of two dimensional digital images; and d. a processor in communication with the memory for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface.
PCT/US1997/023626 1996-12-20 1997-12-19 Apparatus and method for rapid 3d image parametrization WO1998028593A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US3389396P true 1996-12-20 1996-12-20
US60/033,893 1996-12-20

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP19970952572 EP0946856A1 (en) 1996-12-20 1997-12-19 Apparatus and method for rapid 3d image parametrization
CA 2275411 CA2275411A1 (en) 1996-12-20 1997-12-19 Apparatus and method for rapid 3d image parametrization
JP52900598A JP2001507133A (en) 1996-12-20 1997-12-19 Fast 3d image parameter displaying apparatus and method

Publications (1)

Publication Number Publication Date
WO1998028593A1 true WO1998028593A1 (en) 1998-07-02

Family

ID=21873072

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1997/023626 WO1998028593A1 (en) 1996-12-20 1997-12-19 Apparatus and method for rapid 3d image parametrization

Country Status (4)

Country Link
EP (1) EP0946856A1 (en)
JP (1) JP2001507133A (en)
CA (1) CA2275411A1 (en)
WO (1) WO1998028593A1 (en)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8050461B2 (en) 2005-10-11 2011-11-01 Primesense Ltd. Depth-varying light fields for three dimensional sensing
US8150142B2 (en) 2007-04-02 2012-04-03 Prime Sense Ltd. Depth mapping using projected patterns
US8350847B2 (en) 2007-01-21 2013-01-08 Primesense Ltd Depth mapping using multi-beam illumination
US8374397B2 (en) 2005-10-11 2013-02-12 Primesense Ltd Depth-varying light fields for three dimensional sensing
US8390821B2 (en) 2005-10-11 2013-03-05 Primesense Ltd. Three-dimensional sensing using speckle patterns
US8400494B2 (en) 2005-10-11 2013-03-19 Primesense Ltd. Method and system for object reconstruction
US8456517B2 (en) 2008-07-09 2013-06-04 Primesense Ltd. Integrated processor for 3D mapping
US8462207B2 (en) 2009-02-12 2013-06-11 Primesense Ltd. Depth ranging with Moiré patterns
US8494252B2 (en) 2007-06-19 2013-07-23 Primesense Ltd. Depth mapping using optical elements having non-uniform focal characteristics
US8493496B2 (en) 2007-04-02 2013-07-23 Primesense Ltd. Depth mapping using projected patterns
US8717417B2 (en) 2009-04-16 2014-05-06 Primesense Ltd. Three-dimensional mapping and imaging
US8786682B2 (en) 2009-03-05 2014-07-22 Primesense Ltd. Reference image techniques for three-dimensional sensing
US8830227B2 (en) 2009-12-06 2014-09-09 Primesense Ltd. Depth-based gain control
US8982182B2 (en) 2010-03-01 2015-03-17 Apple Inc. Non-uniform spatial resource allocation for depth mapping
US9030528B2 (en) 2011-04-04 2015-05-12 Apple Inc. Multi-zone imaging sensor and lens array
US9066087B2 (en) 2010-11-19 2015-06-23 Apple Inc. Depth mapping using time-coded illumination
US9098931B2 (en) 2010-08-11 2015-08-04 Apple Inc. Scanning projectors and image capture modules for 3D mapping
US9131136B2 (en) 2010-12-06 2015-09-08 Apple Inc. Lens arrays for pattern projection and imaging
US9157790B2 (en) 2012-02-15 2015-10-13 Apple Inc. Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis
US9330324B2 (en) 2005-10-11 2016-05-03 Apple Inc. Error compensation in three-dimensional mapping
US9582889B2 (en) 2009-07-30 2017-02-28 Apple Inc. Depth mapping based on pattern matching and stereoscopic information

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7433024B2 (en) * 2006-02-27 2008-10-07 Prime Sense Ltd. Range mapping using speckle decorrelation
JP2012002780A (en) * 2010-06-21 2012-01-05 Shinko Electric Ind Co Ltd Shape measurement instrument, shape measurement method, and semiconductor package manufacturing method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4148587A (en) * 1977-10-03 1979-04-10 The Boeing Company Laser gauge for measuring changes in the surface contour of a moving part
WO1996025764A1 (en) * 1995-02-16 1996-08-22 Environmental Research Institute Of Michigan System and method for three-dimensional imaging of opaque objects
JPH0914914A (en) * 1994-06-06 1997-01-17 Kishimoto Sangyo Kk Laser light projection method and apparatus therefor in device measuring for moving value by laser speckle pattern

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4148587A (en) * 1977-10-03 1979-04-10 The Boeing Company Laser gauge for measuring changes in the surface contour of a moving part
JPH0914914A (en) * 1994-06-06 1997-01-17 Kishimoto Sangyo Kk Laser light projection method and apparatus therefor in device measuring for moving value by laser speckle pattern
WO1996025764A1 (en) * 1995-02-16 1996-08-22 Environmental Research Institute Of Michigan System and method for three-dimensional imaging of opaque objects

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LULLI A ET AL: "CONTRAST VARIATIONS IN WHITE-LIGHT SPECKLE INTERFEROMETRY WITH APPLICATION TO 3D PROFILOMETRY", OPTICS COMMUNICATIONS, vol. 124, no. 5/06, 15 March 1996 (1996-03-15), pages 550 - 557, XP000583811 *
PATENT ABSTRACTS OF JAPAN vol. 097, no. 005 30 May 1997 (1997-05-30) *

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8400494B2 (en) 2005-10-11 2013-03-19 Primesense Ltd. Method and system for object reconstruction
US8050461B2 (en) 2005-10-11 2011-11-01 Primesense Ltd. Depth-varying light fields for three dimensional sensing
US9066084B2 (en) 2005-10-11 2015-06-23 Apple Inc. Method and system for object reconstruction
US8374397B2 (en) 2005-10-11 2013-02-12 Primesense Ltd Depth-varying light fields for three dimensional sensing
US8390821B2 (en) 2005-10-11 2013-03-05 Primesense Ltd. Three-dimensional sensing using speckle patterns
US9330324B2 (en) 2005-10-11 2016-05-03 Apple Inc. Error compensation in three-dimensional mapping
US8350847B2 (en) 2007-01-21 2013-01-08 Primesense Ltd Depth mapping using multi-beam illumination
US8150142B2 (en) 2007-04-02 2012-04-03 Prime Sense Ltd. Depth mapping using projected patterns
US8493496B2 (en) 2007-04-02 2013-07-23 Primesense Ltd. Depth mapping using projected patterns
US8494252B2 (en) 2007-06-19 2013-07-23 Primesense Ltd. Depth mapping using optical elements having non-uniform focal characteristics
US8456517B2 (en) 2008-07-09 2013-06-04 Primesense Ltd. Integrated processor for 3D mapping
US8462207B2 (en) 2009-02-12 2013-06-11 Primesense Ltd. Depth ranging with Moiré patterns
US8786682B2 (en) 2009-03-05 2014-07-22 Primesense Ltd. Reference image techniques for three-dimensional sensing
US8717417B2 (en) 2009-04-16 2014-05-06 Primesense Ltd. Three-dimensional mapping and imaging
US9582889B2 (en) 2009-07-30 2017-02-28 Apple Inc. Depth mapping based on pattern matching and stereoscopic information
US8830227B2 (en) 2009-12-06 2014-09-09 Primesense Ltd. Depth-based gain control
US8982182B2 (en) 2010-03-01 2015-03-17 Apple Inc. Non-uniform spatial resource allocation for depth mapping
US9098931B2 (en) 2010-08-11 2015-08-04 Apple Inc. Scanning projectors and image capture modules for 3D mapping
US9066087B2 (en) 2010-11-19 2015-06-23 Apple Inc. Depth mapping using time-coded illumination
US9215449B2 (en) 2010-11-19 2015-12-15 Apple Inc. Imaging and processing using dual clocks
US9131136B2 (en) 2010-12-06 2015-09-08 Apple Inc. Lens arrays for pattern projection and imaging
US9167138B2 (en) 2010-12-06 2015-10-20 Apple Inc. Pattern projection and imaging using lens arrays
US9030528B2 (en) 2011-04-04 2015-05-12 Apple Inc. Multi-zone imaging sensor and lens array
US9157790B2 (en) 2012-02-15 2015-10-13 Apple Inc. Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis
US9651417B2 (en) 2012-02-15 2017-05-16 Apple Inc. Scanning depth engine

Also Published As

Publication number Publication date
EP0946856A1 (en) 1999-10-06
CA2275411A1 (en) 1998-07-02
JP2001507133A (en) 2001-05-29

Similar Documents

Publication Publication Date Title
Lenz et al. Techniques for calibration of the scale factor and image center for high accuracy 3D machine vision metrology
US6341016B1 (en) Method and apparatus for measuring three-dimensional shape of object
Reid Automatic fringe pattern analysis: a review
JP2973332B2 (en) Light sensor
US9115986B2 (en) Device for optically scanning and measuring an environment
US8878905B2 (en) Focus scanning apparatus
US9066084B2 (en) Method and system for object reconstruction
US6611617B1 (en) Scanning apparatus and method
US6788210B1 (en) Method and apparatus for three dimensional surface contouring and ranging using a digital video projection system
JP3856838B2 (en) Method and apparatus for measuring the shape of an object
US6268923B1 (en) Optical method and system for measuring three-dimensional surface topography of an object having a surface contour
US7440590B1 (en) System and technique for retrieving depth information about a surface by projecting a composite image of modulated light patterns
US6567682B1 (en) Apparatus and method for lesion feature identification and characterization
KR100815283B1 (en) System for simultaneous projections of multiple phase-shifted patterns for the three-dimensional inspection of an object
US20100046005A1 (en) Electrostatice chuck with anti-reflective coating, measuring method and use of said chuck
US6229913B1 (en) Apparatus and methods for determining the three-dimensional shape of an object using active illumination and relative blurring in two-images due to defocus
Tajima et al. 3-D data acquisition by rainbow range finder
Jalkio et al. Three dimensional inspection using multistripe structured light
Wang et al. Optical vortex metrology for nanometric speckle displacement measurement
US6147760A (en) High speed three dimensional imaging method
US7274470B2 (en) Optical 3D digitizer with enlarged no-ambiguity zone
Kahn-Jetter et al. Three-dimensional displacement measurements using digital image correlation and photogrammic analysis
DE112009001652T5 (en) Multichannel recording
US6252623B1 (en) Three dimensional imaging system
Esteve-Taboada et al. Three-dimensional object recognition by Fourier transform profilometry

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase in:

Ref country code: CA

Ref document number: 2275411

Kind code of ref document: A

Format of ref document f/p: F

Ref document number: 2275411

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 1997952572

Country of ref document: EP

ENP Entry into the national phase in:

Ref country code: JP

Ref document number: 1998 529005

Kind code of ref document: A

Format of ref document f/p: F

WWP Wipo information: published in national office

Ref document number: 1997952572

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1997952572

Country of ref document: EP