WO1998028593A1 - Apparatus and method for rapid 3d image parametrization - Google Patents
Apparatus and method for rapid 3d image parametrization Download PDFInfo
- Publication number
- WO1998028593A1 WO1998028593A1 PCT/US1997/023626 US9723626W WO9828593A1 WO 1998028593 A1 WO1998028593 A1 WO 1998028593A1 US 9723626 W US9723626 W US 9723626W WO 9828593 A1 WO9828593 A1 WO 9828593A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- dimensional
- speckle pattern
- digital images
- speckle
- set forth
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2545—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with one projection direction and several detection directions, e.g. stereo
Definitions
- This invention relates to the measurement and modeling of a three dimensional surface. More particularly, this invention relates to measuring and modeling a three dimensional surface by projecting a speckle pattern upon the three dimensional surface, imaging the speckle pattern with a plurality of cameras to obtain a plurality of two dimensional digital images, and obtaining from the plurality of two dimensional digital images a three dimensional digital image and a model parameter set representing the three dimensional illuminated surface.
- Speckle techniques both photographic and interferometric, are known to produce images of surfaces containing a rich set of spatial frequencies thereby providing spatial information on a wide range of spatial scales.
- a survey of speckle techniques is provided in Jones, Robert, Holographic and speckle interferometry: a discussion of the theory, practice, and application of the techniques, 2nd ed., Cambridge University Press (1989), which is herein incorporated by reference.
- a preferred embodiment of the present invention provides a method for measuring and modeling a three dimensional surface.
- This method has the steps of: (i) illuminating the three dimensional surface with a speckle pattern; (ii) imaging the speckle pattern to obtain a plurality of two dimensional digital images; and (iii) processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the illuminated three dimensional surface.
- a further embodiment includes the step of modeling the illuminated surface, based upon the plurality of two dimensional digital images, to obtain a parameter set characterizing the illuminated surface.
- the embodiment includes performing steps (i)-(iii) as outlined above more than once to provide an ensemble of speckle patterns and an ensemble of two dimensional digital images, wherein the ensemble of speckle patterns contains at least two distinct speckle patterns.
- the method also has the step of modeling the illuminated surface to obtain a parameter set characterizing the illuminated surface, wherein the modeling is based upon the ensemble of two dimensional digital images.
- an apparatus for rapid three dimensional image parametrization of a three dimensional surface has a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; and a processor in communication with the plurality of cameras for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface.
- the processor may further provides a parameter set characterizing the three dimensional surface.
- the speckle pattern generator may have a source of optical radiation coupled through an optical fiber, as well as a speckle pattern shifter for varying the speckle pattern projected upon the three dimensional surface as a function of time.
- the speckle pattern shifter may be a mechanical strain inducer for applying strain to the optical fiber, and the mechanical strain inducer may be a piezoelectric element.
- Yet another embodiment of the present invention is an apparatus for rapid three dimensional image parametrization of a three dimensional surface, where the apparatus comprises a speckle pattern generator for providing a speckle pattern upon the three dimensional surface; a plurality of cameras for imaging the speckle pattern to provide a plurality of two dimensional digital images; a memory in communication with the plurality of cameras for storing the plurality of two dimensional digital images; and a processor in communication with the memory for processing the plurality of two dimensional digital images to obtain a three dimensional digital characterization of the three dimensional surface.
- Fig. 1 illustrates a flow diagram for a preferred embodiment of the invention.
- FIG. 1 An embodiment of a method for measuring and modeling a three dimensional surface of an object is illustrated in Fig. 1.
- An object 10 is illuminated by a laser speckle generator 20.
- the techniques of the present invention are broadly applicable to various optical inspection modalities known in the art, and the application of these techniques to any of such modalities is considered within the scope of the invention and of the appended claims.
- laser speckle generator 20 is a laser coupled to one end of an optical fiber, using optical coupling techniques known to persons of ordinary skill in the art.
- the end of the optical fiber distal to the laser is used to illuminate the object.
- the laser speckle generator projects a speckle pattern upon object 10.
- a speckle pattern is a pattern of illumination in which the intensity profile of the illumination appears as a realization of a random illumination pattern.
- a laser coupled to an inexpensive, low quality optical fiber may provide a speckle pattern, and it is this combination which serves as a speckle generator in a preferred embodiment.
- the speckle may be diffraction limited thereby providing a random pattern including the highest spatial frequencies attainable.
- a mechanical strain may be applied to the optical fiber so that an entire ensemble of uncorrelated speckle patterns can be generated by changing the mechanical strain. This can be accomplished by wrapping the optical fiber around a piezoelectric material, so that a voltage applied to the piezoelectric material causes it to apply a mechanical stress to the optical fiber.
- a plurality of cameras 30 is used to image the speckle patterns illuminated on object 10.
- cameras 30 provide digital images. Due to parallax, the images obtained from cameras 30 will be different from each other. From the differences in the digital images obtained from cameras 30, and from knowledge of the relative positions of the cameras to each other, three dimensional coordinates describing the three dimensional surface of object 10 may be obtained for each speckle illumination. Use is made of the random nature of the speckle illumination pattern in determining the differences in the digital images due to parallax. If the speckle pattern is sufficiently random, then small portions of the speckle pattern will be sufficiently different from other small portions of the speckle pattern, and determining the relative shifts of the digital images will be facilitated by making comparisons among these uniquely identifiable small portions.
- Parameters of a model may be chosen to fit the digital images obtained from cameras 30 according to a model parameter fitting algorithm, as indicated in step 40 of Fig. 1.
- Finite element modeling of a surface is the subject of P. Charette et al., "Large deformation mechanical testing of biological membranes using speckle interferometry in transmission. H Finite element modeling,” Applied Optics, vol. 36(10), pp. 2246-51 (1997), which is incorporated herein by reference.
- the parameters may be obtained by a least squares fit using the finite element method in which the basis functions are cubic- Hermite functions.
- the parameters obtained from model parameter fitting step 40 are used in 50 to obtain a model of the three dimensional surface of object 10. These parameters may be used to display the three dimensional surface of object 10, or they may be used to fabricate or synthesize new three dimensional surfaces which characterize the three dimensional surface of object 10.
- the illumination and imaging process may be repeated a number of times with or without changing the speckle pattern.
- the surface of object 10 may be changing as a function of time, in which case the process must be repeated to obtain parameter sets indexed by time. This is indicated by temporal bandwidth control line 60.
- the spatial bandwidth control line 70 There is also the spatial frequency aspects of the three dimensional surface which must be properly captured. This is indicated by the spatial bandwidth control line 70, in which object 10 is repeatedly illuminated with different speckle patterns generated by laser speckle generator 20 for each illumination step.
- Statistical averaging may be employed over the ensemble of digital images obtained by illuminating object 10 with different speckle patterns, so that a single parameter set is obtained in which the spatial features of the surface of object 10 are properly captured. Numerous modifications may be made to the embodiments described above without departing from the spirit and scope of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP97952572A EP0946856A1 (en) | 1996-12-20 | 1997-12-19 | Apparatus and method for rapid 3d image parametrization |
JP52900598A JP2001507133A (en) | 1996-12-20 | 1997-12-19 | High-speed 3D image parameter display apparatus and method |
CA002275411A CA2275411A1 (en) | 1996-12-20 | 1997-12-19 | Apparatus and method for rapid 3d image parametrization |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US3389396P | 1996-12-20 | 1996-12-20 | |
US60/033,893 | 1996-12-20 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1998028593A1 true WO1998028593A1 (en) | 1998-07-02 |
Family
ID=21873072
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/023626 WO1998028593A1 (en) | 1996-12-20 | 1997-12-19 | Apparatus and method for rapid 3d image parametrization |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP0946856A1 (en) |
JP (1) | JP2001507133A (en) |
CA (1) | CA2275411A1 (en) |
WO (1) | WO1998028593A1 (en) |
Cited By (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8050461B2 (en) | 2005-10-11 | 2011-11-01 | Primesense Ltd. | Depth-varying light fields for three dimensional sensing |
US8150142B2 (en) | 2007-04-02 | 2012-04-03 | Prime Sense Ltd. | Depth mapping using projected patterns |
US8350847B2 (en) | 2007-01-21 | 2013-01-08 | Primesense Ltd | Depth mapping using multi-beam illumination |
US8374397B2 (en) | 2005-10-11 | 2013-02-12 | Primesense Ltd | Depth-varying light fields for three dimensional sensing |
US8390821B2 (en) | 2005-10-11 | 2013-03-05 | Primesense Ltd. | Three-dimensional sensing using speckle patterns |
US8400494B2 (en) | 2005-10-11 | 2013-03-19 | Primesense Ltd. | Method and system for object reconstruction |
US8456517B2 (en) | 2008-07-09 | 2013-06-04 | Primesense Ltd. | Integrated processor for 3D mapping |
US8462207B2 (en) | 2009-02-12 | 2013-06-11 | Primesense Ltd. | Depth ranging with Moiré patterns |
US8494252B2 (en) | 2007-06-19 | 2013-07-23 | Primesense Ltd. | Depth mapping using optical elements having non-uniform focal characteristics |
US8493496B2 (en) | 2007-04-02 | 2013-07-23 | Primesense Ltd. | Depth mapping using projected patterns |
US8717417B2 (en) | 2009-04-16 | 2014-05-06 | Primesense Ltd. | Three-dimensional mapping and imaging |
US8786682B2 (en) | 2009-03-05 | 2014-07-22 | Primesense Ltd. | Reference image techniques for three-dimensional sensing |
US8830227B2 (en) | 2009-12-06 | 2014-09-09 | Primesense Ltd. | Depth-based gain control |
US8982182B2 (en) | 2010-03-01 | 2015-03-17 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US9030528B2 (en) | 2011-04-04 | 2015-05-12 | Apple Inc. | Multi-zone imaging sensor and lens array |
US9066087B2 (en) | 2010-11-19 | 2015-06-23 | Apple Inc. | Depth mapping using time-coded illumination |
US9098931B2 (en) | 2010-08-11 | 2015-08-04 | Apple Inc. | Scanning projectors and image capture modules for 3D mapping |
US9131136B2 (en) | 2010-12-06 | 2015-09-08 | Apple Inc. | Lens arrays for pattern projection and imaging |
US9157790B2 (en) | 2012-02-15 | 2015-10-13 | Apple Inc. | Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis |
US9330324B2 (en) | 2005-10-11 | 2016-05-03 | Apple Inc. | Error compensation in three-dimensional mapping |
US9582889B2 (en) | 2009-07-30 | 2017-02-28 | Apple Inc. | Depth mapping based on pattern matching and stereoscopic information |
CN109635539A (en) * | 2018-10-30 | 2019-04-16 | 华为技术有限公司 | A kind of face identification method and electronic equipment |
US10547830B2 (en) | 2015-11-16 | 2020-01-28 | Samsung Electronics Co., Ltd | Apparatus for and method of illumination control for acquiring image information and depth information simultaneously |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2007096893A2 (en) * | 2006-02-27 | 2007-08-30 | Prime Sense Ltd. | Range mapping using speckle decorrelation |
JP2012002780A (en) * | 2010-06-21 | 2012-01-05 | Shinko Electric Ind Co Ltd | Shape measurement instrument, shape measurement method, and semiconductor package manufacturing method |
KR102159996B1 (en) * | 2013-12-16 | 2020-09-25 | 삼성전자주식회사 | Event filtering device and motion recognition device thereof |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4148587A (en) * | 1977-10-03 | 1979-04-10 | The Boeing Company | Laser gauge for measuring changes in the surface contour of a moving part |
WO1996025764A1 (en) * | 1995-02-16 | 1996-08-22 | Environmental Research Institute Of Michigan | System and method for three-dimensional imaging of opaque objects |
JPH0914914A (en) * | 1994-06-06 | 1997-01-17 | Kishimoto Sangyo Kk | Laser light projection method and apparatus therefor in device measuring for moving value by laser speckle pattern |
-
1997
- 1997-12-19 WO PCT/US1997/023626 patent/WO1998028593A1/en not_active Application Discontinuation
- 1997-12-19 EP EP97952572A patent/EP0946856A1/en not_active Withdrawn
- 1997-12-19 JP JP52900598A patent/JP2001507133A/en active Pending
- 1997-12-19 CA CA002275411A patent/CA2275411A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4148587A (en) * | 1977-10-03 | 1979-04-10 | The Boeing Company | Laser gauge for measuring changes in the surface contour of a moving part |
JPH0914914A (en) * | 1994-06-06 | 1997-01-17 | Kishimoto Sangyo Kk | Laser light projection method and apparatus therefor in device measuring for moving value by laser speckle pattern |
WO1996025764A1 (en) * | 1995-02-16 | 1996-08-22 | Environmental Research Institute Of Michigan | System and method for three-dimensional imaging of opaque objects |
Non-Patent Citations (2)
Title |
---|
LULLI A ET AL: "CONTRAST VARIATIONS IN WHITE-LIGHT SPECKLE INTERFEROMETRY WITH APPLICATION TO 3D PROFILOMETRY", OPTICS COMMUNICATIONS, vol. 124, no. 5/06, 15 March 1996 (1996-03-15), pages 550 - 557, XP000583811 * |
PATENT ABSTRACTS OF JAPAN vol. 097, no. 005 30 May 1997 (1997-05-30) * |
Cited By (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8374397B2 (en) | 2005-10-11 | 2013-02-12 | Primesense Ltd | Depth-varying light fields for three dimensional sensing |
US8390821B2 (en) | 2005-10-11 | 2013-03-05 | Primesense Ltd. | Three-dimensional sensing using speckle patterns |
US8400494B2 (en) | 2005-10-11 | 2013-03-19 | Primesense Ltd. | Method and system for object reconstruction |
US9066084B2 (en) | 2005-10-11 | 2015-06-23 | Apple Inc. | Method and system for object reconstruction |
US9330324B2 (en) | 2005-10-11 | 2016-05-03 | Apple Inc. | Error compensation in three-dimensional mapping |
US8050461B2 (en) | 2005-10-11 | 2011-11-01 | Primesense Ltd. | Depth-varying light fields for three dimensional sensing |
US8350847B2 (en) | 2007-01-21 | 2013-01-08 | Primesense Ltd | Depth mapping using multi-beam illumination |
US8493496B2 (en) | 2007-04-02 | 2013-07-23 | Primesense Ltd. | Depth mapping using projected patterns |
US8150142B2 (en) | 2007-04-02 | 2012-04-03 | Prime Sense Ltd. | Depth mapping using projected patterns |
US8494252B2 (en) | 2007-06-19 | 2013-07-23 | Primesense Ltd. | Depth mapping using optical elements having non-uniform focal characteristics |
US8456517B2 (en) | 2008-07-09 | 2013-06-04 | Primesense Ltd. | Integrated processor for 3D mapping |
US8462207B2 (en) | 2009-02-12 | 2013-06-11 | Primesense Ltd. | Depth ranging with Moiré patterns |
US8786682B2 (en) | 2009-03-05 | 2014-07-22 | Primesense Ltd. | Reference image techniques for three-dimensional sensing |
US8717417B2 (en) | 2009-04-16 | 2014-05-06 | Primesense Ltd. | Three-dimensional mapping and imaging |
US9582889B2 (en) | 2009-07-30 | 2017-02-28 | Apple Inc. | Depth mapping based on pattern matching and stereoscopic information |
US8830227B2 (en) | 2009-12-06 | 2014-09-09 | Primesense Ltd. | Depth-based gain control |
US8982182B2 (en) | 2010-03-01 | 2015-03-17 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US9098931B2 (en) | 2010-08-11 | 2015-08-04 | Apple Inc. | Scanning projectors and image capture modules for 3D mapping |
US9215449B2 (en) | 2010-11-19 | 2015-12-15 | Apple Inc. | Imaging and processing using dual clocks |
US9066087B2 (en) | 2010-11-19 | 2015-06-23 | Apple Inc. | Depth mapping using time-coded illumination |
US9167138B2 (en) | 2010-12-06 | 2015-10-20 | Apple Inc. | Pattern projection and imaging using lens arrays |
US9131136B2 (en) | 2010-12-06 | 2015-09-08 | Apple Inc. | Lens arrays for pattern projection and imaging |
US9030528B2 (en) | 2011-04-04 | 2015-05-12 | Apple Inc. | Multi-zone imaging sensor and lens array |
US9157790B2 (en) | 2012-02-15 | 2015-10-13 | Apple Inc. | Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis |
US9651417B2 (en) | 2012-02-15 | 2017-05-16 | Apple Inc. | Scanning depth engine |
US10547830B2 (en) | 2015-11-16 | 2020-01-28 | Samsung Electronics Co., Ltd | Apparatus for and method of illumination control for acquiring image information and depth information simultaneously |
US11153551B2 (en) | 2015-11-16 | 2021-10-19 | Samsung Electronics Co., Ltd | Apparatus for and method of illumination control for acquiring image information and depth information simultaneously |
CN109635539A (en) * | 2018-10-30 | 2019-04-16 | 华为技术有限公司 | A kind of face identification method and electronic equipment |
Also Published As
Publication number | Publication date |
---|---|
CA2275411A1 (en) | 1998-07-02 |
EP0946856A1 (en) | 1999-10-06 |
JP2001507133A (en) | 2001-05-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6101269A (en) | Apparatus and method for rapid 3D image parametrization | |
EP0946856A1 (en) | Apparatus and method for rapid 3d image parametrization | |
Sansoni et al. | A novel, adaptive system for 3-D optical profilometry using a liquid crystal light projector | |
KR100858521B1 (en) | Method for manufacturing a product using inspection | |
Zhang et al. | High-resolution, real-time three-dimensional shape measurement | |
Zhang et al. | High dynamic range scanning technique | |
US5967979A (en) | Method and apparatus for photogrammetric assessment of biological tissue | |
Zhang | High-resolution, real-time 3-D shape measurement | |
EP0551955A1 (en) | System for determining the topography of a curved surface | |
EP1266187A1 (en) | System for simultaneous projections of multiple phase-shifted patterns for the three-dimensional inspection of an object | |
JP2002213931A (en) | Instrument and method for measuring three-dimensional shape | |
US20180234600A1 (en) | Ear canal modeling using pattern projection | |
JPH09218022A (en) | Contour determination method for diffusion surface of work | |
Dirckx et al. | Optoelectronic moiré projector for real-time shape and deformation studies of the tympanic membrane | |
Zhou et al. | Three-dimensional shape measurement using color random binary encoding pattern projection | |
JPH10246612A (en) | Three-dimensional-shape measuring method | |
JP3370418B2 (en) | 3D shape measurement system | |
CN112325799A (en) | High-precision three-dimensional face measurement method based on near-infrared light projection | |
Muyshondt et al. | A calibrated 3D dual-barrel otoendoscope based on fringe-projection profilometry | |
Rioux | Color 3-D electronic imaging of the surface of the human body | |
JP2010210573A (en) | Estimating method of thickness of object | |
Dirckx et al. | Video moire topography for in-vitro studies of the eardrum | |
Levchenko et al. | Portable low-cost setup for outdoor implementation of dynamic speckle technique | |
Shakher et al. | Image processing and analysis of digital speckle pattern interferometric images for monitoring/surface vibration/tilt | |
Tognola et al. | Three-dimensional laser scanning and reconstruction of ear canal impressions for optimal design of hearing aid shells |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): CA JP |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
ENP | Entry into the national phase |
Ref document number: 2275411 Country of ref document: CA Ref country code: CA Ref document number: 2275411 Kind code of ref document: A Format of ref document f/p: F |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1997952572 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref country code: JP Ref document number: 1998 529005 Kind code of ref document: A Format of ref document f/p: F |
|
WWP | Wipo information: published in national office |
Ref document number: 1997952572 Country of ref document: EP |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 1997952572 Country of ref document: EP |