US12019160B2 - Mechanically resonant photoelastic modulator for time-of-flight imaging - Google Patents
Mechanically resonant photoelastic modulator for time-of-flight imaging Download PDFInfo
- Publication number
- US12019160B2 US12019160B2 US16/971,127 US201916971127A US12019160B2 US 12019160 B2 US12019160 B2 US 12019160B2 US 201916971127 A US201916971127 A US 201916971127A US 12019160 B2 US12019160 B2 US 12019160B2
- Authority
- US
- United States
- Prior art keywords
- window
- photoelastic
- optical
- modulation
- acoustic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000003384 imaging method Methods 0.000 title claims description 29
- 230000003287 optical effect Effects 0.000 claims abstract description 100
- 230000000694 effects Effects 0.000 claims abstract description 31
- 230000005284 excitation Effects 0.000 claims abstract description 29
- 239000000463 material Substances 0.000 claims abstract description 20
- 230000010287 polarization Effects 0.000 claims description 69
- GQYHUHYESMUTHG-UHFFFAOYSA-N lithium niobate Chemical compound [Li+].[O-][Nb](=O)=O GQYHUHYESMUTHG-UHFFFAOYSA-N 0.000 claims description 33
- 239000013078 crystal Substances 0.000 claims description 15
- 230000003068 static effect Effects 0.000 claims description 14
- 238000005259 measurement Methods 0.000 claims description 12
- 230000001902 propagating effect Effects 0.000 claims description 9
- 230000010363 phase shift Effects 0.000 claims description 8
- 230000003595 spectral effect Effects 0.000 claims description 5
- 230000035559 beat frequency Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000000576 coating method Methods 0.000 claims description 3
- 239000011248 coating agent Substances 0.000 claims description 2
- RKTYLMNFRDHKIL-UHFFFAOYSA-N copper;5,10,15,20-tetraphenylporphyrin-22,24-diide Chemical compound [Cu+2].C1=CC(C(=C2C=CC([N-]2)=C(C=2C=CC=CC=2)C=2C=CC(N=2)=C(C=2C=CC=CC=2)C2=CC=C3[N-]2)C=2C=CC=CC=2)=NC1=C3C1=CC=CC=C1 RKTYLMNFRDHKIL-UHFFFAOYSA-N 0.000 claims description 2
- 235000012431 wafers Nutrition 0.000 description 92
- 238000001514 detection method Methods 0.000 description 13
- 238000000034 method Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 10
- 238000004088 simulation Methods 0.000 description 10
- 238000002156 mixing Methods 0.000 description 9
- 229940125730 polarisation modulator Drugs 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000005457 optimization Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 6
- 230000005684 electric field Effects 0.000 description 6
- 229910052782 aluminium Inorganic materials 0.000 description 5
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 241000220225 Malus Species 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- NCGICGYLBXGBGN-UHFFFAOYSA-N 3-morpholin-4-yl-1-oxa-3-azonia-2-azanidacyclopent-3-en-5-imine;hydrochloride Chemical compound Cl.[N-]1OC(=N)C=[N+]1N1CCOCC1 NCGICGYLBXGBGN-UHFFFAOYSA-N 0.000 description 1
- 239000004593 Epoxy Substances 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 239000004677 Nylon Substances 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 239000003989 dielectric material Substances 0.000 description 1
- 230000005670 electromagnetic radiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- AMGQUBHHOARCQH-UHFFFAOYSA-N indium;oxotin Chemical compound [In].[Sn]=O AMGQUBHHOARCQH-UHFFFAOYSA-N 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005305 interferometry Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 229920001778 nylon Polymers 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000000206 photolithography Methods 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 239000012780 transparent material Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
- G01S17/36—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
- G01S7/4816—Constructional features, e.g. arrangements of optical elements of receivers alone
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/491—Details of non-pulse systems
- G01S7/4912—Receivers
- G01S7/4915—Time delay measurement, e.g. operational details for pixel components; Phase measurement
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/50—Systems of measurement based on relative movement of target
- G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
Definitions
- This invention relates to optical modulators, especially in connection with distance and velocity imaging.
- An optical modulator with a wide aperture, high acceptance angle, low required drive voltages and high operating frequency is provided.
- the modulator relies on the photoelastic effect and makes use of mechanical resonance in order to improve efficiency.
- the acoustic excitation and optical propagation path are nominally co-aligned, so the required symmetry breaking is provided by having the modulator material be optically and/or mechanically anisotropic.
- Such a modulator can be used to enable efficient and low-cost per-pixel optical ranging and speed sensing.
- Applications include but are not limited to science, tracking, and robotics.
- a major application of this system is for autonomy, where a low-cost and complexity perception module as proposed here can provide situational awareness, imaging, and possibly localization using prior maps.
- the system can also assist other imaging systems to detect object features with high range and depth resolution and providing tracking capability.
- the system uses a combination of optical techniques with signal processing methods to allow for tracking and localization of number of targets in a scene (assuming that the scene is not very complex). It could have applications in scientific discovery, indoor tracking of objects (in low complexity scenes), long range detection or other related applications.
- Combination with hyper-spectral techniques, as described in the invention and with minimal additional hardware, can be used to further differentiate and identify targets in various lighting and environmental conditions.
- Spectral interference filter(s) can be placed in front of the light source and/or receiver to narrow the spectral width of the source and reject ambient noise from the environment at the cost of increased driving power for the optical source(s).
- the intensity modulation of the optical source(s) in the transmitter side can be done by several different methods, including: modulation of the current, a polarization interferometer at the transmitting chain similar to the one used for the receiver.
- the system can also be operated in the homodyne mode by having the intensity modulation frequency of the optical source equal to the demodulation frequency used in the polarization modulator.
- a photoelastic window can be used for the polarization modulator. Driving the photoelastic window near its acoustic resonance frequency modulates the polarization of light passing through the window at the applied drive frequency. Multiple such materials with same/different acoustic resonances can be used/stacked for multi-frequency operation.
- the acoustic resonance frequencies of the photoelastic window used for polarization modulation can be designed to match the optical intensity modulation frequencies of the optical source used to illuminate the scene and have beat frequencies that fall to desired regions in the optical bandwidth of the image sensor.
- Transparent electrodes can be used for the longitudinal mode operation possibly also with anti-reflection coatings.
- a key feature of this work is the use of a polarization interferometer in the receiver chain to determine velocity, depth, spectral information (color) by intensity modulating the received beam from the scene. This effectively performs as a downconverter of the intensity modulated optical source in the transmitter to be recorded and processed by an image sensor.
- FIGS. 1 A-B schematically show operation of a heterodyne optical ranging system.
- FIG. 2 A shows frequency translation in the system on FIGS. 1 A-B
- FIG. 2 B shows a preferred alignment of modulation frequencies with acoustic resonances.
- FIG. 3 A shows a polarization modulator according to principles of this work.
- FIG. 3 B is an end-view of an exemplary photoelastic modulator element.
- FIGS. 4 A-B show volume average strain vs. frequency for the modulator element of FIG. 3 B .
- FIGS. 5 A-B show the fundamental peaks of FIGS. 4 A-B in higher frequency resolution.
- FIG. 6 shows intensity modulation depth as a function of phase modulation.
- FIGS. 7 A-C show simulation results for distance estimation error.
- FIGS. 8 A-C show the mechanical frequency response of the wafer of FIG. 3 B .
- FIGS. 9 A-D show measured heterodyne detection results.
- Section A describes general principles relating to various embodiments of the invention.
- Section B provides a detailed description of some exemplary embodiments of the invention.
- FIG. 1 A shows operation of a lidar system based on a free space optical mixer according to principles of this work.
- An optical source 102 is configured to illuminate at least one target 104 with source light 103 that is intensity modulated at one or more source modulation frequencies f i .
- Source modulation frequencies are preferably in a range from 500 kHz to 500 MHz.
- optical radiation as electromagnetic radiation having a free space wavelength in a range from 100 nm to 15 ⁇ m.
- Any kind of optical source can be employed, such as a laser or light emitting diode (LED).
- Light 105 from the target(s) is imaged onto an imaging detector array 108 by optics (not shown) configured to form an image of the target(s) on the imaging detector array.
- the imaging optics aren't shown here because they are wholly conventional.
- Any kind of imaging detector array can be used, such as CMOS (complementary metal-oxide-semiconductor) image sensors and CCD (charge-coupled device
- the system also includes a receiver modulator 106 that is modulated at one or more receiver modulation frequencies f j * to provide intensity modulation of light 105 .
- Receiver modulation frequencies are preferably in a range from 500 kHz to 500 MHz. Details within block 106 are described below. The effect of this double modulation scheme is shown on FIG. 1 B .
- the signals of interest at detector array 108 are low frequency heterodyne signals at frequencies
- Processor 110 can determine per-pixel distances and velocities 120 .
- FIG. 2 A is another view of the same frequency shift concept. Note that the envelope phase shifts are translated to lower frequencies without being changed, which enables optical ranging using the heterodyne signals. Moving objects in the target(s) will lead to Doppler shifts in frequency, and these Doppler frequency shifts are also translated to lower frequencies without being changed. This enables velocity imaging.
- modulator 106 is an opto-mechanical device having one or more mechanical resonances a 1 , a 2 , etc. as shown on FIG. 2 B .
- receiver modulation frequencies are chosen such that these frequencies are at or near the frequencies of corresponding acoustic resonances.
- the source modulation frequencies are chosen such that each source modulation frequency has a corresponding nearby receiver modulation frequency.
- the differences between each pair of corresponding source and receiver frequencies is distinct. The purpose here is to recover distinct heterodyne signals from each pair of source/receiver frequencies, while ensuring that cross terms from other frequency combinations are out of band for the receiver.
- these acoustic resonances are in a range from 500 kHz to 500 MHz. Throughout this work, acoustic excitation, resonance etc. and mechanical excitation, resonance etc. are regarded as being synonymous terms.
- a mechanical resonance among a 1 , a 2 , etc. can be excited and the intensity modulation frequency for the source light 103 could be equal to the excited mechanical resonance frequency.
- a homodyne measurement can be performed in this case. Sequential homodyne measurements can be performed for the different mechanical resonance frequencies (where the intensity modulation frequency is equal to the mechanical resonance frequency for each measurement) as an alternative to simultaneous multi-frequency heterodyne measurements. Sequential heterodyne measurements could also be performed, where one of the mechanical resonances is excited and the intensity modulation frequency for source light 103 is at a single frequency and slightly different from the mechanical resonance frequency excited.
- FIG. 3 A shows an example of this approach.
- 302 , 304 , 306 , 308 schematically show the state of light beam 300 as it passes through the optical elements of the modulator.
- element 312 is the core of the modulator.
- This element is made of a photoelastic material configured as a photoelastic window 312 having a thickness d separating first and second window surfaces 321 and 323 , and having lateral dimensions substantially greater than d.
- the window having lateral dimensions substantially greater than thickness d means lateral dimensions are preferably 5 d or greater, and more preferably the lateral dimensions are 10 d or greater.
- the window thickness is preferably 4 mm or less.
- a first acoustic excitation structure 322 is disposed on the first window surface 321 .
- a second acoustic excitation structure 324 is disposed on the second window surface 323 .
- These acoustic excitation structures can be electrodes and/or acoustic transducers.
- a controller 320 is connected to the first and second acoustic excitation structures 322 and 324 and is configured to excite one or more acoustic resonances having resonant frequencies determined by d ( FIG. 2 B ).
- the optical path through the photoelastic window 312 is substantially perpendicular to the first and second window surfaces. Substantially perpendicular means perpendicular to within +/ ⁇ 10 degrees.
- Light passing through the photoelastic window 312 and propagating substantially along the optical path experiences a time-varying birefringence responsive to excitation of the one or more acoustic resonances to provide polarization-modulated output light. This amounts to providing a polarization modulation of incident light, e.g., changing the linear polarization of 304 to a more variable state of polarization schematically shown by 306 .
- the acceptance angle for the photoelastic modulator depends on the thickness of the photoelastic window (assuming it is birefringent). The acceptance angle is approximately plus or minus 40 degrees for a lithium niobate wafer thickness of 100 ⁇ m. This is one of the advantages of this modulation scheme compared with other methods. Since the device relies on acoustic resonance rather than optical resonance (e.g. Fabry-Perot Interferometer), the acceptance angle is very large.
- photoelastic window 312 is followed by a polarizer 314 in order to convert polarization modulation 306 into intensity modulation 308 .
- the polarization modulator can optionally include an input polarizer 316 to convert an unpolarized state 302 to a polarized state 304 .
- an input polarizer is not needed if source 102 provide polarized light to the target(s) 104 and the target(s) do not significantly affect the state of polarization.
- the physical effect relied upon in this work is a variable retardance induced in the photoelastic window by an acoustic excitation in a longitudinal excitation geometry (i.e., optical and acoustic paths substantially aligned).
- This symmetry breaking can be mechanical and/or optical.
- a suitable mechanical asymmetry can arise from a crystal structure in which a longitudinal acoustic wave will generate a variable retardance because of the mechanical anisotropy of the crystal.
- a suitable optical asymmetry can arise when light propagating through the photoelastic window experiences (a static) birefringence to break the symmetry and allow the device to function as a controllable wave plate.
- a compensating window 318 is disposed parallel to the photoelastic window 312 to mitigate the effect of the static birefringence on the acceptance angle of the device.
- Retardance is the phase shift between the two polarization modes on passage through a birefringent window.
- the retardances of the photoelastic window and compensating window net out to a substantially 90 degree (i.e., +/ ⁇ 60-120 degrees) retardance. The sign of this net retardance does not matter.
- the transverse acoustic input provides the required symmetry breaking. But here the longitudinal acoustic input does not break the symmetry as needed. Accordingly, the optical path through the window (i.e., along the surface normal of the two surfaces of the window) should not coincide with an optical axis of the window, for designs having static birefringence.
- a uniaxial material has one such optical axis, while biaxial materials have two such optical axes.
- the crystal cut of the window is optimized to maximize its controllable wave plate behavior.
- the input polarization to the window is preferably such as to excite both polarization modes of the window, preferably substantially equally (i.e., mode intensities equal within +/ ⁇ 50%).
- the input polarization could be linear at a 45 degree angle with respect to the relevant axes of the crystal coordinate system.
- a compensating window 318 is disposed parallel to the photoelastic window 312 .
- the compensating window 318 preferably has a retardance of substantially 90 degrees.
- the photoelastic material of window 312 is mechanically anisotropic to provide the required symmetry breaking.
- the photoelastic window 312 can also be piezoelectric.
- the first and second excitation structures can be electrodes configured to excite the acoustic resonances via the piezoelectric effect in the photoelastic window.
- FIG. 3 B shows an exemplary end view of 312 of FIG. 3 A .
- window 312 is made of lithium niobate, so acoustic excitation structures 322 and 324 are electrodes disposed in the optical path of the device. Transparency of the electrodes is provided by making them wire mesh electrodes, as shown by image 332 .
- 322 a and 324 a are connections to electrodes 322 and 324
- 304 schematically shows the incident state of polarization relative to optic axis 330 of window 312 .
- one or more acoustic transducers can be configured to excite the acoustic resonances.
- 324 on FIG. 3 A is optional, since a single acoustic transducer can suffice to provide the required excitation.
- 324 on FIG. 3 A can be present with 322 omitted (i.e., for single acoustic transducer embodiments, it does not matter which side of the window the transducer is on).
- first and/or second excitation structures are transparent, either by being fabricated of transparent materials like indium tin oxide, or by being configured as a wire mesh electrode.
- Any photoelastic material can be used for window 312 , including lithium niobate.
- the photoelastic effect is birefringence induced by mechanical stress. All dielectric materials have this property.
- the crystal family of the photoelastic material is preferably non-cubic (i.e., triclinic, monoclinic, orthorhombic, tetragonal or hexagonal). Here the trigonal and hexagonal lattices are both included in the hexagonal crystal family.
- Anti-reflection coating(s) can be disposed on the first and/or second window surfaces.
- the first and/or second window surfaces can be curved to enhance the one or more acoustic resonances, analogous to how an optical laser mode is formed in an optical cavity with curved mirrors.
- an exemplary system for providing simultaneous optical distance and/or speed measurements of at least one target includes source 102 and imaging detector array 108 as described above.
- Receiver modulator 106 includes one or more polarization modulators 107 , 107 a , . . . as described above, followed by an output polarizer 109 configured to convert the polarization-modulated output light into intensity modulated output light.
- all frequencies can be applied to a single polarization modulator 107 , or each frequency can be applied individually to a corresponding polarization modulator 107 , 107 a , etc. Any intermediate partitioning of frequencies among the polarization modulators is also possible.
- Imaging detector array 108 has an electrical bandwidth B such that the imaging detector array is non-responsive to signals having frequency greater than B.
- Each receiver modulation frequency f j * also has a corresponding acoustic resonance a j (e.g., as shown on FIG. 2 B ).
- the system can further include a processor 110 configured to determine distance and/or speed ( 120 on FIG. 1 B ) of the at least one target from analysis of received Doppler frequency shifts and/or phase shifts of modulation envelopes at one or more beat frequencies
- the distance and/or speed can be determined individually for one or more pixels of the imaging detector array.
- a preferred distance range for distance/speed ranging is 0-200 m.
- the system can also further include a bandpass spectral filter 112 configured to selectively pass light having the source wavelength and disposed to filter the image.
- a bandpass spectral filter 112 configured to selectively pass light having the source wavelength and disposed to filter the image.
- the human visual system and standard image sensors form high-resolution images of their surroundings. These systems are effective in forming images of the surrounding scene but do not provide accurate estimates of depth. Many applications, however, rely on accurate depth images in a scene, including machine vision, tracking, autonomous vehicles and robotics. The need for generating accurate depth images in a scene necessitates new generation of image sensors.
- Depth imaging in a scene can be achieved through the ToF (time of flight) imaging technique.
- a scene is illuminated with a controlled light source and the interaction of this light with the scene is captured and processed for estimating the depth in the scene.
- the most basic method for ToF imaging involves sending a focused beam of light pulse to a particular location in a scene and measuring the time delay of the returned pulse to the optical detector. Scanning the beam allows depth images to be generated. Scanning of the beam can be realized through mechanical scanning or non-mechanical scanning (solid state).
- Non-mechanical scanning usually uses optical phased arrays with full control of the phase and frequency of a laser beam, although recently solid state optomechanical steering has also been proposed.
- Flash lidar captures depth images through illuminating a part of the scene with a modulated light source. Flash lidar avoids scanning the beam by capturing a part of the scene at a single shot, making it a possible low cost, fast and effective way of measuring depth images.
- flash lidars operate in time domain by measuring the ToF for each sensor pixel after flashing the scene with a light source. Each point in the scene is focused to a specific image sensor pixel with the use of an optical lens. The ToF for the light to arrive at each sensor pixel is used to determine the distance of each point to the sensor.
- These flash lidars have high unambiguous range and depth resolution, but are limited by cost or spatial resolution since they require pulsed lasers and specialized pixels with high bandwidths.
- Another class of flash lidar sends intensity modulated light to a scene and measures the phase of the reflected light from the scene with respect to the illumination light phase, similar to the operation of stepped frequency continuous wave (SFCW) radar.
- This technique has also been referred to as radio frequency interferometry (RFI), since light is modulated at typical radar operating frequencies and the envelope of the light is used for estimating distances.
- RFID radio frequency interferometry
- Standard image sensors do not have the bandwidth to capture the phase of megahertz frequencies.
- the standard method is to demodulate the incoming megahertz frequency to a lower frequency before sampling, similar to the working principle of a superheterodyne receiver.
- phase-shift based ToF imaging sensors rely on the photonic mixer device (PMD). Megahertz modulation frequencies are measured by electronic demodulation inside every pixel. These pixels are referred to as demodulation pixels. Homodyne detection is usually used to sample four different phases for the illumination. Since phase is measured, there is an ambiguity in the distance when a single frequency is used, and there is a trade-off between unambiguous range and depth resolution due to the intensity modulation frequency selected. To significantly improve the unambiguous range while retaining the depth resolution, the phase of light at multiple intensity modulation frequencies can be measured, and signal processing techniques similar to SFCW radar can be used.
- PMD photonic mixer device
- the ToF camera using PMD technology or similar architectures use an image sensor with specialized pixels, limiting the spatial resolution. Since these systems use non-standard image sensors, they are expensive. Additionally, detecting multiple frequencies simultaneously requires multi-heterodyne detection, and this requires increasingly complex “smart pixels” with large sizes, leading to large image pixels and therefore reducing spatial resolution. Standard ToF cameras measure the phase at each frequency by stepping the frequency and measuring the phase, increasing the measurement time.
- FIGS. 1 A-B shows the three main components of the ToF imaging system: modulated light source 102 , free-space optical mixer 106 , and the CMOS image sensor 108 .
- Such an architecture would allow the use of the most advanced state of the art image sensors, which are low cost and have high spatial resolution.
- Such an architecture ideally requires a free-space optical mixer with wide acceptance angle, low cost, low power consumption, and centimeter level aperture to be placed in front of the image sensor for performing the heterodyne detection.
- the function of the optical mixer is shown in FIG. 2 A , in which the megahertz level intensity modulated light reflected from the scene is downconverted by the optical mixer to hertz level beat tones. This allows the image sensor to detect the beat tones, which are used to estimate distance and velocity in the scene using signal processing techniques.
- the applied voltage to the photoelastic modulator will be related to the change in the polarization state of light passing through the modulator.
- the polarization modulation will be determined by calculating the modulated index ellipsoid for the photoelastic modulator.
- the index ellipsoid relates the refractive index to the electric field direction of light (polarization).
- the index ellipsoid can be modulated by using the photoelastic effect, therefore modulating the refractive indices for different polarization states.
- strain can be generated in a wafer to control the polarization state of light electronically by modulating the index ellipsoid.
- the polarization modulation should be such that the two in-plane refractive indices for the wafer are modulated by different amounts to result in an in-plane polarization rotation for light propagating perpendicularly to the wafer.
- Photoelastic modulators are used commercially to control the polarization state of light, but they generally use a nonpiezoelectric and isotropic material. Piezoelectric transducers are used to generate strain in the sample, which travels perpendicularly to the propagation direction of light through the modulator. This configuration breaks in-plane symmetry and leads to in-plane polarization modulation.
- the fundamental mechanical resonance frequencies for these devices are usually in the kilohertz range due to the centimeter scale optical aperture of the modulator. Higher order mechanical modes can be used to drive the modulator, but as the mode order increases, the volume average for strain in the sample decreases due to the varying sign of the strain in the sample. Therefore, photoelastic modulators that generate strain traveling perpendicularly to the propagation direction of light and having centimeter scale apertures usually operate at kilohertz mechanical frequencies, limiting the depth resolution of an imaging system.
- Lithium niobate is an anisotropic material, with two different refractive indices corresponding to the two polarization eigenstates.
- Using a lithium niobate wafer leads to a static polarization rotation for light due to anisotropy.
- the polarization rotation is different for rays incident on the wafer at different angles.
- Not correcting for this static birefringence which depends on the angle the incoming ray makes with the wafer normal, will lead to a limited acceptance angle for the wafer.
- another wafer is placed parallel to the original wafer but rotated in plane by 90°. FIG.
- 3 A demonstrates the polarization modulation by the photoelastic modulator.
- 302 , 304 , 306 , 308 schematically show the state of optical beam 300 as it passes through the optical elements of the modulator.
- Unpolarized light ( 302 ) is polarized ( 304 ) after passing through the first polarizer, 316 in FIG. 3 A .
- the polarized light passes through the correcting lithium niobate wafer, element 318 in FIG. 3 A , to remove the static polarization part.
- Polarization state of light is modulated ( 306 ) when passing through the photoelastic modulator, element 312 in FIG. 3 A .
- the polarization modulation 306 is converted to intensity modulation 308 after light passes through the output polarizer, element 314 in FIG. 3 A .
- the unmodulated index ellipsoid for the lithium niobate wafer can be written as in Eq. (1), where n o and n e are the ordinary and extraordinary refractive indices of lithium niobate, respectively
- the wafer will be separated into infinitesimal volumes which have an infinitesimal thickness dy along the y direction of the crystal and other dimensions equal to the wafer cross-section.
- polarization modulation can be determined for each of these infinitesimal volumes using the photoelastic effect.
- the effective index ellipsoid for the wafer is the arithmetic average of the index ellipsoids for these infinitesimal volumes.
- the effective index ellipsoid can be expressed as in Eq. (2), where S ij is the volume average for strain component in the wafer for (i, j) ⁇ x, y, z ⁇ and p kl are the photoelastic constants of lithium niobate for (k,l) ⁇ 1,2, . . . , 6 ⁇ .
- volume average of strain For the rest of the calculations. To determine the volume average strain tensor components generated in the lithium niobate wafer when voltage is applied through surface electrodes, we simulate the wafer using the mechanics and piezoelectric modules in COMSOL simulation platform in frequency domain. The relationship between the generated strain and the applied voltage to the wafer can be determined using the piezoelectric equations.
- the electrodes only cover half of the surface area of the wafer to limit clamping losses when the wafer is tested experimentally, as schematically shown in FIG. 3 B .
- clamping losses are usually the dominant loss mechanism.
- the wafer will be clamped from the sides, therefore only the center part is deposited with aluminum wire grids and the light is passed through this section for polarization modulation.
- the strain tensor components are calculated in the frequency domain from (0.1-25) MHz with a frequency stepping of 10 kHz. Since the net polarization rotation of light is important, we calculate the volume average for the strain components. It is seen from COMSOL simulations that S yy and S zy with respect to crystal axis are the strain components which have a significant non-zero volume average for strain. The effective index ellipsoid can therefore be expressed as Eq. (3).
- FIGS. 4 A-B show the volume average of the strain components in the wafer S yy ( FIG. 4 A ) and S zy ( FIG. 4 B ) corresponding to the region covered with surface electrodes.
- the volume average for the S yy ( FIG. 5 A ) and S zy ( FIG. 5 B ) strain components around the fundamental mechanical resonance frequency inside the wafer are shown. If the wafer is driven at one of its mechanical resonance frequencies f i *, an acoustic standing wave forms inside the wafer.
- the standing wave causes the two strain components contributing to polarization modulation S yy and S zy to be in-phase or anti-phase.
- the modified index ellipsoid in this case can be expressed as in Eq. (7).
- the polarization modulation ⁇ (t) as a function of time will be derived assuming the incoming beam is perpendicular to the wafer (actually at an angle ⁇ to the normal of the wafer) and the wafer is driven at its fundamental mechanical resonance frequency of f i *.
- Another lithium niobate wafer parallel and rotated in plane by 90° is placed before the photoelastic modulator to correct for static polarization rotation of light.
- the incoming beam sees the refractive indices n z ′(t) and n x (t) when passing through the photoelastic modulator, where refractive index along the x and z′ directions are modulated by the photoelastic effect as in Eq. (7) and the coordinate system with respect to the crystal orientation is used.
- n x ⁇ ( t ) n 0 1 + n 0 2 ⁇ ( p 1 ⁇ 2 ⁇ A 1 ⁇ cos ⁇ ( 2 ⁇ ⁇ ⁇ ⁇ f i * ⁇ t ) + 2 ⁇ p 1 ⁇ 4 ⁇ A 2 ⁇ cos ⁇ ( 2 ⁇ ⁇ ⁇ ⁇ f i * ⁇ t ) ) ( 8 )
- n x ⁇ ( t ) ⁇ n 0 - n 0 3 ⁇ ( 1 2 ⁇ p 1 ⁇ 2 ⁇ A 1 + p 1 ⁇ 4 ⁇ A 2 ) ⁇ cos ⁇ ( 2 ⁇ ⁇ ⁇ f i * ⁇ t ) ( 9 )
- n z ′ ⁇ ( t ) n e 1 + n e 2 ⁇ p 1 ⁇ 3 ⁇ A 1 ⁇ cos ⁇ ( 2 ⁇ ⁇ ⁇ ⁇ f i * ⁇ t ) ( 10 ) n z ′ ⁇ ( t ) ⁇ n e - 1 2 ⁇ n e 3 ⁇ p 1 ⁇ 3 ⁇ A 1 ⁇ cos ⁇ ( 2 ⁇ ⁇ ⁇ f i * ⁇ t ) ( 11 )
- phase difference between the two in-plane electric field components (crystal x and z′ directions) of light with wavelength ⁇ after passing through the wafer of thickness L changes according to Eq. (13).
- ⁇ ⁇ ( t ) 2 ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ n ⁇ ( t ) ⁇ L ( 13 )
- the acceptance angle for this system can be calculated by finding the largest incoming angle with respect to the wafer normal, such that the static birefringence between the ordinary and extraordinary rays is 45°.
- the acceptance angle in this case is approximately equal to 2 ⁇ max , where ⁇ max is the largest ⁇ satisfying the expression in Eq. 14.
- the variables used in Eq. 14 are defined in Eqs. 15-17 to have a simplified expression in Eq. 14, where ⁇ r is the refraction angle for a wave that is traveling perpendicular to the optic axis of the lithium niobate wafer and with electric field parallel to the optic axis of lithium niobate.
- ⁇ e is the refraction angle for the extraordinary wave with corresponding refractive index n(ee), where the propagation direction is not parallel nor perpendicular to the optic axis of lithium niobate.
- the acceptance angle is roughly 20°.
- a thinner wafer can be used to increase the acceptance angle.
- a wafer thickness of 0.1 mm has an acceptance angle of approximately 50°, but the wafer has different mechanical resonance frequencies.
- the relationship between the depth of polarization modulation D as a function of the applied peak-to-peak voltage V p to the photoelastic modulator and the quality factor (Q) of the fundamental mechanical resonance mode of the wafer will be derived.
- Loss is added in simulation to the system by making the density of lithium niobate material complex. Since the photoelastic effect is in the linear regime for weak driving, using the added loss component to the material allows the clamping loss effect to be taken into account. The simulation with the added loss is used to determine the strain components and therefore the depth of modulation at a given mechanical Q and voltage applied to the electrodes. In the linear regime for the photoelastic effect, the depth of polarization modulation is proportional to V p and Q. To determine the proportionality constant, relating D to V p and Q, a known peak-to-peak voltage is applied to a lithium niobate wafer with known mechanical Q at the driving frequency.
- the Q for the wafer in the simulation with the added loss is roughly 9000 (calculated based on 3 dB cutoff points for the strain around the fundamental mechanical resonance frequency). Based on these results, and using the linearity of the photoelastic effect in the weak driving regime, the depth of polarization modulation D for an incident beam along the y direction of the wafer can be calculated roughly as in Eq. 18 for light of wavelength 630 nm:
- the depth of polarization modulation is independent of wafer thickness L to first order.
- the amplitude E of the electric field generated inside the wafer by the surface electrodes is inversely proportional to wafer thickness, but this is compensated by the larger path traversed by the light when passing through the wafer.
- the linear electro-optic effect can also modulate the polarization of light in lithium niobate.
- the phase difference between the two in-plane electric field components (crystal x and z′ directions) of light is modulated by the electro-optic effect, which is expressed in Eq. 20.
- the difference between the in-plane refractive indices of the lithium niobate wafer is modulated at the applied frequency to the surface electrodes.
- the electro-optic effect is negligible compared to the photoelastic effect due to the high mechanical resonance exhibited by the wafer (Q ⁇ 9000).
- Polarization modulation can be converted into intensity modulation by sandwiching the photoelastic modulator between two polarizers.
- Malus' law governs the transmitted intensity of light after passing through a polarizer: the transmitted intensity of light after passing through a polarizer is scaled by cosine squared of the angle between the polarization direction of light and the transmission axis of the polarizer. Since standard polarizers have high extinction ratios, high modulation depth can be realized.
- the intensity modulation is a cosine inside a cosine (similar to frequency modulation).
- This expression can be expanded by the Jacobi-Anger expansion, causing an infinite number of equally spaced frequencies.
- the fundamental tone is downconverted into the bandwidth of the image sensor, and the fundamental tone is used for signal processing; the other tones are low-pass filtered by the image sensor.
- the scene is illuminated with intensity modulated light l(t) with intensity I 0 at frequencies f 1 , f 2 , . . . , f n , expressed in Eq. 21.
- the frequencies f 1 , f 2 , . . . , f n are slightly detuned from the frequencies used to drive the photoelastic modulator f 1 *,f 2 *, . . . ,f n *.
- the light reflected from location (x k , y l ) with reflectivity r(x k , y l ) in the scene is represented as s(t,x k , y l ) and assumed to be unpolarized, which is Doppler shifted by ⁇ f 1 , ⁇ f 2 , . . . , ⁇ f n and phase shifted by ⁇ 1 , ⁇ 2 , . . . , ⁇ n , as shown in Eq. 22.
- the received heterodyne beat signal at image sensor pixel (k,l) corresponding to scene location (x k , y l ) is represented as p(t,x k , y l ), which carries the phase and Doppler information at a frequency ( ⁇ Hz) and falls within the bandwidth of the image sensor.
- p(t,x k , y l ) can be expressed as in Eq. 23, which represents the multiple beat frequencies detected by a single image sensor pixel.
- the angle between the polarization axis of the second polarizer (element 314 in FIG. 3 A ) and the optic axis of the photoelastic modulator 312 is ⁇ 0 .
- ⁇ i 4 ⁇ ⁇ ⁇ f i c ⁇ d ⁇ ( x k , ⁇ l ) is the phase shift at the receiver of the intensity-modulated light that illuminates the scene, where d(x k , y l ) is the distance of the receiver to the scene location and c the speed of light in the scene.
- the distance and velocity of each point in the scene can be efficiently computed by performing a fast Fourier transform (FFT) with respect to time per image sensor pixel and using the phase and frequency shift information.
- FFT fast Fourier transform
- Eq. 25 Due to optical mixing, which takes place in Eq. 23, many tones are generated.
- Optimum depth of modulation can be calculated by optimizing J 0 n-1 (D)J 1 (D)sin( ⁇ 0 )sin(2 ⁇ 0 ), assuming the depth of polarization modulation D is the same for all frequencies f i *.
- the modulation signal to the photoelastic modulator is turned off, the outcoming beam before passing through the second polarizer should be circularly polarized light.
- FIG. 6 shows the depth of modulation for different D values and number of frequencies n used to drive the photoelastic modulator.
- the intensity modulation depth drops as the number of frequencies used to drive the photoelastic modulator is increased. Since the wafer is multi-moded and multiples of these modes are driven with the source, many mixing terms appear in the spectrum, reducing the depth of modulation.
- the intensity modulation frequencies for the illumination light f 1 , f 2 , . . . , f n and the mechanical resonance frequencies used for the wafer f 1 *,f 2 *, . . . ,f n * should be selected such that the only beat tones falling within the temporal bandwidth of the image sensor are , . . . , .
- the polarization modulation depth for higher-order mechanical modes is smaller compared to lower-order modes.
- An alternative to using a single wafer driven at multiples of its resonance frequencies is to have wafers of different thicknesses that are placed in front of and parallel to each other. Each wafer can then be driven at its fundamental mechanical resonance frequency, or possibly by driving another higher-order mode.
- Using a single frequency for distance measurements limits the unambiguous range or the depth resolution.
- the unambiguous range is limited to half the wavelength
- the phase of multiple frequencies can be used after the round-trip of light, similar to the operation of SFCW radar.
- the standard CMOS image sensor has high angular resolution due to the wavelength of light. We assume a single target per image sensor pixel due to the high angular resolution of the sensor. Therefore, the constraints that apply to SFCW radar do not apply for our system. This allows achieving high depth resolution and unambiguous range despite measuring the phase of the returned light at several discrete frequencies.
- MPI multi-path interference
- Standard ToF cameras measure the phase at each frequency using homodyne detection.
- each phase can be modeled as a Gaussian distribution:
- this unambiguous range should be determined based on the signal-to-noise ratio (SNR) and the modulation frequencies, but that problem is not dealt with in this work.
- SNR signal-to-noise ratio
- One possible approach to solve the optimization problem is by separating the optimization problem into bounded least squares problems through constraining the distance such that within each of the regions, the objective function is convex (possibly also with some approximations). The global maximum among the local maxima would then be equivalent to solving the non-convex optimization problem.
- the distance is estimated by minimizing the objective function in Eq. 28, and this procedure is applied for each image sensor pixel (k,l) to estimate the distance in the scene d*(x k , y l ).
- the average estimation error in the range (1-100) m using 2000 photons per pixel per frame and 200 frames used per distance estimate is around 0.8 cm.
- Velocity estimation in a scene is not considered in this work, but essentially the Doppler shift of the tones is used.
- a Y-cut lithium niobate wafer of 0.5 mm thickness and 5.08 cm diameter is coated with aluminum wire grid on both surfaces with alignment to attain near uniform electric field inside the wafer (pointing along the y direction of the crystal) and to retain optical transparency.
- Photolithography with a lift-off process is used to deposit 100 nm thick aluminum grid wire on an area of 2.04 cm diameter and centered on both front and back sides of the wafer through back-side alignment.
- Each aluminum wire is 4 ⁇ m thick and separated by 40 ⁇ m.
- Wirebonding is used from the top and bottom electrode connections stretching from the center part coated with aluminum wire grid to the side of the wafer to connect to a printed circuit board (PCB) plane.
- PCB printed circuit board
- FIGS. 8 A-C show the mechanical frequency response for the device (S 11 parameter measured with respect to 50 ⁇ ).
- the fundamental mechanical resonance frequency shows up around 4.023400 MHz, and the other resonance modes are spaced by around 8 MHz, double the fundamental resonance frequency.
- the wafer supports modes up to 100 MHz, but the focus for the rest of this section will be on the fundamental mechanical resonance frequency at 4.023400 MHz and the higher-order mode at 19.580970 MHz.
- CMOS image sensor To observe optical mixing on the CMOS image sensor and downconvert megahertz-level intensity modulation frequencies to hertz range, we intensity modulate a light-emitting diode (LED) emitting light of wavelength 630 nm at a frequency slightly offset from the mechanical resonance frequency of the wafer.
- the light passes through the optical mixer, which includes the aluminum-deposited lithium niobate wafer.
- the system includes the intensity-modulated LED, polarizer, aluminum-deposited lithium niobate wafer (photoelastic modulator) driven at one or more resonance frequencies, a 90° rotated lithium niobate wafer, and another polarizer.
- FIGS. 9 A-D show that the photoelastic effect is what causes the optical mixing, because when the frequency supplied to the photoelastic modulator is swept around the fundamental mechanical resonance frequency ( FIG. 9 D ), the beat tone signal level (appearing at 100 Hz) changes and shows a resonance behavior.
- FIGS. 9 A-D show the detected beat tones with the CMOS camera.
- the wafer is driven at its fundamental mechanical resonance frequency at 4.023400 MHz, and the LED intensity modulation frequency is offset by 100 Hz from 4.023400 MHz.
- the beat signal at 100 Hz is visible after plotting the power spectrum.
- FIG. 9 A the wafer is driven at its fundamental mechanical resonance frequency at 4.023400 MHz, and the LED intensity modulation frequency is offset by 100 Hz from 4.023400 MHz.
- the beat signal at 100 Hz is visible after plotting the power spectrum.
- the wafer is driven at its higher order mode at 19.580970 MHz, and the intensity modulation for the LED is offset by 60 Hz from 19.580970 MHz.
- the beat signal at 60 Hz is visible after plotting the power spectrum.
- 4.023527 MHz and 19.589670 MHz are used simultaneously to drive the wafer, and the LED intensity modulation frequencies are 4.023587 MHz and 19.589770 MHz with the beat signals occurring at 60 Hz and 100 Hz, respectively. 100,000 image sensor pixels are spatially averaged to improve the SNR.
- FIG. 9 D the frequency difference between the LED and the photoelastic modulator is fixed at 100 Hz, and both frequencies are swept around the fundamental mechanical resonance frequency from 4.02 MHz to 4.03 MHz.
- the beat signal level appearing at 100 Hz is plotted, which shows that the photoelastic effect is dominant over the electro-optic effect, since resonance behavior is observed.
- the discrepancy between the expected and measured depth of modulation could be due to the misalignment between the wafers (leading to constructive and destructive interferences as a result of static polarization).
- Another possible source could be the operation of the photoelastic modulator as an open-loop system. Since the fundamental mechanical mode has a high Q, to achieve high modulation depth, the device needs to be operated at resonance, and even small frequency drifts in the fundamental mode should be tracked with a closed-loop system (e.g., phase-locked loop).
- the observed optical mixing shows that the photoelastic modulator is a promising optical mixer. Depth of modulation can be improved through closed-loop driving to track any resonance drifts, aligning the optical components and refabricating the device to attain higher mechanical Q.
- the working principle of a prototype phase-shift-based ToF imaging system using an optical mixer, including a photoelastic modulator sandwiched between polarizers, and placed in front of a standard CMOS image sensor is demonstrated.
- the photoelastic modulator is a Y-cut lithium niobate wafer, which has a thickness of 0.5 mm and a diameter of 5.08 cm.
- the photoelastic modulator is significantly more efficient than an electro-optic modulator for polarization modulation owing to the high mechanical Q and the strong piezoelectricity and photoelasticity of lithium niobate.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Optical Modulation, Optical Deflection, Nonlinear Optics, Optical Demodulation, Optical Logic Elements (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
Description
Δn(t)=(n 0 −n e)−(n x(t)−n z′(t))=(n 0 3(½p 12 A 1 +p 14 A 2)−½n e 3 p 13 A 1)cos(2πf i *t) (12)
is the phase shift at the receiver of the intensity-modulated light that illuminates the scene, where d(xk, yl) is the distance of the receiver to the scene location and c the speed of light in the scene.
is the Doppler shift for the received light due to motion with velocity v(xk, yl) in the scene location (xk, yl). The distance and velocity of each point in the scene can be efficiently computed by performing a fast Fourier transform (FFT) with respect to time per image sensor pixel and using the phase and frequency shift information.
I(t)=I 0Σi=1 n cos(2πf i t) (21)
s(t,x k ,y i)=r(x k ,y l)I 0Σi=1 n cos(2π(f i +Δf i)t+ψ i) (22)
p(t,k,l)=s(t,x k y l)¼ sin(2θ0)cos(ϕ(t)) (23)
ϕ(t)=DΣ i=1 n cos(2πf i *t)+ϕ0 (24)
p(t,k,l)≈(x k ,y l)J 0 n-1(D)J 1(D)sin(ϕ0 sin(2θ0)Σi=1 n cos(2π t+ψ i) (25)
corresponding to the frequency used.
the measured phase for the beat tone ψi∈[0,2π) is used as in Eq. 26 to estimate the distance d(xk, yl) corresponding to image sensor pixel (k,l):
with d(xk, yl) the distance of the location in the scene (xk, yl) to the receiver, c the speed of light in the scene, and σ2 the noise variance. Due to 2π phase wrapping, even if multiple frequencies are used and perfect phase information is retrieved, there will always be an ambiguous range at the least common multiple of the wavelengths corresponding to the modulation frequencies. This presents an ill-posed optimization problem due to multiple solutions. As a way around this problem, we define an unambiguous range, which is smaller than the least common multiple of the modulation frequencies. In fact, this unambiguous range should be determined based on the signal-to-noise ratio (SNR) and the modulation frequencies, but that problem is not dealt with in this work.
where dm(xk, y1)=r×m, m ∈. The distance is estimated by minimizing the objective function in Eq. 28, and this procedure is applied for each image sensor pixel (k,l) to estimate the distance in the scene d*(xk, yl).
Claims (18)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/971,127 US12019160B2 (en) | 2018-04-19 | 2019-04-19 | Mechanically resonant photoelastic modulator for time-of-flight imaging |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201862659871P | 2018-04-19 | 2018-04-19 | |
| US201862750534P | 2018-10-25 | 2018-10-25 | |
| US16/971,127 US12019160B2 (en) | 2018-04-19 | 2019-04-19 | Mechanically resonant photoelastic modulator for time-of-flight imaging |
| PCT/US2019/028345 WO2019204744A1 (en) | 2018-04-19 | 2019-04-19 | Mechanically resonant photoelastic modulator for time-of-flight imaging |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2019/028345 A-371-Of-International WO2019204744A1 (en) | 2018-04-19 | 2019-04-19 | Mechanically resonant photoelastic modulator for time-of-flight imaging |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US18/429,005 Continuation-In-Part US20240210744A1 (en) | 2018-04-19 | 2024-01-31 | Optically Isotropic Piezoelectric Resonant Collinear Acousto-Optic Modulator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20210109223A1 US20210109223A1 (en) | 2021-04-15 |
| US12019160B2 true US12019160B2 (en) | 2024-06-25 |
Family
ID=68240370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US16/971,127 Active 2041-10-21 US12019160B2 (en) | 2018-04-19 | 2019-04-19 | Mechanically resonant photoelastic modulator for time-of-flight imaging |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US12019160B2 (en) |
| WO (1) | WO2019204744A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210156881A1 (en) * | 2019-11-26 | 2021-05-27 | Faro Technologies, Inc. | Dynamic machine vision sensor (dmvs) that performs integrated 3d tracking |
| CN113075641A (en) * | 2020-01-03 | 2021-07-06 | 华为技术有限公司 | TOF depth sensing module and image generation method |
| US11828851B2 (en) * | 2020-01-20 | 2023-11-28 | The Board Of Trustees Of The Leland Stanford Junior University | Wide-field resonant electro-optic imaging devices and applications |
| US11808859B2 (en) * | 2020-12-10 | 2023-11-07 | GM Global Technology Operations LLC | System and method of depth imaging based on rolling shutter cmos image sensor and down conversion techniques |
| CN113959683B (en) * | 2021-10-25 | 2024-02-23 | 北京航空航天大学 | Deep modulation optical rotation angle detection device and method based on photoelastic modulator |
| CN115718380B (en) * | 2022-11-25 | 2023-09-05 | 中国科学院合肥物质科学研究院 | GHz photoelastic modulator |
| CN119596279B (en) * | 2024-12-17 | 2025-08-29 | 深圳市湾澳科技有限公司 | Optical imaging system and method of resonant cavity |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3655265A (en) * | 1970-08-07 | 1972-04-11 | Hewlett Packard Co | Resonant piezoelectric acousto-optic light filter and apparatus using same |
| WO1994028452A1 (en) | 1993-05-25 | 1994-12-08 | Honeywell Inc. | Opto-electro-mechanical device or filter, process for making, and sensors made therefrom |
| US5422966A (en) | 1994-06-10 | 1995-06-06 | Gopalakrishnan; Ganesh K. | Microwave electro-optic mixer |
| US6057909A (en) | 1995-06-22 | 2000-05-02 | 3Dv Systems Ltd. | Optical ranging camera |
| US6282010B1 (en) * | 1998-05-14 | 2001-08-28 | Texas Instruments Incorporated | Anti-reflective coatings for spatial light modulators |
| US6825455B1 (en) | 1996-09-05 | 2004-11-30 | Rudolf Schwarte | Method and apparatus for photomixing |
| US6970278B1 (en) | 1998-03-16 | 2005-11-29 | Hinds Instruments, Inc. | Controlling resonant photoelastic modulators |
| US20060227316A1 (en) | 2005-04-06 | 2006-10-12 | Phillip Gatt | Three-dimensional imaging device |
| US8983240B2 (en) * | 2011-12-08 | 2015-03-17 | Universitaet Stuttgart | Electro-optical phase modulator with an optically isotropic material which becomes optically anisotropic under an electric field |
-
2019
- 2019-04-19 WO PCT/US2019/028345 patent/WO2019204744A1/en not_active Ceased
- 2019-04-19 US US16/971,127 patent/US12019160B2/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3655265A (en) * | 1970-08-07 | 1972-04-11 | Hewlett Packard Co | Resonant piezoelectric acousto-optic light filter and apparatus using same |
| WO1994028452A1 (en) | 1993-05-25 | 1994-12-08 | Honeywell Inc. | Opto-electro-mechanical device or filter, process for making, and sensors made therefrom |
| US5422966A (en) | 1994-06-10 | 1995-06-06 | Gopalakrishnan; Ganesh K. | Microwave electro-optic mixer |
| US6057909A (en) | 1995-06-22 | 2000-05-02 | 3Dv Systems Ltd. | Optical ranging camera |
| US6825455B1 (en) | 1996-09-05 | 2004-11-30 | Rudolf Schwarte | Method and apparatus for photomixing |
| US6970278B1 (en) | 1998-03-16 | 2005-11-29 | Hinds Instruments, Inc. | Controlling resonant photoelastic modulators |
| US6282010B1 (en) * | 1998-05-14 | 2001-08-28 | Texas Instruments Incorporated | Anti-reflective coatings for spatial light modulators |
| US20060227316A1 (en) | 2005-04-06 | 2006-10-12 | Phillip Gatt | Three-dimensional imaging device |
| US8983240B2 (en) * | 2011-12-08 | 2015-03-17 | Universitaet Stuttgart | Electro-optical phase modulator with an optically isotropic material which becomes optically anisotropic under an electric field |
Non-Patent Citations (9)
Also Published As
| Publication number | Publication date |
|---|---|
| US20210109223A1 (en) | 2021-04-15 |
| WO2019204744A1 (en) | 2019-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12019160B2 (en) | Mechanically resonant photoelastic modulator for time-of-flight imaging | |
| US7193720B2 (en) | Optical vibration imager | |
| US11561084B2 (en) | Polarization sensitive devices, methods and applications | |
| EP3521809B1 (en) | Terahertz full-polarization-state detection spectrograph | |
| US11880114B2 (en) | Ferroelectric liquid crystals Dammann grating for light detection and ranging devices | |
| US11822157B2 (en) | Energy efficient, high resolution light detection and ranging imaging receiver with large field-of-view | |
| US11573428B2 (en) | Imaging method and apparatus using circularly polarized light | |
| US6388754B1 (en) | Shape measuring system and method | |
| KR101424665B1 (en) | Range Measuring Apparatus | |
| CN112368566B (en) | Device and method for determining the presence of a gas | |
| JPWO2003074967A1 (en) | Polarization direction detection type two-dimensional light reception timing detection device and surface shape measurement device using the same | |
| JP5347722B2 (en) | Optical filter and image photographing apparatus | |
| KR102007004B1 (en) | Apparatus for measuring three dimensional shape | |
| Atalar et al. | Time-of-flight imaging based on resonant photoelastic modulation | |
| WO2020259193A1 (en) | Laser detection device, method and system | |
| CN105576495B (en) | Residual amplitude modulation stabilising arrangement based on angle of wedge electro-optic crystal | |
| CN105675134A (en) | FLC-based hyperspectral all-polarization imaging device and method | |
| JP5468836B2 (en) | Measuring apparatus and measuring method | |
| US9983052B1 (en) | Reduction of noise using coherent light-source squeezing | |
| US20100039646A1 (en) | Polarimetric imaging system having a matrix of programmable waveplates based on a material with an isotropic electrooptic tensor | |
| CN108896181B (en) | A spectral polarization imaging device based on elastic-optical modulator and acousto-optical | |
| Atalar et al. | A Time-of-Flight Imaging System Based on Resonant Photoelastic Modulation | |
| CN113884019B (en) | Three-dimensional imaging system and method | |
| JP2015137980A (en) | observation device | |
| Atalar | Resonant photoelastic modulation for time-of-flight imaging with standard image sensors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
| AS | Assignment |
Owner name: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ATALAR, OKAN;SAFAVI-NAEINI, AMIR H.;ARBABIAN, MOHAMMAD AMIN;SIGNING DATES FROM 20200820 TO 20210209;REEL/FRAME:055199/0204 Owner name: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ATALAR, OKAN;SAFAVI-NAEINI, AMIR H.;ARBABIAN, MOHAMMAD AMIN;SIGNING DATES FROM 20190419 TO 20210209;REEL/FRAME:055200/0697 |
|
| AS | Assignment |
Owner name: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ATALAR, OKAN;SAFAVI-NAEINI, AMIR H.;ARBABIAN, MOHAMMAD AMIN;SIGNING DATES FROM 20190419 TO 20210209;REEL/FRAME:055906/0395 |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| ZAAB | Notice of allowance mailed |
Free format text: ORIGINAL CODE: MN/=. |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |