EP3769037A1 - Rideaux de lumière programmables - Google Patents

Rideaux de lumière programmables

Info

Publication number
EP3769037A1
EP3769037A1 EP19772623.5A EP19772623A EP3769037A1 EP 3769037 A1 EP3769037 A1 EP 3769037A1 EP 19772623 A EP19772623 A EP 19772623A EP 3769037 A1 EP3769037 A1 EP 3769037A1
Authority
EP
European Patent Office
Prior art keywords
light
plane
line
sensor
sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19772623.5A
Other languages
German (de)
English (en)
Other versions
EP3769037A4 (fr
Inventor
Srinivasa Narasimhan
Jian Wang
Aswin C. Sankaranarayanan
Joseph BARTELS
William Whittaker
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carnegie Mellon University
Original Assignee
Carnegie Mellon University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carnegie Mellon University filed Critical Carnegie Mellon University
Publication of EP3769037A1 publication Critical patent/EP3769037A1/fr
Publication of EP3769037A4 publication Critical patent/EP3769037A4/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/04—Systems determining the presence of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481—Constructional features, e.g. arrangements of optical elements
    • G01S7/4817—Constructional features, e.g. arrangements of optical elements relating to scanning

Definitions

  • 3D sensors play an important role in the deployment of many autonomous systems, including field robots and self-driving cars.
  • a self-guided vehicle on a road or a robot in the field does not need a full-blown 3D depth sensor to detect potential collisions or monitor its blind spot.
  • a self-guided vehicle on a road or a robot in the field does not need a full-blown 3D depth sensor to detect potential collisions or monitor its blind spot.
  • the vehicle is able to detect if any object comes within a pre-defmed perimeter of the vehicle to allow for collision avoidance. This is a much easier task than full depth scanning and object identification.
  • Various embodiments are generally directed to a device that monitors the presence of objects passing through or impinging on a virtual shell near the device, which is referred to herein as a“light curtain”.
  • Light curtains offer a lightweight, resource-efficient and programmable approach for proximity awareness for obstacle avoidance and navigation. They also have additional benefits in terms of improving visibility in fog as well as flexibility in handling light fall-off
  • the light curtains are created by rapidly rotating a line sensor and a line laser in synchrony.
  • the embodiment is capable of generating light curtains of various shapes with a range of 20-30m in sunlight (40m under cloudy skies and 50m indoors) and adapts dynamically to the demands of the task.
  • light curtains may be implemented by triangulating an illumination plane, created by fanning out a laser, with a sensing plane of a line sensor. In the absence of ambient illumination, the sensor senses light only from the intersection between these two planes, which, in the physical world, is a line. The light curtain is then created by sweeping the illumination and sensing planes in synchrony.
  • the light curtains may have a programmable shape to allow for the detection of objects along a particular perimeter, such as for detecting a vehicle impinging on a lane for a self-driving vehicle.
  • the capability of the light curtain can be enhanced by using correlation-based time-of-flight (ToF) sensors.
  • ToF time-of-flight
  • the light curtain is capable of detecting the presence of objects that intersect a virtual shell around the system. By detecting only the objects that intersect with the virtual shell, many tasks pertaining to collision avoidance and situational awareness can be solved with little or no computational overhead.
  • Light curtains provide a novel approach for proximity sensing and collision avoid-ance that has immense benefits on autonomous devices.
  • the shape of the light curtain can be changed on the fly and can be used to provide better detections, especially under strong ambient light (like sunlight) as well as global illumination (like fog).
  • FIG. 1 is a schematic view of the components of the light curtain device.
  • FIG. 2 is a diagram showing the intersecting planes of the light projector the sensor and the formation of the light curtain by synchronously scanning the projector and sensor such as to move the intersecting line of the planes in the desired shape.
  • FIG. 3 is a top view of the diagram of FIG. 2.
  • FIG. 4 is a diagram showing (a) the viewing an illumination geometry of a light curtain generated by rotating the laser light plane and sensor plane about a parallel axis are; (b) a view of the coordinate frame showing various parameters; and (c) a top view of the coordinate system.
  • FIG. 5 is a diagram showing the thickness of the light curtain.
  • FIG. 6 shows two applications of the light curtain.
  • FIG. 6(a) shows a light curtain being used for a tilted plane a proximity arc and a path check.
  • FIG. 6(b) shows the application of a light curtain in a self-driving vehicle sensing for upcoming traffic adjacent traffic and an adjacent laying check.
  • FIG. 7(a-d) are illustrations of light curtains resulting when the sensor and laser rotates about an axis and about a point.
  • the device consists of a line scan laser (Illumination module 100) and a line scan sensor (Sensor Module 120) as shown in a top view in FIG. 1.
  • illumination module 100 uses a laser diode 102 is a light source.
  • the laser diode 102 may be, for example, a 638nm laser diode with the peak power of 700mW.
  • the light emitted from laser diode 102 is collimated using collimation lens 104.
  • the light is 11 stretched into a line with line lens 106, which may, in some embodiments, be a 45° Powell lens.
  • a steerable galvo mirror 108 is used.
  • the galvo mirror has a dimension of 11 mm x 7 mm and has a 22.5° mechanical angle and can give the sensor and laser a 45° field of view.
  • the galvo mirror 108 takes 500 ps to rotate through a .2° optical angle.
  • a micro-controller is used to synchronize the sensor, the laser and the galvo mirrors 108, 126.
  • the galvo mirror 108 used for the illumination module 100 and the galvo mirror 126 use for sensor module 120 will be identical.
  • a mechanical motor may be used to steer the light beam and sensor.
  • a 2D sensor and with a rolling shutter or a region of interest mask may be used to effectively emulate a faster line sensor.
  • Sensor module 120 comprises a line sensor 122, lens 124 and steerable galvo mirror 126.
  • line sensor 122 is a line scan intensity sensor.
  • the line scan intensity sensor is a 6mm fl 2 S-mount lens having a diagonal field-of-view of 45° and an image circle 7mm in diameter.
  • the line sensor may have 2048x2 total pixels with the pixel size being approximately 7 pm x 7 pm. In preferred embodiments, only the central 1000 pixels of the sensor are used due to the limited circle of illumination of the lens.
  • the line scan sensor may be capable of scanning 95,000 lines per second and may be fitted with an optical bandpass filter having a 630nm center wavelength and a 50nm bandwidth suppress ambient light.
  • the rotation axes our aligned to be parallel and fixed with a baseline of 300 mm.
  • the resulting field-of-view of the system is approximately 45° by
  • the Powell lens 106 fans the laser beam into a planar sheet of light and the line sensor
  • the line sensor 122 senses light from a single plane.
  • the two planes intersect at a line in 3D, as shown in FIG. 2, and, in the absence of ambient and indirect illuminations, the line sensor 122 measures light scattered by any object on the line.
  • the intersecting line can be swept to form any ruled surface.
  • This ruled surface, on which presence of objects can be detected, is the light curtain.
  • the resulting device is programmable, in terms of its light curtain shape, and flexible, in terms of being able to vary laser power and sensor exposure time to suit the demands of an application.
  • FIG. 4(a) shows the viewing and illumination geometry of a light curtain generated by rotating the laser light plane and sensor plane about parallel axes r.
  • the intersection line is also parallel to the two rotation axes, as shown in FIG. 4(b).
  • the coordinate frame of FIG. 4(b) and top view of FIG. 4(c) show various parameters of interest. Note that changing 6 C and q r synchronously generates light curtains with different shapes.
  • FIG. 4(c) and FIG. 5 show the finite sizes of sensor pixels and finite thickness of laser sheet leads to a thick light curtain upon triangulation.
  • the senor 122 When operated in strong ambient light, for example, sunlight, the sensor 122 also
  • galvo mirrors 108, 122 may take time to stabilize after rotation.
  • the stabilization time may be as much as 500ps, before the mirrors are stable enough to capture a line. This limits the overall frame-rate of the device. Adding two lOOps exposures for laser on and off to filter out ambient light allows a display of 1400 lines per second. If the light curtains are designed to contain 200 lines, the entire light curtain can be refreshed at a rate of 5.6 fps. Galvo mirrors which stabilize and time shorter than 500 ps would allow the curtain refresh rates to reach 20 to 30 fps.
  • the light curtain device can also be configured with line sensor 122 and laser 102 rotating over non-parallel axes or with each of them enjoying full rotational degrees of freedom. These configurations have their own unique advantages. When the devices have full rotational freedom, i.e., capable of rotating around a point with no restrictions, then any ruled surface (including for example, a mobius strip) can be generated as a light curtain. Full rotational freedom, however, is hard to implement since multi-axis galvos or gimbals are needed and are often cost-prohibitive.
  • FIG. 7(a,b,c) are illustrations of light curtains resulting when each of the sensor 122 and laser 102 rotates about an axis.
  • FIG. 7(d) is an illustration of a light curtain resulting when the sensor 122 and laser 102 rotate about a point.
  • the lines in light curtain should also be parallel to l c or l p , thus r(t) is a constant value and is in the direction of l c .
  • the lines in light curtain should also go through A, thus:
  • r(t) can be derived as follows.
  • FIG. 7(d) shows a mobius strip as an example. The proof is trivial, that any line in the ruled surface and the rotation center form a plane, which will determine the rotation of line sensor 122 or line laser 102.
  • Optimizing Light Curtains - Parameters of interest in practical light curtains can be quantified, for example, their thickness and SNR of measured detections, and approaches to optimize them are presented herein.
  • Of particular interest is the minimizing of the thickness of the curtain as well as optimizing exposure time and laser power for improved detection accuracy when the curtain spans a large range of depths.
  • Thickness of light curtain The light curtain produced by device described herein has a finite thickness due to the finite size of the sensor pixels and finite thickness of the laser illumination.
  • the laser spot has a thickness of A L meters and each pixel has an angular extent of 5 C radians.
  • the thickness of the light curtain is given as an area of a parallelogram shaded in FIGS. 4(c) and 5, which evaluates to: where r c and r p is the distance between the intersected point and the sensor and laser, respectively.
  • thickness by triangulation can be formalized as following: where r c (r) and r p (t) are the distance from r(t) to sensor rotation center [C, 0,0] and laser rotation center [ P , 0,0] respectively, z(t) is the depth of r(t), and (— , - ) is the range through which the rotate center of the sensor and laser can position. For simplicity, only consider a cross- section of a light curtain in the xz plane.
  • a key advantage of the light curtain device is that the power of the laser or the exposure time can be adapted for each intersecting line to compensate for light fall-off, which is inversely proportional to the square of the depth.
  • points close to be sensor get saturated easily.
  • system of the present invention has an additional degree of freedom wherein the power of the laser and/or the exposure time of the sensor can be adjusted according to depth such that light fall-off is compensated to the extent possible under the device constraints and with respect to eye safety.
  • the laser can send small amounts of light to just overcome the readout noise of the sensor or the photon noise of ambient light, and only a l-bit sensor is required.
  • CW-TOF sensors measures phase to obtain depth.
  • a CW-TOF sensor works by
  • phase difference f and the depth d of the scene point are related as:
  • the depth resolution of a TOF sensor is constant and independent of depth. Further, the depth resolution increases with the frequency of the amplitude wave.
  • TOF-based depth recovery has a phase wrapping problem due to the presence of the mod( ) operator, which implies that the depth estimate has an ambiguity problem and this problem gets worse at higher frequencies.
  • traditional triangulation- based depth estimation has no ambiguity problem, but at the cost of quadratic depth uncertainty.
  • Triangulation and phase are fused by measuring the phase (as with regular correlation-based ToF) in addition the usual measurement of intensity.
  • phase-based depth gating using appropriate codes at illumination and sensing.
  • the use of triangulation automatically eliminates the depth ambiguity of phase-based gating provided the thickness of the triangulation is smaller than the wavelength of the amplitude wave. With this, it is possible to create thinner light curtains over a larger depth range.
  • the senor receives first-bounce light reflected from the object as well as a lot of single-scattered light. With light curtains, the line sensor 102 avoids single-scattered light and only receives multi-scattered light. The ratio between first-bounce light and global light is much higher, thus contrast is better.
  • the light curtain method and device described herein has many benefits.
  • the shape of a light curtain is programmable and can be configured dynamically to suit the demands of the immediate task.
  • light curtains can be used to determine whether a vehicle is changing lanes in front, whether a pedestrian is in the crosswalk, or whether there are vehicles in neighboring lanes.
  • a robot might use a curtain that extrudes its planned (even curved) motion trajectory.
  • FIGS. 6(a,b) show various light curtains for use in robots and cars respectively.
  • the optical design of the light curtain shares similarities with confocal imaging in that small regions are selectively illuminated and sensed. When imaging in scattering media, such as fog and murky waters, this has the implicit advantage that many multi-bounce light paths a. re optically avoided thereby providing images with increased contrast.
  • a key advantage of light curtains is that illumination and sensing can be concentrated to a thin region. Together with the power and exposure adaptability, this provides significantly better performance under strong ambient illumination, including direct sunlight, at large distances (i.e., 20-30m). The performance increases under cloudy skies and indoors to 40m and 50m respectively.
  • the senor only captures a single line of the light curtain that often has small depth variations and hence, little variation in intensity fall-off Thus, the dynamic range of the measured brightness can be low. A such, even a one-bit sensor with a programmable threshold would be ample for the envisioned tasks.
  • CMOS intensity sensors
  • CCD CCD
  • IuGaAs time-of- flight sensors
  • SPAD correlation, SPAD
  • DRS neuromorphic sensors
  • the system may be run under control of one or more microprocessors in communication with memory containing software implementing the functions of the system.
  • the movement of the galvo mirrors 108, 126 is under the control of the software to define the shape of the light curtain.
  • the software may be configurable to allow the definition of light curtains of various shapes.
  • the software may control the cycling of light source 102 as well as the timing of the reading of the data from line sensor 122 and the application of any filtering to the data, for example, the filtering of ambient light.
  • Objects may be detected by breaking the light curtain, causing a variance in the light in the line of pixels sensed by line sensor 122. Upon detection of an object that has breached the light curtain, an alert may be raise and communicated off-unit.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Input (AREA)

Abstract

Selon des modes de réalisation décrits ci-inclus la présente invention concerne d'une manière générale un dispositif qui surveille la présence d'objets passant à travers ou frappant une coque virtuelle près du dispositif, désignée ci-inclus par rideau de lumière, qui est créée en faisant tourner rapidement un capteur de ligne et un laser en ligne de manière synchrone. Les limites du rideau de lumière sont définies par une ligne de balayage définie par l'intersection des plans de détection et d'éclairage.
EP19772623.5A 2018-03-23 2019-03-11 Rideaux de lumière programmables Pending EP3769037A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862761479P 2018-03-23 2018-03-23
PCT/US2019/021569 WO2019182784A1 (fr) 2018-03-23 2019-03-11 Rideaux de lumière programmables

Publications (2)

Publication Number Publication Date
EP3769037A1 true EP3769037A1 (fr) 2021-01-27
EP3769037A4 EP3769037A4 (fr) 2021-11-24

Family

ID=67987449

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19772623.5A Pending EP3769037A4 (fr) 2018-03-23 2019-03-11 Rideaux de lumière programmables

Country Status (3)

Country Link
EP (1) EP3769037A4 (fr)
CA (1) CA3094199A1 (fr)
WO (1) WO2019182784A1 (fr)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU1410600A (en) 1998-11-27 2000-06-19 Hamamatsu Photonics K.K. Driver sensor and apparatus for controlling air bag
US6415051B1 (en) * 1999-06-24 2002-07-02 Geometrix, Inc. Generating 3-D models using a manually operated structured light source
JP5528910B2 (ja) * 2010-06-02 2014-06-25 株式会社Ｐｆｕ オーバーヘッド型画像読取装置
JP6001960B2 (ja) * 2012-08-23 2016-10-05 大塚電子株式会社 配光特性測定装置および配光特性測定方法
US9188775B2 (en) * 2013-08-28 2015-11-17 United Sciences, Llc Optical scanning and measurement
NO20131296A1 (no) * 2013-09-27 2015-01-19 Megalink As System og metode for bestemmelse av posisjonen til et kuleprosjektil på et blinkplan
WO2016131036A1 (fr) * 2015-02-13 2016-08-18 Carnegie Mellon University Système d'imagerie avec commande dynamique synchronisée de source lumineuse à faisceau orientable et photo-capteur masqué de façon reconfigurable

Also Published As

Publication number Publication date
CA3094199A1 (fr) 2019-09-26
WO2019182784A1 (fr) 2019-09-26
JP2021518536A (ja) 2021-08-02
EP3769037A4 (fr) 2021-11-24

Similar Documents

Publication Publication Date Title
Wang et al. Programmable triangulation light curtains
US10649072B2 (en) LiDAR device based on scanning mirrors array and multi-frequency laser modulation
Fuchs Multipath interference compensation in time-of-flight camera images
JP7133554B2 (ja) 焦点調整可能な画像センサを含む距離センサ
US9285477B1 (en) 3D depth point cloud from timing flight of 2D scanned light beam pulses
JP6528447B2 (ja) 視差演算システム及び距離測定装置
JP6387407B2 (ja) 周辺検知システム
CN107209265B (zh) 光探测和测距装置
US10782392B2 (en) Scanning optical system and light projecting and receiving apparatus
CN111344647A (zh) 具有低延时运动规划更新的智能激光雷达系统
ES2512965B2 (es) Sistema y método para escanear una superficie y programa de ordenador que implementa el método
US8860930B2 (en) Three dimensional surface mapping system using optical flow
JP7140474B2 (ja) ステレオ三角測量のためのシステム
US10162171B2 (en) Scanning optical system and light projecting and receiving apparatus
KR101545971B1 (ko) 복합 영상 센싱 시스템
JP7403860B2 (ja) 三角測量ライトカーテンを使用したアジャイルな深度感知
KR20200071960A (ko) 카메라 및 라이다 센서 융합을 이용한 객체 검출 방법 및 그를 위한 장치
English et al. TriDAR: A hybrid sensor for exploiting the complimentary nature of triangulation and LIDAR technologies
KR20230028265A (ko) 회전 금속 모터 프레임에 유리 거울의 부착
CN114375408A (zh) 用于修改lidar视场的系统和方法
US20190377073A1 (en) Distance-measuring apparatus with polarizing filter
US11493634B2 (en) Programmable light curtains
EP3769037A1 (fr) Rideaux de lumière programmables
JP7570099B2 (ja) プログラム可能ライトカーテン
US10818024B2 (en) Ranging objects external to an aircraft using multi-camera triangulation

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200923

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIN1 Information on inventor provided before grant (corrected)

Inventor name: SANKARANARAYANAN, ASWIN C.

Inventor name: WANG, JIAN

Inventor name: NARASIMHAN, SRINIVASA

Inventor name: WHITTAKER, WILLIAM

Inventor name: BARTELS, JOSEPH

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40045304

Country of ref document: HK

A4 Supplementary search report drawn up and despatched

Effective date: 20211022

RIC1 Information provided on ipc code assigned before grant

Ipc: G01S 7/481 20060101ALI20211018BHEP

Ipc: G01S 17/931 20200101ALI20211018BHEP

Ipc: G01S 17/32 20200101ALI20211018BHEP

Ipc: G01S 17/04 20200101ALI20211018BHEP

Ipc: G01B 11/25 20060101AFI20211018BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230123