JP7844571B2 - LIDAR system and method - Google Patents

Description

(Cross-reference of related applications)
[001] This application is in accordance with U.S. Provisional Patent Application No. 62/396, filed on September 20, 2016.
U.S. Provisional Patent Application No. 858, U.S. Provisional Patent Application No. 62/396,863 filed on September 20, 2016, U.S. Provisional Patent Application No. 62/396,864 filed on September 20, 2016, U.S. Provisional Patent Application No. 62/397,379 filed on September 21, 2016, U.S. Provisional Patent Application No. 62/405,928 filed on October 9, 2016, U.S. Provisional Patent Application No. 62/412,294 filed on October 25, 2016, U.S. Provisional Patent Application No. 62/414,740 filed on October 30, 2016, U.S. Provisional Patent Application No. 62/414,740 filed on November 7, 2016 We claim priority rights to U.S. Provisional Patent Application No. 62/418,298, U.S. Provisional Patent Application No. 62/422,602 filed November 16, 2016, U.S. Provisional Patent Application No. 62/425,089 filed November 22, 2016, U.S. Provisional Patent Application No. 62/441,574 filed January 3, 2017, U.S. Provisional Patent Application No. 62/441,581 filed January 3, 2017, U.S. Provisional Patent Application No. 62/441,583 filed January 3, 2017, and U.S. Provisional Patent Application No. 62/521,450 filed June 18, 2017. All of the aforementioned applications are incorporated in their entirety by reference.

[002] The disclosure generally relates to survey techniques for scanning the surrounding environment, and more specifically to systems and methods for detecting objects in the surrounding environment using LIDAR technology.

[003] With the advent of driver assistance systems and autonomous vehicles, automobiles need to be equipped with systems that can reliably detect and interpret their surroundings, including identifying obstacles, hazards, objects, and other physical parameters that may affect the vehicle's navigation. For this purpose, many different technologies have been proposed, including radar, lidar, and camera-based systems that operate independently or redundantly.

[004] One consideration for driver assistance systems and autonomous vehicles is the system's ability to judge its surroundings in a variety of conditions, including rain, fog, darkness, bright light, and snow. Light detection and ranging (LIDAR) is an example of a technology that can function well in a variety of conditions by measuring the distance to an object by shining light onto the object and measuring the reflected pulses with a sensor. Lasers are an example of a light source that can be used in LIDAR systems. As with all detection systems, for LIDAR-based detection systems to be widely adopted by the automotive industry, the system must provide reliable data that enables the detection of distant objects. However, the maximum illumination power of LIDAR systems is currently limited by the need to make LIDAR systems safe for the eye (i.e., to prevent damage to the human eye that could occur if the projected light emission is absorbed by the cornea and lens of the eye and causes thermal damage to the retina).

[005] The systems and methods of the present disclosure are intended to improve the performance of a LIDAR system while complying with eye safety regulations.

[006] Embodiments according to this disclosure provide a system and method for detecting objects in the surrounding environment using LIDAR technology.

[007] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light in scanning a field of view using light from at least one light source; control at least one light deflector so as to deflect light from at least one light source for scanning a field of view; use a first detected reflection associated with scanning a first portion of the field of view to determine that a first object is present in the first portion at a first distance, and that no object is present in a second portion of the field of view at a first distance, detect a first reflection, and after determining that no object is present in the second portion, modify the light source parameters so that more light is projected toward the second portion of the field of view than is projected toward the first portion of the field of view; and use a second detected reflection in the second portion of the field of view to determine that a second object is present at a second distance greater than the first distance.

[008] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light in a scan of a field of view using light from at least one light source, to control the projection of at least first light emission directed toward a first portion of the field of view to determine that there is no object at a first distance in the first portion of the field of view, and, if it is determined based on at least first light emission that there is no object in the first portion of the field of view, to control the projection of at least second light emission directed toward the first portion of the field of view to enable the detection of an object at a second distance greater than a first distance in the first portion of the field of view, and to control the projection of at least third light emission directed toward the first portion of the field of view to determine that there is an object at a third distance greater than a second distance in the first portion of the field of view.

[009] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light in a scan of a field of view including a first portion and a second portion; to receive a signal from at least one sensor for each pixel, the signal representing at least one of a combination of ambient light, light from at least one light source reflected by an object in the field of view and noise associated with at least one sensor; to estimate noise in at least a portion of the signal associated with the first portion of the field of view; to change the sensor sensitivity to reflections associated with the first portion of the field of view based on the estimation of noise in the first portion of the field of view; to estimate noise in at least a portion of the signal associated with the second portion of the field of view; to change the sensor sensitivity to reflections associated with the second portion of the field of view based on the estimation of noise in the second portion of the field of view, wherein the changed sensor sensitivity for reflections associated with the second portion is different from the changed sensor sensitivity for reflections associated with the first portion.

[010] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the light intensity in scanning a field of view using light from at least one light source, to control at least one light deflector so as to deflect light from at least one light source for scanning a field of view, to acquire identification of at least one distinct region of interest in the field of view, and to increase the allocation of light to at least one distinct region of interest more than to other regions such that, after a first scan cycle, the light intensity of at least one subsequent second scan cycle at locations associated with at least one distinct region of interest is higher than the light intensity of the first scan cycle at locations associated with at least one distinct region of interest.

[011] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light in scanning a field of view using light from at least one light source; control at least one optical deflector so as to deflect light from at least one light source for scanning a field of view; receive a reflected signal from at least one sensor indicating light reflected from an object in the field of view; determine, based on the reflected signal of the initial light emission, whether an object is located within a threshold distance from at least one optical deflector in the intermediate area of the LIDAR system, the threshold distance being associated with a safety distance; control at least one light source so as to project an additional light emission toward the intermediate area if no object is detected in the intermediate area, thereby enabling the detection of an object further away from the intermediate area; and regulate at least one of the at least one light source and at least one optical deflector so as not to exceed the maximum allowable exposure density of light in the intermediate area if an object is detected in the intermediate area.

[012] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control the light emission of a light source and scan a field of view by repeatedly moving at least one optical deflector positioned in the outbound path of the light source, wherein at least one optical deflector is instantaneously positioned in a plurality of positions during a single scan cycle of the field of view, and while at least one deflector is in a particular instantaneous position, the processor receives the reflection of a single optical beam spot along a return path to a sensor via at least one deflector, and receives a signal from the sensor for each beam spot associated with an image of each optical beam spot, wherein the sensor includes a plurality of detectors, and the size of each detector is smaller than the image of each optical beam spot, such that an image of each optical beam spot is incident on the plurality of detectors for each beam spot, and determines at least two different range measurements associated with an image of a single optical beam spot from the signals produced by the incident on the plurality of detectors.

[013] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one deflector so as to deflect light from a plurality of light sources toward a plurality of regions forming a field of view along a plurality of outbound paths while at least one deflector is in a particular instantaneous position, and to control at least one deflector so as to receive light reflections from the field of view in at least one common area of at least one deflector while at least one deflector is in a particular instantaneous position, in which at least a portion of light reflections from at least some of the plurality of light sources overlap and incident toward at least one common area, and to receive at least one signal from each of a plurality of detectors indicating light reflections from at least one common area while at least one deflector is in a particular instantaneous position.

[014] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to access an optical budget stored in memory, the optical budget being associated with at least one light source and defining the amount of light that can be emitted by the at least one light source within a given time period, receiving information indicating platform conditions for the LIDAR system, dynamically allocating the optical budget to the field of view of the LIDAR system based on the received information, based on at least two of scan rate, scan pattern, scan angle, spatial light distribution, and temporal light distribution, and outputting signals to control at least one light source so that the light beam can be varied in scanning the field of view according to the dynamically allocated optical budget.

[015] According to the disclosed embodiments, a vibration suppression system for a LIDAR configured for use in a vehicle may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source in scanning a field of view; to control the positioning of at least one optical deflector so as to deflect light from at least one light source for scanning a field of view; to acquire data indicating vehicle vibration; to determine, based on the acquired data, adjustments to the positioning of at least one optical deflector to compensate for vehicle vibration; and to implement the determined adjustments to the positioning of at least one optical deflector, thereby suppressing at least a portion of the effect of vehicle vibration on scanning a field of view in at least one optical deflector.

[016] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source in a scanning cycle of a field of view, the light projected from at least one light source is directed to at least one deflector to scan a field of view, a reflected signal is received from at least one sensor indicating light reflected from an object in the field of view, and the beam of light and the scan are coordinated so as to generate at least three sectors of the field of view in a scanning cycle, the first sector having a first beam of light and associated first detection range, the second sector having a second beam of light and associated second detection range, the third sector having a third beam of light and associated third detection range, the second beam of light is greater than each of the first and third beams, and the processor detects an object in the second sector located at a distance greater than the first and third detection ranges, based on input from at least one sensor.

[017] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source in a plurality of scans of a field of view including near-field and far-field portions; to control at least one light deflector so as to deflect light from at least one light source to scan a field of view; to implement a first scan rate for a first frame associated with a scan cycle covering the near-field portion and a second scan rate for a second frame associated with a scan cycle covering the far-field portion, wherein the first scan rate is greater than the second rate; and to control at least one light source so as to change the light source parameters after projecting light that enables the detection of objects in a plurality of sequential first frames associated with the near-field portion, thereby projecting light that enables the detection of objects in second frames associated with the far-field portion.

[018] According to the disclosed embodiments, a LIDAR system for use in a vehicle may include at least one processor configured to control at least one light source so as to vary the luminous flux of at least one light source in scanning a field of view, to control at least one light deflector so as to deflect light from at least one light source for scanning a field of view, to receive an input indicating the current driving environment of the vehicle, and to coordinate the control of at least one light source and at least one light deflector so as to dynamically adjust the instantaneous detection distance by varying the amount of light projected and the spatial light distribution of light in scanning a field of view based on the current driving environment.

[019] According to the disclosed embodiments, a LIDAR system for use in a vehicle may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source in a scanning cycle of the field of view; to control at least one light deflector so as to deflect light from at least one light source for scanning the field of view; to acquire an input indicating that a vehicle is likely to change direction across a lane; and in response to the input indicating that a vehicle is likely to change direction across a lane, to increase the beam of light on the side opposite to the direction of the vehicle's lane change, including the far lane into which the vehicle is about to merge, compared to the rest of the field of view, and to temporarily expand the detection range on the side opposite to the direction of the vehicle's lane change than the detection range in the direction of the lane change.

[020] According to the disclosed embodiments, a LIDAR system for use with a roadway vehicle traveling on a highway may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source during a scanning cycle of a field of view, and to control at least one optical deflector so as to deflect light from at least one light source for scanning a field of view, wherein the field of view can be divided into a central region generally corresponding to the highway on which the vehicle is traveling, a right peripheral region generally corresponding to the area to the right of the highway, and a left peripheral region generally corresponding to the area to the left of the highway, and to take input that the vehicle is in a mode corresponding to driving on a highway, and in response to input that the vehicle is in a mode corresponding to driving on a highway, the processor may coordinate the control of at least one light source and the control of at least one optical deflector so that during scanning of a field of view including the central region, the right peripheral region, and the left peripheral region, more light is directed to the central region than to the right peripheral region and the left peripheral region.

[021] According to the disclosed embodiments, the LIDAR system may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source in scanning a field of view, to control at least one light deflector so as to deflect light from at least one light source for scanning a field of view, to receive information from at least one sensor indicating ambient light in a field of view, to identify in the received information an indication of a first portion of a field of view containing more ambient light than a second portion of the field of view, and to modify the light source parameters such that, when scanning a field of view, there is more beam of light projected toward the first portion of the field of view than beam of light projected toward the second portion of the field of view.

[022] According to the disclosed embodiments, a LIDAR system for use in a vehicle may include: at least one light source configured to project light toward a field of view to illuminate a plurality of objects in the environment of the vehicle; and at least one processor configured to control the at least one light source so as to vary the beam of light from the at least one light source in scanning a plurality of parts of the field of view; to receive information indicating that heat is being radiated from at least one system component and that the temperature associated with the at least one system component exceeds a threshold, and to change the illumination ratio between two parts of the field of view in response to the received information indicating that the temperature exceeds a threshold, such that less light is delivered to the field of view in at least one subsequent scan cycle than in the preceding scan cycle.

[023] According to the disclosed embodiments, a LIDAR system may include a window for receiving light, a micro-electromechanical (MEMS) mirror for deflecting light to produce deflected light, a frame, an actuator, and interconnection elements mechanically connected between the actuator and the MEMS mirror, each actuator including a body and a piezoelectric element, the piezoelectric element being configured to bend the body and move the MEMS mirror when an electric field is applied, and the MEMS mirror is oriented with respect to the window when the MEMS mirror is positioned in an idle position.

[024] In accordance with other disclosed embodiments, the method may include one or more of the steps performed by the processor described above and/or any of the steps described herein.

[025] In a further disclosed embodiment, a non-temporary computer-readable storage medium may store program instructions which are executed by at least one processing device to carry out any of the methods described herein.

[026] The general descriptions above and the detailed descriptions below are for illustrative purposes only and do not limit the scope of the claims.

[026] The general statements above and the detailed statements below are for illustrative and explanatory purposes only and do not limit the scope of the claims.

[027] The accompanying drawings, which are incorporated into and form part of this disclosure, illustrate various embodiments of the disclosure.

[028] A diagram showing an exemplary LIDAR system according to the disclosed embodiments. [029] An image showing an exemplary output of a single scan cycle of a vehicle-mounted LiDAR system according to the disclosed embodiment. [030] Another image showing a representation of a point cloud model determined from the output of a LIDAR system according to the disclosed embodiment. [031] This figure shows the configuration of a projection unit according to an embodiment of the present disclosure. [031] This figure shows the configuration of a projection unit according to an embodiment of the present disclosure. [031] This figure shows the configuration of a projection unit according to an embodiment of the present disclosure. [031] This figure shows the configuration of a projection unit according to an embodiment of the present disclosure. [032] This figure shows the configuration of a scan unit according to an embodiment of the present disclosure. [032] This figure shows the configuration of a scan unit according to an embodiment of the present disclosure. [032] This figure shows the configuration of a scan unit according to an embodiment of the present disclosure. [032] This figure shows the configuration of a scan unit according to an embodiment of the present disclosure. [033] This figure shows the configuration of a detection unit according to an embodiment of the present disclosure. [033] This figure shows the configuration of a detection unit according to an embodiment of the present disclosure. [033] This figure shows the configuration of a detection unit according to an embodiment of the present disclosure. [033] This figure shows the configuration of a detection unit according to an embodiment of the present disclosure. [033] This figure shows the configuration of a detection unit according to an embodiment of the present disclosure. [034] Includes four exemplary figures showing the emission pattern of a single portion of the field of view in a single frame time. [035] Includes three illustrative diagrams showing the emission scheme in a single frame time across the entire field of view. [036] This figure shows the actual light emission and received reflection projected over a single frame of the entire field of view. [037] This figure shows a first exemplary embodiment according to an embodiment of the present disclosure. [037] This figure shows a first exemplary embodiment according to an embodiment of the present disclosure. [037] This figure shows a first exemplary embodiment according to an embodiment of the present disclosure. [038] This figure shows a second exemplary embodiment according to an embodiment of the present disclosure. [039] This flowchart shows an exemplary method for detecting an object using a LIDAR system according to an embodiment of the present disclosure. [040] This figure shows an example of a two-dimensional sensor according to some embodiments of the present disclosure. [041] This figure shows an example of a one-dimensional sensor according to some embodiments of the present disclosure. [042] Block diagram showing an exemplary LIDAR device having transmit and receive alignment according to some embodiments of the present disclosure. [043] Block diagram showing another exemplary LIDAR device having transmit and receive alignment according to some embodiments of the present disclosure. [044] A figure showing some exemplary first field of view (FOV) and second FOV according to some embodiments of the present disclosure. [045] This figure shows an exemplary scan pattern of a second FOV over the entire area of a first FOV, according to some embodiments of the present disclosure. [046] This figure shows another exemplary scan pattern of a second FOV over the entire area of a first FOV, according to some embodiments of the present disclosure. [047] The present invention provides a schematic diagram of the field of view of a LIDAR system and a related depth map scene representation according to the embodiments disclosed herein. [048] The present invention provides a schematic diagram of a field of view and a related depth map scene representation generated using a LIDAR system with dynamically variable luminous beam function according to the embodiments disclosed herein. [049] A flowchart is provided of a method for dynamically varying the light beam in the scanning field of a LIDAR system according to the embodiments disclosed herein. [050] The present invention provides a schematic diagram of the field of view of a LIDAR system and a related depth map scene representation according to the embodiments disclosed herein. [051] The present invention provides a schematic diagram of a field of view and a related depth map scene representation generated using a LIDAR system with a dynamically variable luminous beam function according to the embodiments disclosed herein. [052] A flowchart is provided of a method for dynamically varying the light beam in the scanning field of a LIDAR system according to the embodiments disclosed herein. [053] A flowchart shows an exemplary method for changing the sensor sensitivity in a LIDAR system according to some embodiments of the present disclosure. [054] This figure shows an example of a received signal according to some embodiments of the present disclosure, along with a function for estimating the predicted signal. [055] This figure shows an example of a received signal according to some embodiments of the present disclosure, along with a function for estimating noise. [056] This flowchart shows a first example of a method for detecting an object in a region of interest using a LIDAR system. [057] This flowchart shows a second example of a method for detecting an object in a region of interest using a LIDAR system. [058] Another figure showing an exemplary LIDAR system according to the disclosed embodiments. [059] This is a schematic diagram of a LIDAR system according to an embodiment of the present disclosure. [060] A flowchart of an exemplary process for controlling light emission according to an embodiment of the present disclosure. [061] This is a flowchart illustrating an exemplary implementation of the process shown in Figure 24, according to an embodiment of the present disclosure. [062] This flowchart shows an exemplary method for detecting an object according to some embodiments of the present disclosure. [063] A flowchart shows an exemplary method for detecting an object using LIDAR according to some embodiments of the present disclosure. [064] A flowchart shows another exemplary method for detecting an object using LIDAR according to some embodiments of the present disclosure. [065] A flowchart shows yet another exemplary method for detecting an object using LIDAR according to some embodiments of the present disclosure. [066] A diagram of a LIDAR system having multiple light sources and a common deflector, according to some embodiments of the present disclosure. [067] A diagram of another LIDAR system having multiple light sources and a common deflector, according to some embodiments of the present disclosure. [068] A block diagram of the LiDAR system 100 is provided, along with various information sources that can be used when allocating the available optical and/or computational budgets for the LiDAR system 100. [069] A flowchart is shown that gives an example of a method 3000 for controlling a LIDAR system based on an allocated budget, according to the disclosed embodiments. [070] A flowchart illustrating an exemplary method for controlling a LIDAR system according to the embodiments disclosed herein is shown. [071] This is a schematic diagram of a situation in which an uneven allocation of the optical budget is appropriate according to the embodiments disclosed herein. [072] A figure showing a vehicle according to an exemplary embodiment disclosed, and a vibration compensation system according to some embodiments. [072] This figure shows a vibration compensation system according to several embodiments. [072] This figure shows a steering device, a central processing unit (CPU), and an actuator mirror according to several embodiments. [072] This figure shows an actuator mirror according to several embodiments. [072] This figure shows a two-axis MEMS mirror according to several embodiments. [072] This figure shows a single-axis MEMS mirror according to several embodiments. [072] This figure shows a round MEMS mirror according to several embodiments. [073] A schematic diagram of a LiDAR system installation capable of compensating for motion detected along a road, according to an exemplary embodiment disclosed. [074] This is a flowchart showing how to use the vehicle vibration compensation system. [075] A schematic diagram of different detection ranges in different sectors according to the embodiments disclosed herein. [075] A schematic diagram of different detection ranges in different sectors according to the embodiments disclosed herein. [075] A schematic diagram of different detection ranges in different sectors according to the embodiments disclosed herein. [075] A schematic diagram of different detection ranges in different sectors according to the embodiments disclosed herein. [076] This is a schematic diagram showing different sectors in the field of view according to the embodiments disclosed herein. [077] A flowchart shows an example of a method for detecting an object in a region of interest using a LIDAR system according to an embodiment disclosed herein. [078] This is a schematic diagram of the field of view of a LIDAR system according to an embodiment of the present disclosure. [079] This is a schematic diagram of an exemplary field of view of a LIDAR system according to an embodiment of the present disclosure. [080] This is a flowchart illustrating an exemplary implementation of the scanning process according to an embodiment of the present disclosure. [080] This is a flowchart illustrating an exemplary implementation of the scanning process according to an embodiment of the present disclosure. [081] A flowchart illustrating an exemplary method for detecting an object in the path of a vehicle using LiDAR, according to some embodiments of the present disclosure. [082] A flowchart shows another exemplary method for detecting an object in the path of a vehicle using LIDAR, according to some embodiments of the present disclosure. [083] This figure shows an example of a vehicle in an urban environment according to some embodiments of the present disclosure. [084] This figure shows an example of a vehicle in a rural setting according to some embodiments of the present disclosure. [085] This figure shows an example of a vehicle in a traffic jam according to some embodiments of the present disclosure. [086] This figure shows an example of a vehicle in a tunnel according to some embodiments of the present disclosure. [087] This figure shows an example of a vehicle exiting a tunnel according to some embodiments of the present disclosure. [088] Figures 42A, 42B, and 42C show exemplary vehicles from different angles according to some embodiments of the present disclosure. [089] This figure shows an exemplary LIDAR system having multiple light sources aimed at a common area of at least one light deflector. [090] A flowchart shows an exemplary method of a LIDAR detection scheme for lane crossing direction change according to some embodiments of the present disclosure. [091] This figure shows an example of a LIDAR detection scan scheme according to some embodiments of the present disclosure. [092] This figure shows an example of a LIDAR detection scheme for lane crossing direction changes and other situations according to some embodiments of the present disclosure. [092] This figure shows an example of a LIDAR detection scheme for lane crossing direction changes and other situations according to some embodiments of the present disclosure. [093] A schematic diagram of a vehicle traveling in a trunk road environment with the assistance of a LIDAR system according to an exemplary embodiment disclosed. [094] A schematic diagram of dynamic light allocation by a LIDAR system in a trunk road environment according to an exemplary embodiment disclosed. [094] A schematic diagram of dynamic light allocation by a LIDAR system in a trunk road environment according to an exemplary embodiment disclosed. [094] A schematic diagram of dynamic light allocation by a LIDAR system in a trunk road environment according to an exemplary embodiment disclosed. [094] A schematic diagram of dynamic light allocation by a LIDAR system in a trunk road environment according to an exemplary embodiment disclosed. [095] A method for detecting an object in the path of a vehicle using a LIDAR system is shown according to an exemplary embodiment disclosed. [096] This figure shows an example of a detection configuration of a LIDAR system according to an exemplary disclosed embodiment. [097] This is a schematic diagram showing various parts of the LIDAR field of view. [098] This flowchart shows an example of a method for detecting an object in a region of interest using a LIDAR system. [099] This is a schematic diagram of a LIDAR system according to an embodiment of the present disclosure. [0100] This is a flowchart illustrating an exemplary implementation of a temperature reduction process according to an embodiment of the present disclosure. [0101] This is a flowchart illustrating an exemplary implementation of a temperature reduction process according to an embodiment of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure. [0102] This figure shows an example of a MEMS mirror and related components incorporated into a scan unit of a LIDAR system according to some embodiments of the present disclosure.

[0103] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. While several exemplary embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, components shown in the drawings may be replaced, added, or modified, and the exemplary methods described herein may be modified by replacing, rearranging, removing, or adding steps to the disclosed methods. Accordingly, the following detailed description is not limited to the disclosed embodiments and examples. The appropriate scope is defined by the appended claims.

Definitions of Terms [0104] The disclosed embodiments may include optical systems. As used herein, the term “optical system” broadly includes any system used for generating, detecting, and/or manipulating light. For example, an optical system may include one or more optical components for generating, detecting, and/or manipulating light. For instance, light sources, lenses, mirrors, prisms, beam splitters, collimators, polarizing optics, optical modulators, optical switches, optical amplifiers, optical detectors, optical sensors, optical fibers, and semiconductor optical components may each be part of an optical system, though not necessarily essential. In addition to one or more optical components, an optical system may also include other non-optical components, such as electrical components, mechanical components, chemical reaction components, and semiconductor components. Non-optical components can work in conjunction with the optical components of the optical system. For example, an optical system may include at least one processor for analyzing detected light.

[0105] In accordance with this disclosure, the optical system may be a LiDAR system. As used herein, the term “LIDAR system” broadly includes any system that can determine the value of a parameter indicating the distance between a pair of tangible objects based on reflected light. In one embodiment, a LiDAR system can determine the distance between a pair of tangible objects based on the reflection of light emitted by the LiDAR system. As used herein, the term “determine distance” broadly includes generating an output indicating the distance between a pair of tangible objects. The determined distance may represent the physical dimensions between a pair of tangible objects. As merely one example, the determined distance may include a line of flight distance between the LiDAR system and another tangible object within the LiDAR system’s field of view. In another embodiment, a LiDAR system can determine the relative velocity between a pair of tangible objects based on the reflection of light emitted by the LiDAR system. Examples of outputs indicating the distance between a pair of tangible objects include the number of standard units of length between the tangible objects (e.g., meters, inches, kilometers, millimeters), the number of arbitrary units of length (e.g., the length of the LiDAR system), the ratio of distance to another length (e.g., the ratio to the length of an object detected within the LiDAR system's field of view), a time quantity (e.g., given in standard units, arbitrary units, or ratios, e.g., the time required for light to travel between the tangible objects), one or more locations (e.g., defined using approved coordinates, defined for known locations), and others.

[0106] A Lidar system can determine the distance between a pair of tangible objects based on reflected light. In one embodiment, a Lidar system can process the detection results of a sensor to generate time information indicating the time period between the emission of an optical signal and its detection by the sensor. This time period is sometimes referred to as the "time of flight" of the optical signal. In one example, the optical signal is a short pulse, and its rise time and/or fall time can be detected upon reception. By processing information regarding the time of flight of the optical signal using known information about the speed of light in the relevant medium (usually air), the distance the optical signal travels between emission and detection can be provided. In another embodiment, a Lidar system can determine the distance based on frequency phase shifts (or multi-frequency phase shifts). Specifically, a Lidar system can process information indicating one or more modulation phase shifts of an optical signal (for example, by solving a system of equations to give a final measure). For example, an emitted optical signal can be modulated by one or more constant frequencies. At least one phase shift of the modulation between the emitted signal and the detected reflection can indicate the distance the light traveled between emission and detection. Modulation can be applied to continuous-wave optical signals, quasi-continuous-wave optical signals, or other types of emitted optical signals. It should be noted that additional information can be used by the LIDAR system to determine the distance. This additional information may include, for example, location information (e.g., relative position) between the signal projection location and the detection location (especially if they are far apart), and other factors.

[0107] In some embodiments, a LiDAR system can be used to detect multiple objects within the LiDAR system environment. The term “detecting objects within the LiDAR system environment” broadly includes generating information indicating an object that reflects light toward a detector associated with the LiDAR system. If two or more objects are detected by the LiDAR system, the information generated about different objects, such as a car driving on the road, a bird perched on a tree, a person touching a bicycle, or a van moving toward a building, can communicate with each other. The dimensions of the environment in which the LiDAR system detects objects may vary depending on the implementation. For example, a LiDAR system can be used to detect multiple objects within the environment of a vehicle on which the LiDAR system is installed, up to a horizontal distance of up to 100 m (or 200 m, 300 m, etc.) and up to a vertical distance of up to 10 m (or 25 m, 50 m, etc.). In another example, a LiDAR system can be used to detect multiple objects within the vehicle environment or within a predetermined horizontal range (e.g., 25°, 50°, 100°, 180°, etc.) and a predetermined vertical height (e.g., ±10°, ±20°, +40° to 20°, ±90°, or 0° to 90°).

[0108] As used herein, the term “detecting an object” can broadly mean determining the presence of an object (for example, an object may be located in a particular direction relative to a LiDAR system and/or another reference location, or an object may be located within a particular spatial volume). In addition to or instead of this, the term “detecting an object” may mean determining the distance between an object and another location (for example, a location of a LiDAR system, a location on Earth, or a location of another object). In addition to or instead of this, the term “detecting an object” may refer to identifying an object (e.g., classifying the type of object such as a car, tree, or road; recognizing a specific object (e.g., the Washington Monument); determining a vehicle registration number; revealing the composition of an object (e.g., solid, liquid, transparent, or translucent); or determining the motion parameters of an object (e.g., whether it is moving, its speed, direction of movement, or expansion). In addition to or instead of this, the term “detecting an object” may refer to generating a point cloud map. Each of one or more points in the point cloud map corresponds to a location within an object or a location on the object's surface. In one embodiment, the data resolution for the point cloud map representation of a field of view may be associated with a field of view of 0.1° × 0.1° or 0.3° × 0.3°.

[0109] In accordance with this disclosure, the term “object” broadly includes finite compositions from which light can be reflected from at least a portion. For example, an object may be at least partially solid (e.g., a car, a tree), at least partially liquid (e.g., a puddle on a road, rain), at least partially gaseous (e.g., smoke, a cloud), or composed of a number of distinct particles (e.g., a sandstorm, fog, a spray), and may be one or more sizes such as ~1 millimeter (mm), ~5 mm, ~10 mm, ~50 mm, ~100 mm, ~500 mm, ~1 meter (m), ~5 m, ~10 m, ~50 m, ~100 m, ~100 m. Smaller or larger objects, and any size between these examples, can also be detected. It should be noted that for various reasons, a LIDAR system may detect only a portion of an object. For example, in some cases, light is reflected from only a few sides of an object (e.g., only the side facing the LIDAR system is detected). In other cases, light is projected onto only a portion of an object (e.g., a laser beam is projected onto a road or a building). In other cases, an object may be partially obscured by another object between the LIDAR system and the detected object. In yet other cases, the LIDAR sensor may only detect light reflected from a portion of the object. This is because, for example, ambient light or other interference may interfere with the detection of certain parts of the object.

[0110] In accordance with this disclosure, a LiDAR system can be configured to detect objects by scanning the environment of the LiDAR system. The term “scanning the environment of the LiDAR system” broadly includes illuminating the field of view or a portion of the field of view of the LiDAR system. In one example, scanning the environment of the LiDAR system can be achieved by moving or pivoting an optical deflector to deflect light in various directions toward different parts of the field of view. In another example, scanning the environment of the LiDAR system can be achieved by changing the positioning (i.e., location and/or orientation) of a sensor relative to the field of view. In yet another example, scanning the environment of the LiDAR system can be achieved by changing the positioning (i.e., location and/or orientation) of a light source relative to the field of view. In yet another example, scanning the environment of the LiDAR system can be achieved by changing the positions of at least one light source and at least one sensor so that they move rigidly relative to the field of view (i.e., the relative distance and orientation of at least one sensor and at least one light source are maintained).

[0111] As used herein, the term “field of view of a LiDAR system” may broadly encompass the range of the observable environment of a LiDAR system capable of detecting objects. It should be noted that the field of view (FOV) of a LiDAR system can be affected by various conditions. These conditions include, but are not limited to, the orientation of the LiDAR system (e.g., the direction of the optical axis of the LiDAR system), the position of the LiDAR system relative to the environment (e.g., distance from the ground, adjacent terrain, and obstacles), and the operating parameters of the LiDAR system (e.g., emission power, calculation settings, specified operating angle). The field of view of a LiDAR system can be defined, for example, using solid angles (e.g., defined using angles φ and θ, where φ and θ are defined, for example, in a plane perpendicular to the axis of symmetry of the LiDAR system and/or its FOV). In one example, the field of view can be defined within a specific range (e.g., up to 200 m).

[0112] Similarly, the term “instantaneous field of view” can broadly encompass the range of the observable environment in which an object can be detected by the LIDAR system at any given moment. For example, in a scanning LIDAR system, the instantaneous field of view is narrower than the entire FOV of the LIDAR system and can be moved within the FOV of the LIDAR system to enable detection in other parts of the FOV of the LIDAR system. Moving the instantaneous field of view within the FOV of the LIDAR system can be achieved by moving an optical deflector in the LIDAR system (or outside the LIDAR system) to deflect the light beam into and/or from the LIDAR system in different directions. In one embodiment, the LIDAR system can be configured to scan a scene in the environment in which the LIDAR system is operating. As used herein, the term “scene” can broadly encompass some or all of the objects in the field of view of the LIDAR system in their relative positions and current state during the operation of the LIDAR system. For example, a scene may include ground elements (e.g., soil, roads, grass, sidewalks, road signs), the sky, man-made objects (e.g., vehicles, buildings, signs), plants, people, animals, light projection elements (e.g., flashlights, sun, other LIDAR systems), etc.

[0113] The disclosed embodiments may include obtaining information used to generate a reconstructed three-dimensional model. Examples of types of reconstructed three-dimensional models that can be used include point cloud models and polygon meshes (e.g., triangular meshes). The terms “point cloud” and “point cloud model” are well known in the art and should be interpreted as including a set of data points spatially located in a particular coordinate system (i.e., having identifiable locations in the space described by each coordinate system). The term “point cloud point” refers to a point in space (which may be dimensionless or a microcellular space such as 1 ^cm³ ) whose location can be described by a point cloud model using a coordinate set (e.g., (X, Y, Z), (r, φ, θ)). Just as an example, a point cloud model may store additional information for some or all of its points (e.g., color information for points generated from a camera image). Similarly, any other type of reconstructed three-dimensional model may store additional information for some or all of its objects. Similarly, the terms “polygon mesh” and “triangular mesh” are well known in the art and should be interpreted, in particular, to include a set of vertices, edges, and faces that define the shape of one or more 3D objects (such as polyhedra). Faces may include one or more triangles (triangular meshes), quadrilaterals, or other simple convex polygons for the sake of simplifying rendering. Faces may also include more general concave polygons or polygons with holes. Polygon meshes can be represented using various techniques such as vertex-vertex meshes, face-vertex meshes, winged-edge meshes, and render dynamic meshes.
Various parts of a polygon mesh (e.g., vertices, faces, edges) are spatially located, directly and/or relatively, within a specific coordinate system (i.e., each has an identifiable location within the space described by that coordinate system). The generation of a reconstructed 3D model can be carried out using any standard, specialized, and/or novel photogrammetry techniques, many of which are known in the art. It should be noted that other types of environmental models can also be generated using LIDAR systems.

[0114] According to the embodiments disclosed, a LIDAR system may include at least one projection unit using a light source configured to project light. As used herein, the term “light source” broadly refers to any device configured to emit light. In one embodiment, the light source may be a laser such as a solid-state laser, a laser diode, a high-power laser, or an alternative light source such as a light-emitting diode (LED) based light source. Furthermore, the light source 112 shown throughout the drawings may emit light in different formats, such as light pulses, continuous waves (CW), quasi-CW, etc. For example, one type of light source that can be used is a vertical-cavity surface-emitting laser (VCSEL). Another type of light source that can be used is an external cavity diode laser (ECDL). In some examples, the light source is approximately 650 nm
The light source may include a laser diode configured to emit light with wavelengths between 1150 nm and 1150 nm. Alternatively, the light source may include a laser diode configured to emit light with wavelengths between approximately 800 nm and 1000 nm, approximately 850 nm and 950 nm, or approximately 1300 nm and 1600 nm. Unless otherwise indicated, the term "approximately" with respect to a number is defined as a maximum 5% variance from the stated value. Further details of the projection unit and at least one light source are described below with reference to Figures 2A to 2C.

[0115] According to the disclosed embodiments, a LIDAR system may include at least one scanning unit using at least one optical deflector configured to deflect light from a light source to scan a field of view. The term “optical deflector” broadly includes any mechanism or module configured to deviate light from its initial path, such as mirrors, prisms, controllable lenses, mechanical mirrors, mechanical scanning polygons, active diffraction (e.g., controllable LCDs), Risley prisms, non-mechanical electro-optical beam steering (e.g., those made by Vscent), polarizing gratings (e.g., those provided by Boulder Non-Linear Systems), optical phased arrays (OPAs), and others. In one embodiment, the optical deflector may include a plurality of optical components, such as at least one reflective element (e.g., a mirror) and at least one refractive element (e.g., a prism, a lens). In one example, an optical deflector may be movable to deflect light to various angles (e.g., to different angles or within a continuous range of angles). An optical deflector may be optionally controllable in various ways (e.g., deflecting by α degrees, changing the deflection angle by Δα, moving a component of the optical deflector by M millimeters, changing the rate at which the deflection angle changes). Furthermore, an optical deflector may be optionally operable to change the deflection angle within a single plane (e.g., θ coordinates). An optical deflector may be optionally operable to change the deflection angle within two non-parallel planes (e.g., θ and φ coordinates). Alternatively or in addition to these, an optical deflector may be optionally operable to change the deflection angle between predetermined settings (e.g., along a predetermined scan route) or otherwise. Regarding the use of optical deflectors in LIDAR systems, it should be noted that optical deflectors can be used in the outbound direction (transmission direction, also referred to as TX) to deflect light from a light source to at least a portion of the field of view. However, it is also possible to use optical deflectors in the inbound direction (reception direction, also referred to as RX) to deflect light from at least a portion of the field of view to one or more optical sensors. Further details regarding the scan unit and at least one optical deflector are described below with reference to Figures 3A to 3C.

[0116] Embodiments disclosed may include pivoting an optical deflector to scan a field of view. As used herein, the term “pivot” broadly includes rotating an object (in particular a solid object) about one or more axes of rotation while keeping the center of rotation substantially fixed. In one embodiment, pivoting an optical deflector may include, but not necessarily, rotating the optical deflector about a fixed axis (e.g., a shaft). For example, in some MEMS mirror implementations, the MEMS mirror can move by the action of a number of bends (benders) attached to the mirror, and the mirror may produce some degree of spatial translation in addition to rotation. Nevertheless, such a mirror can be designed to rotate about a substantially fixed axis and is therefore considered pivotal in accordance with this disclosure. In other embodiments, certain types of optical deflectors (e.g., non-mechanical electro-optical beam steering, OPA) do not require any moving components or internal movement to change the deflection angle of the deflected light. It should be noted that studies on the movement or pivoting of optical deflectors can also be applied to controlling optical deflectors to change their deflection behavior by making necessary modifications. For example, by controlling the optical deflector, it is possible to change the deflection angle of light beams arriving from at least one direction.

[0117] The disclosed embodiments may include receiving reflections associated with a portion of the field of view corresponding to a single instantaneous position of the optical deflector. As used herein, the term “instantaneous position of the optical deflector” (also referred to as “state of the optical deflector”) broadly refers to the location or position in space of at least one controlled component of the optical deflector at a given instantaneous point in time or over a short period of time. In one embodiment, the instantaneous position of the optical deflector can be measured with respect to a frame of reference. The frame of reference may relate to at least one fixed point in the LIDAR system. Or, for example, the frame of reference may relate to at least one fixed point in the scene. In some embodiments, the instantaneous position of the optical deflector may include some movement of one or more components of the optical deflector (e.g., mirrors, prisms), but this movement is typically limited to the maximum degree of change during a scan of the field of view. For example, a scan of the entire field of view of a LIDAR system may include changing the deflection of light over a range of 30°. Furthermore, the instantaneous position of at least one optical deflector may include an angular shift of within 0.05° of the optical deflector. In other embodiments, the term “instantaneous position of optical deflector” may refer to the position of the optical deflector during the acquisition of light being processed to provide data for a single point in a point cloud (or another type of 3D model) generated by the LIDAR system. In some embodiments, the instantaneous position of the optical deflector may coincide with a fixed position or orientation to which the deflector briefly stops during illumination of a particular sub-region of the LIDAR field of view. In other cases, the instantaneous position of the optical deflector may coincide with a particular position/orientation along the scanning range of the optical deflector's position/orientation to which the optical deflector passes as part of a continuous or semi-continuous scan of the LIDAR field of view. In some embodiments, the optical deflector can be moved so that it is positioned at multiple different instantaneous positions during a scan cycle of the LIDAR FOV. In other words, during the time period in which a scan cycle occurs, the deflector can be moved to a series of different instantaneous positions/orientations, and the deflector can reach different instantaneous positions/orientations at different points in time during the scan cycle.

[0118] According to the disclosed embodiments, a LIDAR system may include at least one sensing unit using at least one sensor configured to detect reflections from objects within a field of view. The term “sensor” broadly includes any device, element, or system capable of measuring electromagnetic wave characteristics (e.g., power, frequency, phase, pulse timing, pulse duration) and generating an output relating to the measured characteristics. In some embodiments, at least one sensor may include multiple detectors constructed from multiple sensing elements. At least one sensor may include one or more types of optical sensors. It should be noted that at least one sensor may include multiple sensors of the same type with different other characteristics (e.g., sensitivity, magnitude). Other types of sensors may also be used. For example, several types of sensors may be used in combination for various reasons such as improved detection in a certain distance range (especially near distance), improved dynamic range of the sensor, improved temporal response of the sensor, and improved detection under various environmental conditions (e.g., ambient temperature, rain, etc.).

[0119] In one embodiment, the sensor includes a silicon photomultiplier (SiPM), which is a solid-state single-photon sensing device constructed from avalanche photodiodes (APDs) and single-photon avalanche diodes (SPADs), functioning as sensing elements on a common silicon substrate. In one example, the typical distance between SPADs is between approximately 10 μm and 50 μm, and each SPAD may have a recovery time between approximately 20 ns and 100 ns. Similar photomultipliers made of other non-silicon materials can also be used. Although the SiPM device operates in digital/switching mode, the SiPM is an analog device because all the microcells are read out in parallel, allowing signals to be generated within a dynamic range from a single photon to hundreds and thousands of photons detected by different SPADs. It should be noted that outputs from different types of sensors (e.g., SPAD, APD, SiPM, PIN diode, photodetector) are combined into a single output which can then be processed by the LIDAR system's processor. Further details of the detection unit and at least one sensor are described below with reference to Figures 4A to 4C.

[0120] According to the disclosed embodiments, a LIDAR system includes or can communicate with at least one processor configured to perform a variety of functions. At least one processor may constitute any physical device having electrical circuits that perform logical operations for one or more inputs. For example, at least one processor may include one or more integrated circuits (ICs) that include an application-specific integrated circuit (ASIC), a microchip, a microcontroller, a microprocessor, all or part of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or other circuits suitable for executing instructions or logical operations. Instructions executed by at least one processor may be preloaded into memory integrated with or embedded in the controller, for example, or stored in separate memory. Memory may include random access memory (RAM), read-only memory (ROM), hard disk, optical disk, magnetic medium, flash memory, other permanent memory, fixed memory, or volatile memory, or any other mechanism capable of storing instructions. In some embodiments, the memory is configured to store information representing data about objects in the environment of the LiDAR system. In some embodiments, at least one processor may include two or more processors. Each processor may have a similar configuration, or they may have different configurations, being electrically connected or disconnected from one another. For example, the processors may be separate circuits or integrated into a single circuit. When two or more processors are used, these processors may be configured to operate independently or in cooperation. These processors may be coupled electrically, magnetically, optically, acoustically, mechanically, or by other means that enable their interaction. Further details of the processing unit and at least one processor are described below with reference to Figures 5A to 5C.

System Overview
[0121] Figure 1A shows a LiDAR system 100 including a projection unit 102, a scanning unit 104, a detection unit 106, and a processing unit 108. The LiDAR system 100 can be mounted on a vehicle 110. According to embodiments of the present disclosure, the projection unit 102 may include at least one light source 112, the scanning unit 104 may include at least one optical deflector 114, the detection unit 106 may include at least one sensor 116, and the processing unit 108 may include at least one processor 118. In one embodiment, at least one processor 118 may be configured to coordinate the operation of at least one light source 112 and the movement of at least one optical deflector 114 in order to scan a field of view 120. During a scan cycle, each instantaneous position of at least one optical deflector 114 can be associated with a particular portion 122 of the field of view 120. Furthermore, the LIDAR system 100 may include at least one optional optical window 124 for guiding light projected toward the field of view 120 and/or receiving light reflected from objects within the field of view 120. The optional optical window 124 can be used for different purposes, such as collimation of projected light and focusing of reflected light. In one embodiment, the optional optical window 124 may be an aperture, a flat window, a lens, or any other type of optical window.

[0122] In accordance with this disclosure, the LiDAR system 100 can be used in autonomous or semi-autonomous road vehicles (e.g., automobiles, buses, vans, trucks, and any other ground vehicles). An autonomous road vehicle equipped with the LiDAR system 100 can scan the environment and drive to a destination vehicle without human input. Similarly, the LiDAR system 100 may also be used in autonomous/semi-autonomous aircraft (e.g., UAVs, drones, quadcopters, and any other aircraft or flying devices), or autonomous or semi-autonomous surface vessels (e.g., boats, ships, submarines, and any other vessels). Autonomous aircraft and surface vessels equipped with the LiDAR system 100 can scan the environment and navigate to a destination autonomously or with the help of a remote human operator. According to one embodiment, a vehicle 110 (road vehicle, aircraft, or surface vessel) can use the LiDAR system 100 to assist in the detection and scanning of the environment in which the vehicle 110 is operating.

[0123] In some embodiments, the LiDAR system 100 may include one or more scanning units 104 for scanning the environment around the vehicle 110. The LiDAR system 100 can be mounted or installed on any part of the vehicle 110. A sensing unit 106 can receive reflections from around the vehicle 110 and transmit reflection signals indicating light reflected from objects in the field of view 120 to a processing unit 108. In accordance with this disclosure, the scanning units 104 can be mounted or incorporated on the bumper, fender, side panel, spoiler, roof, headlight assembly, taillight assembly, rearview mirror assembly, hood, trunk, or any other suitable part of the vehicle 110 that can accommodate at least a portion of the LiDAR system. In some cases, the LiDAR system 100 captures a complete surrounding view of the environment around the vehicle 110. For this purpose, the LiDAR system 100 may have a 360° horizontal field of view. In one example, as shown in Figure 1A, the LiDAR system 100 may include a single scan unit 104 mounted on the roof of the vehicle 110. Alternatively, the LiDAR system 100 may include a plurality of scan units (e.g., two, three, four, or more scan units 104) each having a field of view, and these fields of view combined can cover a horizontal field of view by scanning 360 degrees around the vehicle 110. It will be recognized by those skilled in the art that the LiDAR system 100 may include any number of scan units 104 arranged in any manner, and each unit may have a field of view of 80° to 120° or less, depending on the number of units used. Furthermore, it is also possible to obtain a 360° horizontal field of view by mounting a plurality of LiDAR systems 100, each having a single scan unit 104, on the vehicle 110. Nevertheless, it should be noted that one or more LiDAR systems 100 do not need to provide a complete 360° field of view, and that in some situations a narrower field of view may be useful. For example, vehicle 110 may require a first LiDAR system 100 with a 75° field of view in front of the vehicle, and possibly a second LiDAR system 100 with a similar field of view (optionally having a smaller detection range) in the rear. It should also be noted that various vertical field of view angles are possible.

[0124] Figure 1B is an image showing exemplary output from a single scan cycle of a LiDAR system 100 mounted on a vehicle 110 according to a disclosed embodiment. In this example, the scan unit 104 is incorporated into the right headlight assembly of the vehicle 110. All gray dots in the image correspond to locations in the environment around the vehicle 110, determined from reflections detected by the sensing unit 106. In addition to location, each gray dot can also be associated with various types of information, such as intensity (e.g., how much light returns from that location), reflectivity, proximity to other dots, and others. In one embodiment, the LiDAR system 100 can generate multiple point cloud data entries from reflections detected in multiple scan cycles of the field of view, enabling, for example, the determination of a point cloud model of the environment around the vehicle 110.

[0125] Figure 1C is an image showing a representation of a point cloud model determined from the output of the LIDAR system 100. According to the disclosed embodiments, an ambient view image can be generated from the point cloud model by processing the generated point cloud data entries of the environment around the vehicle 110. In one embodiment, the point cloud model can be provided to a feature extraction module that processes the point cloud information to identify a plurality of features. Each feature may contain data about various aspects of objects (e.g., cars, trees, people, and roads) in the point cloud and/or environment around the vehicle 110. Features may have point cloud models of the same resolution (i.e., the same number of data points arranged in 2D arrays of arbitrarily similar size) or have different resolutions. Features can be stored in any type of data structure (e.g., raster, vector, 2D array, 1D array). Furthermore, virtual features such as representations of the vehicle 110, boundaries, or bounding boxes separating areas or objects within the image (as shown in Figure 1B, for example), and icons representing one or more identified objects can be overlaid on the point cloud model representation to form the final surrounding view image. For example, the symbol for the vehicle 110 can be overlaid in the center of the surrounding view image.

Projection unit
[0126] Figures 2A to 2D show various configurations and roles of the projection unit 102 in the LIDAR system 100. Specifically, Figure 2A shows a projection unit 102 with a single light source, Figure 2B shows multiple projection units 102 with multiple light sources aimed at a common light deflector 114, Figure 2C shows a projection unit 102 with primary and secondary light sources 112, and Figure 2D shows an asymmetric deflector used in some configurations of the projection unit 102. Those skilled in the art will recognize that the illustrated configurations of the projection unit 102 can have many variations and modifications.

[0127] Figure 2A shows an example of a bistatic configuration of a LIDAR system 100 in which the projection unit 102 includes a single light source 112. The term “bistatic configuration” broadly refers to a LIDAR system configuration in which the projected light emitted from the LIDAR system and the reflected light incident on the LIDAR system pass through different optical channels. Specifically, outbound light emission may pass through a first optical window (not shown), and inbound light emission may pass through another optical window (not shown). In the example shown in Figure 2A, the bistatic configuration includes a configuration in which the scan unit 104 includes two optical deflectors. The first optical deflector 114A is for outbound light, and the second optical deflector 114B is for inbound light (the inbound light of the LIDAR system includes emitted light reflected from objects in the scene, and may also include ambient light arriving from other sources). In such a configuration, the inbound and outbound routes are different.

[0128] In this embodiment, all components of the LIDAR system 100 can be housed in a single housing 200 or divided among multiple housings. As shown in the figure, the projection unit 102 is associated with a single light source 112 which includes a laser diode 202A (or one or more laser diodes coupled together) configured to emit light (projected light 204). In one non-limiting example, the light projected by the light source 112 may have a wavelength between about 800 nm and about 950 nm, an average power between about 50 mW and about 500 mW, a peak power between about 50 W and about 200 W, and a pulse width between about 2 ns and about 100 ns. Furthermore, the light source 112 may optionally be associated with an optical assembly 202B used for manipulating the light emitted by the laser diode 202A (e.g., for collimation or focusing). It should be noted that other types of light sources 112 are also available and this disclosure is not limited to laser diodes. Furthermore, the light source 112 can emit light in various formats, such as optical pulses, modulation frequency, continuous wave (CW), quasi-CW, or any other form corresponding to the specific light source used. The projection format and other parameters may be changed by the light source from time to time based on different factors, such as instructions from the processing unit 108. The projected light is directed towards an outbound deflector 114A, which acts as a steering element to guide the projected light into the field of view 120. In this example, the scanning unit 104 also includes a pivotable feedback deflector 114B that guides photons reflected back from object 208 in the field of view 120 (reflected light 206) towards the sensor 116. The reflected light is detected by the sensor 116, and information about the object (e.g., distance to object 212) is determined by the processing unit 118.

[0129] In this figure, the LIDAR system 100 is connected to a host 210. In accordance with this disclosure, the term “host” refers to any computing environment that interfaces with the LIDAR system 100 and may be a vehicle system (e.g., part of a vehicle 110), a test system, a security system, a survey system, a traffic control system, an urban modeling system, or any system that monitors its surroundings. Such a computing environment may include at least one processor and/or be connected to the LIDAR system 100 via the cloud. In some embodiments, the host 210 may also include interfaces to external devices such as cameras and sensors configured to measure various features of the host 210 (e.g., acceleration, steering deflection, vehicle reversing, etc.). In accordance with this disclosure, the LIDAR system 100 may be fixed to a stationary object associated with the host 210 (e.g., a building, a tripod) or to a portable system associated with the host 210 (e.g., a portable computer, a movie camera). In accordance with this disclosure, by connecting the LIDAR system 100 to the host 210, the output of the LIDAR system 100 (e.g., 3D models, reflectivity images) can be provided to the host 210. Specifically, the host 210 can use the LIDAR system 100 to assist in the detection and scanning of the host 210's environment or any other environment. Furthermore, the host 210 can integrate, synchronize, or otherwise use together the output of the LIDAR system 100 with the output of other detection systems (e.g., cameras, microphones, radar systems). In one example, the LIDAR system 100 may be used by a security system. This embodiment is described in detail below with reference to Figure 7.

[0130] The LiDAR system 100 may also include a bus 212 (or other communication mechanism) for interconnecting subsystems and components to transfer information within the LiDAR system 100. Optionally, the bus 212 (or other communication mechanism) can be used to interconnect the LiDAR system 100 with the host 210. In the example in Figure 2A, the processing unit 108 includes two processors 118 for coordinating and regulating the operation of the projection unit 102, the scanning unit 104, and the detection unit 106 based at least partially on information received from internal feedback of the LiDAR system 100. In other words, the processing unit 108 can be configured to operate the LiDAR system 100 dynamically in a closed loop. A closed-loop system is characterized by having feedback from at least one of its elements and updating one or more parameters based on the received feedback. Furthermore, a closed-loop system can receive feedback and update its own operation at least partially based on that feedback. A dynamic system or element is one that can be updated during operation.

[0131] According to some embodiments, scanning the environment around the LIDAR system 100 may include illuminating the field of view 120 with light pulses. The light pulses may have parameters such as pulse duration, pulse angular dispersion, wavelength, instantaneous power, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, and others. Scanning the environment around the LIDAR system 100 may also include detecting and characterizing various aspects of the reflected light. The characteristics of the reflected light may include, for example, time of flight (i.e., time from emission to detection), instantaneous power (e.g., power signature), average power over the entire feedback pulse, and photon distribution/signal over the feedback pulse period. By comparing the characteristics of the light pulses with the corresponding reflection characteristics, distances can be estimated, and possibly physical properties such as the reflectance of object 212 can also be estimated. By repeating this process in a predetermined pattern (e.g., raster, Lissajous, or other pattern) across multiple adjacent portions 122, a complete scan of the field of view 120 can be achieved. As will be discussed in more detail below, in some situations the LiDAR system 100 can direct light to only a portion of the portions 122 of the field of view 120 in each scan cycle. These portions may or may not be adjacent to each other.

[0132] In another embodiment, the Lidar system 100 may include a network interface 214 for communicating with a host 210 (e.g., a vehicle controller). Communication between the Lidar system 100 and the host 210 is represented by dashed arrows. In one embodiment, the network interface 214 may include an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem,
Alternatively, it may include a modem that provides data communication connectivity to a corresponding type of telephone line. In another example, the network interface 214 may include a LAN card that provides data communication connectivity to a compatible local area network (LAN). In another embodiment, the network interface 214 may include an Ethernet port connected to a radio frequency receiver and transmitter and/or an optical (e.g., infrared) receiver and transmitter. The specific design and implementation of the network interface 214 depends on one or more communication networks on which the LiDAR system 100 and host 210 are intended to operate. For example, the network interface 214 can be used to provide outputs of the LiDAR system 100, such as a 3D model or operating parameters of the LiDAR system 100, to an external system. In another embodiment, the communication unit can be used to receive commands from an external system, receive information about the inspected environment, and receive information from another sensor, for example.

[0133] Figure 2B shows an example of a monostatic configuration of a LIDAR system 100 including a plurality of projection units 102. The term “monostatic configuration” broadly refers to a LIDAR system configuration in which the projected light emitted from the LIDAR system and the reflected light incident on the LIDAR system pass through an optical path that is at least partially shared. In one example, the outbound and inbound light beams share at least one optical assembly through which both light beams pass. In another example, the outbound light emission may pass through an optical window (not shown), and the inbound light emission may pass through the same optical window (not shown). A monostatic configuration may include a scan unit 104 that includes a single optical deflector 114 which directs the projected light toward the field of view 120 and the reflected light toward the sensor 116. As shown, both the projected light 204 and the reflected light 206 are incident on an asymmetric deflector 216. The term "asymmetric deflector" refers to any optical device having two sides that can deflect a light beam incident from one side in a different direction than that which deflects a light beam incident from the second side. In one example, the asymmetric deflector does not deflect the projected light 204, but deflects the reflected light 206 towards the sensor 116. An example of an asymmetric deflector may include a polarizing beam splitter. In another example, the asymmetry 216 may include an optical isolator that allows light to pass in only one direction. According to this disclosure, the monostatic configuration of the LIDAR system 100 may include an asymmetric deflector that increases detection sensitivity by preventing reflected light from entering the light source 112 and guiding all reflected light towards the sensor 116.

[0134] In the embodiment shown in Figure 2B, the LIDAR system 100 includes three projection units 102, each having a single light source 112 aimed at a common light deflector 114. In one embodiment, multiple light sources 112 (including two or more light sources) can project light at substantially the same wavelength, and each light source 112 is generally associated with a different area of the field of view (shown in the figure as 120A, 120B, and 120C). This allows for scanning of a wider field of view than could be achieved with a single light source 112. In another embodiment, multiple light sources 102 can project light at different wavelengths, and all light sources 112 can be directed to the same portion (or overlapping portion) of the field of view 120.

[0135] Figure 2C shows an example of a LiDAR system 100 in which the projection unit 102 includes a primary light source 112A and a secondary light source 112B. The primary light source 112A can project light with wavelengths longer than those perceptible to the human eye in order to optimize the SNR and detection range. For example, the primary light source 112A can project light with wavelengths between approximately 750 nm and 1100 nm. In contrast, the secondary light source 112B can project light with wavelengths visible to the human eye. For example, the secondary light source 112B can project light with wavelengths between approximately 400 nm and 700 nm. In one embodiment, the secondary light source 112B can project light along substantially the same optical path as the light projected by the primary light source 112A. Both light sources can be time-synchronized and project light emission in a simultaneous or alternating pattern. The alternating pattern means that the light sources are not active at the same time, which can reduce mutual interference. It will be recognized to those skilled in the art that other combinations of wavelength ranges and activation schedules can also be implemented.

[0136] According to some embodiments, if the secondary light source 112B is too close to the LIDAR optical output port, a person may blink. This can ensure an eye-safe mechanism that cannot be achieved with typical laser sources utilizing the near-infrared spectrum. In another embodiment, the secondary light source 112B can be used for calibration and reliability in POS (point of service). This is done in a manner somewhat similar to the calibration of headlights performed on a vehicle 110 using a special reflector/pattern at a specific height from the ground. The POS operator can check the calibration of the LIDAR by simply visually inspecting a scan pattern on a characteristic target, such as a test pattern board, located at a specified distance from the LIDAR system 100. Furthermore, the secondary light source 112B can provide a means for operational reliability that the LIDAR is working for the end user. For example, the system may be configured so that a person can place their hand in front of the light deflector 114 to test its operation.

[0137] Furthermore, the secondary light source 112B may also have an invisible element that can double as a backup system in the event of a failure of the primary light source 112A. This feature can be useful for fail-safe devices with a high functional safety rank. Given that the secondary light source 112B is visible, and also for cost and complexity reasons, the secondary light source 112B may have less power than the primary light source 112A. Therefore, if the primary light source 112A fails, the functionality of the system will be reduced to the functionality and capability set of the secondary light source 112B. Although the capability of the secondary light source 112B may be inferior to that of the primary light source 112A, the LIDAR system 100 can be designed to ensure that the vehicle 110 arrives safely at its destination.

[0138] Figure 2D shows an asymmetric deflector 216 that may be part of the LIDAR system 100. In the illustrated example, the asymmetric deflector 216 includes a reflective surface 218 (such as a mirror) and a unidirectional deflector 220. Although not always the case, the asymmetric deflector 216 can optionally be a static deflector. The asymmetric deflector 216 can be used in a monostatic configuration of the LIDAR system 100 to enable a common optical path for transmitting and receiving light through at least one deflector 114, as shown, for example, in Figures 2B and 2C. However, typical asymmetric deflectors, such as beam splitters, are characterized by energy loss, particularly in the receiving path, which may be more sensitive to power loss than in the transmitting path.

[0139] As shown in Figure 2D, the LIDAR system 100 may include an asymmetric deflector 216 positioned in the transmission path. The asymmetric deflector 216 includes a unidirectional deflector 220 for separating the transmitted optical signal from the received optical signal. Optionally, the unidirectional deflector 220 may be substantially transparent to the transmitted light and substantially reflective to the received light. The transmitted light is generated by the projection unit 102 and can travel through the unidirectional deflector 220 to the scanning unit 104. The scanning unit 104 deflects the transmitted light toward the optical outlet. The received light travels through the optical inlet to at least one deflection element 114, which deflects the reflected signal toward another path away from the light source toward the detection unit 106. Optionally, the asymmetric deflector 216 may be combined with a polarizing light source 112 that is linearly polarized on the same polarization axis as the unidirectional deflector 220. In particular, the cross-section of the outbound light beam is significantly smaller than that of the reflected signal. Therefore, the LIDAR system 100 may include one or more optical components (e.g., lenses, collimators) for focusing the emitted polarized beam to the dimensions of the asymmetric deflector 216 or otherwise manipulating it. In one embodiment, the unidirectional deflector 220 can be a polarized beam splitter that is substantially transparent to the polarized beam.

[0140] According to some embodiments, the LIDAR system 100 may further include an optical system 222 (e.g., a quarter-wavelength phase difference plate) for changing the polarization of the emitted light. For example, the optical system 222 can change the linear polarization of the emitted light beam to circular polarization. The light reflected from the field of view and returning to the system 100 passes through the deflector 114 and reaches the optical system 222, where it is circularly polarized in the opposite direction to the transmitted light. The optical system 222 then converts the received counter-polarized light into linearly polarized light that is not coaxial with the polarizing beam splitter 216. As described above, due to the optical dispersion of the beam that propagates the distance to the target, the received light portion is larger than the transmitted light portion.

[0141] A portion of the received light enters the unidirectional deflector 220, which reflects the light towards the sensor 106 with some power loss. However, another portion of the received light enters the reflective surface 218 surrounding the unidirectional deflector 220 (e.g., a slit in a polarizing beam splitter). The reflective surface 218 reflects the light towards the detection unit 106 with virtually no power loss. The unidirectional deflector 220 reflects light composed of various polarization axes and directions, which ultimately reach the detector. Optionally, the detection unit 106 may include a sensor 116 that is independent of laser polarization and is primarily sensitive to the amount of incident photons in a specific wavelength range.

[0142] It should be noted that the proposed asymmetric deflector 216 offers far superior performance compared to a simple mirror with through holes. In a mirror with holes, all reflected light reaching the holes is lost from the detector. However, in the deflector 216, a unidirectional deflector 220 deflects a significant portion of the light (e.g., about 50%) towards each sensor 116. In a LiDAR system, the number of photons that reach the LiDAR from a long distance is extremely limited, so improving the photon capture rate is important.

[0143] Devices for beam splitting and steering are described according to several embodiments. A polarized beam can be emitted from a light source having a first polarization. The emitted beam can be guided to pass through a polarized beam splitter assembly. The polarized beam splitter assembly includes a unidirectional slit on the first side and a mirror on the opposite side. The unidirectional slit allows the emitted polarized beam to be directed toward a quarter-wavelength phase plate. The quarter-wavelength phase plate changes the emitted signal from a polarized signal to a linear signal (or vice versa), thereby preventing the reflected beam from passing through the unidirectional slit later.

Scan unit
[0144] Figures 3A to 3D illustrate various configurations and roles of the scan unit 104 in the LIDAR system 100. Specifically, Figure 3A shows the scan unit 104 with a MEMS mirror (e.g., rectangular), Figure 3B shows the scan unit 104 with a MEMS mirror (e.g., circular), Figure 3C shows the scan unit 104 with an array of reflectors used in a monostatic scan LIDAR system, and Figure 3D shows an exemplary LIDAR system 100 that mechanically scans the environment around the LIDAR system 100. The illustrated configurations of the scan unit 104 are merely illustrative and it will be acknowledged to those skilled in the art that there may be many variations and modifications within the scope of this disclosure.

[0145] Figure 3A shows an exemplary scan unit 104 equipped with a single-axis rectangular MEMS mirror 300. In this example, the MEMS mirror 300 functions as at least one deflector 114. As shown, the scan unit 104 may include one or more actuators 302 (specifically 302A and 302B). In one embodiment, the actuator 302 may be made of a semiconductor (e.g., silicon) and includes a piezoelectric layer (e.g., PZT, zirconate titanate, aluminum nitride) that changes size in response to an electrical signal applied by an actuator controller, a semiconductor layer, and a base layer. In one embodiment, the physical properties of the actuator 302 may determine the mechanical stress applied to the actuator 302 when current flows. When the piezoelectric material is activated, it applies force to the actuator 302 and bends the actuator 302. In one embodiment, the resistivity (Ractive) of one or more actuators 302 in the active state when the mirror 300 is deflected to a specific angular position can be measured and compared to the resistivity (Rrest) in the resting state. The feedback, including Ractive, provides information for determining the actual mirror deflection angle compared to the expected angle, and allows for correction of the mirror 300's deflection as needed. The difference between Rrest and Ractive can be correlated to a mirror drive to an angular deflection value that can function to close the loop. This embodiment is used for dynamic tracking of the actual mirror position and can optimize response, amplitude, deflection efficiency, and frequency in both linear and resonant mode MEMS mirror schemes. This embodiment is described in more detail below with reference to Figures 32 to 34.

[0146] During scanning, current can flow from contact 304A to contact 304B (through actuator 302A, spring 306A, mirror 300, spring 306B, and actuator 302B) (represented by dashed lines in the figure). Due to the insulating gap of the semiconductor frame 308, such as insulating gap 310, actuators 302A and 302B can become two separate islands electrically connected via spring 306 and frame 308. The current, or any associated electrical parameters (voltage, current frequency, capacitance, relative permittivity, etc.), can be monitored by associated position feedback. In the event of a mechanical failure where one of the components is damaged, the current flowing through the structure changes and deviates from the functional calibration value. In extreme cases (e.g., if a spring breaks), the current is completely interrupted due to the circuit break of the electrical chain by the failed element.

[0147] Figure 3B shows another exemplary scan unit 104 equipped with a biaxial circular MEMS mirror 300. In this example, the MEMS mirror 300 functions as at least one deflector 114. In one embodiment, the MEMS mirror 300 may have a diameter between about 1 mm and about 5 mm. As shown, the scan unit 104 may include four actuators 302 (302A, 302B, 302C, and 302D), each of which may be of different lengths. In the illustrated example, current (shown as a dashed line in the figure) flows from contact 304A to contact 304D, but in other cases, current may flow from contact 304A to contact 304B, from contact 304A to contact 304C, from contact 304B to contact 304C, from contact 304B to contact 304D, or from contact 304C to contact 304D. According to several embodiments, a biaxial MEMS mirror can be configured to deflect light in the horizontal and vertical directions. For example, the deflection angle of a biaxial MEMS mirror can be between approximately 0° and 30° in the vertical direction and between approximately 0° and 50° in the horizontal direction. Those skilled in the art will recognize that the illustrated configuration of the mirror 300 can have many variations and modifications. In one example, at least the deflector 114 may have a biaxial rectangular mirror or a single-axis circular mirror. Examples of circular and rectangular mirrors are shown merely as examples in Figures 3A and 3B. Any shape can be adopted depending on the system specifications. In one embodiment, an actuator 302 is incorporated as at least an integral part of the deflector 114 to directly provide power for moving the MEMS mirror 300. Furthermore, the MEMS mirror 300 can be connected to the frame 308 by one or more rigid support elements. In another embodiment, at least the deflector 114 may include an electrostatic or electromagnetic MEMS mirror.

[0148] As described above, the monostatic scan Lidar system utilizes at least a portion of the same optical path for the emission of projected light 204 and the reception of reflected light 206. While the outbound light beam can be collimated and focused into a narrow beam, the reflection in the return path spreads out to a larger optical area due to dispersion. In one embodiment, the scan unit 104 may have a large reflection area in the return path and an asymmetric deflector 216 that redirects the reflection (i.e., reflected light 206) towards the sensor 116. In one embodiment, the scan unit 104 may include a MEMS mirror with a large reflection area, with negligible impact on field of view and frame rate performance. Further details of the asymmetric deflector 216 are given below with reference to Figure 2D.

[0149] In some embodiments (as illustrated, for example, in Figure 3C), the scan unit 104 may include a deflector array (e.g., a reflector array) with small optical deflectors (e.g., mirrors). In one embodiment, by implementing the optical deflector 114 as a group of individual small optical deflectors operating synchronously, the optical deflector 114 can perform at a larger deflection angle and a higher scan rate. The deflector array can effectively function as a large optical deflector (e.g., a large mirror) with respect to the effective area. The deflector array can be operated using a shared steering assembly configuration. This configuration allows the sensor 116 to collect reflected photons from substantially the same portion of the field of view 120 that is simultaneously illuminated by the light source 112. The term “simultaneously” means that the two selected functions occur during coincident or overlapping time periods. In this case, one starts and ends during the duration of the other, or the latter starts before the completion of the other.

[0150] Figure 3C shows an example of a scan unit 104 equipped with a reflector array 312 having small mirrors. In this embodiment, the reflector array 312 functions as at least one deflector 114. The reflector array 312 may include a plurality of reflector units 314 that pivot (individually or together) and are configured to direct light pulses toward the field of view 120. For example, the reflector array 312 may be part of the outbound path of light projected from the light source 112. Specifically, the reflector array 312 can direct projected light 204 toward a portion of the field of view 120. Alternatively, the reflector array 312 may be part of the return path of light reflected from the surface of an object located within the illuminated portion of the field of view 120. Specifically, the reflector array 312 can direct reflected light 206 toward the sensor 116 or toward the asymmetric deflector 216. In one example, the area of the reflector array 312 may be between approximately 75 and approximately 150 ^mm² . Here, each reflector unit 314 has a width of approximately 10 μm, and the support structure can be made lower than 100 μm.

[0151] According to some embodiments, the reflector array 312 may include one or more subgroups of steerable deflectors. Each subgroup of electrically steerable deflectors may include one or more deflector units, such as reflector units 314. For example, each steerable deflector unit 314 may include at least one of a MEMS mirror, a reflective surface assembly, and an electromechanical actuator. In one embodiment, each reflector unit 314 may be individually controlled by an individual processor (not shown) to tilt to a specific angle along each of one or more distinct axes. Alternatively, the reflector array 312 may be associated with a common controller (e.g., processor 118) configured to synchronously manage the movement of the reflector units 314 so that at least a portion of the reflector units 314 pivot simultaneously and point in substantially the same direction.

[0152] Furthermore, at least one processor 118 can select at least one reflector unit 314 for the outbound path (hereinafter referred to as the "transmitting mirror") and a group of reflector units 314 for the return path (hereinafter referred to as the "receiving mirror"). Increasing the number of transmitting mirrors according to this disclosure may increase the spread of the reflected photon beam. Furthermore, decreasing the number of receiving mirrors may narrow the receiving field, compensate for ambient light conditions (clouds, rain, fog, extreme heat, and other environmental conditions), and improve the signal-to-noise ratio. Also, as described above, the emitted light beam is typically narrower than the reflected light portion and can be sufficiently detected by a small portion of the deflection array. Furthermore, it is possible to prevent light reflected from the portion of the deflection array used for transmission (e.g., the transmitting mirror) from reaching the sensor 116, thereby reducing the effect of internal reflections of the LIDAR system 100 on system operation. Furthermore, at least one processor 118 can pivot one or more reflector units 314 to overcome mechanical problems and drift, such as thermal and gain effects. In one example, one or more reflector units 314 may move in a manner different from their intended behavior (frequency, rate, speed, etc.), but this movement can be compensated for by appropriately electrically controlling the deflectors.

[0153] Figure 3D shows an exemplary LiDAR system 100 that mechanically scans the environment. In this example, the LiDAR system 100 may include a motor or other mechanism for rotating the housing 200 about an axis of the LiDAR system 100. Alternatively, the motor (or other mechanism) can mechanically rotate the rigid structure of the LiDAR system 100, on which one or more light sources 112 and one or more sensors 116 are mounted, thereby scanning the environment. As described above, the projection unit 102 may include at least one light source 112 configured to project light emission. The projected light emission can travel along an outbound path toward the field of view 120. Specifically, if the projected light 204 travels toward an optional optical window 124, the projected light emission can be reflected by the deflector 114A and pass through the exit aperture 314. The reflected light emission can travel from object 208 along a return path toward the detection unit 106. For example, if the reflected light 206 travels toward the detection unit 106, it is deflected by the deflector 114B. It will be acknowledged by those skilled in the art that a LiDAR system equipped with a rotation mechanism for synchronously rotating one or more light sources or one or more sensors may use this synchronous rotation instead of (or in addition to) manipulating an internal light deflector.

[0154] In embodiments where the scanning of the field of view 120 is mechanical, the projected light emission can be directed to an exit aperture 314, which is part of a wall 316 that separates the projection unit 102 from the rest of the LIDAR system 100. In some examples, the wall 316 can be made of a transparent material (e.g., glass) covered with a reflective material to form a deflector 114B. In this example, the exit aperture 314 may correspond to a portion of the wall 316 that is not covered with the reflective material. In addition to or instead of this, the exit aperture 314 may include a hole or cut in the wall 316. The reflected light 206 can be reflected by the deflector 114B and directed towards the incident aperture 318 of the sensing unit 106. In some examples, the incident aperture 318 may include a filter window configured to allow wavelengths within a specific wavelength range to enter the sensing unit 106 while attenuating other wavelengths. The reflection of object 208 from the field of view 120 can be reflected by the deflector 114B and incident on sensor 116. By comparing several characteristics of the reflected light 206 with the projected light 204, at least one aspect of object 208 can be determined. For example, by comparing the time when the projected light 204 was emitted by the light source 112 with the time when sensor 116 received the reflected light 206, the distance between object 208 and the LIDAR system 100 can be determined. In some examples, other aspects of object 208, such as shape, color, and material, can also be determined.

[0155] In some examples, the LiDAR system 100 (or a part thereof including at least one light source 112 and at least one sensor 116) can be rotated about at least one axis to determine a three-dimensional map of the LiDAR system 100. For example, to scan a field of view 120, the LiDAR system 100 can be rotated about a substantial vertical axis as indicated by arrow 320. Figure 3D shows the LiDAR system 100 being rotated clockwise about the axis as indicated by arrow 320, but in addition to or instead of this, the LiDAR system 100 may be rotated counterclockwise. In some examples, the LiDAR system 100 can be rotated 360 degrees about the vertical axis. In other examples, the LiDAR system 100 can be rotated back and forth along a sector smaller than 360 degrees of the LiDAR system 100. For example, the LIDAR system 100 can be mounted on a platform that oscillates back and forth around its axis without undergoing a complete rotation.

Detection unit
[0156] Figures 4A to 4E illustrate various configurations and roles of the detection unit 106 in the LIDAR system 100. Specifically, Figure 4A shows an exemplary detection unit 106 with a detector array, Figure 4B shows a monostatic scan using a two-dimensional sensor, Figure 4C shows an example of a two-dimensional sensor 116, Figure 4D shows a lens array associated with the sensor 116, and Figure 4E includes three figures showing the lens structure. The illustrated configurations of the detection unit 106 are merely illustrative and may have many alternative variations and modifications consistent with the principles of this disclosure, as will be seen by those skilled in the art.

[0157] Figure 4A shows an example of a detection unit 106 equipped with a detector array 400. In this example, at least one sensor 116 includes the detector array 400. The LiDAR system 100 is configured to detect objects (e.g., a bicycle 208A and clouds 208B) within a field of view 120 located at different distances from the LiDAR system 100 (which may be several meters or more). The object 208 may be a solid object (e.g., a road, a tree, a car, a person), a fluid object (e.g., fog, water, particles in the air), or another type of object (e.g., dust or irradiated powdery material). When photons emitted from the light source 112 strike the object 208, the photons are reflected, refracted, or absorbed. Typically, as shown in the figure, only a portion of the photons reflected from object 208A enter the arbitrarily selected optical window 124. Since a change in distance up to 15 cm results in a 1 ns difference in travel time (because photons travel at the speed of light between the object 208 and the object), the time difference in travel time of different photons striking different objects can be detected by an optical time sensor with a sufficiently rapid response.

[0158] The sensor 116 includes a plurality of detection elements 402 for detecting photons of photon pulses reflected back from the field of view 120. All detection elements may be contained in a detector array 400, which may have a rectangular arrangement (e.g., as shown) or any other arbitrary arrangement. The detection elements 402 may operate simultaneously or partially simultaneously. Specifically, each detection element 402 may provide detection information for each sampling period (e.g., every nanosecond). In one example, the detector array 400 may be a silicon photomultiplier tube (SiPM), which is a solid-state single-photon sensing device constructed from an array of single-photon avalanche diodes (SPADs, which function as detection elements 402) on a common silicon substrate. Similar photomultiplier tubes made of other non-silicon materials may also be used. Although the SiPM device operates in digital/switching mode, the SiPM is an analog device because all the microcells are read out in parallel, allowing signals to be generated within a dynamic range from a single photon to hundreds and thousands of photons detected by different SPADs. As described above, two or more types of sensors can be implemented (e.g., SiPM and APD). In some cases, the detection unit 106 includes at least one APD integrated with a SiPM array and/or at least one APD detector positioned adjacent to the SiPM, on separate or common silicon substrates.

[0159] In one embodiment, the detection elements 402 can be grouped into a plurality of regions 404. These regions are geometric locations or environments within the sensor 116 (e.g., within the detector array 400) and can be formed into various shapes (e.g., rectangles, squares, rings, etc., as shown in the figure, or any other arbitrary shape). Not all individual detectors contained within the geometric area of a region 404 belong to that region, but in most cases, unless some overlap at the boundaries between regions is desirable, those detectors do not belong to other regions 404 covering other areas of the sensor 310. As shown in Figure 4A, these regions may be non-overlapping regions 404, but they may also overlap. All regions can be associated with a region output circuit 406 associated with that region. The region output circuit 406 can provide a region output signal for the corresponding group of detection elements 402. For example, the region of the output circuit 406 may be an adder circuit, but a combination of the outputs of each other detector into a single output can also be employed (whether scalar, vector, or any other arbitrary format). Optionally, each region 404 is a single SiPM, but this is not necessarily the case; a region may be a small portion of a single SiPM, a group of several SiPMs, or a combination of different types of detectors.

[0160] In the illustrated example, the processing unit 108 is located in a separate housing 200B (inside or outside) the host 210 (e.g., in a vehicle 110), and the detection unit 106 may include a dedicated processor 408 for analyzing reflected light. Alternatively, the processing unit 108 may be used to analyze reflected light 206. It should be noted that the LIDAR system 100 can be implemented in multiple housings in ways different from the illustrated example. For example, the light deflector 114 may be located in a housing different from the projection unit 102 and/or the detection module 106. In one embodiment, the LIDAR system 100 may include multiple housings interconnected in different ways, such as by electrical wire connections, wireless connections (e.g., RF connections), fiber optic cables, and any combination of the above.

[0161] In one embodiment, the analysis of the reflected light 206 may include determining the time of flight of the reflected light 206 based on the outputs of individual detectors in different regions. Optionally, the processor 408 may be configured to determine the time of flight of the reflected light 206 based on the output signals of multiple regions. In addition to the time of flight, the processing unit 108 can analyze the reflected light 206 to determine the average power of the entire feedback pulse and to determine the photon distribution/signal ("pulse shape") during the feedback pulse period. In the illustrated example, the output of any detection element 402 is not sent directly to the processor 408, but can be combined (e.g., added) with signals from other detectors in region 404 before being passed to the processor 408. However, this is merely illustrative, and the sensor 116 circuitry may also transmit information from the detection element 402 to the processor 408 via other routes (without going through the region output circuitry 406).

[0162] Figure 4B shows a LiDAR system 100 configured to scan the environment of the LiDAR system 100 using a two-dimensional sensor 116. In the example of Figure 4B, the sensor 116 is a matrix of 4 × 6 detectors 410 (also referred to as "pixels"). In one embodiment, the pixel size can be approximately 1 × 1 mm. The sensor 116 is two-dimensional in the sense that it has two or more sets (e.g., rows, columns) of detectors 410 on two non-parallel axes (e.g., orthogonal axes as shown in the illustrated example). The number of detectors 410 in the sensor 116 can vary in various implementations depending on, for example, the desired resolution, signal-to-noise ratio (SNR), desired detection distance, etc. For example, the sensor 116 may have any number of pixels from 5 to 5,000. In another example (not shown in the figure), the sensor 116 may be a one-dimensional matrix (e.g., 1 × 8 pixels).

[0163] Each detector 410 may include a plurality of detection elements 402. The detection elements 402 are, for example, avalanche photodiodes (APDs), single-photon avalanche diodes (SPADs), a combination of avalanche photodiodes (APDs) and single-photon avalanche diodes (SPADs), or detection elements that measure both the time of flight from a laser pulse transmission event to a reception event and the intensity of the received photon. For example, each detector 410 may include any number of SPADs from 20 to 5,000. The outputs of the detection elements 402 within each detector 410 can be added, averaged, or otherwise combined to provide a unified pixel output.

[0164] In the illustrated example, the detection unit 106 may include a two-dimensional sensor 116 (or a plurality of two-dimensional sensors 116) having a field of view smaller than the field of view 120 of the LIDAR system 100. In this consideration, the field of view 120 (the entire field of view that can be scanned by the LIDAR system 100 without moving, rotating, or swaying in any direction) is referred to as the "first FOV 412," and the field of view of the smaller sensor 116 is referred to as the "second FOV 412" (which can be rephrased as the "instantaneous FOV"). The range of the second FOV 414 relative to the first FOV 412 varies depending on the specific application of the LIDAR system 100, and can be, for example, between 0.5% and 50%. In one example, the second FOV 412 may be elongated vertically between 0.05° and 1°. Even if the LIDAR system 100 includes two or more two-dimensional sensors 116, the combined field of view of those sensor arrays is still smaller than the first FOV 412, and can be, for example, at least one-fifth, at least one-tenth, at least one-twentieth, or at least one-fiftieth.

[0165] To cover the first FOV 412, the scan unit 106 can guide photons arriving from different parts of the environment at different time points to the sensor 116. In the illustrated monostatic configuration, the scan unit 106 can guide reflected light 206 to the sensor 116, provided that the projected light 204 is guided toward the field of view 120 and at least one deflector 114 is positioned at the instantaneous location. Typically, at each time point during scanning of the first FOV 412, the light beam emitted by the LIDAR system 100 covers a portion of the environment larger than the second FOV 414 (at the angular aperture) and includes the portion of the environment focused by the scan unit 104 and the sensor 116.

[0166] Figure 4C shows an example of a two-dimensional sensor 116. In this embodiment, the sensor 116 is an 8 × 5 matrix of detectors 410, each detector 410 containing a plurality of detection elements 402. In one example, detector 410A is located in the second row (labeled "R2") and third column (labeled "C3") of the sensor 116 and contains a matrix of 4 × 3 detection elements 402. In another example, detector 410B is located in the fourth row (labeled "R4") and sixth column (labeled "C6") of the sensor 116 and contains a matrix of 3 × 3 detection elements 402. Thus, the number of detection elements 402 in each detector 410 may be constant or different, and different detectors 410 in a common array may have different numbers of detection elements 402. The outputs of all detection elements 402 in each detector 410 can be added, averaged, or otherwise combined to provide a single pixel output value. In the example shown in Figure 4C, the detectors 410 are arranged in a rectangular matrix (rows and columns of straight lines), but other arrangements such as a circular arrangement or a honeycomb arrangement may also be used.

[0167] According to some embodiments, measurements from each detector 410 make it possible to determine the time of flight from the light pulse emission event to the reception event and the intensity of the received photon. The reception event may be the result of the light pulse being reflected from the object 208. The time of flight may be a timestamp value representing the distance from the reflecting object to an arbitrarily selected optical window 124. The time of flight value can be determined by a photon detection and counting method such as a time-correlated single photon counter (TCSPC), an analog photon detection method such as signal integration and qualification (by analog-to-digital conversion or a simple comparator), or by other means.

[0168] In some embodiments, and also referring to Figure 4B, during a scan cycle, each instantaneous position of at least one optical deflector 114 can be associated with a specific portion 122 of the field of view 120. The design of the sensor 116 allows for the association of reflected light from a single portion of the field of view 120 with multiple detectors 410. Thus, the scan resolution of the LIDAR system can be expressed by multiplying the number of instantaneous positions (per scan cycle) by the number of detectors 410 in the sensor 116. The information from each detector 410 (i.e., each pixel) represents the basic data elements from which the captured field of view is constructed in three-dimensional space. This includes, for example, the basic elements of a point cloud representation, having a spatial position and associated reflectance values. In one embodiment, reflections from a single portion of the field of view 120 detected by multiple detectors 410 may have come back from various objects located within that single portion of the field of view 120. For example, a single portion of the field of view 120 may be larger than 50 × 50 cm in the far field of view and may easily contain two, three, or more objects that partially overlap each other.

[0169] Figure 4D is a cross-sectional view of a portion of the sensor 116 according to an example of the subject disclosed herein. The illustrated portion of the sensor 116 includes a portion of a detector array 400 comprising four detection elements 402 (e.g., four SPADs, four APDs). The detector array 400 may be a photodetector sensor implemented with complementary metal-oxide-semiconductor (CMOS). Each of the detection elements 402 has a detection area positioned within the periphery of the substrate. The sensor 116 can be used in a monostatic LiDAR system having a narrow field of view (e.g., since the scan unit 104 scans different portions of the field of view at different times). A narrow field of view for the incident light beam, if implemented, eliminates the problem of out-of-focus imaging. As illustrated in Figure 4D, the sensor 116 may include multiple lenses 422 (e.g., microlenses), each lens 422 capable of directing incident light toward a different detection element 402 (e.g., toward the active area of the detection element 402), which may be usable when out-of-focus imaging is not a concern. Since most of the light reaching the sensor 116 can be deflected toward the active area of the detection element 402, the optical fill factor and sensitivity of the detector array 400 can be increased using the lenses 422.

[0170] The detector array 400, as illustrated in Figure 4D, can include several layers embedded in a silicon substrate by various methods (e.g., implants), resulting in a detection area, contact elements for the metal layer, and insulating elements (e.g., shallow trench implants (STIs), guard rings, optical trenches, etc.). The detection area is the volumetric element in the CMOS detector, enabling the optical conversion of incident photons into current when the appropriate voltage bias is applied to the device. In the case of APD/SPAD, the detection area, through a combination of electric fields, pulls electrons generated by photon absorption towards the multiplication area, where photon-induced electrons are amplified, causing avalanche breakdown of the multiplied electrons.

[0171] The front-lit detector (e.g., as shown in Figure 4D) has its input optical port on the same side as the metal layer on top of the semiconductor (silicon). The metal layer is necessary to provide electrical connections between the individual photodetector elements (e.g., anode and cathode) and various other elements such as the bias voltage, quenching/ballast elements, and other photodetectors in the common array. The optical port through which photons pass when incident on the detector's sensing area consists of a passage through the metal layer. It should be noted that the passage of light from several directions through this passage can be blocked by one or more metal layers (e.g., the metal layer ML6 shown on the leftmost detector element 402 in Figure 4D). Such blocking reduces the overall light absorption efficiency of the detector.

[0172] Figure 4E shows three detection elements 402, each associated with a lens 422, according to an example of the subject matter disclosed herein. Each of the three detection elements in Figure 4E, denoted 402(1), 402(2), and 402(3), represents a lens configuration that can be implemented in association with one or more detection elements 402 of the sensor 116. It should be noted that combinations of these lens configurations are also possible.

[0173] In the lens configuration illustrated with respect to the detection element 402(1), the focal point of the associated lens 422 can be located above the semiconductor surface. Optionally, the apertures of different metal layers of the detection element can have varying sizes that match the cone shape of the focused light produced by the associated lens 422. Such a structure can improve the signal-to-noise ratio and resolution of the array 400 as a whole device. Larger metal layers may be important for power transmission and ground shielding. This technique may be useful, for example, in monostatic LiDAR designs with a narrow field of view when the incident light beam consists of parallel rays and the imaging focal point has no effect on the detection signal.

[0174] In the lens configuration illustrated with respect to the detection element 402(2), the efficiency of photon detection by the detection element 402 can be improved by identifying the sweet spot. Specifically, a photodetector implemented in CMOS may have a sweet spot within the detection volume area where the probability of a photon undergoing the avalanche effect is highest. Therefore, as illustrated by the detection element 402(2), the focal point of the lens 422 can be positioned at the sweet spot location within the detection volume area. The lens shape and distance from the focal point can take into account the refractive index of all elements through which the laser beam passes along the path from the lens to the detection sweet spot location embedded in the semiconductor material.

[0175] In the lens configuration illustrated with respect to the detection element on the right side of Figure 4E, a diffuser and a reflector element can be used to improve the efficiency of photon absorption in semiconductor materials. Specifically, near-IR wavelengths require a significantly long path in the silicon material to achieve a high probability of absorption of photons traveling along this path. In a typical lens configuration, photons may traverse the detection area and be absorbed, not becoming detectable electrons. A long absorption path that improves the probability of photons producing electrons results in a detection area size that is impractical for CMOS devices manufactured using typical manufacturing processes (e.g., tens of micrometers). The rightmost detector element in Figure 4E illustrates a technique for handling incident photons. The associated lens 422 focuses the incident light onto the diffuser element 424. In one embodiment, the photosensor 116 may further include a diffuser located in a gap away from at least some of the outer surfaces of the detector. For example, the diffuser 424 can direct the light beam laterally (e.g., as vertically as possible) towards the detection area and the reflective optical trench 426. The diffuser is located at the focal point, above the focal point, or below the focal point. In this embodiment, incident light can be focused onto a specific location where the diffuser element is positioned. Optionally, the detector element 422 is designed to optically avoid inactive areas where photon-induced electrons may be lost, reducing effective detection efficiency. A reflective optical trench 426 (or other form of optically reflective structure) causes photons to travel back and forth within the detection area, thereby increasing the likelihood of detection. Ideally, photons are trapped in the cavity consisting of the detection area and the reflective trench indefinitely until they are absorbed and generate electron/hole pairs.

[0176] According to this disclosure, a long path is generated to absorb incident photons and contribute to a high detection probability. Furthermore, optical trenches can also be implemented in the detection element 422 to reduce crosstalk effects of parasitic photons in the avalanche that could leak to other detectors and cause false detection events. According to some embodiments, the photodetector array can be optimized to utilize a higher yield of received signals, that is, to receive as many received signals as possible and minimize signal loss due to internal degradation. The photodetector array can be improved by (a) selectively designing a metal layer on the substrate to shift the focus to a location above the semiconductor surface, (b) directing the focus to the most responsive/sensitive area (i.e., the "sweet spot") of the substrate, (c) adding diffusers above the substrate to guide the signal towards the "sweet spot," and/or adding reflective material to trenches to reflect the deflected signal back to the "sweet spot."

[0177] In some lens configurations, the lens 422 may be positioned so that its focal point is above the center of the corresponding detection element 402, but it should be noted that this is not always the case. In other lens configurations, the focal position of the lens 422 relative to the center of the corresponding detection element 402 is shifted based on the distance of each detection element 402 from the center of the detection array 400. This can be useful in relatively large detection arrays 400 where detector elements further from the center receive light at angles significantly off-axis. Shifting the focal location (for example, towards the center of the detection array 400) allows for correction of the incident angle. Specifically, shifting the focal location (for example, towards the center of the detection array 400) allows for correction of the incident angle while using substantially the same lens 422 for all detection elements positioned at the same angle to the detector surface.

[0178] Adding an array of lenses 422 to the array of detection elements 402 may be useful when using a relatively small sensor 116 that covers only a small portion of the field of view. In such cases, the reflected signals from the scene reach the detector array 400 from substantially the same angle, so all the light can be easily focused onto the individual detectors. In one embodiment, the lenses 422 can also be used in the LIDAR system 100 to increase the detection probability of the entire array 400 (preventing photons from being "wasted" in dead areas between detectors/sub-detectors) at the expense of spatial distinctness. This embodiment is in contrast to conventional implementations such as CMOS RGB cameras that prioritize spatial distinctness (i.e., light propagating in the direction of detection element A cannot be guided by the lens towards detection element B, i.e., it cannot be "bleeded" to another detection element in the array). Optionally, the sensor 116 includes an array of lenses 422, each correlated to a corresponding detection element 402, wherein at least one of the lenses 422 deflects the light propagating to the first detection element 402 toward the second detection element 402 (thereby increasing the detection probability of the entire array).

[0179] Specifically, according to some embodiments of the present disclosure, the optical sensor 116 may include an array of photodetectors (e.g., a detector array 400), where each photodetector (e.g., a detector 410) is configured to conduct an electric current when light passes over the outer surface of each detector. Furthermore, the optical sensor 116 may include at least one microlens configured to guide light toward the array of photodetectors, the at least one microlens having a focal point. The optical sensor 116 may further include at least one layer of conductive material interposed between the at least one microlens and the array of photodetectors, having a gap that allows light to pass from the at least one microlens to the array, the at least one layer being sized to maintain space between the at least one microlens and the array, and positioning a focal point (e.g., the focal point may be planar) within the gap at a location spaced apart from the detection surface of the array of photodetectors.

[0180] In related embodiments, each detector may include a plurality of single-photon avalanche diodes (SPADs) or a plurality of avalanche photodiodes (APDs). The conductive material may be a multilayer metal constriction, and at least one layer of the conductive material may be electrically connected to the detectors in the array. In one example, the at least one layer of the conductive material may include a plurality of layers. Furthermore, the gap may be shaped to converge from at least one microlens toward the focal point and diverge from the focal region toward the array. In other embodiments, the light sensor 116 may further include at least one reflector adjacent to each photodetector. In one embodiment, a plurality of microlenses may be arranged in a lens array, and a plurality of detectors may be arranged in a detector array. In another embodiment, the plurality of microlenses may include a single lens configured to project light to a plurality of detectors in the array.

Processing unit
[0181] Figures 5A to 5C illustrate various functions of the processing unit 108 according to some embodiments of the present disclosure. Specifically, Figure 5A is a diagram showing the emission pattern of a single portion of the field of view within a single frame time, Figure 5B is a diagram showing the emission scheme of the entire field of view within a single frame time, and Figure 5C is a diagram showing the actual light emission projected toward the field of view during a single scan cycle.

[0182] Figure 5A shows four examples of emission patterns within a single frame time of a single portion 122 of the field of view 120 associated with the instantaneous position of at least one optical deflector 114. According to embodiments of this disclosure, the processing unit 108 can control (or coordinate the operation of at least one light source 112 and at least one optical deflector 114) at least one light source 112 and the optical deflector 114 to vary the light beam during scanning of the field of view 120. According to other embodiments, the processing unit 108 controls only at least one light source 112, and the optical deflector 114 can be moved or pivoted in a fixed, predetermined pattern.

[0183] Figures A through D of Figure 5A show the power of light emitted toward a single portion 122 of the field of view 120 over time. In Figure A, the processor 118 can control the operation of the light source 112 so that the initial light emission is projected toward portion 122 of the field of view 120 during scanning of the field of view 120. If the projection unit 102 includes a pulsed light source, the initial light emission may include one or more initial pulses (also referred to as "pilot pulses"). The processing unit 108 can receive pilot information from the sensor 116 about the reflection associated with the initial light emission. In one embodiment, the pilot information may be represented as a single signal based on the output of one or more detectors (e.g., one or more SPADs, one or more APDs, one or more SiPMs, etc.) or as multiple signals based on the output of multiple detectors. In one example, the pilot information may include analog and/or digital information. In another example, the pilot information may include a single value and/or multiple values (e.g., different time points and/or different parts of segments).

[0184] Based on information about the reflection associated with the initial light emission, the processing unit 108 can be configured to determine the type of light emission that will subsequently be projected toward portion 122 of the field of view 120. The subsequent light emission determined for a particular portion of the field of view 120 may occur within the same scan cycle (i.e., within the same frame) or in a subsequent scan cycle (i.e., within a subsequent frame). This embodiment will be described in further detail below with reference to Figures 23 to 25.

[0185] In Figure B, the processor 118 can control the operation of the light source 112 so that light pulses of different intensities are projected toward a single portion 122 of the field of view 120 during scanning of the field of view 120. In one embodiment, the LIDAR system 100 may be operable to generate one or more different types of depth maps. The types of depth maps are, for example, one or more of the following: a point cloud model, a polygon mesh, a depth image (holding depth information for each pixel or 2D array of the image), or any other type of 3D model of the scene. The sequence of depth maps may be a time series in which different depth maps are generated at different points in time. Each depth map in a sequence associated with a scan cycle (which can be rephrased as a "frame") may be generated within a period of the corresponding subsequent frame time. In one example, a typical frame time may last less than one second. In some embodiments, the LIDAR system 100 may have a fixed frame rate (e.g., 10 frames per second, 25 frames per second, 50 frames per second) or the frame rate may be dynamic. In other embodiments, the frame times of different frames within a sequence may not be identical. For example, the LiDAR system 100 may implement a rate of 10 frames per second, generating the first depth map in 100 milliseconds (on average), the second frame in 92 milliseconds, and the third frame in 142 milliseconds.

[0186] In Figure C, the processor 118 can control the operation of the light source 112 so that light pulses associated with different durations are projected toward a single portion 122 of the field of view 120 during scanning of the field of view 120. In one embodiment, the LIDAR system 100 may be operable to generate a different number of pulses in each frame. The number of pulses may vary between 0 and 32 pulses (e.g., 1, 5, 12, 28, or more pulses) and may be obtained based on information obtained from previous emission. The time between light pulses depends on the desired detection range and may be between 500 ns and 5000 ns. In one example, the processing unit 108 may receive information from the sensor 116 about the reflection associated with each light pulse. Based on this information (or lack thereof), the processing unit 108 can determine whether additional light pulses are needed. It should be noted that the processing time and emission time periods in Figures A to D are not to scale. Specifically, the processing time may be significantly longer than the emission time. In Figure D, the processing unit 102 may include a continuous wave light source. In one embodiment, the initial light emission includes a period of time during which light is emitted, and subsequent emissions may be continuous with or discontinuous to the initial light emission. In one embodiment, the intensity of continuous emissions may change over time.

[0187] According to some embodiments of the present disclosure, the emission pattern can be determined for each portion of the field of view 120. In other words, the processor 118 can control the light emission to enable differentiation of illumination of different portions of the field of view 120. In one example, the processor 118 can determine the emission pattern for a single portion 122 of the field of view 120 based on the detection of reflected light from the same scan cycle (e.g., initial emission). This makes the LIDAR system 100 highly dynamic. In another example, the processor 118 can determine the emission pattern for a single portion 122 of the field of view 120 based on the detection of reflected light from a previous scan cycle. For subsequent emission, differences in the pattern of subsequent emission may occur by determining different values of light source parameters, such as one of the following:
a. The overall energy of the subsequent emission b. The energy profile of the subsequent emission c. The number of optical pulse repetitions per frame d. Optical modulation characteristics such as duration, rate, peak, average power, and pulse shape e. Wave characteristics of the subsequent emission such as polarization and wavelength

[0188] In accordance with this disclosure, the differentiation of subsequent emissions can be used for different applications. In one example, the emission power level can be limited in a portion of the field of view 120 where safety is a concern, while a higher power level can be emitted in other portions of the field of view 120 (thus improving the signal-to-noise ratio and detection range). This relates to eye safety, but may also relate to skin safety, optical system safety, detection material safety, and others. In another example, based on detection results from the same or previous frames, greater energy can be directed to a more beneficial portion of the field of view 120 (e.g., region of interest, distant target, low-reflection target, etc.), while limiting the irradiation energy to other portions of the field of view 120. It should be noted that the processing unit 108 can process detection signals from a single instantaneous field of view several times within a single scan frame time. For example, subsequent emissions can be determined after each pulse emission, or after a certain number of pulses.

[0189] Figure 5B shows three examples of emission schemes in a single frame time of the field of view 120. According to embodiments of the present disclosure, at least the processing unit 108 can use acquired information to dynamically adjust the operating mode of the LiDAR system 100 and/or determine parameter values of specific components of the LiDAR system 100. Acquired information can be determined from processing data captured in the field of view 120 or received (directly or indirectly) from the host 210. The processing unit 108 can use acquired information to determine a scan scheme for scanning different parts of the field of view 120. Acquired information may include current light conditions, current weather conditions, the current driving environment of the host vehicle, the current location of the host vehicle, the current trajectory of the host vehicle, the current topography of the road around the host vehicle, or any other conditions or objects that can be detected by light reflection. In some embodiments, the determined scan scheme may include at least one of the following: (a) designation of the portion within the field of view 120 that is actively scanned as part of the scan cycle; (b) projection plan of projection unit 102 defining the light emission profile in different portions of the field of view 120; (c) deflection plan of scan unit 104 defining, for example, the deflection direction and frequency, and specifying idle elements in the reflector array; and (d) detection plan of detection unit 106 defining the sensitivity or response pattern of the detector.

[0190] Furthermore, the processing unit 108 can determine a scan scheme by at least partially obtaining the identification of at least one region of interest and at least one region of non-interest within the field of view 120. In some embodiments, the processing unit 108 can determine a scan scheme by at least partially obtaining the identification of at least one high region of interest and at least one low region of interest within the field of view 120. The identification of at least one region of interest within the field of view 120 may be determined, for example, from processing data captured within the field of view 120, based on data from another sensor (e.g., camera, GPS), received (directly or indirectly) from the host 210, or by any combination of the above. In some embodiments, the identification of at least one region of interest may include the identification of a portion, area, section, pixel, or object within the field of view 120 that is important to monitor. Examples of areas that may be identified as a region of interest may include crosswalks, moving objects, people, nearby vehicles, or any other environmental conditions or objects that may be useful for vehicle navigation. Examples of areas that may be identified as regions of non-interest (or low-interest regions) include static (non-moving) distant buildings, skylines, areas above the horizon, and objects within the field of view. Once at least one region of interest within the field of view 120 has been identified, the processing unit 108 can determine or modify an existing scan scheme. In addition to determining or modifying light source parameters (as described above), the processing unit 108 can allocate detector resources based on the identification of at least one region of interest. In one example, to reduce noise, the processing unit 108 may activate detectors 410 expected to correspond to regions of interest and deactivate detectors 410 expected to correspond to regions of non-interest. In another example, the processing unit 108 may modify detector sensitivity, for example, to increase sensor sensitivity for long-range detection with low reflectance power.

[0191] Figures A to C in Figure 5B show examples of different scan schemes for scanning the field of view 120. Each square in the field of view 120 represents a different portion 122 associated with the instantaneous position of at least one deflector 114. The luminous flux levels represented by the square filling pattern are described in explanatory text 500. Figure A shows a first scan scheme in which all portions have the same importance/priority and are assigned a default luminous flux. The first scan scheme can be used in the startup phase or periodically alternated with another scan scheme to monitor the entire field of view for unexpected/new objects. In one example, the light source parameters of the first scan scheme can be configured to generate light pulses with a constant amplitude. Figure B shows a second scan scheme in which a high luminous flux is assigned to a portion of the field of view 120 and the default and low luminous fluxes are assigned to the rest of the field of view 120. The low luminous flux can be assigned to the portion of the field of view 120 of least interest. Figure C shows a third scanning scheme in which a small vehicle and a bus (see silhouette) are identified within the field of view 120. This scanning scheme allows for high-power tracking of the vehicle and bus contours, while allocating (or not allocating) low levels of light beam to the central portion of the vehicle and bus. This light beam allocation allows for concentrating a large portion of the optical budget on the contours of the identified objects and less on the less important central portion.

[0192] Figure 5C shows the emission of light toward the field of view 120 during a single scan cycle. In the illustrated example, the field of view 120 is represented by an 8x9 matrix, where each of the 72 cells corresponds to a separate portion 122 associated with a different instantaneous position of at least one optical deflector 114. In this exemplary scan cycle, each portion contains one or more white dots representing the number of light pulses projected toward that portion, and some portions contain black dots representing the reflected light from that portion detected by the sensor 116. As shown, the field of view 120 is divided into three portions: sector I on the right side of the field of view 120, sector II in the center of the field of view 120, and sector III on the left side of the field of view 120. In this exemplary scan cycle, sector I is initially assigned one light pulse per portion, sector II, previously identified as a region of interest, is initially assigned three light pulses per portion, and sector III is initially assigned two light pulses per portion. As shown in the figure, a scan of the field of view 120 reveals four objects 208: two free-form objects in the near field of view (e.g., between 5 and 50 meters), a rounded rectangular object in the medium field of view (e.g., between 50 and 150 meters), and a triangular object in the far field of view (e.g., between 150 and 500 meters). While the study in Figure 5C uses pulse count as an example of beam allocation, it should be noted that beam allocation to different parts of the field of view can be performed in other ways. For example, pulse duration, pulse angular dispersion, wavelength, instantaneous power, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, and others can be used. The description of light emission, such as a single scan cycle in Figure 5C, illustrates the various functions of the LIDAR system 100. In the first embodiment, the processor 118 is configured to detect a first object at a first distance (e.g., a rectangular object with rounded corners) using two light pulses, and to detect a second object at a second distance greater than the first distance (e.g., a triangular object) using three light pulses. This embodiment will be described in detail below with reference to Figures 11 to 13. In the second embodiment, the processor 118 is configured to allocate more light to a portion of the field of view where a region of interest is identified. Specifically, in this example, sector II is identified as a region of interest and is therefore allocated three light pulses, while the rest of the field of view 120 is allocated two or fewer light pulses. This embodiment will be described in detail below with reference to Figures 20 to 22. In the third embodiment, the processor 118 is configured to control the light source 112 so that only a single light pulse is projected onto portions B1, B2, and C1 of Figure 5C, which are parts of sector III to which two light pulses were initially allocated per portion. The reason for doing this is that the processing unit 108 detected a near-field object based on the first light pulse. This embodiment is described in detail below with reference to Figures 23 to 25. Furthermore, as a result of other considerations, it is also possible to allocate a pulse amount less than the maximum. For example, if an object at a first distance (e.g., a near-field object) is detected in at least some areas, the overall amount of light emitted to this portion of the field of view 120 can be reduced. This embodiment is described in detail below with reference to Figures 14 to 16. Other reasons for determining power allocation to different areas are discussed further below with reference to Figures 29 to 31, 53 to 55, and 50 to 52.

Further details and examples of the various components of the LIDAR system 100 and their associated functions are included in the applicant's U.S. Patent Application No. 15/391,916, filed December 28, 2016; U.S. Patent Application No. 15/393,749, filed December 29, 2016; U.S. Patent Application No. 15/393,285, filed December 29, 2016; and U.S. Patent Application No. 15/393,593, filed December 29, 2016. These are included in their entirety by reference.

Exemplary implementation: Vehicle
[0193] Figures 6A to 6C show an implementation of the LIDAR system 100 in a vehicle (e.g., vehicle 110). Any embodiment of the LIDAR system 100 described above or below can be incorporated into vehicle 110 to provide a distance-sensing vehicle. Specifically, in this example, the LIDAR system 100 integrates a plurality of scanning units 104 and optionally a plurality of projection units 102 within a single vehicle. In one embodiment, the vehicle can use such a LIDAR system to improve power, range, and accuracy within and beyond the overlapping zone, and to improve redundancy in the highly sensitive parts of the FOV (e.g., in the direction of forward movement of the vehicle). As shown in Figure 6A, vehicle 110 may include a first processor 118A for controlling scanning of field 120A, a second processor 118B for controlling scanning of field 120B, and a third processor 118C for controlling the synchronization of scanning of these two fields. In one example, processor 118C may be a vehicle controller and may have a shared interface between a first processor 118A and a second processor 118B. The shared interface enables data exchange at the intermediate processing level and synchronization of combined field-of-view scanning to form an overlap in temporal and/or spatial space. In one embodiment, the data exchanged using the shared interface may be (a) the time of flight of the received signal associated with the overlapping field of view and/or nearby pixels, (b) the laser steering position status, and (c) the detection status of an object in the field of view.

[0194] Figure 6B shows the overlapping region 600 between field of view 120A and field of view 120B. In the illustrated example, the overlapping region is associated with 24 portions 122 from field of view 120A and 24 portions 122 from field of view 120B. Assuming the overlapping region is defined and known to processors 118A and 118B, each processor can be designed to limit the amount of light emitted into the overlapping region 600 to comply with eye safety limits for light from multiple light sources, or for other reasons such as maintaining an optical budget. Furthermore, processors 118A and 118B can avoid interference between the light emitted from the two light sources by loose synchronization between scan unit 104A and scan unit 104B, and/or by controlling the laser transmission timing, and/or by timing the activation of the detection circuit.

[0195] Figure 6C shows how the overlapping region 600 between field of view 120A and field of view 120B can be used to increase the detection distance of the vehicle 110. In accordance with this disclosure, the effective detection range can be increased by utilizing two or more light sources 112 that project nominal light emissions into the overlapping zone. The term “detection range” may include the approximate distance from the vehicle 110 at which the LIDAR system 100 can clearly detect an object. In one embodiment, the maximum detection range of the LIDAR system 100 is about 300 meters, about 400 meters, or about 500 meters. For example, with a detection range of 200 meters, the LIDAR system 100 can detect an object located 200 meters (or less) from the vehicle 110 with more than 95%, more than 99%, or more than 99.5% of the time, even when the reflectivity of the object is less than 50% (e.g., less than 20%, less than 10%, or less than 5%). Furthermore, the LIDAR system 100 may have a false alarm rate of less than 1%. In one embodiment, projected light from two light sources arranged in a temporal and spatial space can be used to improve the SNR and thus improve the range and/or quality of service of objects located in overlapping areas. Processor 118C can extract high-level information from reflected light in fields 120A and 120B. The term "extract information" may include any process of identifying information related to objects, individuals, locations, events, etc., within the captured image data by any means known to those skilled in the art. Furthermore, by sharing high-level information such as objects (road boundaries, backgrounds, pedestrians, vehicles, etc.) and motion vectors, processors 118A and 118B enable each processor to direct attention to surrounding areas that are likely to soon become areas of interest. For example, it can be determined that a moving object in field 120A will soon enter field 120B.

Exemplary implementation: Survey system
[0196] Figure 6D shows an implementation of the LiDAR system 100 in a survey system. As described above, the LiDAR system 100 can be fixed to a stationary object 650, which may include a motor or other mechanism for rotating the housing of the LiDAR system 100 to obtain a wider field of view. Alternatively, the survey system may include multiple LiDAR units. In the example shown in Figure 6D, the survey system can use a single rotatable LiDAR system 100 to acquire 3D data representing the field of view 120 and process this 3D data to detect people 652, vehicles 654, changes in the environment, or any other data important for security.

[0197] According to several embodiments of this disclosure, 3D data can be analyzed to monitor retail processes. In one embodiment, 3D data can be used in retail processes requiring physical security (e.g., detection of intrusion into retail facilities, vandalism inside or around retail facilities, unauthorized access to protected areas, and suspicious activity around vehicles in parking lots). In another embodiment, 3D data can be used for public safety (e.g., detection of people slipping and falling on store premises, spilling hazardous liquids on store floors or resulting obstruction of passage, attacks or kidnappings in store parking lots, obstruction of fire emergency exits, and congestion inside or outside store areas). In another embodiment, 3D data can be used for collecting business confidential data (e.g., tracking people passing through store areas to reveal how many people pass through, where they stop, how long they stop, and how their shopping habits compare to purchasing habits).

[0198] In accordance with other embodiments of this disclosure, 3D data can be analyzed and used for traffic violation enforcement. Specifically, 3D data can be used to identify vehicles traveling at speeds exceeding the legal limit or any other road traffic law requirement. In one example, the LiDAR system 100 can be used to detect vehicles that have crossed a stop line or designated stopping point when the traffic light is red. In another example, the LiDAR system 100 can be used to identify vehicles traveling in lanes reserved for public transport. In yet another example, the LiDAR system 100 can be used to identify vehicles making a U-turn at an intersection where certain U-turns are prohibited on a red light.

[0199] Examples of various disclosed embodiments are described above and below in relation to the control unit controlling the deflector scan, but it should be noted that the various features of the disclosed embodiments are not limited to such systems. Techniques for allocating light to different parts of the LIDAR FOV are applicable to types of light-based sensing systems (LIDA or others) where it may be desirable or necessary to guide different amounts of light to different parts of the field of view. In some cases, such light allocation techniques have a positive effect on the sensing function as described herein, but other advantages may also be obtained.

[0200] Scanning LiDAR based on detector array

[0201] Many existing LiDAR systems apply a laser flash to a scene, which then generates reflections, and these reflections are used to construct an image of the scene. However, such systems often produce low levels of detail (e.g., low resolution) and may lack redundancy in the measurement.

[0202] Therefore, the system and method of this disclosure enable the use of a moving (or scanning) laser spot using multiple detectors. Thus, compared to existing systems, finer detail can be obtained in addition to the multiplicity of measurements at each spot. Such multiplicity provides further detail and/or redundant measurements, which can be used, for example, in error correction.

[0203] Figure 7 shows an exemplary method 700 for detecting an object using a LiDAR system. Method 700 can be performed by one or more processors (for example, at least one processor 118 of the processing unit 108 of the LiDAR system 100 shown in Figure 1A, and/or two processors 118 of the processing unit 108 of the LiDAR system 100 shown in Figure 2A).

[0204] In step 701, the processor 118 controls the light emission from the light sources (e.g., the light source 112 in Figure 1A, the laser diode 202 of the light source 112 in Figure 2A, and/or the multiple light sources 102 in Figure 2B). For example, the processor 118 can increase or decrease the power of the light sources. Furthermore, the processor 118 can vary the timing of the pulses from the light sources. Alternatively or simultaneously, the processor 118 can vary the length of the pulses from the light sources. As another example, the processor 118 can, either or simultaneously, vary the spatial dimensions of the light pulses emitted from the light sources (e.g., length or width, or by changing the cross-sectional area in other ways). In yet another example, the processor 118 can, either or simultaneously, vary the amplitude and/or frequency of the pulses from the light sources. In yet another example, the processor 118 can change the parameters of the continuous wave (CW) or quasi-CW light emission from the light sources (e.g., its amplitude, modulation, phase, etc.). The light source can be called a "laser," but alternative light sources may be used instead of or simultaneously with a laser. For example, a light-emitting diode (LED) based light source or similar can be used as the light source. In some embodiments, controlling the light emission may include controlling the operation of other components in the emission path in addition to the light source itself. For example, the processor 118 can further control the light source by controlling the collimation optics and/or other optical components on the transmission path of the LIDAR system.

[0205] In step 703, the processor 118 scans the field of view (e.g., the field of view 120 in Figures 1A and 2A) by repeatedly moving at least one optical deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or unidirectional deflector 114 in Figure 2B) located in the outbound path of the light source. In some embodiments, at least one optical deflector may include a pivotable MEMS mirror (e.g., the MEMS mirror 300 in Figure 3A).

[0206] In some embodiments, the processor 118 can move at least one optical deflector so that at least one optical deflector is positioned at a plurality of different instantaneous positions during a single scan cycle of the field of view (for example, the deflector may be controlled to move from one instantaneous position to another instantaneous position, or through one instantaneous position, during a scan of the LIDAR FOV). For example, at least one optical deflector may be moved continuously or discontinuously from one of a plurality of positions to another during a scan cycle (optionally, additional positions and/or repetitions may also be used). As used herein, the term “move” may refer to the physical movement of the deflector, or to a change in the electrical or optical properties of the deflector (for example, if the deflector includes a MEMS mirror or other piezoelectric or thermoelectric mirror, or if the deflector includes an optical phased array (OPA), etc.). Movement of at least one deflector may also be performed on a light source in combination with at least one deflector. For example, if a LiDAR system includes a vertical cavity surface-emitting laser (VCSEL) array or any other type of emitter array, moving at least one deflector may involve changing the combination of active lasers in the array. In such an implementation, the instantaneous position of the deflector may be determined by a specific combination of active light sources in the VCSEL array (or other type of emitter array).

[0207] In some embodiments, the processor 118 can coordinate at least one optical deflector with a light source such that the outbound path of the light source coincides at least partially with the return path when at least one optical deflector is positioned at a specific instantaneous location. For example, as shown in Figure 2B, the projected light 204 and the reflected light 206 coincide at least partially. In such embodiments, at least one deflector may include a pivotable MEMS mirror.

[0208] Similarly, in some embodiments, the overlapping portion of the outbound and return paths may include a common optical deflection element. For example, as shown in Figure 2B, the optical deflector 114 directs the projected light toward the field of view 120 and the reflected light toward the sensor 116. In some embodiments, the common optical deflection element can be a movable deflection element (i.e., a deflection element that can be controllably moved between multiple instantaneous positions). In some embodiments, the overlapping portion may include a portion of the surface of the common optical deflection element. Thus, in certain embodiments, even if the projected light does not incident on the entire (or nearly entire) surface of the common optical deflection element, one or more reflections may cover the entire (or nearly entire) surface of the common optical deflection element.

[0209] Alternatively, or simultaneously, at least one optical deflector may include at least one outbound deflector and at least one feedback deflector. For example, as shown in Figure 2A, the outbound deflector 114A directs the projected light 204 toward the field of view 120, and the feedback deflector 114B directs the reflected light 206 from an object 208 within the field of view 120. In such an embodiment, the processor 118 can receive reflections of a single optical beam spot along a feedback path to the sensor that does not coincide with the outbound path via at least one feedback deflector. For example, as shown in Figure 2A, the projected light 205 travels along a path that does not coincide with the reflected light 206.

[0210] The optical paths, such as the outbound and return paths described above, may be at least partially within the housing of the LIDAR system. For example, the outbound path may include a portion of the space between a light source within the housing and at least one optical deflector and/or a portion of the space between at least one optical deflector and the housing aperture. Similarly, the return path may include a portion of the space between at least one optical deflector (or at least one separate optical deflector) within the housing and the housing aperture and/or a portion of the space between a sensor and at least one optical deflector (or at least one separate optical deflector).

[0211] In step 705, while at least one deflector is in a specific instantaneous position, the processor 118 receives a reflection of a single light beam spot along the return path to the sensor (e.g., at least one sensor 116 of the detection unit 106 in Figures 1A, 2A, 2B, and 2C) via at least one deflector. As used herein, the term “beam spot” can refer to a portion of a light beam from a light source that can produce one or more reflections from the field of view. A “beam spot” may include a single pixel or multiple pixels. Thus, a “beam spot” can illuminate a portion of a scene that is detected by a single pixel of the LIDAR system or by several pixels of the LIDAR system. Each portion of the scene may cover an entire pixel, but it is not necessarily required that the entire pixel be covered for detection by the pixel. Furthermore, the size of the “beam spot” may be larger than, approximately the same as, or smaller than at least one deflector (e.g., if the beam spot is deflected by a deflector on the outbound path).

[0212] In step 707, the processor 118 receives a signal from the sensor associated with the image of each light beam spot, for each beam spot. For example, the sensor can absorb the reflection of each beam spot and transmit it to the processor 118, thus converting the absorbed beam spot into an electronic (or other digital) signal. Therefore, the sensor may include a SiPM (silicon photomultiplier tube) or any other solid-state device constructed from an array of avalanche photodiodes (APD, SPAD, etc.) on a common silicon substrate, or any other device capable of measuring electromagnetic wave characteristics (e.g., power, frequency) and generating an output (e.g., a digital signal) associated with the measured characteristics.

[0213] In some embodiments, the sensor may include multiple detectors (e.g., the detection elements 402 of the detection array 400 in Figure 4A). In certain embodiments, the size of each detector can be smaller than the image of each light beam spot, so that an image of each light beam spot is incident on multiple detectors for each beam spot. Therefore, as used herein, the multiple light beam spots do not need to include all the spots on which an image is projected, but include at least two spots that are larger than the multiple detectors.

[0214] In some embodiments, each detector of a sensor's multiple detectors may include one or more sub-detectors. For example, a detector may include SiPMs, and each SiPM may include multiple individual single-photon avalanche photodiodes (SPADs). In such an example, the sensor may include multiple (e.g., 5, 10, 20, etc.) SiPM detectors, and each SiPM may include multiple (e.g., tens, hundreds, thousands) SPADs. Thus, in certain embodiments, the detector includes a minimum group having outputs that are converted into a single data point in the generated output model (e.g., a single data point in the generated 3D model).

[0215] In some embodiments, the LIDAR system may include multiple light sources. In such embodiments, the processor 118 can simultaneously control the light emission of multiple light sources aimed at a common light deflector. For example, as shown in Figure 2B, multiple light sources 102 are aimed at a common light deflector 114. Furthermore, the processor 118 can receive signals associated with images of different light beam spots from multiple sensors, each positioned along a different return path. For example, as further shown in Figure 2B, multiple sensors 116 receive reflections along different return paths. Thus, the multiple sensors 116 can generate signals associated with images of different light beam spots. In some embodiments, it is also possible to deflect the reflections by at least one scanning light deflector. For example, as shown in Figure 2B, multiple scanning light deflectors 214 deflect the reflections received along different return paths before they reach the multiple sensors 116.

[0216] In some embodiments, the sensor may include a one-dimensional array of detectors (e.g., having at least four individual detectors). In other embodiments, the sensor may include a two-dimensional array of detectors (e.g., having at least eight individual detectors).

[0217] In step 709, the processor 118 determines at least two different range measurements associated with an image of a single light beam spot from the signals generated by incidence on multiple detectors. For example, at least two different range measurements may correspond to at least two different distances. In a similar example, the sensor may be configured to detect reflections associated with at least two different times of flight for a single light beam spot. In such an example, the time-of-flight measurements may differ due to differences in the distance traveled by the light beam spot to different detectors. Similarly, the frequency phase shift measurements and/or modulation phase shift measurements may differ due to differences in distance and/or angle of incidence to different detectors.

[0218] In some embodiments, at least two different range measurements are obtained from detection information acquired by two or more detectors of one or more sensors (e.g., two or more independently sampled SiPM detectors). Furthermore, the at least two different range measurements can be associated in two different directions with respect to the LIDAR system. For example, a first range measurement detected by a first detector of one or more sensors can be converted to a first detection location (e.g., spherical coordinates _θ1 , _φ1 , _D1 ), and a second range measurement detected by a second detector of one or more sensors from the reflection of the same beam spot can be converted to a second detection location (e.g., spherical coordinates _θ2 , _φ2 , _D2 ). At least two pairs of coordinates are different in the first and second detection locations (e.g., _θ2 ≠ _{θ2, D1 ≠ D2} , etc.).

[0219] In another example where at least two different range measurements correspond to at least two different distances, the at least two different range measurements may include a first distance measurement to a part of an object and a second distance measurement to elements in the object's environment. For example, if a beam spot covers both an object (tree, building, vehicle, etc.) and elements in the object's environment (road, person, fog, water, dust, etc.), the first range measurement may indicate the distance to a part of the object (branch, door, headlight, etc.), and the second range measurement may indicate the distance to the background elements. In another example where at least two different range measurements correspond to at least two different distances, the at least two different range measurements may include a first distance measurement to a first part of an object and a second distance measurement to a second part of an object. For example, if the object is a vehicle, the first range measurement may indicate the distance to a first part of the vehicle (bumper, headlight, etc.), and the second range measurement may indicate the distance to a second part of the vehicle (trunk handle, wheels, etc.).

[0220] Alternatively, or simultaneously, at least two different range measurements associated with an image of a single light beam spot may correspond to at least two different intensities. Similarly, the at least two different range measurements may include a first intensity measurement associated with a first part of the object and a second intensity measurement associated with a second part of the object.

[0221] In some embodiments, the processor 118 can simultaneously determine a first set of range measurements associated with the image of the first light beam spot and a second set of range measurements associated with the image of the second light beam spot. In some embodiments, the first set of range measurements may be greater than the second set of range measurements. For example, if the first light beam spot contains more detail than the second light beam spot, the processor 118 can determine more range measurements from the first light beam spot than from the second light beam spot. In such an example, if the second light beam spot is at least partially directed to the sky or contains the sky, the measurements from the first light beam spot may include 8 measurements and the measurements from the second light beam spot may include 5 measurements.

[0222] In some embodiments, the number of different range measurements determined in a scan cycle may be greater than the number of instantaneous positions. For example, the processor 118 can determine at least two different range measurements for each instantaneous position, as described above with reference to step 709. For example, if the sensor includes N detectors and the LIDAR system detects ranges at M instantaneous positions of the deflector in each scan cycle, the number of range measurements determined may be up to N × M. In some embodiments, even if multiple pulses are emitted for each portion of the FOV, the number of pulses emitted in each scan cycle may be less than the number of point cloud data points (or other 3D model data points) generated. For example, if P pulses are emitted at each instantaneous position, the number of pulses in one scan cycle may be P × M. In the embodiments described above, P < M.

[0223] Method 700 may include additional steps. For example, Method 700 may include generating output data (e.g., a 3D model) associated with different measurements in different directions relative to the LiDAR. In such an example, the processor 118 can generate a 3D model frame (etc.) from information of many pixels at different angles of light beams and FOV.

[0224] In some embodiments, different directions may differ with respect to the optical window or LIDAR aperture through which the reflected signal passes as it travels to at least one detector. For example, in spherical coordinates, at least one of φ or θ may differ between two measurements.

[0225] In some embodiments, method 700 may further include generating multiple point cloud data entries from the reflection of a single light beam spot. For example, processor 118 can generate multiple point cloud data entries from the reflection of a single light beam spot, as shown in Figure 1B. Processor 118 may generate multiple entries such as 2, 3, 4, 8, etc., from a single light beam spot, for example, using at least two range measurements from step 709.

[0226] Multiple point cloud data entries can define a two-dimensional surface. For example, multiple point cloud data entries can form part of a point cloud model, as shown in Figure 1C.

[0227] Method 700 can be performed so that different parts of the field of view are detected at different times. Therefore, illuminating a portion of the field of view and determining multiple range measurements as a result is performed while at least the other parts of the field of view are not illuminated by the light source. Thus, Method 700 scans a portion of the field of view, and this is repeated for different portions of the field of view, resulting in multiple "scan-in-scan" operations of the field of view, as shown in the examples in Figures 10B and 10C.

[0228] As described above, Figure 4C shows an example of a two-dimensional sensor 116 that may be used in the method 700 of Figure 7. Figure 8A shows an alternative sensor 800 that can be used in place of or in combination with the sensor 116 of Figure 4C. For example, the detector 410 in the example of Figure 4C is rectangular, while the example of Figure 8A shows multiple hexagonal pixels (e.g., pixel 844) consisting of individual detectors (e.g., detector 846). Similarly, Figure 8B shows an alternative one-dimensional sensor 850 that can be used in place of or in combination with the sensor 116 of Figure 4C and/or the sensor 800 of Figure 8A. In the example of Figure 8B, the one-dimensional column of pixels (e.g., pixel 854) consists of individual detectors (e.g., detector 856). Although shown as a one-dimensional vertical column in Figure 8B, another embodiment may include a one-dimensional horizontal row of detectors.

[0229] Figures 9A and 9B are block diagrams illustrating exemplary LiDAR devices having transmission and reflection alignment. LiDAR systems 900 and 900' in Figures 9A and 9B represent implementations of LiDAR system 100 in Figure 1. Therefore, the functions and modifications discussed with respect to Figures 1A to 1C can be similarly applied to the embodiments in Figures 9A and 9B, and vice versa. For example, as shown in Figures 9A and 9B, LiDAR systems 900 and 900' may include at least one photon pulse emitter assembly 910 for emitting one or more photon inspection pulses. The emitter 910 may include a projection unit 102 having at least one light source 112 in Figure 1A, Figure 2A, or Figure 2B.

[0230] As further shown in Figures 9A and 9B, the Lidars 900 and 900' may include at least one photon steering assembly 920 for guiding the photon inspection pulses in the direction of the scene segment being scanned and for directing reflected photons toward the photon detection assembly 930. The steering assembly 920 may include controllably maneuverable optical systems (e.g., rotatable/movable mirrors, movable lenses, etc.) and may also include fixed optical components such as beam splitters. For example, the steering assembly 920 may include deflectors 114A and 114B in Figure 2A and/or a common optical deflector 114 and multiple scanning optical deflectors 214 in Figure 2B. Some optical components (e.g., used for collimation of laser pulses) may be part of the emitter 910, and other optical components may be part of the detector 930.

[0231] In Figures 9A and 9B, the LiDARs 900 and 900' may further include at least one photon detection assembly 930 for detecting photons of the photon inspection pulse reflected back from objects in the scanned scene. The detection assembly 930 may include a two-dimensional sensor 932, such as the sensor 116 in Figure 4C and/or the sensor 800 in Figure 8. In some embodiments, the detection assembly 930 may include multiple two-dimensional sensors.

[0232] As shown in Figures 9A and 9B, the LiDARs 900 and 900' may include a controller 940 for controlling the steering assembly 920 and/or the emitter 910 and/or the detector 930. For example, the controller 940 may include at least one processor (e.g., the processor 118 of the processing unit 108 of the LiDAR system 100 shown in Figure 1A, and/or the two processors 118 of the processing unit 108 of the LiDAR system shown in Figure 2A). The controller 940 can control the steering assembly 920 and/or the emitter 910 and/or the detector 930 in various coordinated ways, as described both above and below. Accordingly, the controller 940 can perform all or part of the methods disclosed herein.

[0233] As shown in Figure 9B, the LIDAR 900' also includes at least one vision processor 950. The vision processor 950 can acquire data from the photon detector 930 and process it to generate data from the acquired data. For example, the vision processor 950 (optionally in combination with the controller 940) can generate point cloud data entries from the acquired data and/or generate a point cloud data model therefrom.

[0234] In both LIDAR 900 and 900', the controller 940 (and optionally the vision processor 950 of LIDAR 900') is used to coordinate transmission (TX) and reflection (RX). This coordination may include synchronously deflecting both transmission and reflection using coordination between multiple optical deflectors, and/or synchronously deflecting both transmission and reflection using coordination within a shared deflector. Synchronization may involve the physical movement of one or more optical deflectors and/or piezoelectric and/or thermoelectric adjustments.

[0235] Figure 10A shows an example of the first FOV 1000 and several examples of the second FOVs 1002 and 1004. The magnitude of the FOV angles and the pixel dimensions may differ from the examples given in Figure 10A.

[0236] Figure 10B shows an exemplary scan pattern of the second FOV 1002 in Figure 10A across the entire area of the first FOV 1000 in Figure 10A. As shown in Figure 10B, the second FOV 1002 can be scanned in FOV 1000, moving from right to left in a horizontal pattern, and then moving from left to right in a diagonal pattern. Such patterns are merely illustrative. The systems and methods of this disclosure can use any pattern to move the second FOV across the entire area of the first FOV.

[0237] Figure 10C shows an exemplary scan pattern of the second FOV 1006 across the entire first FOV 1000 in Figure 10A. As shown in Figure 10C, the second FOV 1006 can be scanned across the FOV 1000 and moved from right to left in a horizontal pattern. Such patterns are merely illustrative. The systems and methods of this disclosure can use any pattern to move the second FOV across the entire first FOV. Referring to the examples in Figures 10B and 10C, it should be noted that the array sizes used in these figures are merely non-limiting examples, as are the non-limiting examples in Figure 10A, and that the number of pixels in each array can be significantly small (e.g., 2×3, 3×5, etc.), significantly large (e.g., 100×100, etc.), or any number in between.

[0238] In the example in Figure 10C, the second FOV 1006 has a height corresponding to the height of the first FOV 1000. Therefore, as shown in Figure 10C, by matching at least one corresponding dimension of the second FOV to that of the first FOV, the required scan rate of the second FOV across the entire area of the first FOV can be reduced. Accordingly, a LIDAR system according to this disclosure may include a 1D sensor array having dimensions corresponding to at least one dimension of the first FOV.

[0239] Returning to the example in Figure 10B, the second FOV 1002 has smaller dimensions in both directions than the first FOV 1000. This allows the LIDAR system to concentrate energy over a smaller area, improving the signal-to-noise ratio and/or detection distance.

[0240] By scanning the second FOV across the entire area of the first FOV, the system of this disclosure can generate a 3D model with relatively low object smear. One reason for this is that the majority of the object can be detected simultaneously by the maneuverable 2D sensor array. The reduction in smear level can be achieved with respect to one or more axes of the sensor array (e.g., (X, Y), (φ, θ)) and/or the depth axis (e.g., Z, r). The low smear level results in improved detection accuracy.

[0241] Furthermore, scanning the second FOV across the entire first FOV can enable scanning at relatively low frequencies (for example, if 10 vertical pixels are implemented in the sensor array, the movement of the mirrors along the vertical axis will be at approximately 10 times lower frequencies). This reduction in scan rate allows for the use of larger mirrors.

[0242] Furthermore, scanning the entire first FOV with the second FOV may enable the use of weaker light sources than existing systems. This could reduce power consumption, miniaturize the LIDAR system, and/or improve eye safety and other safety issues. Similarly, scanning the entire first FOV with the second FOV may enable the use of relatively smaller sensors compared to existing systems. This could reduce the size, weight, cost, and/or complexity of the system. It could also enable the use of more sensitive sensors than existing systems.

[0243] Furthermore, scanning the second FOV across the entire range of the first FOV may reduce the ambient light and/or noise collected. This may improve the signal-to-noise ratio and/or detection distance compared to existing systems.

[0244] Selective irradiation in Lidar based on detection results

[0245] As described above, a depth map of objects detected in a scene can be generated using a LiDAR system. Such a depth map may include a point cloud model, a polygon mesh, a depth image, or any other type of 3D model of the scene. However, in some cases, fewer objects than all objects in a particular scene or within a particular distance range may be detected by the LiDAR system. For example, a 3D reconstruction of a scene may detect and include objects that are relatively close to the LiDAR system, while other objects (including, for example, smaller, less reflective, or farther away objects) may not be detected for a particular set of operating parameters of the LiDAR system. In addition, the system's signal-to-noise ratio may in some cases be lower than desired or lower than the level required to enable the detection of objects within the LiDAR system's field of view.

[0246] In a particular embodiment, the LiDAR system described herein, including any of the configurations described above, allows for the dynamic variation of the amount of light flux to different sections of the LiDAR FOV, thereby enabling the modification of one or more operating parameters of the LiDAR system during the current scan of the FOV or subsequent scans of the FOV. To accomplish this, the LiDAR system 100 may have the capability to increase the number of objects detected within the FOV. The LiDAR system 100 may also be capable of increasing the signal-to-noise ratio (e.g., the ratio of illumination from the LiDAR system to other noise sources or interference sources such as sunlight (or other illumination sources) or electrical noise associated with the detection circuit). Increasing the signal-to-noise ratio can improve the system sensitivity and resolution. Such dynamic variation of the light flux applied to different regions of the LiDAR FOV can generate a depth map (or other representation of the scene) from the environment of the LiDAR system, including representations of one or more objects that might otherwise be undetectable.

[0247] There are many different possibilities for the dynamic variation of the luminous flux in a specific area of the scanned FOV, some of which will be discussed in more detail below. Such dynamic variations in luminous flux may include reducing or maintaining the luminous flux level where objects are detected within the scanned FOV, and increasing the luminous flux in areas where no objects are detected. Such increases in luminous flux may enable the detection of objects that are further away or less reflective. Such increases in luminous flux may also enable an improvement in the signal-to-noise ratio in a specific area of the scanned FOV. As described above, these effects enable the generation of a depth map of objects in the environment of the LIDAR system, providing more complete information. In addition to objects detectable within a specific range associated with a fixed luminous flux scan, the system can identify other objects through dynamic luminous flux adjustments during scanning.

[0248] As described above, the LIDAR system 100 may include a projection unit 102 which includes at least one light source 112. The processing unit 108 can be configured to coordinate the operation of the light source 112 and any other available light sources. In some examples, the processing unit 108 controls the light source 112 so that it can vary the beam of light in scanning the field of view of the LIDAR system using light from at least one light source 112. The processing unit 108 can also control the deflector 114 of the scanning unit 104 to deflect the light from at least one light source 112 for scanning the field of view. Also, as described above, the processing unit 108 can interact with the detection unit 106 to monitor reflections from various objects in the scanned FOV (for example, based on signals generated by one or more sensors on which reflections are incident). Based on the monitored reflections, the processing unit 108 can generate a depth map or other reconstruction of the scene associated with the scanned FOV. For example, the processing unit 108 may determine, using a first detected reflection associated with scanning a first portion of the field of view, that a first object exists within the first portion at a first distance. During the FOV scan, the processing unit 108 may determine that an object is absent from a second portion of the field of view at a first distance. For example, to determine that an object is absent from the second portion, the processing unit 108 may begin measuring from the time of light emission into the second portion of the field of view and process the received sensor detection signals over a time period that matches or includes the travel time of light to a distance equal to twice the first distance (because if an object is within the first range, the reflected light must travel to the object and then return). After detecting the first reflection and determining that an object is absent from the second portion, the processing unit 108 may, for example, modify the light source parameters associated with at least one light source 112 so that more light is projected into the second portion of the field of view than is projected into the first portion. The processing unit 108 may then determine, using a second detected reflection in the second portion of the field of view, that a second object exists at a second distance greater than the first distance.

[0249] During scanning of the LIDAR system FOV, the processing unit 108 can control one or more parameters associated with the available light sources 112 to change the amount of light flux supplied (e.g., projected) to a particular area of the FOV. In some examples, changing the light flux may include increasing the amount of light flux supplied to one area of the FOV compared to the amount of light flux supplied to another area of the FOV. Changing the light flux may also include increasing the amount of light supplied (e.g., within a particular area of the FOV) over a particular period of time compared to another period. The increase in light flux corresponding to projecting or supplying more light to a particular area may include various quantitative characteristics of the projected light. For example, an increase in luminous flux may result in or be accompanied by a corresponding increase in power per solid angle, irradiance per FOV portion, number of projected light pulses, power per pixel, number of photons per unit time, number of photons per scan cycle, total energy for a particular time period, total energy for a single scan cycle, luminous flux density (e.g., measured in W/ ^m² ), number of emitted photons per data point in the generated point cloud model, total energy per data point in the generated point cloud model, or any other feature that increases the luminous flux.

[0250] The deflector 114, controlled by the processing unit 108 to deflect light from at least one light source 112, may include any suitable optical elements for altering the optical path of at least a portion of the light incident on the deflector. For example, in some embodiments, the deflector 114 may include a MEMS mirror (e.g., a pivotable MEMS mirror). The deflector may include other types of mirrors, prisms, controllable lenses, mechanical mirrors, mechanical scan polygons, active diffraction elements (e.g., controllable LCDs), Risley prisms, non-mechanical electro-optical beam steering devices, polarizing gratings, optical phased arrays (OPAs), or any other suitable optical steering elements.

[0251] Optionally, after the detection of the first object, the processing unit 108 may control at least the light source 112 and/or at least the deflector so that all emission occurs toward the second region of the FOV used for detecting the second object before additional light is emitted toward the first portion (e.g., in a later scan cycle).

[0252] Referring to Figures 11 and 12, this technique of scanning a LiDAR FOV and dynamically varying the beam of light applied to a specific region of the LiDAR FOV will be described in more detail. Figure 11 is a schematic diagram of an FOV 120 that can be scanned using a processing unit 108 that controls one or more light sources 112 and at least one deflector 114. For example, as mentioned above, the FOV can be scanned by moving the deflector 114 across a plurality of instantaneous positions, each corresponding to a specific portion 122 of the FOV 120. It should be noted that the FOV 120 may include a plurality of substantially equal portions 122 (e.g., defined by the same solid angle). However, this is not necessarily the case. It should be noted that during scanning of the FOV 120, the deflector 114 may remain stationary at each instantaneous position for a predetermined amount of time. During that time, light such as a continuous wave, single pulse, or multiple pulses can be projected onto the corresponding portion 122 of the FOV 120. Furthermore, during a predetermined stop time at a specific instantaneous position, (for example, in a monostatic embodiment) light reflected from objects in the scene can be guided to one or more detector units using the deflector 114. Alternatively, the deflector 114 can continuously (or semi-continuously) move through multiple instantaneous positions during the scanning of the FOV 120. During such continuous or semi-continuous scanning, light such as a continuous wave, single pulse, or multiple pulses can be projected onto the instantaneous portion 122 of the FOV 120. Also, during such continuous or semi-continuous scanning, (for example, in a monostatic embodiment) light reflected from objects in the scene can be guided to one or more detectors using the deflector 114.

[0253] Scanning the FOV 120 proceeds, for example, by projecting light from the light source 112 onto region 1102 of the FOV 120, collecting reflected light from region 1102, and performing time-of-flight analysis based on the projected and reflected light to determine the distance to one or more objects in region 1102 that produced the reflection. After collecting reflected light from region 1102, the processing unit 108 can move the deflector 114 to another region of the FOV 120 (e.g., an adjacent region or some other region) and repeat the process. The entire FOV 120 can be scanned in this way (for example, by moving from region 1102 to 1104, scanning all other rows, and ending at region 1106). The scan pattern associated with Figure 11 proceeds from left to right, then right to left, starting from the top row and continuing through each subsequent row. However, any suitable scan pattern can be used for scanning the FOV 120 (e.g., one row at a time in one or two horizontal directions, one column at a time in one or two vertical directions, diagonally, or by selecting individual regions or subsets of regions). Furthermore, as mentioned earlier, a depth map or any type of reconstruction can be generated based on the determination of reflections and distances to objects.

[0254] The processing unit 108 can control the deflector 114 in any suitable manner so that the deflector 114 redirects light from the light source 112 to various regions 122 of the FOV 120. For example, in some embodiments, at least one processor 118 can be configured to control at least one optical deflector 114 so that at least one optical deflector pivots along two orthogonal axes or along two substantially perpendicular axes. In other embodiments, at least one processor 118 can be configured to control at least one optical deflector 114 so that at least one optical deflector pivots along two linearly independent axes, thereby enabling two-dimensional scanning. Such deflector movement can be achieved by any of the techniques described above. Furthermore, the processing unit 108 may optionally control a rotatable motor for maneuvering at least one optical deflector.

[0255] As the scan of the FOV 120 progresses, it becomes possible to determine the presence of an object in a specific region of the FOV by using light reflections from some of the specific regions of the FOV (e.g., region 122). For example, during the scan of region 1108, the processing unit 108 can determine the presence of an object (i.e., the car) in region 1108 and the distance to this object by using reflections received from the car 1110. As a result of scanning region 1108, the processing unit 108 can determine that an object exists in that region and that this object is at a distance D1 from the host of the LIDAR system 100. It should be noted that the processing unit 108 may optionally determine two or more distances for a region of the FOV. This occurs, for example, when two or more objects reflect light within the same field of view (for example, in region 1120 of Figure 12, reflections from both the car 1202 and the road can be received and analyzed), or when a reflective object is positioned to reflect light from various distances (for example, from an inclined surface).

[0256] However, scans of other regions of the FOV 120 may not return any observable reflections. In such cases, the processing unit 108 does not detect the presence of an object in those regions. For example, as shown in Figure 11, regions 1120 and 1122 of the FOV 120 do not return any observable reflections during each scan of these regions. As a result, the depth map generated from the received reflections and distance analysis performed for these regions of the FOV 120 does not indicate the presence of an object in region 1120 or 1122. Based on the absence of available reflections in regions 1120 and 1122 (and other non-reflective regions), the processing unit 108 can determine that there is no object in those regions within the distance range detectable by the LIDAR system 100 with a given set of operating parameters. For example, the processing unit 108 can determine that the automobile 1110 (or at least a part of the automobile) is present at a distance D1 within region 1108. Based on the reflection received from this region, it can then determine that no object exists within region 1120 at a distance D1 relative to the LIDAR system 100. This determination can be based on the assumption that if any object exists in region 1120 at distance D1 (and assuming such an object has similar reflective properties to the automobile 1110), the processing unit 108 should identify the presence of those objects in the same way it did in region 1108.

[0257] In particular, the determination that an object is absent within a specific area of the FOV 120 may be based on the detection capabilities of the LIDAR system in a specific set of operating parameters. By changing these operating parameters, particularly to increase the detection sensitivity of the LIDAR system (e.g., by increasing the signal-to-noise ratio), it may become possible to identify objects in areas where objects were not detected before the change in operating parameters.

[0258] In a region of the FOV (e.g., region 1120 or 1122) where the processing unit 108 has determined that an object at distance D1 is absent, the processing unit 108 may modify the light source parameters to project more light to one or more regions of the FOV than is projected to the region of the FOV where an object was detected. In the example shown in Figure 11, after failing to detect an object (at distance D1 or other distances) in region 1120, the processing unit 108 may modify the light source parameters to project more light to region 1120 than is directed to region 1108 (where part of the automobile 1110 was detected). Such an increase in the amount of light supplied to region 1120 increases the signal-to-noise ratio in that region, improves the sensitivity of the LIDAR system in that region, enables the system to detect objects with lower reflectivity, and/or enables the system to detect objects that may be located further away (e.g., at a distance greater than D1).

[0259] Various light source parameters can be changed to increase the amount of light supplied to a specific area of the FOV. For example, the processing unit 108 can extend the duration of continuous wave emission projected onto a specific area of the FOV, increase the total number of light pulses, increase the total light energy of the emitted light, and/or increase the power of one or more light pulses (e.g., peak power, average power, etc.). In addition to or instead of this, the number of light pulse repetitions per scan can be increased so that more light pulses are supplied to one area of the FOV than to another area of the FOV. More broadly, for example, after it is determined that no object is present in a specific area at a particular distance or range of distances, or based on any other arbitrary criterion regarding the detection result for a specific area of the FOV, any light source parameters can be adjusted so that the beam of light directed to one part of the field of view (e.g., area 1120) is different from (e.g., increased) the beam of light directed to another part of the field of view (e.g., area 1108). As mentioned above, an increase in the amount of light projected onto a particular region may include at least one of the following: an increase in the power supplied per solid angle, an increase in irradiance relative to the size of the region, an increase in light pulses, an increase in power per pixel, an increase in photons over a given time period, an increase in total energy over a predetermined time period, an increase in luminous flux density (W/ ^m² ), an increase in the number of photons per scan cycle, an increase in total energy over a specific time period, an increase in the number of emitted photons per data point in the generated point cloud model, or an increase in total energy per data point in the generated point cloud model.

[0260] Various changes to the light source parameters can be made in response to the observation of any arbitrary criteria. For example, at least one pilot pulse can be emitted, and the detection result based on the acquired reflection of that at least one pilot pulse can be observed. If the detection result of a particular region from which at least one pilot pulse has been emitted indicates the absence of objects, the absence of objects at a particular distance (e.g., D1), the presence of fewer objects than expected, the absence of objects beyond a particular distance range, the detection of objects with low reflectivity, a low signal-to-noise ratio (or any other arbitrary appropriate criterion), the processing unit 108 can provide more light to the particular region using any of the techniques described above, or any other technique that provides more light to the particular region (e.g., a long continuous wave, an additional pulse, higher power). In some examples, “pilot pulse” refers to a pulse of light intended to determine subsequent light emission (e.g., to the same region of the FOV during the same scan) using the detected reflection. It should be noted that the pilot pulse may, but not necessarily, have less energy than the pulse of subsequent light emission. In some cases, the pilot pulse corresponds to the initial light pulse (in the emission sequence) delivered to a specific region of the LiDAR FOV.

[0261] Furthermore, various changes to light source parameters can also be performed as part of a predetermined operation sequence. For example, in some embodiments, a predetermined irradiation sequence for one or more regions 122 of the FOV 120 may include applying a specified series of light pulses to one or more regions 122 of the FOV 120. The first light pulse may contain a relatively low power, and one or more subsequent pulses may contain a higher power level than the first emitted pulse. In some cases, these light pulses may be emitted with gradually increasing power levels. However, it should be noted that the series of pulses may have similar power levels per pulse, and the increase in light intensity is achieved by the accumulated emission amount during scanning.

[0262] Increasing the amount of light supplied to a specific region of the FOV can be performed at various points in time during the operation of the LIDAR system 100. In some embodiments, increasing the amount of light supplied to a specific region of the FOV relative to another region can be performed during a simultaneous scan of the FOV. That is, during a particular scan of the FOV, the amount of light supplied to the portion of the FOV associated with a specific instantaneous position of the deflector 114 can be greater than the amount of light supplied to the portion of the FOV corresponding to a different instantaneous position of the deflector 114 during this particular scan of the FOV. Thus, in some embodiments, the processing unit 108 can be configured to change the light source parameters to project more light (e.g., higher irradiance per solid angle) to a specific region of the FOV in the same scan cycle in which the absence of an object (e.g., at a specific distance) in another region of the FOV is determined.

[0263] Alternatively, increasing the amount of light supplied to a particular FOV region compared to other regions of the FOV can be performed during different scans of the FOV. In other words, after performing a full scan of the FOV, during subsequent scans of the FOV, more light can be supplied to one or more regions of the FOV than to other regions of the FOV, based on the results of the previous scan of the FOV. In some cases, it is not necessary to perform a full scan of the FOV and then return to a particular part of the FOV to increase the amount of light supplied to that particular region compared to the amount of light supplied to other parts of the FOV. Such an increase can be performed by performing a partial scan of the FOV, then returning to a particular region of the FOV, and increasing the amount of light supplied to that region compared to the amount of light supplied to that region during the previous scan (or partial scan) or to other regions of the FOV.

[0264] Increasing the amount of light supplied to a specific region of the FOV, such as region 1120, can result in additional reflections that may enable object detection. Considering the increased light supply to a specific region, the signal-to-noise ratio can be increased for the light collected from that region. As a result of this improved signal-to-noise ratio, and/or considering the additional light available for detection (at least in some cases, including higher-power light), it may be possible to detect objects at a greater distance from the LIDAR system than would have been possible with less light in a particular FOV region.

[0265] As an example for illustrative purposes, in some embodiments, the scan of the FOV 120 in Figure 11 can be started using a predetermined or default amount of light given to each region 122 of the FOV. If an object is detected in a particular region, such as region 1108, the processing unit 108 can move the deflector 114 to a different instantaneous position to examine another area of the FOV 120 without emitting additional light to the aforementioned region (e.g., region 1108). If no object is detected in a particular region, such as region 1120, the processing unit 108 can provide an additional amount of light to that region. The increase in light intensity can be performed during the current scan of the FOV and before moving the deflector 114 to a new instantaneous position. Alternatively, the increase in light intensity for region 1120 can be performed during a subsequent scan or partial scan of the FOV 120. In some cases, the increase in light intensity given to a region such as region 1120 may detect an object that was not detected during the examination of a particular FOV region using less light. In an example where additional light is supplied during a partial scan of the FOV120, a pilot pulse is supplied to each region of one row while scanning in the first direction. Additional light is emitted while scanning back in the opposite direction to the region where the absence of an object is determined, after which the scan of another row begins.

[0266] In an example for illustrative purposes, if, based on one or more pilot pulses, no object is detected in region 1120, or if no object is detected in region 1120 beyond a certain distance (e.g., the distance D1 at which the automobile 1110 is determined to be located), the amount of light to region 1120 can be increased using one of the techniques described above. For example, in one embodiment, one or more additional light pulses can be provided to region 1120 (optionally at a higher power level than the pilot pulses). Each of these additional light pulses may produce subsequent reflections that can be detected by the detection unit 106. As a result, an object such as the automobile 1202 may be detected, as shown in Figure 12. In some cases, this object, such as the automobile 1202, is located at a distance D2 greater than the distance at which an object was previously detected in the same or a different region (e.g., the automobile 1110 is located at a distance D1 and occupies region 1108 and the surrounding region of the FOV). Therefore, in this specific (and non-limiting) example for explanation, the initial pilot pulse of light applied to region 1120 and its subsequent reflections may not detect an object at distance D1 (for example, the distance at which car 1110 was detected in region 1108 based on the pilot pulse applied to region 1108). In response to the absence of an object observed at distance D1 in region 1120, one or more additional pulses (optionally, higher-power pulses or longer continuous waves, but not necessarily) can be applied to region 1120 to provide more light. Based on these one or more additional pulses applied to region 1120 and their subsequent reflections, the presence of an object such as car 1202 can be determined at a distance D2 greater than the distance D1 at which car 1110 was detected in region 1180.

[0267] It should be noted that the light intensity increase and the specific protocol selected to provide the light intensity increase are specific to a particular region of the FOV, such as region 1120, which may correspond to a specific instantaneous position of the deflector 114. Alternatively, the specific protocol selected to increase the light may not be limited to a specific region of the FOV, such as region 1120, but may be shared by multiple regions of the FOV. For example, region 1204 of FOV 120 (outlined in Figure 12) may contain four specific regions of the FOV, each corresponding to a different instantaneous position of the deflector 114. In some embodiments, the protocol selected to increase the light to any of the subregions of FOV region 1204 may be the same for all subregions. As a result, by applying a common light increase protocol to each subregion of FOV region 1204, multiple objects or a single object may be detected at similar distances or distance ranges in these subregions. For example, in Figure 12, by applying a common light amplification protocol to sub-regions of the FOV region 1204, a portion of the vehicle 1202 can be detected in each of the different sub-regions of the FOV region 1204 (for example, at a distance D2 greater than the distance D1 at which the vehicle 1110 was detected). It should be noted that, optionally, the decision of whether or not to amplify light emission in one region of the FOV may depend (at least partially) on the reflection detection information from another region of the FOV.

[0268] In addition to giving the ability to detect objects at greater distances by adding light to a specific region of the FOV, other objects may also be detected as a result of the increase in light. For example, as shown in Figure 11, a pilot pulse of light supplied to region 1122 may not detect an object in region 1122. However, a subsequent increase in light supplied to region 1122 and at least one reflection resulting therefrom may enable the detection of an object in region 1122, such as a turtle 1206, as shown in Figure 12. The turtle 1206 may be located at a distance D3 that is smaller than D1 (for example, closer to the LIDAR system than the car 1110), but since the turtle 1206 has a lower reflectivity than the car 1110, it may not have been detected in response to the initial pilot pulse supplied to region 1122 (the low reflectivity of the turtle 1206 may be a result of, for example, its partial size relative to region 112 and/or the even lower reflectivity of its shell). For example, one or more subsequent light pulses applied to region 1122, and their respective reflections, can detect a turtle 1206 with low reflectivity. Such techniques for increasing the light to the FOV region enable the detection of various objects that would not be detected based on the initial amount of light supplied to the FOV region. For example, this technique can be used to detect objects such as distant curbs 1208 and 1210 and/or the horizon 1212 of the road 1214 (at a distance D4 greater than D1, D2, and D3). Simultaneously, in regions of the FOV where objects such as a car 1110 or a tree 1216 are detected using the initial pulse (or part of the available optical budget), additional light amplification may not be necessary, and the FOV scan can continue at different instantaneous positions of the deflector 114.

[0269] After detecting an object within a specific region of the FOV, the deflector 114 can be moved to a new instantaneous position without further light emission, although in some cases it may emit additional light while the deflector continues to guide light towards the corresponding specific region of the FOV. Reflections obtained from this supplemental light can provide further information about the specific region of the FOV and/or confirm detections made based on lower levels of light applied to the same region. For example, in the illustrative examples for Figures 11 and 12, the deflector 114 can be positioned to apply a pilot pulse to region 1120. The resulting reflection may not detect an object within region 1120. Next, a second pulse can be applied to region 1120 (e.g., at a higher power level than the pilot pulse). Reflections from the second pulse enable the detection of vehicle 1202 at a distance D2 greater than the distance D1 at which vehicle 1110 was detected based on the reflection of the pilot pulse applied to region 1108. However, instead of moving away from region 1120 after detecting the car 1202 in that region, the deflector 114 can remain at the instantaneous position corresponding to region 1120. A third pulse (optionally, with higher power than the second pulse) can be applied to region 1120. Subsequent reflections of the third pulse may not detect any additional objects in region 1120 (although detection is possible). However, the reflection of the third pulse allows confirmation of the decision (based on the reflection of the second pulse) that there is no object at distance D1 in region 1120. The reflection of the third pulse also allows confirmation of the detection of the car 1202 at distance D2, which was determined based on the reflection of the second pulse applied to region 1120. It should be noted that in some cases, none of the reflections of the first, second, and third pulses may enable the detection of an object in their respective regions, but detection may be possible by combining all of these reflections. This may be a result of SNR improvement. Another example is a decision algorithm that can check the consistency of detection across several pulses.

[0270] It should be noted that increasing the amount of light supplied to a specific region of the FOV 120 may be done in accordance with any desired protocol, whether it is in relation to the amount of light supplied to other regions of the FOV, or to the amount of light supplied to a specific region of the FOV during the same scan of the FOV or during earlier scans. Such protocols may be applied to all regions of the FOV, some regions of the FOV, or a single region of the FOV. Protocols for selectively increasing the light to any part of the FOV may be predetermined or may be created during the FOV scan based on various criteria applied during the FOV scan (e.g., detection of an object in a specific region). If an object is detected in a specific region as a result of a specific amount of projected light, a similar amount of light may be supplied to that region during subsequent scans of the FOV. Such a technique can speed up object detection and FOV scanning by potentially eliminating the need to search for the object using the increased light. However, in some cases, it may be desirable to revert to applying a low level of light to a specific FOV region during subsequent scans of the FOV. For example, if the LIDAR system is moving towards an object it has already detected, it can detect the object again using a smaller light intensity than the one used for the initial detection.

[0271] By varying the amount of light supplied to different regions during scanning of the LiDAR system's field of view (FOV), the detection capabilities and/or resolution of the LiDAR system can be improved. Furthermore, based on objects detected in each of the different regions of the FOV, a three-dimensional map or reconstruction of the scene associated with the FOV can be generated using any appropriate technique. For example, in some embodiments, a point cloud can be generated showing some or all of the objects detected within the FOV. Returning to the example in Figure 12, the point cloud (or other 3D construction) may show car 1110 at distance D1, car 1202 at distance D2, and turtle 1206 at distance D3, where D3 < D1 < D2. Each detected object represented in the 3D construction can be associated with a specific detection direction (φ/θ or x/y) or angular range.

[0272] By selectively emitting light to different parts of the FOV based on the detection or absence of objects in those parts, the LIDAR system 100 can achieve several functions, such as one or more of the following: (a) speeding up object detection and FOV scanning by potentially eliminating the need to search for objects using increased light intensity; (b) reducing the total energy used for detection across the entire FOV; (c) redirecting energy allocation to areas that may have a significant impact; (d) reducing the environmental impact of the LIDAR system by reducing excess light emission in directions where objects are known to be present; and (e) reducing the processing demands for handling detection signals beyond what is necessary.

[0273] Figure 13 provides a flowchart of method 1302 for detecting an object using a LIDAR system. During operation of the LIDAR system 100 according to the embodiments disclosed herein, any or all steps may include controlling at least one light source to vary the beam of light in scanning a field of view using light from at least one light source. Alternatively, any or all operating steps may include controlling at least one light deflector to deflect light from at least one light source for scanning a field of view. In step 1308, method 1302 may determine that a first object at a first distance is present in the first portion of the field of view using a first detection reflection associated with scanning the first portion of the field of view. In step 1310, method 1302 may include determining that no object at the first distance is present in a second portion of the field of view. In step 1312, method 1302 may include modifying the light source parameters so that more light is projected to the second part of the field of view than is projected to the first part of the field of view, after the detection of the first reflection and the determination that no object is present in the second part. In step 1314, method 1302 may include using the second detected reflection in the second part of the field of view to determine that a second object is present at a second distance greater than the first distance. It should be noted that the increase in the luminous flux for detecting the second object in the second part can be performed during the current scan of the FOV or during a subsequent scan of the FOV. Optionally, if steps 1310, 1312, and 1314 are performed after the detection in step 1308, all emission to the second FOV part used for detecting the second object occurs before any additional light is sent to the first part (e.g., in a later scan cycle).

[0274] In some embodiments, this method may include scanning the FOV 120 over multiple scan cycles. In this case, a single scan cycle includes moving at least one optical deflector to multiple instantaneous positions. While at least one optical deflector is positioned at a particular instantaneous position, this method may include deflecting a light beam from at least one light source toward an object in the field of view, and deflecting reflections received from the object toward at least one sensor.

[0275] Increased flux allocation for Lidar detection

[0276] As described above, if no object at a first distance D1 is detected in a second region of the LIDAR FOV, where one or more objects have been detected in different regions of the LIDAR FOV, the luminous flux to the second region can be varied. However, in addition to this, in some embodiments, the luminous flux to a particular region of the LIDAR FOV can be varied based on whether or not an object is detected at an arbitrary distance in that region. For example, the processor 118 can determine, based on a first amount of light supplied to a particular region of the LIDAR FOV, that no object exists within a distance S1 from the LIDAR system 100 in that region. In response to such a determination, the processor 118 can supply more light to that particular part of the FOV. With the increase in light, the processor 118 can determine that no object exists within a distance S2 from the LIDAR system in that particular region, where S2 > S1. In response to such a decision, more light can be directed to a specific region of the LiDAR FOV, and in response to this increase in light, the processor 118 can detect the presence of one or more objects at a distance S3 from the LiDAR system, where S3 > S2 > S1. Therefore, such an increase in light directed to a specific region of the LiDAR FOV can enhance the detection capabilities of the LiDAR system within that region. Furthermore, the disclosed progressive illumination scheme makes it possible to achieve long-range detection with limited power consumption.

[0277] Figure 14 provides a schematic diagram of the LiDAR field of view 1410 that can be generated by the LiDAR system 100 and a related depth map scene representation. As shown in the figure, objects in the scene can be detected at distances within a range S0 relatively close to the LiDAR system (viewpoint in Figure 14). The range S0 can cover various distance intervals depending on the operating parameters of the LiDAR system 100. In some cases, S0 represents a range from 0m to 10m. In other cases, S0 corresponds to a range from 0m to 20m, 30m, 50m, etc.

[0278] In some examples, the LIDAR system may determine that an object is absent in a first portion of the field of view at a first distance S1. For example, as shown in Figure 14, based on light projected onto a specific region 1412 of the LIDAR FOV 1410 (e.g., a first light emission), the LIDAR system can identify various foreground objects such as the surface of the road 1414, curb 1416, and/or sidewalk 1418. However, the processor 118 of the LIDAR system 100 may not detect an object at a distance S1 within the region 1412. That is, the processor 118 may determine that an object is absent in the region 1412 at a distance S1 (and possibly beyond that distance). In some embodiments, distance S1 can be greater than distance S0. For example, S0 may include a range from 0 m to S1, which is up to 20 m. In some examples, S1 can be equal to and/or less than distance S0. For example, if a first light emission under relevant conditions (e.g., ambient light conditions) allows the LIDAR system to detect an object of a given reflectivity up to approximately 40 meters away, then the first light emission may allow the system to determine that no such object exists at distances of 20 m, 30 m, 39 m, and possibly 40 m.

[0279] There are several possible reasons why the LIDAR system 100 might not detect an object at distance S1 within region 1412. For example, in some cases, there may be no objects at distance S1 within that region at all. However, in other cases, the amount of light projected onto region 1412 may be insufficient to detect an object at distance S1. This could be because these objects are characterized by low reflectivity, or because distance S1 falls outside the operating range of the LIDAR system 100 for a specific set of operating parameters (e.g., duration, intensity, power level of light projected onto region 1412).

[0280] If no object is detected in region 1412 based on the first light emission, rather than abandoning the detection of an object at distance S1 in region 1412, the processor 118 can supply additional light to region 1412 to detect an object at a distance greater than S1, if possible. In other words, if the processor 118 determines, based on the first light emission, that there are no objects to detect in the first portion 1412 of the field of view 1410, the processor 118 can control the projection of at least a second light emission directed into region 1412 of the field of view 1410 so that an object at a second distance S2, greater than the first distance S1, can be detected in region 1412. The second emission not only potentially increases the ability to detect an object at distance S2, but can also increase the likelihood that the LIDAR system 100 will detect an object at distance S1.

[0281] In some cases, the processor 118 can cause the light projector 112 and the deflector 114 to project additional light into region 1412. For example, the processor 118 can control the projection of at least a third light emission guided into region 1412 of the field of view 1410 so as to determine that there is an object in region 1412 at a third distance S3 greater than a second distance S2 greater than distance S1. As shown in Figure 15, the third light emission into region 1412 may enable the detection of a pedestrian 1510 (or at least a portion thereof) at a distance S3 from the LIDAR system 100. The pedestrian 1510 may not have been detected in response to the first or second light emission guided into region 1412, but the third emission into region 1412 made it possible to determine the presence of the pedestrian 1510 at distance S3. Furthermore, the second and third light emissions, respectively, which enable the detection of objects at a second distance S2 or a third distance S3 within region 1412, allow the LIDAR system 100 to detect objects beyond the distance range S0 (e.g., curbs 1416, road surfaces 1414, and/or sidewalks 1418). For example, as shown in Figure 15, such objects are detected and mapped at distances up to and beyond S3.

[0282] Therefore, as described above, the processor 118 can project additional light emissions onto a specific region of the LiDAR FOV based on whether or not an object is detected at various distances within that region. For example, in one embodiment, the processor 118 may be configured to control the projection of at least a third light emission directed onto that portion/region of the LiDAR FOV if it is determined, based on the detection of at least one of the first and second light emissions, that an object at a first distance (e.g., S1 in Figure 14) is absent within a specific portion/region of the LiDAR FOV. Furthermore, the processor 118 may be configured to control the projection of at least a third light emission directed onto that portion/region of the LiDAR FOV if it is determined, based on the detection of at least a second light emission, that an object at a second distance (e.g., S2 in Figure 14) is absent within a specific region/region of the LiDAR FOV.

[0283] Distances S1, S2, and S3 may depend on specific operating parameters of the LIDAR system 100. These operating parameters can be selected to suit specific deployments of the LIDAR system 100 (e.g., in vehicles, buildings, aircraft, etc.), specific climatic conditions (e.g., sunny, rainy, snowy), or any other arbitrary environmental conditions (e.g., rural and urban environments). However, in some embodiments, distance S1 may be 20 m or less from the LIDAR system 100. Distance S3 may include distances greater than 100 m, and distance S2 may be between distances S1 and S3. To further examine the concept of detection distance of the LIDAR system, it should be noted that the aforementioned detection distances (S0, S1, S2, S3) are not necessarily predetermined, and these distances may be determined based on the light emission energy scheme used by the LIDAR system. Furthermore, detection distance may also depend on other factors such as climate, ambient light conditions, visibility, and the reflectivity of the target. It should be noted that in Figures 14 and 15, the detection range S0 is illustrated as extending uniformly from the vertical plane on which the LIDAR system is located, for the sake of simplification. However, as noted, none of the detection ranges are necessarily uniform across different parts of the FOV, and these distances can be measured radially from points located on the LIDAR's optical window, rather than from the zero-distance plane (as shown in the illustrations).

[0284] Herein, for any of the embodiments disclosed, a particular portion or region of the LIDAR FOV may, in some embodiments, refer to a single pixel of the FOV scan. In such embodiments, a particular portion of the FOV may correspond to a single instantaneous position of the deflector 114 as the deflector 114 moves across a range of positions/orientations to scan the LIDAR FOV. Referring, for example, to Figure 14, region 1412 (e.g., a portion of the LIDAR FOV) may represent a single pixel of the LIDAR FOV 1410. However, in other embodiments, a particular region of the LIDAR FOV may include multiple pixels. For example, region 1520 of the LIDAR FOV may include multiple pixels, each corresponding to a different instantaneous position of the deflector 114. Individual pixels included in one region or portion of the FOV, such as the pixels included in region 1520, may be continuous or discontinuous. In some cases, the FOV portion represents a specific region of interest within the LiDAR FOV, and similar light emission protocols can be used.

[0285] In some embodiments, the relative positions of specific portions of the LiDAR FOV can vary in various ways. For example, in some examples, the deflector 114 can be moved continuously across a plurality of instantaneous positions, each corresponding to a specific region of the LiDAR FOV (e.g., in a sweep pattern, a raster pattern, randomly, or pseudo-randomly). In the process described above, there may be some time between the first light emission to a specific portion of the FOV and the second light emission to the same portion of the FOV. During that time, the deflector 114 may move such that the exact instantaneous position of the deflector during the first emission is different from the exact instantaneous position during the second emission. Similarly, the exact instantaneous position of the deflector during the third emission may be different from the exact instantaneous positions during the first and second emissions. As a result, the regions of the LiDAR FOV irradiated by the first, second, and third emissions may be slightly different from each other. However, for the purposes of this disclosure, grouped light emissions or multiple light emissions projected onto substantially overlapping regions of the LiDAR FOV are considered to be directed to the same region of the LiDAR FOV. In other words, in some embodiments, a specific portion of the LiDAR FOV may correspond to a single instantaneous position of the deflector 114. In other embodiments, a specific portion of the LiDAR FOV may correspond to two or more instantaneous positions of the deflector 114.

[0286] As described above, by monitoring and/or processing the output of one or more sensors such as sensor 116, the processor 118 can determine both whether an object is present within a specific area of the LiDAR FOV or whether an object is absent within the same area. For example, as shown in Figure 14, the reflection of light projected onto the FOV area 1412 enables the detection and depth mapping of objects such as the road surface 1414, curb 1416, or sidewalk 1418, particularly at distances within the range S0. On the other hand, the same projection of light onto area 1412 may not provide object detection or depth mapping capabilities at distances S1 or even further away from the LiDAR system, such as S2. In such cases, based on the information obtained from the available reflections of light from area 1412, the processor 118 can determine that no object is present within area 1412 at distances beyond S0, such as S1, S2, etc.

[0287] The decision that an object is absent does not necessarily mean that there is actually nothing in a particular region of the LIDAR FOV. As mentioned above, such a decision may be made when the detector 116 receives insufficient light reflection from that region to detect an object in that region. Alternatively, the decision that an object is absent may be made, for example, when reflections are collected, but the information is insufficient to determine distance information for at least one reflection source or to generate a depth map based on one or more reflections. However, as described with respect to Figures 14 and 15, increasing the light beam level to a particular region of the FOV may result in the detection of objects that were not previously detected. Furthermore, object detection may not involve a binary process (for example, reflections from an object are received or no reflections are received at all). Rather, detection may require the detector 116 to receive sufficient light reflections for the processor 118 to recognize that an object is present in the particular region onto which the light is projected. Therefore, whether or not detection occurs may depend on various factors such as the reflectivity of the object and the distance to the object. Light projections described herein as enabling detection at distinct distances may correspond to light projections that result in positive detection in a certain proportion of cases involving objects with a specific level of reflectivity (e.g., reflectivity levels of at least 2.5%, 5%, 10%, etc.) (e.g., at least 50%, 75%, 90%, 99%, or more of cases involving light projections of a particular set of features).

[0288] Using the process described above to increase the luminous flux to a particular region of the LIDAR FOV based on whether or not an object is detected at various distances within that region, a scan of the LIDAR FOV can be performed, and multiple objects can be detected using light projection from anywhere within the light projection sequence associated with a particular FOV region. For example, in some cases, an object can be detected in one region of the LIDAR FOV using only the first light emission. Only after providing second light emissions, third light emissions, etc., to other regions of the LIDAR FOV can objects be detected in those regions by scanning those parts. In the exemplary embodiments shown in Figures 14 and 15, the road surface 1414 can be detected by the first light emission to the FOV region 1530. In region 1412, objects such as the sidewalk 1418, curb 1416, and road 1414 can be detected by the first light emission within at least a certain range (S0 or S1 shown in Figure 14). Subsequent (e.g., second) light emissions guided to region 1412 allow for detection of the sidewalk 1418, curb 1416, and road 1414 over a wider area (e.g., S2). A third light emission guided to region 1412 allows for detection of a pedestrian 1510 at a distance S3. Similar results can be obtained in one or more other regions of the FOV 1410. Of course, some regions may receive only one light emission (or no light emission at all), while others may receive multiple light emissions. As a result, a particular scan of the LIDAR FOV may include objects detected based on the first, second, and/or third light emissions, depending on the number of light emissions projected onto a particular region.

[0289] There are various techniques that can be used to increase the beam of light to a specific region of the LiDAR FOV, including those described above or below. In some cases, the processor 118 can control the light projector 112 to vary the beam of light to a specific region of the FOV (e.g., its aiming direction, power level, light intensity, wavelength, pulse width, duration of continuous wave application, etc.). In other cases, the processor 118 can control at least one light deflector 114 to vary the beam (e.g., by controlling its orientation and thus the direction of projection to a specific region of the FOV, by controlling the amount of time that light is projected onto a specific region of the FOV, etc.).

[0290] Furthermore, the processor 118 can control both at least one aspect of the light projector 112 and at least one aspect of the deflector 114 in order to control the amount of light received by a specific region of the FOV. For example, in some embodiments, the processor 118 can control the light projector 112 to emit multiple light energy emissions. The processor 118 can also control the light deflector 114 such that the first, second, and third light emissions provided by the light projector 112 are all projected onto a specific portion of the LIDAR FOV corresponding to a single instantaneous position of the light deflector 114 (or at least an intimate instantaneous position of the deflector). Each of the first, second, and third light emissions may have similar characteristics (e.g., power level, duration, number of pulses, wavelength, etc.). Alternatively, one or more of the first, second, and third light emissions may have different characteristics. For example, one or more of these emissions may exhibit a higher power level than the others. In some embodiments, the power levels associated with the first, second, and third light emissions may gradually increase with each emission. Also, in some embodiments, the processor 118 may be configured to control the light projector 112 (which may include one multi-wavelength light source, or multiple light sources, each capable of emitting light of different wavelengths) such that the first light emission projected onto a specific region of the FOV has a different wavelength from both the second and third light emissions induced onto the specific region of the FOV. In some examples, each of the first, second, and third light emissions contains a single pulse (optionally, these pulses may have similar characteristics). In some examples, each of the first, second, and third light emissions contains the same number of pulses (optionally, these pulses may have similar characteristics). In some examples, each of the first, second, and third light emissions contains one or more pulses (optionally, these pulses may have similar characteristics).

[0291] In some embodiments, each of the light emissions projected onto a specific region of the Lidar FOV may have a similar light intensity (e.g., substantially the same light intensity). However, in other embodiments, the processor 118 can make the light intensities of the various light emissions from the light projector 112 different. For example, the processor 118 can be configured to control the light projector 112 such that the second light emission has a greater light intensity than the first light emission provided by the light projector 112 to a specific region of the FOV. Similarly, the processor 118 can control the light projector 112 such that the third light emission from the light projector 112 to a specific region of the FOV has a greater light intensity than the second light emission.

[0292] Similarly, each of the light emissions projected onto a specific region of the LiDAR FOV may have a similar power level. However, in other embodiments, the processor 118 can make the light power levels of the various light emissions from the light projector 112 different. For example, the processor 118 can be configured to control the light projector 112 such that a second light emission has a higher power level than a first light emission provided by the light projector 112 to a specific region of the FOV. Similarly, the processor 118 can be configured to control the light projector 112 such that a third light emission from the light projector 112 to a specific region of the FOV has a higher power level than a second light emission. In yet another case, the power levels associated with one or more light emissions following the first light emission to a specific region of the LiDAR FOV may be lower than the power levels associated with the first light emission. As a result of the additional light emissions provided to a specific region of the LiDAR FOV, the stored light energy may increase with each emission. This increases the likelihood of object detection in that area, including the gradual increase in detection distance.

[0293] Considering the effect of accumulating light energy applied to a specific portion of the Lidar FoV, different light emissions or pulses can be used together to detect objects in that portion of the FOV. For example, in some embodiments, the processor 118 can use a third light emission projected onto a specific region of the FOV together with one or both of the first or second light emissions projected onto that region to determine the presence of an object in that region. Furthermore, the accumulated light energy allows for an extension of the detection distance. By using the first emission and one or both of the second or third emission, the processor 118 can detect objects at a greater distance (e.g., S3) than the detection distance associated with the second emission alone (e.g., S2) or the first emission alone (e.g., S0).

[0294] In addition to detecting objects using multiple light emissions, multiple light emissions can also be used to generate data points used to create a depth map representing objects in the scene. For example, in some embodiments, data points for the depth map can be generated based solely on a first light emission projected onto a specific region of the LiDAR FOV. In other embodiments, data points for the depth map can be generated based on a combination of a first emission, a second emission, and/or a third emission (or more) projected onto a specific region of the FOV.

[0295] Furthermore, specific objects can be detected using different combinations of light emissions at different points in time (e.g., in different scans of the LiDAR FOV). In some examples, at time T0, it is necessary to combine multiple light emissions (e.g., two, four, or more emissions) to detect the presence of pedestrian 1510 (Figure 15). As the distance to pedestrian 1510 decreases (e.g., as the vehicle on which the LiDAR system 100 is deployed approaches pedestrian 1510), the amount of light emission required to detect pedestrian 1510 may decrease. For example, if the distance to pedestrian 1510 is less than S0, pedestrian 1510 can be detected based on a single light emission to a specific area of the LiDAR FOV during subsequent scans.

[0296] In the described embodiment for dynamically varying the amount of light beam delivered to a specific region of a LiDAR FOV during scanning of the FOV, more light can be projected onto a specific region of the FOV than is projected onto one or more other regions of the FOV during scanning of the LiDAR FOV. For example, the processor 118 can be configured to change the light source parameters related to the light projected onto the first portion of the LiDAR FOV such that the amount of light beam directed to the first portion is greater than the amount of light beam directed to at least one other portion of the LiDAR FOV during the same scan cycle of the FOV. The processor 118 can also monitor the amount of light delivered to various regions of the FOV to ensure compliance with applicable regulations. For example, the processor 118 can be configured to control the light projector 112 so that the accumulated energy density of light projected onto a specific area of the LiDAR FOV does not exceed the maximum permissible exposure limit (either within a single scan of the FOV or across multiple scans of the FOV).

[0297] For example, the processing unit 108 can control the gradual projection of light onto portions of the LiDAR FOV to intermittently determine whether an object has been detected in each portion of the LiDAR FOV, and if an object has been detected, it can control the light emission to that portion of the FOV to be maintained within a safe light emission limit that does not harm the detected object. These techniques can be implemented complimentarily. That is, in each of one or more portions of the FOV, the processing unit 108 can implement stop conditions (preventing the removal of the maximum permissible exposure limit) while complimentarily and continuously checking whether additional light is needed (for example, by determining that the light projected so far onto the portion of the LiDAR FOV is insufficient to effectively detect an object).

[0298] The LIDAR system 100 may include a preliminary signal processor (not shown) for high-speed processing of the reflected signals of earlier light emissions (e.g., first light emission, second light emission). This is to enable a rapid decision regarding later light emissions, in particular when later light emissions (e.g., second light emission, third light emission) occur within the same scan cycle, or in particular when later light emissions occur while the light deflector is still in substantially the same instantaneous position. A rapid decision regarding later emissions may include determining whether further emissions (e.g., second light emission, third light emission) are needed, and may also include determining the parameters of subsequent emissions for each segment. It should be noted that some or all of the circuitry of the preliminary signal processor may differ from the circuitry of the range estimation module used to determine various points in the 3D model. This is because a rapid decision does not necessarily require accurate range estimation (e.g., determining only the presence or absence of an object may suffice). Another reason for using different circuits is that the main range estimation circuit may not be fast enough to make a decision at the rate necessary to provide further light emission at the same instantaneous position of at least one optical deflector. It should be noted that the processing results of such a secondary signal processor may be insufficient for range estimation. Optionally, the secondary signal processor can be an analog processor that processes analog detection signals (e.g., voltage), and the main range estimator module can be a digital signal module that processes detection information after analog-to-digital conversion (or includes such a module). Furthermore, it should be noted that, to prevent excessive light energy emission (e.g., for eye safety reasons), the same (or similar) secondary signal processing module can be implemented in the LIDAR system 100 and used for detecting objects in adjacent areas of the LIDAR system. This will be discussed in more detail below.

[0299] The increase in the beam of light delivered to a specific portion of the LiDAR FOV can proceed according to any suitable protocol. For example, in some embodiments, as described above, the first, second, and third light emissions (or more or fewer emissions) can be projected onto a specific region of the LiDAR FOV, and then the deflector 114 can be moved to different instantaneous positions to scan different regions of the FOV. In other words, the processor 118 can be configured to control the deflector 114 so that the first, second, and third light emissions are projected onto a specific portion of the LiDAR FOV in a single scan cycle.

[0300] In other cases, multiple light emissions designated for a specific region of the LiDAR FOV can be projected onto that portion of the FOV during different scans of the FOV. For example, the processor 118 can be configured to control the deflector 114 so that one or more of the first, second, and third light emissions are projected onto a specific portion of the LiDAR FOV in different scan cycles.

[0301] Using the disclosed embodiments, a method for detecting an object using a LIDAR system can be performed. For example, as described above, detecting an object with the LIDAR system 100 may include controlling at least one light source so that the beam of light can be varied in a scan of a LIDAR field of view using light from at least one light source. As shown in Figure 16, a method for detecting an object with the LIDAR system 100 may include controlling the projection of at least a first light emission directed to a first portion of the field of view (step 1620) to determine that an object at a first distance is absent in the first portion of the field of view (step 1630). The method may also include, if the absence of an object in the first portion of the field of view is determined based on at least a first light emission, controlling the projection of at least a second light emission directed to the first portion of the field of view to enable the detection of an object at a second distance greater than a first distance in the first portion of the field of view (step 1640). Furthermore, this method may include controlling the projection of at least a third light emission directed to a first portion of the field of view to determine that an object at a third distance greater than a second distance exists within the first portion of the field of view (step 1650).

[0302] Adaptive noise reduction for different areas of the field of view

[0303] In a LiDAR system according to an embodiment of the present disclosure, the captured signal may contain noise. Noise can originate from a wide variety of sources. For example, some noise may originate from a detector (e.g., the detection unit 106 in Figures 4A to 4C) and may include background noise, amplification noise, etc. Furthermore, some noise may originate from the environment and may include ambient light, etc. For example, when the LiDAR system projects light into the air, towards an object that is very far away, or towards another area where reflection is minimal, ambient noise may be greater than that of the reflected signal. On the other hand, when the LiDAR system projects light towards an object located in a dark area of the field of view, ambient noise may be less than that of the reflected signal. In one example, ambient light may include light that reaches the LiDAR system directly from an external light source (e.g., the sun, car headlights, electric lighting devices). As another example, ambient light may include light from an external light source that reaches the LIDAR system after being deflected (e.g., reflected) by objects within the FOV (e.g., reflection of light from metallic or non-metallic surfaces, deflection by the atmosphere, glass, or other transparent or opaque objects).

[0304] The system and method of this disclosure can collect data pixel by pixel (for example, for the detection unit 106). Furthermore, a LIDAR system according to an embodiment of this disclosure can handle noise from various sources, and this can also be done pixel by pixel.

[0305] As used herein, the term “pixel” is used broadly to include portions of the FOV that are processed into elements of a model obtained from objects within the FOV of a LiDAR system. For example, when processing sensor detection data to give a point cloud model, “pixels” of the FOV may correspond to portions of the FOV that are converted into single data points of the point cloud model. In one example, the dimensions of a pixel can be given using a solid angle or two angles (e.g., φ and θ) of its angular size. In some embodiments, a single “pixel” of the FOV can be detected by multiple sensors (e.g., multiple SiPM detectors) to give a number of corresponding data points of a 3D model. In a scanning system, pixels of the FOV may have substantially the same angular size as the laser beam projected onto the scene, or (e.g., if the same beam covers several pixels) may be smaller than the angular size of the beam. Pixels the same size as the laser beam spot means that most of the photons of the laser beam emitted within a portion of the FOV (e.g., more than 50%, more than 70%, more than 90%, etc.) are defined as each pixel. In some embodiments, any two pixels in the FOV may be completely non-overlapping. However, as an optional choice, some pairs of pixels may partially overlap.

[0306] The systems and methods of this disclosure may enable the estimation, reduction, and, in some cases, removal of noise by, for example, changing the detector (e.g., the detection unit 106 in Figures 4A to 4C). Figure 17 shows an exemplary method 1700 for changing the sensor sensitivity in a LiDAR system. Method 1700 can be performed by at least one processor (e.g., the processor 118 of the processing unit 108 of the LiDAR system 100 shown in Figure 1A, and/or two processors 118 of the processing unit 108 of the LiDAR system shown in Figure 2A).

[0307] In step 1701, the processor 118 controls at least one light source (e.g., light source 112 in Figure 1A, the laser diode 202 of light source 112 in Figure 2A, and/or multiple light sources 102 in Figure 2B) so that the beam can be varied during scanning of the field of view (e.g., the field of view 120 in Figures 1A and 2A). For example, the processor 118 can vary the timing of pulses from at least one light source. Alternatively or simultaneously, the processor 118 can vary the length of pulses from at least one light source. In another example, the processor 118 can vary the magnitude of pulses from at least one light source (e.g., length or width, or by changing the cross-sectional area in other ways) instead or simultaneously. In yet another example, the processor 118 can vary the amplitude and/or frequency of pulses from at least one light source instead or simultaneously. In yet another example, the processor 118 can change the parameters of continuous wave (CW) or quasi-CW light emission (e.g., its amplitude, modulation, phase, etc.).

[0308] In some embodiments, the field of view (e.g., the field of view 120 in Figures 1A and 2A) may include at least a first and a second portion. For example, the first and second portions may include half, a quarter, or other proportions of the area covered by the field of view. In other examples, the first and second portions may include irregular portions rather than symmetrical and/or any proportion of the area covered by the field of view. In yet another example, the first and second portions may include discontinuous portions of the area covered by the field of view. In some examples, the first portion of the FOV may be one FOV pixel, and the second portion of the FOV may be another pixel. In yet another example, the first portion of the FOV may include several FOV pixels, and the second portion of the FOV may include different groups of the same number of pixels. In some embodiments, the first and second portions of the FOV may partially overlap. Alternatively, the first and second portions may be completely non-overlapping.

[0309] Step 1701 may further include controlling at least one optical deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or one-way deflector 214 in Figure 2B) to scan the field of view. For example, processor 118 may scan the field of view by mechanically moving at least one optical deflector. Alternatively, or simultaneously, processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector to scan the field of view.

[0310] In some embodiments, a single scan cycle of the field of view may include moving at least one optical deflector so that at least one deflector is positioned at a plurality of different instantaneous positions during the scan cycle (for example, the deflector is controlled to move from or through one instantaneous position to another instantaneous position during the scan of the LIDAR FOV). For example, at least one optical deflector may move continuously or discontinuously from one of the plurality of positions to another during the scan cycle (optionally, additional positions and/or repetitions may also be used).

[0311] In such embodiments, the processor 118 can coordinate at least one optical deflector with at least one light source such that, when at least one optical deflector is positioned at a particular instantaneous location, the optical beam is deflected by at least one optical deflector from at least one light source toward the field of view, and reflections from objects in the field of view are deflected by at least one optical deflector toward at least one sensor. Thus, at least one optical deflector can guide the optical beam toward the field of view and receive reflections from the field of view. For example, in the examples shown in Figures 1A, 2B, and 2C, the deflector guides the optical beam toward the field of view and receives reflections from the field of view. In some embodiments, the reflection may be caused by the optical beam guided toward the field of view. In other embodiments, the optical beam from at least one light source can be guided toward the field of view by at least one optical deflector separate from at least one other optical deflector that receives reflections from the field of view. For example, in the example shown in Figure 2A, one deflector guides the light beam towards the field of view, while another deflector receives the reflected light from the field of view.

[0312] In step 1703, the processor 118 receives a signal for each pixel from at least one sensor (e.g., the detection unit 106 in Figures 4A to 4C). For example, the signal may represent at least one of ambient light and light from at least one light source reflected by an object in the field of view. As described above, in some embodiments, such as when the object is dark and/or far away, ambient light may occupy a larger portion of the signal than reflected light. In other embodiments, such as when the object is bright and/or close, ambient light may occupy a smaller portion of the signal than reflected light.

[0313] The received signal may further represent at least one of the following: ambient light, light from at least one light source reflected by an object in the field of view, and noise associated with at least one sensor. For example, background noise and amplification noise may be combined with ambient light and/or reflected light in the signal. In particular, the signal from at least one sensor may include noise generated by the amplification electronic equipment.

[0314] The received signals may be related to various parts of the field of view. For example, at least one signal may be related to a first part of the field of view, and at least one other signal may be related to a second part of the field of view. In some embodiments, each signal may be related to a specific part of the field of view. In other embodiments (for example, in embodiments where parts of the field of view have overlapping sections), some and/or all signals may be related to multiple parts of the field of view.

[0315] In some embodiments, step 1703 may further include receiving signals for different pixels at different time points. For example, if at least one deflector is moved during the scan cycle as described above, the processor 118 may receive signals corresponding to different pixels at different time points depending on when at least one deflector is in a particular instantaneous position.

[0316] In step 1705, the processor 118 estimates the noise in at least one signal associated with the first portion of the field of view. The processor 118 can estimate the noise in at least one signal using a variety of noise estimation techniques, individually or in combination. Examples of noise estimation techniques are discussed below with reference to Figures 18 and 19.

[0317] In some embodiments, the processor 118 can estimate noise in each portion of the field of view based on reflections associated with a single position of at least one optical deflector (each portion may be less than 10%, less than 5%, less than 1%, or less than 0.1% of the field of view). For example, the processor 118 can extrapolate the estimated noise from a single position to other positions in the same portion. In some embodiments, extrapolation may include copying the estimated noise from a single position to other positions.

[0318] In other embodiments, extrapolation may include applying one or more functions to the estimated noise from a single location to generate an output of estimated noise for other locations. For example, the functions may depend on the distance between the other locations and the single location, the difference between the actual and/or predicted brightness of the other locations and the single location, the difference between previously estimated noise at the other locations and the currently estimated noise at the single location, etc. The function may output the estimate at the other locations directly, or it may output adjustment coefficients (e.g., for addition, subtraction, multiplication, etc.) to convert the estimated noise at the single location to estimates at the other locations, or it may perform convolution or other operations on the estimated noise at the single location to generate estimates or adjustment coefficients at the other locations. Similarly, in some examples, the processor 118 may also estimate the noise at a single location based on noise estimates (or initial signals) from multiple other parts of the FOV, for example, by averaging noise estimates from locations surrounding the FOV portion.

[0319] In some embodiments, each portion may comprise less than 10% of the field of view. In some embodiments, each portion may comprise less than 5% of the field of view. For example, each portion may comprise less than 1% of the field of view. As another example, each portion may comprise less than 0.1% of the field of view.

[0320] Alternatively, or simultaneously, the processor 118 may estimate noise in the signals associated with a particular portion of the field of view based on a comparison of signals associated with that portion of the field of view received in at least one previous scan cycle. For example, the processor 118 may apply one or more functions to at least one previous signal to generate an output of estimated noise for other locations. For example, the function may depend on the time between the previous signal and the current signal, the difference between the actual and/or predicted brightness of the previous signal and the current signal, the noise previously estimated in the previous signal, etc. The function may directly output the noise estimate for the current signal, or it may output an adjustment coefficient (for example, addition, subtraction, multiplication, etc.) to convert the estimated noise of the previous signal to the estimate of the current signal, or it may perform convolution or other operations on the estimated noise of the previous signal to generate the estimate or adjustment coefficient for the current signal.

[0321] In step 1707, the processor 118 can change the sensor sensitivity to reflections associated with the first portion of the field of view based on the noise estimation in the first portion of the field of view. For example, the sensor sensitivity may be based on a signal threshold. In some embodiments, the processor 118 can increase the signal threshold of the first portion relative to the signal threshold of the second portion. The processor 118 can do this, for example, when the noise estimation of the first portion is greater than the noise estimation of the second portion. Therefore, the larger the signal threshold of the first portion, the greater the estimated noise that can be removed.

[0322] In some embodiments, the sensor sensitivity can be modified in one or more detectors of the sensor. Alternatively, or simultaneously, the sensor sensitivity can be modified in the processor 118. For example, the signal threshold can be modified with respect to pre-processed or post-processed data. In one example, the sensor outputs analog data, which can be converted to digital sampling (e.g., amplitude versus time). After the digital sampling is correlated with a function representing the expected signal (e.g., by convolution, etc.) (as described later with reference to Figure 18), a signal threshold can be applied to the output of the correlation.

[0323] In some embodiments, the processor 118 can change the sensor sensitivity to reflections related to a portion of the FOV by changing the operating parameters of the processor 118. Changing the operating parameters in such cases can modify the sensor sensitivity by changing the sensitivity of detection to signal levels and/or noise levels acquired by at least one sensor. For example, the processor 118 can change the sensor sensitivity by changing the post-convolution threshold, as discussed in the previous paragraph (e.g., steps 1707 and/or 17011). However, in addition to or instead of this, it is also possible for the processor 118 to change other operating parameters of the processor 118 in response to noise levels in order to change the sensor sensitivity.

[0324] As an additional example, the processor 118 can estimate the noise level due to background noise and/or amplified noise, and change the sensor sensitivity to have a minimum threshold higher than this estimated noise level. Therefore, the estimated noise can be minimized, if not eliminated, by setting the minimum threshold accordingly.

[0325] In some embodiments, the processor 118 can change the sensor sensitivity to reflections associated with a portion of the field of view (e.g., a first portion) corresponding to a single instantaneous position of at least one optical deflector. For example, the processor 118 can change the sensor sensitivity only during the time when at least one optical deflector is at a particular instantaneous position. In other embodiments, the processor 118 can change the sensor sensitivity to reflections associated with a portion of the field of view (e.g., a first portion) corresponding to multiple instantaneous positions of at least one optical deflector. For example, the processor 118 can change the sensor sensitivity at various times when at least one optical deflector is at different positions among the multiple instantaneous positions. In some embodiments, the changed sensitivity may be equivalent for the multiple instantaneous positions. That is, the processor 118 can similarly change the sensor sensitivity during the time when at least one optical deflector is at one of the multiple instantaneous positions. In other embodiments, the changed sensitivity may differ for the multiple instantaneous positions. In other words, the processor 118 can change the sensor sensitivity so that when at least one optical deflector is at one of the multiple instantaneous positions, it is different from when at least one optical deflector is at another of the multiple instantaneous positions.

[0326] Alternatively, or simultaneously, step 1707 may further include individually modifying the sensor sensitivity to reflections associated with the first and second parts such that, for the same amount of light projected onto the first and second parts, the detection distance associated with the first part is greater (e.g., at least 50%) than the detection distance associated with the second part. Thus, the detection distance can be increased by increasing the sensor sensitivity of the first part (and/or by decreasing the minimum threshold and/or increasing the maximum threshold).

[0327] Alternatively, or simultaneously, step 1707 may further include individually modifying the sensor sensitivity to reflections associated with the first and second parts such that, for the same amount of light projected onto the first and second parts, the resolution associated with the first part is greater than the resolution associated with the second part. Thus, the resolution can be increased by increasing the sensor sensitivity of the first part (and/or by reducing the minimum threshold and/or increasing the maximum threshold).

[0328] In some embodiments, step 1707 can be performed only after step 1705 has been completed. Furthermore, in some embodiments, a change in sensor sensitivity for a portion of the FOV (e.g., steps 1707 and 1711) can be performed after the corresponding noise estimation for each portion of the FOV underlying this change (e.g., steps 1705 and 1709, respectively) without measuring other portions of the FOV. Similarly, in some embodiments, a change in sensor sensitivity for a portion of the FOV (e.g., steps 1707 and 1711) can be performed after the corresponding noise estimation for each portion of the FOV underlying this change (e.g., steps 1705 and 1709, respectively) without moving at least one deflector to a different instantaneous position.

[0329] In step 1709, the processor 118 estimates noise in at least some of the signals associated with the second portion of the field of view. As discussed above, the processor 118 can estimate noise in at least some of the signals using a wide variety of noise estimation techniques individually or in combination. The processor 118 may use one or more of the same noise estimation techniques in steps 1705 and 1709, or it may use one or more different noise estimation techniques in steps 1705 and 1709. In some embodiments, the processor 118 may determine that a particular noise estimation technique is appropriate for the first portion and a different noise estimation technique is appropriate for the second portion. For example, the processor 118 may determine that the noise contribution from amplification is larger in the first portion. This could be because, for example, the amplification is greater in the first portion than in the second portion because it is darker. In such an example, the processor 118 can use different techniques to estimate the noise in the first portion that is the source of the large amplification noise. Examples of noise estimation techniques are discussed below with reference to Figures 18 and 19. Optionally, the noise estimation of the second portion of the FOV in step 1709 may depend on the result of step 1705. Alternatively, the noise estimations of the first and second portions of the FOV (steps 1705 and 1709, respectively) may be completely independent and unrelated to each other.

[0330] In some embodiments, the processor 118 may report the noise estimates generated in steps 1705 and/or 1709 to another system (e.g., an external server). Furthermore, the processor 118 may report one or more noise-indicating parameters and/or one or more noise-related parameters based on each noise estimate obtained by the processor 118. Each parameter may be specific to a portion of the FOV, or specific to a larger portion of the FOV that includes each portion of the FOV. Examples of reported parameters, but not limited to, include noise estimates, one or more sensitivity settings, detection distance, detection quality index, etc. In some embodiments, the report may include one or more parameters indicating the modified sensor sensitivity from steps 1707 and/or 1711.

[0331] In step 1711, the processor 118 can change the sensor sensitivity to reflections associated with the second portion of the field of view based on the noise estimation in the second portion of the field of view. For example, the sensor sensitivity may include a signal threshold. In some embodiments, the processor 118 can increase the signal threshold of the second portion relative to the signal threshold of the first portion. The processor 118 can do this, for example, when the noise estimation of the second portion is greater than the noise estimation of the first portion. Therefore, the larger the signal threshold of the first portion, the greater the estimated noise that can be removed.

[0332] As an example, the processor 118 can estimate the noise level due to ambient noise and/or amplified noise, and modify the sensor sensitivity to have a minimum threshold higher than this estimated noise level. Therefore, the estimated noise can be minimized, if not eliminated, by setting a minimum threshold. The modified sensor sensitivity for reflections related to the second part may differ from the modified sensor sensitivity for reflections related to the first part.

[0333] In some embodiments, as discussed above, the processor 118 can change the sensor sensitivity to reflections associated with a portion of the field of view (e.g., a second portion) corresponding to a single instantaneous position of at least one optical deflector. In other embodiments, as discussed above, the processor 118 can change the sensor sensitivity to reflections associated with a portion of the field of view (e.g., a second portion) corresponding to multiple instantaneous positions of at least one optical deflector.

[0334] In some embodiments, the processor 118 can change the sensor sensitivity to a first reflection associated with a first portion received in a first scan cycle, and to a second reflection associated with a second portion in a second scan cycle. For example, steps 1705 and 1707 can be performed in the first scan cycle, and steps 1709 and 1711 can be performed in the second scan cycle. In some embodiments, the first scan cycle can be performed chronologically before the second scan cycle, or the second scan cycle can be performed chronologically before the first scan cycle.

[0335] In other embodiments, the processor 118 can change the sensor sensitivity to the first reflection associated with the first part and the second reflection associated with the second part. The first and second reflections are received in a single scan cycle. For example, steps 1705 and 1707 can be performed in the same scan cycle as steps 1709 and 1711.

[0336] Steps 1707 and/or 1711 may further include making different changes to the sensor sensitivity to reflections associated with the first and second parts so that, after detecting an external light source at a first distance in the first part, an object at a second distance greater than the first distance can be detected in the second part. Thus, by increasing the sensor sensitivity of the second part (and/or decreasing the minimum threshold and/or the maximum threshold), external light sources in the first part that may introduce noise into the second part can be compensated for. In another example, after detecting an object at a first distance in the first part, the processor 118 may change the sensor sensitivity in the second part to enable detection of objects further away than this object. In yet another example, the processor 118 may change the sensor sensitivity so that an object that was not visible in the first part due to increased noise can be detected in the second part.

[0337] As a further example, steps 1707 and/or 1711 may further include making different changes to the sensor sensitivity to reflections associated with the first and second parts so that, after detecting an external light source at a first distance in the second part, an object at a second distance greater than the first distance can be detected in the first part. Thus, the sensor sensitivity of the first part can be increased to compensate for an external light source in the second part that may introduce noise into the first part.

[0338] Alternatively, or simultaneously, step 1711 may further include individually modifying the sensor sensitivity to reflections associated with the first and second parts such that, for the same amount of light projected onto the first and second parts, the detection distance associated with the second part is greater than the detection distance associated with the first part. Thus, the detection distance can be increased by increasing the sensor sensitivity of the second part (and/or by decreasing the minimum threshold and/or increasing the maximum threshold).

[0339] Alternatively, or simultaneously, steps 1707 and 1711 may further include individually modifying the sensor sensitivity to reflections associated with the first and second parts such that, for the same amount of light projected onto the first and second parts, the resolution associated with the second part is greater than the resolution associated with the first part. Thus, the resolution can be increased by increasing the sensor sensitivity of the second part (and/or by reducing the minimum threshold and/or increasing the maximum threshold).

[0340] For each portion of the FOV whose sensitivity setting has been modified by the processor 118 (for example in steps 1707 and/or 1711), the processor 118 can detect objects in that portion of the FOV using the modified sensitivity setting. For each portion of the FOV whose sensitivity setting has been modified by the processor 118, the processor 118 can generate data points in a model of the scene contained in the FOV (for example, a 2D or 3D model such as a point cloud model).

[0341] Method 1700 may include additional steps. For example, Method 1700 may further include changing the sensor sensitivity to reflections associated with a third portion of the field of view different from the first and second portions, based on the estimation of noise in the first portion. For example, as described above, the processor 118 can extrapolate the noise estimated from the first portion to the third portion. Alternatively, Method 1700 may further include changing the sensor sensitivity to reflections associated with a third portion of the field of view different from the first and second portions, based on the estimation of noise in the first and second portions.

[0342] In some embodiments, extrapolation may include copying the estimated noise from the first and/or second parts to the third part. In other embodiments, extrapolation may include applying one or more functions to the estimated noise from the first and/or second parts to generate an output of the estimated noise for the third part. For example, the function may depend on the distance between the first and/or second part and the third part, the difference between the actual and/or predicted brightness of the first and/or third parts, the difference between the noise previously estimated in the third part and the noise currently estimated in the first and/or second part, etc. The function may output the estimate for the third part directly, or an adjustment coefficient (e.g., for addition, subtraction, multiplication, etc.) for converting the estimated noise of the first and/or second parts to the estimate for the third part, or it may perform convolution or other operations on the estimated noise of the first and/or second parts to generate the estimate or adjustment coefficient for the third part.

[0343] In addition to changing the sensor sensitivity, the processor 118 may also change one or more operating characteristics of at least one light source for each part of the FOV based on the noise estimation in each part of the FOV. For example, the processor 118 may change the light source parameters associated with the first part (e.g., pulse timing, pulse length, pulse size, pulse amplitude, pulse frequency, etc.) such that the amount of light beam directed to the first part is greater than the amount of light beam directed to at least one other part of the field of view. Alternatively, the processor 118 may change the light source parameters associated with the first part such that the amount of light beam directed to the first part is less than the amount of light beam directed to at least one other part of the field of view. The processor 118 may change the light source parameters based on the noise estimation in steps 1705 and/or 1709. For example, the processor 118 may decide to reduce the amount of light beam directed to the first part because there is little noise in the reflection from the first part. As another example, the processor 118 may decide to increase the amount of light beam directed to the first part because there is a lot of noise in the reflection from the first part. Therefore, independently of or in combination with changing the sensor sensitivity, the processor 118 can further address noise by varying the light beam directed to the field of view.

[0344] As another example, the processor 118 can increase the amount of light projected onto the first part compared to the amount of light projected onto the second part. The processor 118 can do this, for example, when the noise estimate in the first part is greater than the noise estimate in the second part. Therefore, as described above, the processor 118 can deal with noise by varying the amount of light projected. Alternatively, the processor 118 can decrease the amount of light projected onto the first part compared to the amount of light projected onto the second part. The processor 118 can do this, for example, when the noise estimate in the first part is greater than the noise estimate in the second part. Therefore, as described above, the processor 118 can deal with noise by varying the amount of light projected.

[0345] Numerous noise estimation techniques can be used in conjunction with method 1700 in Figure 17. Figure 18 shows an example of a received signal along with a function for estimating the predicted signal. As shown in Figure 18, the received signal 1801 represents the entire signal of a portion of the field of view, including the received noise. The received signal 1801 is a discretized measurement and is therefore represented as a function with discontinuities in the gradient.

[0346] As further shown in Figure 18, function 1803 can represent an estimate of a noise-free predicted signal. For example, function 1803 can be constructed based on past measurements and/or known characteristics that have been measured. For example, function 1803 can be constructed by processor 118 based on signals previously received in a portion of the field of view and/or on the characteristics of an object in a portion of the field of view (e.g., the known location of the object, the known brightness of the object, etc.). Processor 118 can derive the characteristics of the object based on the previously received signals.

[0347] To adjust the received signal 1801 to address noise, the processor 118 can adapt the received signal 1801 to function 1803. In other embodiments, function 1803 may represent a function that performs convolution or other operations on the received signal 1801 to remove noise.

[0348] Figure 19 shows an example of a received signal, along with a function for estimating the predicted signal. As shown in Figure 19, similar to Figure 18, the received signal 1901 represents the entire signal, including the received noise. The received signal 1901 is a discretized measurement and is therefore represented as a function with discontinuities in the gradient.

[0349] As further shown in Figure 19, function 1903 can represent an estimate of expected noise. For example, function 1903 can be constructed based on past measurements and/or known characteristics of at least one sensor. For example, function 1903 can be constructed by processor 118 based on signals previously received in a portion of the field of view and/or characteristics of at least one sensor (e.g., known background noise, known amplified noise, etc.). Processor 118 can derive the characteristics of at least one sensor based on manufacturing specifications and/or previous measurements.

[0350] To adjust the received signal 1901 to address noise, the processor 118 may subtract function 1903 from the received signal 1901. In other embodiments, function 1903 may represent a function that performs convolution or other operations on the received signal 1901 to estimate noise from the received signal 1901.

[0351] Systems and methods relating to this disclosure may include any suitable noise estimation technique and are not limited to the examples in Figures 18 and 19.

[0352] Variable beam allocation within Lidar FOV to improve detection in the region of interest

[0353] By detecting laser beam reflections from real-world surroundings within the environment, the LIDAR system 100 can generate a 3D reconstruction of objects in the environment within the LIDAR system's FOV. Such LIDAR systems can have applications across a wide range of technologies. In particular, one such technology is the field of autonomous or semi-autonomous vehicles. As interest in autonomous driving technology grows, LIDAR systems are increasingly seen as important components for the operation of autonomous vehicles. As LIDAR systems are adopted in the automotive industry, the system must provide a reliable reconstruction of surrounding objects. Therefore, as the operational capabilities of LIDAR systems improve, they can become important in greatly contributing to the realization of autonomous navigation. Such improvements may include increased scan resolution, expanded detection range, and/or increased receiver sensitivity. Such performance improvements can be achieved by using high-energy lasers. However, the use of high-energy lasers is currently impractical for various reasons, such as cost and operating temperature in automotive environments. The maximum irradiation power of LiDAR systems is limited by the need to make them safe for the eyes (e.g., to avoid potential damage to the retina or other parts of the eye if projected light emission is absorbed by the eye). Therefore, there is a need for LiDAR systems that comply with eye safety regulations while simultaneously providing performance characteristics that enhance the system's usefulness in integrated technology platforms (e.g., autonomous vehicles).

[0354] Generally, the disclosed LiDAR systems and methods can improve system performance while complying with eye safety regulations. For example, the disclosed systems can improve the quality of detection and subsequent reconstruction in a region of interest (ROI) by allocating variable optical power in the field of view of the LiDAR system. By allocating power to the field of view based on the level of interest in a particular ROI, the efficiency of the system can be improved while maintaining high quality and useful data from the ROI. Furthermore, separating the FOV into different levels of ROI and allocating power to the field of view based on the level of interest in a particular ROI can offer many advantages. For example, this allows the LiDAR system to utilize its optical budget more efficiently by avoiding the expenditure of optical projection and detection resources in areas of low interest. It can also reduce interference with the surrounding environment (e.g., other LiDAR systems or pedestrians on the street). Furthermore, it can simplify the computational complexity of preparing and analyzing the results and reduce the associated costs. The region of interest may correspond to the area or sub-area of the LiDAR FOV. In some cases, the ROI may be determined to include a rectangular region of the LiDAR FOV, or a region of the FOV having any other arbitrary shape. In some embodiments, the ROI extends in an irregular pattern, which may include multiple discontinuous segments across the entire LiDAR FOV. Furthermore, the ROI does not need to be aligned with a specific axis of the FOV and can be defined in a free form relative to the FOV.

[0355] According to the disclosed embodiments, the LIDAR system 100 in Figure 22 may include, for example, at least one processor 118 in a processing unit 108. The at least one processor 118 can control at least one light source 102 so that the light intensity can be varied in scanning the field of view 120 using light from at least one light source. The at least one processor 118 can also control at least one light deflector 114 to deflect light from at least one light source for scanning the field of view 120. Furthermore, in some embodiments (for example, as shown in Figure 3), the at least one light deflector 114 may include a pivotable MEMS mirror 300. The at least one processor 118 can obtain identification of at least one distinct region of interest within the field of view 120. Subsequently, at least one processor 118 may increase the allocation of light to at least one clear region of interest compared to other regions, such that, after the first scan cycle, the light intensity of at least one subsequent second scan cycle at locations associated with at least one clear region of interest is higher than the light intensity of the first scan cycle at locations associated with at least one clear region of interest. For example, increasing the light intensity can be achieved by increasing the power per solid angle, increasing the irradiance to the FOV portion, emitting additional light pulses, increasing the power per pixel, emitting additional photons per unit time, increasing the total energy over a specific time period, emitting additional photons per data point in the generated point cloud model, increasing the total energy per data point in the generated point cloud model, or any other characteristic that increases the luminous flux.

[0356] In a disclosed system such as the LIDAR system 100, at least one processor 118 can control at least one light source 112. For example, at least one processor 118 can control at least one light source 112, for example, LIDAR
In response to the level of the associated region of interest within the FOV, more or less luminous flux can be generated. Areas of low interest within the FOV (e.g., areas 30 meters from the road or the skyline) can be allocated low levels of luminous flux or no luminous flux at all. However, other areas of high interest (e.g., areas including pedestrians or moving vehicles) can be allocated high levels of luminous flux. Such allocation avoids the expenditure of light projection and detection resources in areas of low interest while improving resolution and other performance characteristics in areas of high interest. Furthermore, light allocation can be varied not only per object but also per object-part. For example, in some cases, having clearly defined information regarding the location of object edges (e.g., the outer edge or exterior of an object like a vehicle) is more important and useful. Therefore, it may be desirable to allocate more luminous flux to the FOV region where the vehicle's edge is located, and less luminous flux to the FOV region containing parts of the object within its exterior. As an example for illustrative purposes only, as shown in Figure 5C, at least one processor 118 can be configured to emit two light pulses of projection light for use in analyzing an FOV region that includes the edges of an object (e.g., a rounded rectangular object with coordinate units (B,1), (B,2), (C,1), (C,2) in Figure 5C). On the other hand, in an FOV region related to the interior of an object, or within an exterior defined at least by the detected outer edge, such as the central part of the rounded rectangular shown in Figure 5C (coordinate (C,5)), less light flux can be supplied to these regions. As another example for illustrative purposes shown in Figure 5C, after the first scan cycle, only one light pulse is supplied to the interior region of the rounded rectangular. It should be noted that any suitable technique can be used to increase the light flux for a particular region of interest. For example, the light flux can be increased by emitting additional light pulses, emitting longer-duration light pulses or continuous waves, increasing the light power, etc.

[0357] At least one processor 118 can control various aspects of the movement of at least one optical deflector 114 (e.g., angular orientation of the deflector, rotation angle of the deflector along two or more axes) to change the deflection angle, for example. Furthermore, at least one 118 can control the movement speed of at least one deflector 114, the amount of time at least one deflector is stopped at a specific instantaneous position, the translation of at least one deflector, etc. By controlling at least one optical deflector 114, at least one processor 118 can guide projected light to one or more regions of interest in the LIDAR FOV in a unique way that allows the LIDAR system to scan regions of interest in the field of view at a desired detection sensitivity level, signal-to-noise ratio, etc. As described above, the optical deflector 114 may include a pivotable MEMS mirror 300, and at least one processor 118 can control the deflection angle, deflection speed, stop time, etc. of the MEMS mirror. These factors can all affect the FOV range and/or the frame rate of the LiDAR system. The processing unit 108 can control at least one optical deflector 114 to change the deflection angle of the light emitted by the LiDAR system 100 and/or the light reflected from the scene within the FOV and returned to the LiDAR system 100.

[0358] During operation, at least one processor 118 may acquire, or otherwise determine or identify, at least one region of interest within the field of view 120. The identification of at least one region of interest within the field of view 120 may be determined by analysis of signals collected from the detection unit 106, by detecting one or more objects, parts of objects, or types of objects within the FOV 120, based on any of the various detection characteristics recognized during the scan of the FOV 12, and/or based on information received (directly or indirectly) from the host 210, or based on any other appropriate criteria.

[0359] As shown in Figure 22, the processing unit 108 can receive information not only from the detection unit 106 and other components of the LiDAR system 100, but also from various other systems. In some embodiments, the processing unit 108 can receive input from, for example, the GPS 2207, the vehicle navigation system 2201, the radar 2203, another LiDAR unit 2209, one or more cameras 2205, and/or any other arbitrary sensors or information systems. In addition to determining one or more specific regions of interest within the FOV 120 based on detections from the LiDAR system 100, the processing unit 108 can identify one or more regions of interest within the FOV 120 based on one or more outputs from the GPS 2207, the vehicle navigation system 2201, the radar 2203, the LiDAR unit 2209, the cameras 2205, etc. At least one region of interest may include a portion, area, section, region, subregion, pixel, etc., related to the FOV 120.

[0360] After identifying at least one region of interest within the field of view 120, at least one processor 118 may determine a new scan scheme or modify an existing scan scheme in relation to the light projection and subsequent detection of this at least one region of interest. For example, after identifying at least one region of interest, at least one processor 118 may determine or modify one or more parameters related to the light projector 112 (as described above). For example, at least one processor 118 may determine the amount of light flux provided to a particular region of interest, the number of light pulses to project, the power level of the light projection, the projection time of the continuous wave, or any other characteristics that may affect the amount of light flux given to a particular identified region of interest.

[0361] At least one processor 118 can determine specific instantaneous positions to which the deflector 114 moves during scanning of at least one region of interest. The processing unit 108 can also determine the stop times associated with the determined instantaneous positions and/or the movement characteristics for moving the deflector 114 between the determined instantaneous positions.

[0362] Before identifying a region of interest within the Lidar FOV, in some embodiments, a default light intensity can be projected onto each region of the FOV during scanning. For example, if all parts of the FOV have the same importance/priority, a default light intensity can be assigned to each part of the FOV. Delivering a default light intensity to each region of the FOV may include, for example, controlling the available light sources so that each FOV region is provided with a similar number of light pulses having a similar amplitude during scanning. However, after identifying at least one region of interest, at least one processor 118 may increase the light intensity supplied to at least one region within the region of interest compared to one or more other regions of the FOV. In the example illustrating Figure 5C, sector II can represent an identified region of interest (for example, sector II may be determined to contain high-density objects, specific types of objects (such as pedestrians), objects within a specific distance range relative to the LIDAR system (e.g., within 50m or 100m), objects determined to be near or within the host vehicle's path, or any other characteristic indicating a region of greater interest than at least one other area within the FOV 120). Considering the state of sector II as the region of interest, more light can be supplied to the small areas contained within sector II than to the rest of the FOV. For example, as shown in Figure 5C, three light pulses can be assigned to the small areas within sector II (areas other than those determined to be occupied by objects). Other areas of less interest, such as sectors I and III, may receive fewer pulses, such as one or two pulses each.

[0363] In some embodiments, designation of the region of interest may depend on the distance of the target object to the LIDAR system. The further the target object is from the LIDAR system, the longer the laser pulse must travel, which can lead to greater potential laser signal loss. Therefore, distant targets may require higher energy light emission than nearby targets to maintain a desired signal-to-noise ratio. Such light energy can be achieved by modulating the power output, pulse width, pulse repetition rate, or any other parameter affecting the output energy of the light source 112. Nearby objects are easily detectable, and therefore, in some cases, designation of the region of interest may not be appropriate for such nearby objects. On the other hand, more distant objects may require greater light energy to achieve a suitable signal-to-noise ratio that enables target detection. For such distant objects, designation of the region of interest and increasing the light energy supplied to each region of the FOV where those objects are located may be appropriate. For example, in Figure 5C, a single light pulse can be assigned to detect a first object at a first distance (e.g., any of the near-field objects located near the bottom of Figure 5C), two light pulses can be assigned to detect a second object at a second distance greater than the first distance (e.g., a mid-field object with rounded corners), and three light pulses can be assigned to detect a third object at a third distance greater than both the first and second distances (e.g., a far-field triangular object).

[0364] On the other hand, the available laser energy levels may be limited by eye safety regulations and by potential thermal and electrical limitations. Therefore, in order to ensure eye safety during use of the LIDAR system 100, at least one processor 118 may set an upper limit (cap) on the accumulated light in at least one distinct region of interest based on an eye safety threshold. For example, at least one processor 118 may be programmed to limit the amount of light projected (e.g., over a specific time period) within a distinct region of the FOV in order to comply with eye safety limits. Herein, the upper limit may refer to a threshold amount of light over a specific time period corresponding to an upper limit of light for eye safety (or a limit set lower than the upper limit to give a safety margin). At least one processor 118 may control at least one light source 112 so as not to exceed the upper limit during operation of the LIDAR system 100. Therefore, for example, a region of low interest may be defined in an area where the eyes are likely to be located. This is, for example, the driver's area of a vehicle or an area at a certain height relative to sidewalks and bicycle lanes. Low interest in these areas does not necessarily mean that detection is less important in these areas compared to other areas; rather, it means that detection is less important (or of less interest) than maintaining the safe operation of the system.

[0365] At least one processor 118 can determine the identification of at least one distinct region of interest based on information received from a suitable source. In some embodiments, at least one processor 118 can receive the identification of at least one region of interest from at least one sensor configured to detect reflections of light associated with a first scan cycle. Such a sensor may include, for example, a sensing unit 106 comprising one or more photosensitive objects and one logic device (e.g., a processor, DSP, gate array, etc.) configured to generate at least one identifier of a region of interest or a potential region of interest. In some embodiments, at least one processor 118 can identify a region of interest based, for example, the output of the sensing unit 106. In other embodiments, at least one processor 118 can receive the identification of a region of interest or at least one index of a region of interest from one or more sources around the LIDAR system 100. For example, as shown in Figure 22, such identifications or indicators may be received from a vehicle navigation system 2201, radar 2203, camera 2205, GPS 2207, or another LiDAR 2209. Such indicators or identifiers can be associated with mapped objects or features, directional headings, etc., from the vehicle navigation system, or with one or more objects or groups of objects detected by radar 2203, LiDAR 2209, or camera 2205, etc. Optionally, at least one processor 118 can determine the identification of at least one distinct region of interest based on information received from a remote source located outside the platform on which the LiDAR system 100 is installed. For example, if the LiDAR system 100 is used for mapping an environment by multiple vehicles, it can receive instructions on how to determine the region of interest from a server that coordinates the operation of various vehicles.

[0366] In some examples, the region of interest can be determined during or after the first scan of the FOV, and the increase in light to the identified region of interest can be achieved during one or more subsequent scans of the FOV. As a result, the light intensity of at least one subsequent second scan cycle at locations associated with at least one clear region of interest can be higher than the light intensity of the first scan cycle at locations associated with at least one clear region of interest. In some examples, at least one subsequent second scan cycle of the FOV includes multiple subsequent second scan cycles. In such cases, the total light intensity over multiple second scan cycles in the area of at least one clear region of interest of the FOV can be greater than the total light intensity over multiple second scan cycles in other regions of no interest.

[0367] By identifying regions of interest and increasing the amount of light supplied to those regions compared to regions of low interest, more objects and/or objects at greater distances can be detected in the regions of interest than in the regions of low interest. For example, in the regions of interest, the reflection of projected light can determine that a first object at a first distance exists in at least one distinct region of interest. Furthermore, this first distance can be greater than a second distance at which no object was detected in the regions of non-interest.

[0368] At least one processor 118 can change the illumination resolution of at least one distinct region of interest with respect to other regions. The spatial resolution of the 3D representation of at least one distinct region of interest in at least one subsequent second scan cycle is higher than the spatial resolution of the 3D representation of at least one distinct region of interest in the first scan cycle. Furthermore, at least one processor 118 can change the illumination timing of at least one distinct region of interest with respect to other regions such that the temporal resolution of the 3D representation of at least one distinct region of interest in at least one subsequent second scan cycle is higher than the temporal resolution of the 3D representation of at least one distinct region of interest in the first scan cycle. For example, higher temporal resolution can be achieved by increasing the frame rate, though not limited to this, and vice versa.

[0369] In the example above, if more light is allocated to at least one region of interest, higher spatial and/or temporal resolution can also be obtained. On the other hand, in regions of no interest and low interest, the allocation of light to those regions is reduced, and therefore lower spatial and/or temporal resolution can be achieved. Increasing spatial and/or temporal resolution can be achieved, but is not limited to, by using higher light intensity (e.g., by reducing the area of the region of interest to increase the allocation of luminous flux per unit area). Similarly, decreasing spatial and/or temporal resolution can be achieved, but is not limited to, by using lower light intensity (e.g., by increasing the area of the region of interest to decrease the allocation of luminous flux per unit area).

[0370] Figure 20 is a flowchart of an exemplary method 2000 for detecting an object in a region of interest using a LIDAR system. In step 2001, a processor (e.g., processor 118) controls at least one light source (e.g., light source 112) to vary the light intensity in scanning a field of view (e.g., field of view 120) using light from at least one light source (e.g., light source 112). In step 2002, the processor (e.g., processor 118) controls at least one optical deflector (e.g., optical deflector 114) to deflect light from at least one light source (e.g., light source 112) to scan the field of view (e.g., field of view 120). The deflection of at least one light source in step 2002 may also affect the deflection of reflected light reaching from the field of view (e.g., field of view 120) toward at least one sensor (e.g., sensor 116). In step 2003, the processor (e.g., processor 118) obtains identification of at least one distinct region of interest within the field of view (e.g., field of view 120). Furthermore, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) can be received from at least one sensor (e.g., detection unit 106) configured to detect light reflections associated with the first scan cycle. In some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) may be based on the current driving mode of the vehicle in which the LiDAR system is deployed. In some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) may be based on an object detected in at least one distinct region of interest. In some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) can be received from at least one of GPS, a vehicle navigation system, radar, LiDAR, and a camera.

[0371] In step 2004, the processor (e.g., processor 118) increases the allocation of light to at least one distinct region of interest compared to other regions, so that after the first scan cycle, the light intensity of at least one subsequent second scan cycle in locations associated with at least one distinct region of interest is higher than the light intensity of the first scan cycle in locations associated with at least one distinct region of interest. Furthermore, the at least one subsequent second scan cycle comprises multiple subsequent second scan cycles, and the total light intensity over multiple second scan cycles in the area of at least one distinct region of interest is greater than the total light intensity over multiple second scan cycles in other non-interest regions.

[0372] In step 2005, the processor (e.g., processor 118) adjusts the light allocation so that, in a single scan cycle, more light is projected towards at least one distinct region of interest than towards other regions. Depending on the circumstances, the processor (e.g., at least one processor 118) may allocate less light to multiple regions identified as regions of no interest in at least one subsequent second scan cycle than the amount of light projected towards those regions in the first scan cycle.

[0373] Figure 21 is a flowchart of an exemplary method 2100 for detecting objects in multiple regions of interest using a LIDAR system. In step 2001, a processor (e.g., processor 118) controls at least one light source (e.g., light source 112) to vary the light intensity in scanning a field of view (e.g., field of view 120) using light from at least one light source (e.g., light source 112). In step 2002, the processor (e.g., processor 118) controls at least one optical deflector (e.g., optical deflector 114) to deflect light from at least one light source (e.g., light source 112) to scan the field of view (e.g., field of view 120). In step 2003, the processor (e.g., at least one processor 118) obtains identification of at least one distinct region of interest in the field of view (e.g., field of view 120). Furthermore, the obtained identification of at least one distinct region of interest in the field of view (e.g., field of view 120) can be received from at least one sensor (e.g., sensing unit 106) configured to detect reflections of light associated with a first scan cycle. Furthermore, in some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) may be based on the current driving mode of the vehicle in which the LiDAR system is deployed. In some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) may be based on an object detected in at least one distinct region of interest. In some embodiments, the acquired identification of at least one distinct region of interest within the field of view (e.g., field of view 120) may be received from at least one of GPS, a vehicle navigation system, radar, LiDAR, and a camera.

[0374] In step 2101, the processor (processor 118) can rank multiple regions of interest. For example, the scene can be scanned by a LiDAR system. Regions of interest in the scene can be designated as regions of non-interest (RONI) or as regions of interest (ROI) with interest levels between low and high levels. For example, road boundaries and vertical surfaces of buildings can be designated as high-interest regions (R2), pedestrians and moving vehicles can be designated as medium-interest regions (R1), and the rest of the scene can be considered as a low-interest region (R0). The skyline can be designated as a RONI (R3).

[0375] In step 2103, the processor (processor 118) allocates light based on ranking. The amount of light allocated to the highest-ranked region of interest may be greater than the amount of light allocated to lower-ranked regions of interest. For example, the power or resource allocation for the exemplary scene described above can be determined by the processor. Based on the rank, the processor may allocate the largest power to the highest-ranked region of interest R2, then to the medium-ranked region of interest R1, and the smallest to the lowest-ranked region of interest R0. For RONI R3, some power may be allocated to periodically confirm that it is still RONI. In this example, region 2 (R2) can be defined as the region of greatest interest. This can be associated with the highest quality of service, the largest laser power, the highest receiver sensitivity, the highest angular scan resolution, the highest range resolution, and the highest frame rate, suggesting the longest distance detection capabilities.

[0376] In step 2004, the processor (e.g., processor 118) increases the allocation of light to at least one distinct region of interest compared to other regions, so that after the first scan cycle, the light intensity of at least one subsequent second scan cycle in locations associated with at least one distinct region of interest is higher than the light intensity of the first scan cycle in locations associated with at least one distinct region of interest. Furthermore, the at least one subsequent second scan cycle includes multiple subsequent second scan cycles, and the total light intensity over multiple second scan cycles in the area of at least one distinct region of interest can be greater than the total light intensity over multiple second scan cycles in other non-interest regions.

[0377] In step 2005, the processor (e.g., processor 118) adjusts the light allocation so that in a single scan cycle, more light is projected towards at least one distinct region of interest than towards other regions. Depending on the circumstances, the processor (e.g., at least one processor 118) may allocate less light to multiple regions identified as regions of no interest in at least one subsequent second scan cycle than the amount of light projected towards those regions in the first scan cycle.

[0378] Adaptive Lidar Irradiation Technique Based on Intermediate Detection Results

[0379] Figure 23 shows an exemplary embodiment of a LiDAR system 2300 that emits light and detects photons reflected from the LiDAR's field of view. In some embodiments, the LiDAR system 2300 can operate as described above with reference to the LiDAR system 100. Based on the detection results, the LiDAR can generate a series of depth maps. As previously stated, the LiDAR may be capable of generating one or more different types of depth maps. The types of depth maps may be, for example, one or more of a point cloud model (PC), a polygon mesh, a depth image (holding depth information for each pixel or 2D array of the image), or any other type of 3D model of the scene.

[0380] The generated depth maps may include temporal characteristics. For example, depth maps may be generated in a time series in which different depth maps are generated at different points in time. Each depth map (which can be rephrased as a “frame”) in this sequence may be generated within the duration of a scan of the LiDAR FOV. In some embodiments, such a scan may occur within a period of several seconds, within about one second, or less than one second.

[0381] In some embodiments, the LIDAR system 2300 (which can be rephrased as "LIDAR") may have a fixed frame rate (e.g., 10 frames per second (FPS), 25 FPS, etc.) or a dynamic frame rate throughout the sequence. The frame times of different frames are not necessarily the same throughout the sequence. For example, a 10 FPS LIDAR may generate one depth map in 100 milliseconds (on average), generate the next frame in 92 milliseconds, generate a third frame in 142 milliseconds, and generate additional frames at various rates, averaging these to a 10 FPS specification.

[0382] Frame time can refer to a time span beginning with the first optical projection that is detected and generates frame detection information, and ending with the completion of each depth map ("frame"). "Frame illumination duration" is a time span beginning with the first optical projection that is detected and generates frame detection information, and ending when the last photon that is detected and affects the frame detection information is emitted (i.e., "frame illumination duration" is the first part of each frame time, followed by time for at least some processing of the frame detection information to generate each depth map). In some embodiments, all actions, processes, or events described in this disclosure as occurring within the same frame time may need to occur within the same frame illumination duration (i.e., stricter time constraints may be implemented).

[0383] In some embodiments, frame times may partially overlap (for example, processing of the Nth depth map may continue until the illumination of the (N+1)th frame), but may also be completely non-overlapping, at an optional rate. In some embodiments, there may be a time gap between the frame times of different frames.

[0384] The number of depth maps in a sequence can be three or more, but a significantly longer frame sequence may be generated by the LiDAR. For example, a sequence may contain more than 10 depth maps. For example, a sequence may contain more than 100 depth maps. For example, a sequence may contain more than 1,000 depth maps. It should be noted that a sequence does not necessarily contain all frames generated by the LiDAR. Optionally, a sequence of depth maps may contain all depth maps generated by the LiDAR between the first depth map and the last depth map in the sequence.

[0385] System 2300 may include at least a sensor interface 2350 and a light source controller 2360, but may also include additional components such as those discussed below. The sensor interface 2350 may be configured or operable to receive detection information from one or more sensors of the LiDAR (e.g., sensors 2344(1), 2344(2), and 2344(3)) indicating one or more light quantities (e.g., the number of detected photons, the stored energy of the detected light, etc.) detected by each of the one or more sensors. The light detected by the sensors may include photons emitted by the LiDAR, reflected from the scene, and returned to one or more detectors of the LiDAR, for at least some segments of the LiDAR's field of view (FOV).

[0386] The FOV of a Lidar may include several segments (two or more, up to several hundred or several thousand, and possibly more) illuminated at different time points. Each segment may include one or more items of the depth map (e.g., one or more polygons, one or more point cloud points, one or more depth image pixels) and may be covered by one or more sensors (generating one or more detection signals). In some embodiments, the segments of the FOV may include non-overlapping segments. In other embodiments, some of the segments of the FOV may partially overlap with each other. Optionally, the depth map may not include items for one or more segments (e.g., because the photons were not reflected within a possible time frame, or because the SNR was too low to detect). In such cases, the depth map may, but not necessarily, include indications of corresponding data loss.

[0387] In some embodiments, the depth map generated by the LiDAR may include depth information based on the detection of light from illuminated segments without pre-illumination processing (e.g., with regard to an optional distinction between central and peripheral segments). The depth map generated by the LiDAR may also include depth information for parts (or segments) of the FOV that are not illuminated and/or not based on light detection. For example, some items of the depth map (pixels, PC points, polygons, or parts thereof) may be based on interpolation or averaging of values determined in the illuminated portion of the FOV.

[0388] In an exemplary embodiment, the sensor interface 2350 is operable to receive preliminary detection information (from one or more sensors of the LiDAR) for each of the multiple segments of the LiDAR's field of view during each frame time of the sequence, for each segment that was emitted by the LiDAR during each frame time and reflected (or otherwise scattered) from each segment. In some segments, the light projected by the LiDAR is not reflected (e.g., if there is no target within the LiDAR's detection range), but in at least some segments, the preliminary detection information can indicate the amount of projected light reflected from the scene and detected by one or more sensors of the LiDAR. The signals generated by one or more sensors, along with the detection information (including the preliminary detection information) provided by one or more sensors, may include contributions from, for example, external radiation (e.g., sunlight, flash light, and light sources/radiation sources other than the LiDAR system 100) and sensor noise (e.g., dark current).

[0389] Preliminary detection information can be acquired as a single signal (based on the output of one or more sensors, e.g., one or more SPADs, one or more APDs, one or more SiPMs, etc.) or as multiple signals (e.g., the outputs of multiple sensors). Preliminary detection information may include analog and/or digital information. Preliminary detection information may include a single value and/or multiple values (e.g., from different time points and/or different segment portions). Preliminary detection information may be associated with one or more items in the depth map (e.g., one or more polygons, one or more point cloud points, one or more depth image pixels, etc.). It should be noted that the preliminary information may later be used to determine the distance to at least one object within the FOV.

[0390] The light source controller 2360 may be configured and operable to control the light source 2310 of the LIDAR, and in particular to control the emission of light by the light source 2310. The light source controller 2360 may, but is not necessarily, be the sole entity controlling the emission of light by the light source 2310 of the LIDAR. If the LIDAR includes two or more light sources, the light source controller 2360 may be configured and operable to control one or more, and possibly all, of these light sources. Furthermore, various controllers other than controller 2360 may control or influence at least one operating mode of the light sources associated with the LIDAR system 100.

[0391] In some embodiments, the light source controller 2360 is configured to control the subsequent light emission by the LiDAR at each frame time of the sequence. The subsequent emission is emitted after the preliminary light emission (an emission used for preliminary detection information) (if this emission is permitted by the light source controller 2360). If the LiDAR emits pulsed light, the subsequent light emission may include one or more light pulses.

[0392] The light source controller 2360 can be configured to control the subsequent light emission by the LiDAR for each segment during each frame time of the sequence, based on preliminary detection information for each of the multiple segments. That is, during each frame time, the light source controller 2360 can control the subsequent light emission for each of the multiple segments based on the detection and processing of light emitted by the LiDAR during the same frame time and detected in the same segment.

[0393] By controlling the subsequent light emission for each segment of the FOV, it is possible to distinguish the light projection onto different segments of the LIDAR FOV based on the detection of reflected light from the same frame, which shows the detection result (for example, of a target in different parts of the FOV) with a nearly instantaneous input. Using this distinction, various goals can be achieved, such as:
a. Eye safety (as well as other safety considerations such as safety of the air, safety of optical systems, and safety of highly sensitive materials and objects): It is possible to limit the emitted power level in one or more parts of the LIDAR FOV where safety should be considered, while emitting a higher power level in other parts of the FOV (this may improve the signal-to-noise ratio and detection range).
b. Power Management: It is possible to limit the irradiated energy delivered to other parts of the FOV while directing a large amount of energy to a highly useful area of the LIDAR FOV (e.g., region of interest, distant target, low-reflectance target, etc.). Such light allocation for eye safety or power management (or any other purpose) may be based on detection results from the current frame or any preceding frame.

[0394] In some embodiments, controlling the subsequent light projection emission to a particular segment or region of the FOV may involve controlling (e.g., changing) one or more parameters of the light source to affect the subsequent light emission. Such changes may affect various characteristics of the projected light, including (but not limited to) any one of the following:
a. Increase, decrease, limit, or elimination of light projection onto one or more LIDAR FOV segments during the current or subsequent scan of the FOV. b. Total light energy supplied to the entire FOV or any part of the FOV. c. Energy profile of the light supplied to any part of the FOV. d. Duration of light emission. e. Wave properties of the light projected onto any part of the FOV, such as polarization and wavelength.

[0395] Furthermore, Figure 23 shows multiple segments of the FOV. It will be apparent to those skilled in the art that each segment may represent a three-dimensional conical section (essentially a cone or frustum of a cone). For the sake of illustration simplicity, only the cross-section of each segment is shown. Furthermore, the number of segments and their spatial configuration can vary considerably. For example, the segments in the figure are arranged in a 3x6 2D rectangular array, but other non-rectangular arrays can also be used, as can 1D arrays.

[0396] The system 2300 can be adapted to control (and possibly inspect) the inspection of a region or segment of a scene (indicated here as a specific field of view (FOV) to be scanned) using light pulses (or other forms of transmitted light such as CW laser irradiation). The characteristics of the irradiation (initial irradiation, subsequent irradiation, or any other irradiation by LIDAR) can be selected (and possibly even during LIDAR operation) as a function of one or more of the following parameters (among others):
a. Optical characteristics of the scene segment being inspected b. Optical characteristics of scene segments other than the one being inspected c. Scene elements present in or near the scene segment being inspected d. Scene elements present in or near the scene segment other than the one being inspected e. Operating mode of the scan or steering device f. Situational characteristics/features of the host platform on which the scan or steering device is operating.

[0397] The LIDAR light source 2310 (which can be rephrased as “emitter” and “emitter assembly”) may include one or more individual emitters (e.g., one or more lasers, one or more LEDs), which may operate using similar or different operating parameters (e.g., wavelength, power, focus, divergence, etc.). The light source controller 2360 may control one, some, or all of the individual emitters of the light source 2310. In some embodiments, the light source 2310 can be operated to emit photon inspection pulses toward the FOV. In some embodiments, the light source and deflector can be combined. For example, the LIDAR system may include a vertical cavity surface-emitting laser or an optical phased array.

[0398] The LIDAR sensor assembly 2340 (which can be rephrased as “sensor array,” “sensor,” “detector array,” and “detector assembly”) may include one or more photosensitive detectors 2344, each of which may contain an individual detection unit. For example, each detector 2344 may be a silicon photomultiplier tube (SiPM) containing multiple single-photon avalanche diodes (SPADs). The sensor assembly detects photons emitted by the LIDAR and reflected back from objects in the scanned scene.

[0399] In some embodiments, the LIDAR may further include a steering assembly 2330 for guiding emitted light in the direction of the scene segment being scanned and/or directing reflected photons towards the sensor array 2340. The steering assembly 2330 may include controllably steerable optical systems (e.g., rotating/movable mirrors, rotating/movable lenses, etc.) and may also include fixed optical components such as beam splitters, mirrors, and lenses. Some optical components (e.g., used for collimation of laser pulses) may be part of the emitter, while other optical components may be part of the detector assembly. In some embodiments, the steering assembly 2330 may include an array of mirrors.

[0400] In some embodiments, the light source controller 2360 can be connected to the light source 2310 in different ways, such as an electrical circuit or other wired or wireless connections. The light source controller 2360 can also be connected to the steering assembly 2330 to control the steering direction of emitted and/or reflected light based on the analysis of preliminary detection information. For example, if no further illumination is needed in a given segment, the steering assembly can be instructed to immediately change to a different steering state to illuminate another segment of the scene.

[0401] The LiDAR controller 2320 may be implemented to control the sensing array 2340, the steering assembly 2330, and/or other components of the LiDAR. The controller 2320 may include a light source controller 2360, but the light source controller 2360 may be external to and/or independent of the controller 2320 (e.g., the host 230). In the latter case, the light source can be controlled by both the controller 2320 and the light source controller 2360. Optionally, the controller 2320 can be used to coordinate and regulate the operation of the emitter 2310, the steering assembly 2330, and the sensor assembly 2340, and further optionally, to adjust them according to scene segment inspection characteristics (e.g., based on internal feedback, host information, or other sources).

[0402] According to some embodiments, LIDAR-based inspection of a scene segment may include illuminating the scene segment (which can be rephrased as "segment," "region," and "scene region") with transmitted light (e.g., photon pulses). The emitted light may have known parameters. These parameters may include, for example, duration, angular dispersion, wavelength, instantaneous power, photon density at different distances from the emitter, average power, power intensity, pulse width, pulse repetition rate, pulse sequence, duty cycle, wavelength, phase, polarization, and others.

[0403] Furthermore, the inspection of the region may include detecting reflected photons and characterizing the various aspects of these reflected photons. Reflected photons may include photons of emitted light that have been reflected from illuminated elements present within the scanned scene segment and returned to the LIDAR.

[0404] Since reflected photons can be generated by the inspection photon and the scene element from which the inspection photon was reflected, the received reflected photons can be analyzed accordingly. By comparing the characteristics of the emitted light with the characteristics of the corresponding reflected detection signal, the distance of one or more scene elements present in the scanned scene segment, and possibly other physical characteristics (such as reflection intensity), can be estimated. By repeating this process across multiple parts of the FOV (e.g., with a raster pattern, Lissajous pattern, or other patterns), the entire scene can be scanned to generate a depth map of the scene.

[0405] A "scene segment" or "scene region" can be defined, for example, using the angle in a spherical coordinate system corresponding to a light beam in a given direction. Furthermore, a light beam having a center radial vector in a given direction can also be characterized by its angular divergence, spherical coordinate range, and other properties.

[0406] In some embodiments, the different segments defined in the illumination context are not necessarily identical in size to the parts or portions of the FOV (e.g., “pixels” or depth maps) distinguished in the detection context. For example, a LiDAR can generate an N×M depth map (e.g., a 100×100 depth image), but the same FOV may be divided into fewer segments with respect to illumination (e.g., 10×10 or 20×1). In another example, the illumination segments may be smaller than the angular resolution of the detection in at least one dimension.

[0407] In some embodiments, the range estimator 2390 acquires detection information obtained by the sensor array 2340 and processes this information to generate a depth map. This processing can be performed based on time-of-flight analysis or by any other method known in the art.

[0408] Preliminary detection information can be based on the detection of simultaneous emission (e.g., one or more pulses, or spatially continuous irradiation) by multiple detectors (e.g., pixels, SiPMs). The light source controller 2360 can determine how to collectively control subsequent emission that is detectable by all detectors, based on the preliminary detection information generated by the multiple detectors (e.g., 2340). In some embodiments, the light source controller 2360 may prevent subsequent emission to an entire segment even if only one or some (but not all) of the detectors indicate that projection to each segment is unsafe.

[0409] Figure 24 is a flowchart showing an example of Method 2400 according to the embodiments disclosed herein. Method 2400 is a method for controlling the operation of a LiDAR (Light Detection and Ranging) device that generates a sequence of depth maps. Each depth map in the sequence can be generated with a corresponding frame time of less than one second. In some embodiments, Method 2400 may be performed pixel by pixel or beam spot by beam spot.

[0410] Referring to the example described in the preceding diagram, Method 2400 can be performed by System 2300. Method 2400 may include performing any functions, processes, capabilities, etc., considered with respect to System 2300, even if not explicitly stated. Similarly, System 2300 may be configured, adapted, and/or operable to incorporate any step or variation of Method 2400, even if not explicitly stated.

[0411] Method 2400 may include performing at least steps 2440 and 2450 on each segment from a plurality of segments of the LIDAR field of view at each frame time of the sequence. In some embodiments, Method 2400 may or may not include performing steps 2440 and 2450 on all segments within the FOV. In other embodiments, Method 2400 may or may not include performing steps 2440 and 2450 on all described segments of the FOV.

[0412] Step 2440 may include obtaining preliminary detection information (e.g., in one or more signals) based on the light emitted by the LIDAR during each frame time and reflected from each segment. Obtaining preliminary detection information may include obtaining detection information for a single pixel (or another type of item in the depth map, e.g., a PC point or polygon, a surface, face, edge, or vertex of a polygon mesh) or for two or more pixels (or items). Referring to the example described in the preceding diagram, step 2440 can be performed by the sensor interface 2350 and/or the sensor assembly 2340.

[0413] Step 2450 may include selectively controlling the subsequent light emission to each segment by the LIDAR during the same frame time, based on preliminary detection information (of the same segment during the same frame time in Step 2440). Referring to the example described with respect to the aforementioned figures, Step 2450 can be performed, for example, by the light source controller 2360. The control of Step 2450 may include, for example, any form of control considered with respect to the light source controller 2360. In some embodiments, Step 2450 may include controlling the steering assembly of the LIDAR (e.g., steering assembly 2330) to induce subsequent emission to each segment.

[0414] In some embodiments, at each frame time, the acquisition of preliminary detection information (for all segments) and selective control are performed within the same frame irradiation duration (the time from the emission of the first photon within the frame time to the emission of the last photon detected and affecting the frame's depth map). Optionally, selective control and subsequent emission are completed before the processing of detection information for generating the frame time's depth map begins.

[0415] In some embodiments, irradiation and analysis of different segments can be carried out in various orders. For example, after performing preliminary irradiation of each segment, acquiring preliminary detection information (step 2440), and selectively controlling subsequent irradiation to the same segment (step 2450), the execution of these steps on another segment can proceed.

[0416] In another embodiment, another segment may be irradiated between the pre-irradiation and the subsequent irradiation (by subsequent emission) of the first segment. In some embodiments, the segment dark time (LI) of the single segment is irradiated before the subsequent emission of the single segment.
There is a period of time when the DAR does not project light onto that segment. During this time, another segment among the multiple segments is illuminated by the LIDAR.

[0417] Method 2400 can be used to ensure that the LIDAR system 100 is safe for the eyes (e.g., operates in accordance with the requirements of relevant eye safety regulations). In some embodiments, prior to selective control of irradiation, there is a step (not shown) based on preliminary detection information to determine that no person is within at least an eye-safe range within the projection field (e.g., sector, cone, or frustum) for at least a predetermined number of frames. Thus, the LIDAR system 100 can prevent subsequent emissions with power exceeding the safety threshold in the portion of the FOV where a person was present. The eye-safe range (e.g., the “range threshold” in Figure 23) can be, but is not necessarily, a predetermined range. In some cases, the processor 118 can be configured to adjust a threshold related to the safety distance based on reflected signals received based on one or more light projections to a specific region of the LIDAR FOV (based on either the initial light projection or subsequent light projections having at least one modified feature relative to the initial light projection).

[0418] Depending on the detected conditions or circumstances, the selective control in step 2450 may include controlling the projection of subsequent light emissions into the projection field that are below or not below the eye-safe irradiation limit, but in all cases, irradiation control can be performed in compliance with eye-safe regulations. For example, if LIDAR detection indicates that no eye-bearing individual (person or other) is present in one or more specific areas of the LIDAR FOV, the subsequent light projection in that one or more areas may proceed at a level that is normally not safe for the eye. If an eye-bearing individual is subsequently detected to have entered, for example, one or more areas in which such an individual was not previously present, one or more parameters of the light projector can be changed so that the subsequent light emission into the occupied area is safe for the individual's eyes. In another example, one or more eye-bearing individuals may be detected within a specific area of the LIDAR FOV but at a distance exceeding the eye-safe threshold (e.g., ocular hazard distance). In such cases, light can be projected onto the area so that it is not safe for the eye within the eye safety threshold, but is safe for the eye beyond the eye safety threshold at which the individual was detected. In yet another example, a person and/or animal may be detected within a range of the intermediate area of the LiDAR system (e.g., within a predetermined eye safety threshold distance). In such cases, the light projection can be modified to maintain eye safety within those areas of the intermediate area of the LiDAR where an individual with one or more eyes was detected. Eye safety protocols can define a maximum power level or a threshold for accumulated energy over time. For example, if subsequent light emission includes a group of pulses, compliance with eye safety may require that the accumulated energy of those pulses does not exceed a predetermined threshold level. In some examples, if an object (e.g., a person) is detected in the intermediate area of the LiDAR system, the processor 118 can be configured to prevent further light emission into the portion of the intermediate area associated with the detected object. In other examples, if an object is detected in the intermediate area, at least one processor can be further configured to restrict at least one of at least one light source and at least one light deflector to emit visible light into the intermediate area. It should be noted that visible light can be emitted from a light source separate from the light source used in determining distance.

[0419] The term “intermediate area” is widely used in the art and should be interpreted broadly to include the area adjacent to the LiDAR system. The size of the intermediate area may depend on the power setting of the LiDAR system (which affects the potential fault distance of the LiDAR system). The intermediate area has substantially the same diameter in all directions of the FOV (from which the LiDAR system can emit light), and may have a difference of up to 50%, for example, but is not necessarily so. Optionally, the intermediate area of the LiDAR system is defined in all directions of the FOV from which the LiDAR system can emit light.

[0420] In some embodiments, based on the light projected onto a selected area of the LiDAR FOV, a processor 118 can receive a reflected signal from at least one sensor indicating light reflected from an object in the LiDAR FOV. Based on the reflected signal resulting from the initial light emission, the processor 118 can determine whether an object is located in the intermediate area of the LiDAR system (e.g., an area associated with a particular segment of the LiDAR FOV or a group of segments of the FOV, and within a threshold distance from at least one light deflector). The threshold distance can be associated with a safety distance, such as the eye's safety distance. If no object is detected in the intermediate area of the FOV, the processor 118 can control at least one light source to project an additional light emission into the intermediate area, thereby enabling the detection of an object further away from the intermediate area. In such a case, for example, at least one processor can be configured to use the initial light emission and the additional light emission to determine the distance of an object located further away from the intermediate area. It should be noted that the term "reflected signal" is broadly interpreted to include any form of reflection and scattering of light, including specular reflection, diffuse reflection, and other forms of light scattering.

[0421] If an object is detected in the intermediate area, the processor 118 can regulate at least one of at least one light source and at least one light deflector so that the accumulated energy density of the light projected onto the intermediate area does not exceed the maximum allowable exposure. For example, by changing various parameters of the light projection unit and/or light deflection unit, additional light emission can be given to the LIDAR FOV segment such that at least one aspect differs from the initial light emission (e.g., at least one aspect differs with respect to eye safety parameters). The additional light emission can be given to a specific LIDAR FOV segment during the same FOV scan in which the initial light emission occurred, or during any subsequent FOV scan.

[0422] If the LIDAR system can determine the distance to a detected object, this information can be used by the LIDAR system to comply with eye safety regulations. For example, once an object is detected, the processor 118 can determine the distance to this object (e.g., based on time-of-flight analysis). The processor 118 can calculate the intensity of the projected light at the detected object (e.g., based on the detected distance and known characteristics of the projected light from the light source 112/deflector 114). Based on this calculation, the processor 118 can determine an eye-safe exposure time at this distance to the object. The processor 118 can then control at least one of the light source 112 and the deflector 114 so as not to exceed this exposure time. Similarly, the processor 118 can be configured to determine a value related to the maximum permissible exposure. This determination can be made based on the determined distance between at least one light deflector and the object detected in the intermediate area of the LIDAR system.

[0423] In addition to or instead of determining the exposure time, the processor 118 can determine the eye-safe acceptable light energy at a distance to the object, based on the intensity calculations described above. It should be noted that, in some examples, the processor 118 can indirectly determine both the exposure time and the acceptable light energy by determining values that represent each parameter. It should also be noted that, even if not explicitly detailed, the determination of the acceptable light energy (if implemented) can be used in the same way as the determined exposure time, with necessary modifications.

[0424] It should also be noted that the distance between at least one light deflector and the object can be determined directly or indirectly. This indirect determination of the distance can be achieved, for example, by determining another distance, such as the distance between at least one light source and the object.

[0425] In embodiments where the FOV is divided into multiple segments or sectors to perform a scan of the LIDAR FOV, for example, each segment or sector can be associated with a different intermediate area with respect to the LIDAR system. That is, each segment or sector can define a distinct intermediate area in the vicinity of the LIDAR system, along with a threshold distance for eye safety. In some embodiments, the processor 118 can be configured to determine whether an object is located in each of the intermediate areas associated with the multiple sectors, based on reflected signals obtained from the initial light emission to each sector. In some cases, the processor 118 can be configured to detect an object in a first intermediate area associated with a first sector, based on reflected signals received from a particular sector via a sensor unit. Similarly, the processor can be configured to determine whether an object is absent in a second intermediate area associated with a second sector. In such cases, at least one processor 118 can be configured to control at least one light source to project an additional light emission to the second intermediate area (e.g., in a single scan cycle). Furthermore, the processor 118 can regulate at least one of the light sources and/or light deflectors so that the stored light energy density in the first intermediate area does not exceed the maximum allowable exposure.

[0426] In some embodiments in which the LIDAR FOV is divided into multiple sectors, the processor 118 can be configured to determine whether an object is located in each of the intermediate areas associated with the multiple sectors, based on the reflected signals associated with the initial light emission from each sector. When the processor 118 detects an object in a first intermediate area associated with a first sector and determines that no object is present in a second intermediate area associated with a second sector, it can control at least one light source to project an additional light emission to the second intermediate area in a single scan cycle. The processor 118 can also restrict at least one of the at least one light source and at least one light deflector so that the stored light energy density in the first intermediate area does not exceed the maximum allowable exposure.

[0427] It should be noted that any of the above-described LIDAR system embodiments can be used in conjunction with the eye-safe light projection protocol described herein. For example, in some embodiments, the eye-safe LIDAR includes a monostatic deflector configuration in which the deflector directs projected light to a specific segment of the field of view, and the same deflector can guide light reflected from objects within that specific segment of the field of view to one or more sensors. Furthermore, the light deflector may include multiple light deflectors, and the processor 118 can be configured to have the multiple light deflectors cooperate to scan the LIDAR FOV. In some embodiments, at least one light deflector includes a single light deflector, and at least one light source may include multiple light sources aimed at the single light deflector.

[0428] The LIDAR system 100 can employ a variety of different light sources. For example, in some cases, the light source can be configured to project light with wavelengths such as less than 1000 nm or between 800 nm and 1000 nm.

[0429] In some embodiments, the LIDAR system 100 may include two or more light sources. In such cases, each light source can be associated with a different area of the LIDAR FOV. The processor 118 can be configured to coordinate the operation of at least one light deflector and multiple light sources so that when an object is detected in a first area of the field of view at a distance greater than the safety distance, the energy density of light projected onto a second area of the field of view by different light sources does not exceed the maximum permissible exposure associated with the second area of the field of view.

[0430] Furthermore, the processor 118 can be configured to coordinate at least one light deflector and at least one light source so that, if another object is detected in a different area at a distance greater than the safety distance, the energy density of light projected onto this other part of the field of view by at least one light source does not exceed the maximum permissible exposure associated with this other part of the field of view. In some embodiments, the safety distance is the nominal ocular hazard distance (NOHD).

[0431] In some embodiments, the selective control of step 2450 may include preventing subsequent emissions having power exceeding a safety threshold for a projection field in which a person has been present for at least a predetermined number of frame times, in at least one segment during at least one frame time. In some embodiments, the selective control of at least one FOV segment during at least one frame time may include reducing the amount of stored energy while maintaining or increasing the optical projection power level applied to that at least one FOV segment. For example, in the case of a pulsed laser, one or more pulses of a preceding irradiation may have the same peak power (or a lower power level) as one or more pulses of one or more subsequent emissions. However, the stored energy of the subsequent irradiation may still be lower than the stored energy of the preceding one or more emissions. In this way, it is also possible to increase the signal-to-noise ratio and/or detection range while operating in compliance with eye safety regulations. Of course, in other cases, it is possible to vary the power level, stored energy characteristics, or any other combination of optical emission parameters in order to achieve the LIDAR detection goal while complying with eye safety regulations.

[0432] In some embodiments, the selective control of step 2450 may include stopping (or preventing) subsequent light emission to a specific FOV segment or group of segments within a given frame time in order to comply with eye safety regulations. Such control may also be implemented to reduce or eliminate the risk of saturation of the detector or any other component of the detection and/or processing chain. Furthermore, such control may address power saving issues (e.g., not consuming energy when unnecessary, such as when an object can be detected and/or its range determined based on previous emission without continuing the emission).

[0433] In some embodiments, selective control of at least one segment of the LiDAR FOV in at least one frame time may include preventing the emission of subsequent emissions if preliminary detection information satisfies a predetermined detection criterion. In some embodiments, after selective control, the preliminary detection information may be further processed (for each segment, without using subsequent detection information) to generate depth information for the segment.

[0434] With respect to eye safety (for example), Method 2400 can be used to prevent irradiation of a potentially harmful emission into an FOV area based on a determination of the likelihood that one or more objects may contain persons and/or animals if one or more objects are detected in that FOV area. Potentially harmful emissions into a particular FOV area may be suspended even if it is unlikely that one or more objects may contain individuals with eyes. Potentially harmful emissions into a particular FOV may also be suspended (or otherwise modified) even if no individuals (or objects) are detected, if the FOV is determined to be an area where individuals with eyes are generally present (based on the detected circumstances, e.g., near a stopped bus, near a crosswalk, near a sidewalk, near a building entrance). Higher power emissions may be given to other areas of the FOV that are not determined to contain individuals with eyes or are not expected/predicted to contain such individuals. As a result, the generated depth map becomes more useful due to detection in areas not subject to eye safety limitations (e.g., due to high power emission), and the overall quality of the depth map can be improved compared to when all light emission across the entire FOV is performed at power levels that assume the presence of an individual with eyes.

[0435] Method 2400 may include performing selective control of subsequent emission to various FOV segments having power levels at least twice as different from each other, based on corresponding preliminary detection information, within a single frame time of the sequence (for example, within the same frame time, a subsequent emission to one FOV segment may have a power level at least twice as large as a subsequent emission to another segment). A depth map of this frame time can be generated (e.g., step 580). This can, for example, enable high SNR or long-range detection in some parts of the FOV while maintaining compliance with eye safety in other areas of the FOV or across the FOV as a whole (e.g., considering the accumulated energy threshold).

[0436] In some embodiments, step 2450 may include selectively controlling subsequent radiation to prevent saturation of the detection path from which sensor detection information is acquired. This may include any component of the sensor or LIDAR in the detection and/or processing path, such as an amplifier or an analog-to-digital converter. Saturation prevention can be used in advanced processing of the detection results (for example, to estimate the reflectance level of the detected target).

[0437] Method 2400 may include limiting (or otherwise controlling) the emission level for a given FOV segment at a given frame time based on detection results in one or more preceding frames of the same segment or other segments. Method 2400 may also include limiting (or otherwise controlling) the emission level for a given segment at a given frame time based on detection results in another segment (at the same frame time or preceding frame time).

[0438] Method 2400 may include controlling subsequent emission of FOV to segments (e.g., within the same frame time) based on preliminary detection information from the same FOV segment or another FOV segment acquired within the same frame time. For example, detection of a target within the intermediate area of the LiDAR, particularly a target corresponding to an individual with eyes, within a specific FOV segment may affect subsequent light emission to the same FOV segment or to one or more surrounding FOV segments. Such a target may span two or more FOV segments, or may be expected to move to a neighboring FOV segment.

[0439] Method 2400 may include selectively controlling pre-emissions for a particular FOV segment prior to acquiring pre-detection information, based on detection information collected during previous frame times. In some embodiments, different light sources can be used for pre-irradiation and subsequent irradiation. For example, if subsequent emission can be projected by the main light source of the LiDAR, pre-irradiation may be projected by another light source (e.g., a visible light source, or possibly a light source of another system). Optionally, pre-detection information is based on the detection of at least one photon emitted by at least one light source of the LiDAR that does not project during each subsequent emission. These different light sources can be controlled by a single light source controller or by different controllers.

[0440] The detection of preliminary detection information and subsequent detection information may be performed by different sensors. For example, preliminary detection information may be based on detection by at least one sensor optimized for short-range detection, and method 2400 may include processing detection information of reflected photons of subsequent emission detected by at least one other sensor optimized for long-range detection. The use of different types of sensors may, but is not necessarily, be combined with the use of different types of light sources (e.g., optimized for different sensors, or vice versa). In one example, sensor 116 may include an avalanche photodiode (APD) detector for short-range object detection in addition to (or instead of) an array of single-photon avalanche diodes (SPADs).

[0441] Pre-irradiation of one FOV segment can be used on several segments of the FOV (e.g., if the pre-irradiation is below a threshold level, such as the eye safety threshold). Irradiation of other segments of the FOV (e.g., due to energy levels exceeding the threshold level) can be managed by analyzing the preliminary detection information of the relevant frames. For example, if the FOV region around the periphery of the FOV indicates a low risk to individuals with eyes, the periphery of the FOV can be analyzed using a preliminary low-level survey signal, and the center of the FOV can be scanned using a higher-power optical projection.

[0442] Method 2400 may include performing a scan step within the frame time of the FOV. The scan step may include, for example, acquiring perimeter detection information based on light emitted by the LIDAR within the frame time and reflected from one or more objects in at least one segment located on the perimeter of the FOV. These steps may also include selectively controlling the emission of light to the central segment of the FOV based on the perimeter detection information.

[0443] Referring to Method 2400 as a whole, and further to its variations discussed above, it should be noted that Method 2400 can be embodied in computer-readable code (instruction set) that can be executed by a processor (e.g., a LIDAR controller). This discloses a non-transient computer-readable medium for controlling the operation of a light-detecting and ranging (LIDAR) device that generates a sequence of depth maps (each depth map in the sequence is generated in a corresponding frame time of less than one second). The non-transient computer-readable medium may contain stored instructions, which, when executed on a processor, may perform steps including (a) obtaining preliminary detection information of light emitted by the LIDAR and reflected from each segment during each frame time, and (b) selectively controlling subsequent light emission by the LIDAR to each segment during each frame time based on the preliminary detection information. Any other steps of Method 2400 can also be implemented as instructions stored in the computer-readable medium and executable by a processor.

[0444] Figure 25A is a flowchart illustrating an example of Method 2500 according to the subject matter disclosed herein. Method 2500 is one possible implementation of Method 2400. As shown in Figure 25A, optionally, selective control of further light emission by the LiDAR (within a given segment at a given frame time) based on results from the same frame time can be repeated several times within a safe frame time. For example, this sequence of emission, detection, analysis, and selective control can be repeated for each pulse emitted for a particular FOV segment.

[0445] Step 2500 may, in some embodiments, include the step of detecting an object within the range threshold of the Lidar and setting a subsequent light emission based on whether or not an object has been detected. The Lidar system 100, or the Lidar described above with reference to Figure 23, can control one or more light sources 112 to emit light pulses into the intermediate area. The light pulses can be directed to a specific segment of the FOV by one or more deflectors 114. If an object is present within a specific segment of the FOV, the Lidar system 100 can receive light reflected from the object via one or more sensors 116 or sensor arrays. The processor 118 or range estimator 2390 can use this reflected light to determine the distance between the object and the Lidar system 100. If the object is within the threshold distance, the processor 118 can restrict at least one of the light sources 112 and at least one of the light deflectors 114 so that the accumulated energy density of the light projected into the intermediate area does not exceed the maximum allowable exposure. If no object is detected, a subsequent light pulse is emitted in the same segment to detect whether or not an object exists beyond the intermediate area.

[0446] For example, if a pedestrian is detected, the subsequent light emission characteristics can be determined to account for the presence of this pedestrian. In some embodiments, light emission to a specific FOV segment in which a pedestrian is determined to be present may be limited to power levels, integrated energy levels, duration, etc., in accordance with applicable eye safety regulations. Advantages of this embodiment include increasing the safety of the LIDAR to pedestrians or other persons in this area by reducing the emitted power to a level considered safe by local or federal regulations.

[0447] Figure 25B shows an exemplary method 2500 for detecting an object. Method 2500 can be performed by at least one processor (for example, the processor 118 of the processing unit 108 of the LIDAR system 100 shown in Figure 1A, and/or two processors 118 of the processing unit 108 of the LIDAR system shown in Figure 2A). In step 2502, the processor 118 can control at least one light source (for example, the light source 112 in Figure 1A, the laser diode 202 of the light source 112 in Figure 2A, and/or multiple light sources 102 in Figure 2B) so as to vary the beam of light from at least one light source during a scan cycle of the field of view (for example, the field of view 120 in Figures 1A and 2A). For example, the processor 118 can vary the timing of pulses from at least one light source. Alternatively or simultaneously, the processor 118 can vary the length of pulses from at least one light source. As another example, the processor 118 may, instead of or simultaneously, vary the magnitude of pulses from at least one light source (e.g., by changing the length or width, or by otherwise changing the cross-sectional area). In yet another example, the processor 118 may, instead of or simultaneously, vary the amplitude and/or frequency of pulses from at least one light source.

[0448] Step 2504 may further include the processor 118 controlling at least one deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or unidirectional deflector 214 in Figure 2B) to deflect light from at least one light source in order to scan the field of view. For example, the processor 118 may scan the field of view by mechanically moving at least one optical deflector. Alternatively, or simultaneously, the processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector in order to scan the field of view.

[0449] In step 2506, the processor 118 can receive a reflected signal from at least one sensor (e.g., sensor 116 in Figure 1A) indicating light reflected from an object in the field of view. In one embodiment, the reflected signal can be associated with a single portion of the field of view (e.g., the second FOV 414 in Figure 4B). In step 2508, based on the reflected signal of the initial light emission, the processor 118 can determine whether an object is located within a threshold distance from at least one light deflector in the intermediate area of the field of view. According to one embodiment, the threshold distance can be associated with the eye injury distance. In other words, the processor 118 can determine whether the amount of projected light could harm an individual located in the intermediate area. According to another embodiment, the threshold distance can be associated with the sensor saturation distance. In other words, the processor 118 can determine whether the reflected light resulting from the amount of projected light could overflow the sensor 116.

[0450] If no object is detected in the intermediate area, i.e., in step 2510, the processor 118 can control at least one light source to project additional light emission into the intermediate area, thereby enabling the detection of objects further away from the intermediate area. Furthermore, if an object is detected in the intermediate area, i.e., in step 2512, the processor 118 can adjust at least one of the at least one light source and at least one light deflector so that the stored light energy density in the intermediate area does not exceed the maximum allowable exposure. According to the two embodiments described above, the maximum allowable exposure can be related to the amount of projected light that could damage an object located in the intermediate area, or the amount of projected light that could cause reflected light to overflow the sensor 116 and impair its function.

[0451] Parallel Scene Scanning in Lidar Using a Common Steerable Deflector

[0452] LiDAR systems according to embodiments of the present disclosure can utilize multiple light sources. For example, the use of multiple light sources enables simultaneous scanning of different parts of the field of view and/or scanning of the field of view using light of different wavelengths and intensities. Furthermore, LiDAR systems according to embodiments of the present disclosure can utilize a common deflector aimed at light from multiple light sources. For example, LiDAR systems according to embodiments of the present disclosure can use a single deflector to simultaneously guide light from multiple light sources to different directions of the field of view. Using a common deflector can reduce the size, cost, and/or complexity of the LiDAR system. In some embodiments, LiDAR systems according to embodiments of the present disclosure can use the same deflector to guide light from multiple light sources and to guide reflections received from the field of view. In some embodiments, one of the multiple light sources may include multiple individual light sources that operate simultaneously (e.g., to increase the luminous flux). Such multiple sources may include similar individual light sources, but may also include individual light sources of different types and/or different characteristics.

[0453] Figures 26A, 26B, and 26C illustrate (respectively) methods 2600, 2630, and 2660 for detecting an object using a LiDAR (particularly one with a common deflector) according to an example of the subject matter disclosed herein. Any one of methods 2600, 2630, and 2660 can be performed by at least one processor (for example, the processor 118 of the processing unit 108 of the LiDAR system 100 shown in Figure 1A, the two processors 118 of the processing unit 108 of the LiDAR system shown in Figure 2A, at least one processor 2702 of the LiDAR system 2700 in Figure 27, and/or at least one processor 2802 of the LiDAR system 2800 in Figure 28). At least one processor may be located within the body of a vehicle (e.g., vehicle 4201 in Figure 42A, vehicle 4205 in Figure 42B, vehicle 4209 in Figure 42C, etc.). Any combination of methods 2600, 2630, and/or 2660 can also be implemented by at least one processor. Such combinations may include any combination of two or more steps from any of the methods 2600, 2630, and/or 2660 discussed below. Furthermore, methods 2600, 2630, and/or 2660 can be implemented by any LIDAR system according to the disclosed embodiments, as an option. The considerations regarding the components of the LIDAR system in any one of these methods (2600, 2630, 2660) are non-limitingly applicable to the other two methods with necessary modifications.

[0454] In step 2601, at least one processor 118 or 2702 or 2802 directs light from multiple light sources (e.g., light sources 2706, 2708, and 2710 of the LIDAR system 2700 in Figure 27, and/or light sources 2806 and 2808 of the LIDAR system 2800 in Figure 28) along multiple outbound paths, while at least one deflector (e.g., deflector 2704 of the LIDAR system 2700 in Figure 27 and/or deflector 2804 of the LIDAR system 2800 in Figure 28) is in a specific instantaneous position, to multiple regions (e.g., regions 2712a, 2712b, and 2712 in Figure 27) that form a field of view (e.g., field of view 2712 in Figure 27 and/or field of view 2810 in Figure 28) c, and/or regions 2810a and 2810b in Figure 28, are controlled to deflect the light. For example, as shown in Figures 2B, 27, and 28, at each instantaneous position of at least one deflector, the light from each of the different light sources can be directed to a corresponding portion of the region. The aforementioned multiple regions (e.g., 2712a, 2712b) can form a field of view by extending to a field of view larger than any one of the regions. In some embodiments, the formed FOV may be continuous (e.g., as shown in Figures 27 and 28). Alternatively, or simultaneously, the formed FOV may include two or more distinct FOV regions.

[0455] One or more characteristics of multiple light sources may differ. For example, as described below, the wavelengths of light emitted from at least two of multiple light sources may differ. Another example is that the maximum power, duty cycle, pulse timing, pulse length, etc., of at least two of multiple light sources may differ. On the other hand, one or more characteristics of multiple light sources may be identical or at least substantially similar.

[0456] For example, as shown in Figure 27, the multiple light sources may include at least three separate light sources. In some embodiments, the relative angles of light from the multiple light sources may be fixed throughout the scan. For example, light from the first light source may be incident on the surface of the deflector at an angle _α1 (t) at different time points t, and light from the second light source may be incident on the same surface of the deflector at an angle _α2 (t) at different time points t, such that _α1 (t) _–α2 (t) is constant at all different time points t. In embodiments where the light sources are not arranged on a single plane, two angles can be used to identify the angle of incidence of each light source on the surface of the deflector. In some embodiments, each light source may be associated with a generally different region of the field of view. For example, at least a first region of the field of view (e.g., region 2712a in Figure 27) may be adjacent to at least a second region (e.g., region 2712b in Figure 27) and separated from at least a third region (e.g., region 2712c in Figure 27). Alternatively, or simultaneously, in such examples, at least each region of the field of view can be associated with a different angular portion of the field of view. In some embodiments, the different regions can be of similar size (e.g., similar angular size). This can be defined, for example, by the range of instantaneous positions of each deflector 114 in Figure 1A.

[0457] In some embodiments, at least one deflector may include a single deflector configured to pivot along two separate axes, as shown in Figures 27 and 28. Thus, a single deflector can guide and/or deflect light along two separate axes. Alternatively, or simultaneously, at least one deflector may include a deflector array comprising multiple deflectors configured to pivot independently, as shown in Figure 3B. As used herein, the term “pivot” refers to the mechanical movement of a deflector or reflector within a deflector, but may also include any other method of movement between different instantaneous positions, as will be further examined with respect to the deflector 114 in Figure 1A (e.g., with respect to OPA, VCSEL array, MEMS array, etc.).

[0458] In some embodiments, during a scan cycle, at least one optical deflector may be positioned at multiple different instantaneous positions. For example, at least one optical deflector may move between different instantaneous positions in a continuous sweeping motion. Alternatively, at least one deflector may move between multiple discrete instantaneous positions during a scan cycle, rather than in a continuous sweeping motion.

[0459] In some embodiments, at least one deflector is two-dimensional and can therefore define two axes. In such embodiments, the field of view regions may be arranged corresponding to one axis of at least one deflector, or to both axes. Alternatively, or simultaneously, two, three, or more regions of the FOV may be arranged along a straight line, parallel to each other, for example, so that each covers substantially the same area. Thus, scans within a scan cycle may be performed parallel to at least one axis in another (substantially perpendicular) direction.

[0460] In some embodiments, at least one optical deflector can be housed in the same housing as at least one of the multiple light sources. In other embodiments, at least one optical deflector can be housed in a separate housing from the multiple light sources.

[0461] In step 2603, at least one processor 118 or 2702 or 2802 controls at least one deflector such that light reflections from the field of view are received in at least one common area of the at least one deflector (e.g., common area 2714 of deflector 2704 in Figure 27, common area 2748 in Figure 27, and/or common area 2812 of deflector 2804 in Figure 28) while at least one deflector is in a particular instantaneous position. In some embodiments, at least some of the light reflections of at least some of the multiple light sources are incident on each other in at least one common area. For example, the area where one light reflection is received may at least partially overlap with the area where another light reflection is received. Such incidences may occur simultaneously or at different times. For example, light reflections that have traveled different distances may arrive at different times. As another example, light from multiple light sources may be emitted at different times, thus producing reflections at different times. Such timings may vary considerably among the multiple light sources.

[0462] In step 2605, at least one processor 118 or 2702 or 2802 receives at least one signal from each of the plurality of detectors (e.g., the detectors of sensors 2716, 2718, and 2720 in Figure 27, and/or the detector of sensor 2814 in Figure 28) indicating light reflection from at least one common area while at least one deflector is at a specific instantaneous position. For example, the plurality of detectors can absorb light reflection and convert the absorbed reflection into an electrical signal, digital signal, etc., for transmission to at least one processor 118 or 2702 or 2802. Thus, the plurality of detectors may include any photosensitive device capable of measuring electromagnetic wave characteristics (e.g., power, frequency) and generating an output (e.g., a digital signal) related to the measured characteristics. In some embodiments, the received light reflection may be from a common incident area, i.e., an area into which reflections from two or more light sources, as described above, are incident.

[0463] In some embodiments, the multiple detectors may be part of a single sensor configured to measure at least two different distances associated with a specific position of at least one optical deflector, as shown, for example, in Figure 28. For example, the at least two different distances may be associated with different portions of the field of view (FOV). Alternatively, or simultaneously, the at least two distances may be determined within the same pixel by photons reflected from different targets, for example, partially obscuring each other. In other embodiments, the multiple detectors may be part of separate, distinct sensors located at different locations within a LiDAR system.

[0464] In examples where the multiple light sources include at least three distinct light sources (and in some examples where the multiple light sources include two light sources), each light source can be associated with a different region of the field of view, where at least a first region of the field of view is adjacent to at least a second region and spaced apart from at least a third region, and the multiple detectors can be associated with at least three distinct sensors (or, for example, two in the case of two light sources). In such embodiments, each of the at least three distinct sensors can be associated with a different light source, as shown, for example, in Figure 27. Furthermore, in such embodiments, the multiple detectors can be configured to simultaneously detect a first object located in the first region of the field of view and a second object located in the third region of the field of view. Such detection may occur simultaneously, for example, within the scan time of one of the pixels in the first region. However, such simultaneous detection may occur instantaneously.
) These events may not occur simultaneously. For example, as mentioned above, reflected light from different distances may arrive at different times. Another example is when light from multiple light sources is emitted at different times, thus generating reflections at different points in time.

[0465] Method 2600 may include additional steps. For example, Method 2600 may further include scanning the field of view by repeatedly moving at least one deflector, as described above. During a single scanning cycle of the field of view, at least one deflector can be positioned at multiple different instantaneous positions (at different times). Thus, at different instantaneous positions, the deflection and/or reflection of light from multiple light sources may also be different. Thus, light may be directed to different regions of the field of view, and/or reflections may be received from different regions of the field of view.

[0466] As another example, method 2600 may include determining a plurality of distance measurements related to different regions of a field of view based on light reflection from at least one common area. For example, one distance measurement may be for a first vehicle in a first region, and another distance measurement may be for a second vehicle in a second region. As yet another example, one distance measurement may be for a vehicle in a first region, and another distance measurement may be for a road in a second region (which may be, for example, behind, in front of, or adjacent to the vehicle). As yet another example, one distance measurement may be for a first vehicle in a first region, and another distance measurement may be for an object in a second region (which may be, for example, located beside the road). In certain embodiments, several objects may be detected in different regions. This is because, for example, those regions partially overlap, and/or the object is located on the boundary between the two regions.

[0467] Method 2600 may further include controlling a plurality of light sources such that, at a first instantaneous position of at least one deflector (i.e., while at least one deflector remains at the first instantaneous position), the first light source emits more light flux to at least one deflector than the second light source, and at a second instantaneous position of at least one deflector, the second light source emits more light flux to at least one deflector than the first light source. As an example, at least one processor 118 or 2702 or 2802 may increase the light flux from the first light source at a first instantaneous position in which light from the first light source is guided straight, and increase the light flux from the second light source at a second instantaneous position in which light from the second light source is guided straight. Thus, more energy can be expended to see an object straight ahead than an object to the side. As described above, at least one processor 118 or 2702 or 2802 can vary the timing of pulses from multiple light sources, vary the length of pulses from multiple light sources, vary the magnitude of pulses from multiple light sources (e.g., length or width, or by changing the cross-sectional area in other ways), vary the amplitude and/or frequency of pulses from multiple light sources, and/or change the parameters of continuous wave (CW) or quasi-CW light emission from multiple light sources (e.g., amplitude, modulation, phase, etc.). In some embodiments, luminous flux management in different regions (by different light sources) can be managed separately from other regions. Alternatively, or simultaneously, at least one processor 118 or 2702 or 2802 can balance the optical budget and/or power budget of two or more regions (e.g., to avoid exceeding maximum power consumption).

[0468] Referring to method 2630 shown in Figure 26B, in step 2631, at least one processor 118 or 2702 or 2802 moves at least one deflector so that, while at least one deflector is at a particular instantaneous position, it deflects light from multiple light sources along multiple outbound paths toward multiple regions forming a field of view. The term “move” should be interpreted broadly, as will be further considered using some of the examples above. For example, at each instantaneous position of at least one deflector, light from each of different light sources can be directed toward the corresponding portion of the region, as shown in Figures 2B, 27, and 28.

[0469] As described above with reference to Method 2600 in Figure 26A, the characteristics of one or more of the multiple light sources may vary considerably. Furthermore, similar to Method 2600 in Figure 26A, the multiple light sources may include at least two (e.g., as shown in Figure 28), at least three (e.g., as shown in Figure 27), or more distinct light sources. In such examples, each light source may be associated with a generally different region of the field of view. Alternatively, or simultaneously, these regions may overlap at least partially (e.g., to improve performance, to increase the maximum luminous flux in that area, for calibration between two areas, as a backup in case of failure of a critical part of the FOV, or when operating at different wavelengths). Furthermore, in such examples, at least a first region of the field of view may be adjacent to at least a second region and separated from at least a third region. Alternatively, or simultaneously, in such examples, at least each region of the field of view may be associated with a different angular portion of the field of view.

[0470] In some embodiments, similar to method 2600 in Figure 26A, at least one optical deflector can be housed in the same housing as at least one of the multiple light sources. In other embodiments, at least one deflector can be housed in a separate housing from the multiple light sources.

[0471] In step 2633, while at least one deflector is in a specific instantaneous position, at least one deflector can receive light reflections from multiple light sources from objects in the field of view on at least one common area of the at least one deflector. In some embodiments, at least some of the light reflections are incident on each other in at least one common area. For example, as described above with reference to method 2600 in Figure 26A, the area where one light reflection is received may at least partially overlap with the area where another light reflection is received. Such incidences may occur simultaneously or at different times. For example, light reflections that have traveled different distances may arrive at different times. As another example, light from multiple light sources may be emitted at different times, thus producing reflections at different times. Such timings may vary considerably among the multiple light sources.

[0472] In step 2635, each of the multiple detectors (e.g., the detectors of sensors 2716, 2718, and 2720 in Figure 27, and/or the detector of sensor 2814 in Figure 28) can receive light reflections from at least one common area when at least one optical deflector is in its instantaneous position. For example, as described above with reference to method 2600 in Figure 26A, the multiple detectors may include any photosensitive devices capable of measuring electromagnetic wave characteristics (e.g., power, frequency) and generating an output (e.g., a digital signal) related to the measured characteristics. In some embodiments, the received light reflections may be from a common incident area, i.e., an area into which reflections from two or more light sources are incident, as described above.

[0473] In some embodiments, similar to method 2600 in Figure 26A, the multiple detectors may be part of a single sensor configured to measure at least two different distances associated with specific positions of at least one optical deflector, as shown, for example, in Figure 28. For example, the at least two different distances may be associated with different portions of the field of view (FOV). Alternatively, or simultaneously, the at least two distances may be determined within the same pixel by photons reflected from different targets that partially obscure each other, for example.

[0474] Method 2630 may include additional steps such as any combination of one or more steps considered with respect to Method 2600 (but not limited to these steps).

[0475] Referring to method 2660 shown in Figure 26C, in step 2661, at least one processor 118 or 2702 or 2802 controls multiple light sources aimed at at least one deflector. For example, similar to step 2601 of method 2600, at least one processor 118 or 2702 or 2802 can control the timing of pulses from the multiple light sources, control the length of pulses from the multiple light sources, control the magnitude of pulses from the multiple light sources (e.g., length or width, or by changing the cross-sectional area in other ways), control the amplitude and/or frequency of pulses from the multiple light sources, and/or control the parameters of continuous wave (CW) or quasi-CW light emission from the multiple light sources (e.g., amplitude, modulation, phase, etc.).

[0476] In step 2663, at least one processor 118 or 2702 or 2802 receives data from a plurality of detectors configured to detect reflections from a plurality of light sources. In some embodiments, similar to method 2600 in Figure 26A, the plurality of detectors may be part of a single sensor configured to measure at least two different distances associated with a particular position of at least one optical deflector, as shown in Figure 28. For example, the at least two different distances may be associated with different parts of the FOV. Alternatively, or simultaneously, the at least two distances may be determined by photons reflected from different targets within the same pixel, for example, partially obscuring each other.

[0477] In step 2665, at least one processor 118 or 2702 or 2802 moves at least one deflector so that, while at least one deflector is in a specific instantaneous position, it deflects light from multiple light sources along multiple outbound paths toward multiple regions forming a field of view. The term “move” should be interpreted broadly, as will be further considered using some of the examples above.

[0478] Similar to method 2600 in Figure 26A, as shown in Figure 27, the multiple light sources may include at least three distinct light sources. In such examples, each light source can be associated with a generally different region of the field of view. Furthermore, in such examples, at least a first region of the field of view may be adjacent to at least a second region and separated from at least a third region. Alternatively, or simultaneously, in such examples, at least each region of the field of view may be associated with a different angular portion of the field of view.

[0479] In step 2667, at least one processor 118 or 2702 or 2802 controls at least one deflector such that, while at least one deflector is in a specific instantaneous position, light reflections from the field of view are received on at least one common area of at least one deflector. In some embodiments, at least some of the light reflections from at least some of multiple light sources overlap and are incident on each other in at least one common area. An example for illustrative purposes is given above with reference to method 2600 in Figure 26A.

[0480] In step 2669, at least one processor 118, 2702, or 2802 receives from each of the plurality of detectors at least one signal indicating light reflection from at least one common area while at least one deflector is at a specific instantaneous position. An example for illustrative purposes is given above with reference to method 2600 in Figure 26A.

[0481] Method 2660 may include additional steps such as any combination of one or more steps considered with respect to Methods 2600 and/or 2630 (but not limited to these steps).

[0482] As another example, similar to method 2600 in Figure 26A, method 2660 may include determining a plurality of distance measurements related to different regions of the field of view based on light reflection from at least one common area. Method 2660 may further include, similar to method 2600 in Figure 26A, controlling a plurality of light sources such that, at a first instantaneous position of at least one deflector, the first light source emits more light flux to at least one deflector than the second light source, and at a second instantaneous position of at least one deflector, the second light source emits more light flux to at least one deflector than the first light source.

[0483] While methods 2600, 2630, and/or 2660 describe using completely separate regions to form the field of view, these methods may be implemented using at least one pair of partially overlapping FOV regions. This may be implemented for different applications, such as (but not limited to) improving performance, increasing the maximum luminous flux in that area, calibration between the two regions, backup in case of failure of a critical part of the FOV, or when operating at different wavelengths. Some of these applications will be discussed in more detail below.

[0484] In embodiments where the multiple light sources may include at least two separate light sources, each light source may be configured to project light at a different wavelength (e.g., the light sources in Figures 27 and 28). A light source operating at a given wavelength may emit light in a certain wavelength band, which may be narrowband (e.g., a light source with a wavelength of 600 nm may emit a non-negligible amount of light in a bandwidth of ±2 nm, i.e., 598–602 nm). In some embodiments, the light sources of the LIDAR systems 100, 200, 2700, and/or 2800 may be coupled with filters, for example, to limit the wavelength range of the projected light. In some embodiments, at least the first light source may be configured to project light with wavelengths of 400–800 nm and/or 800–1000 nm, and at least the second light source may be configured to emit light with wavelengths greater than 800 nm (or 850 nm or 900 nm, etc.) and/or 1500 nm. In some embodiments, the first light source may be configured to project light of wavelengths such that both the light from the first light source and the light from the second light source are completely outside the visible range.

[0485] In such embodiments, at least two light sources may be configured to project light onto substantially overlapping areas of the field of view. Using different wavelengths in substantially overlapping areas makes it possible to detect objects that were not visible (or at least deemed invisible) at one wavelength using another wavelength.

[0486] Furthermore, in embodiments where multiple light sources are configured to project light of the same (or substantially the same) wavelength, the multiple light sources may be configured to project light into substantially overlapping field-of-view areas. This overlap allows for matching the results, reducing calibration or detection errors, and/or suppressing noise from one light source when a second light source is used. Furthermore, such overlap enables the detection of faulty equipment, such as a faulty light source or detector.

[0487] Although described as using completely separate light sources, the multiple light sources used in methods 2600, 2630, and/or 2660 may include a single light source coupled to a beam splitter. Thus, the beam splitter projects light beams from different directions to at least one deflector, thereby functioning similarly to completely separate light sources.

[0488] Figure 27 shows an exemplary LiDAR system 2700 having multiple light sources and a common deflector. As shown in Figure 27, light from multiple light sources can be incident on the overlapping area of at least one optical deflector. In addition to or instead of this, light emitted from multiple light sources and reflected back from the scene can be incident on the overlapping area of at least one optical deflector. As shown in Figure 27, at least one processor 2702 of system 2700 controls at least one deflector 2704. At least one deflector 2704 may be positioned at a specific instantaneous position during a scan cycle. At least one processor 2702 can further control multiple light sources (e.g., light sources 2706, 2708, and 2710).

[0489] Multiple light sources 2706, 2708, and 2710 can be directed toward the field of view 2712. As shown in Figure 27, the field of view 2712 includes a first region 2712a adjacent to the second region 2712b and spaced apart from the third region 2712c. Although shown as non-overlapping in Figure 27, some embodiments may include two or more regions with substantially overlapping areas.

[0490] The inset in Figure 27 shows the surface of the deflector 2704 (e.g., a single rectangular mirror), and further shows the illumination of light beams from three light sources 2706, 2708, and 2710 incident on this surface as light beam cross-sections 2742, 2744, and 2746. Different shaded levels represent the accumulated illumination levels of each area of the deflector 2704, whether illumination from a single light source, two light sources, or three light sources (e.g., area 2748). As shown in the inset, in area 2748, light from the three light sources 2706, 2708, and 2710 is incident on a common area of the deflector (not necessarily, but sometimes simultaneously). As further shown in the inset, some light from one or more of the light sources 2706, 2708, and 2710 may not be incident on the deflector (represented by dashed ellipses). For example, the light beam may be larger than the dimensions of the surface (shown on the vertical axis) and therefore smaller than the corresponding dimensions of the surface. The inset further shows that the light beams 2742, 2744, and 2746 may be of different sizes and/or at least partially mismatched with each other. In some embodiments, the reflected wavefront may be much larger than the optical aperture of the LIDAR system (e.g., system 2700), so the deflector 2704 can be set to a size and/or position such that light reflected from multiple light sources enters the entire active deflection area of the deflector.

[0491] In the embodiment of Figure 27, at least one deflector 2704 has a common area 2714. Multiple light sources 2706, 2708, and 2710 can be aimed at the common area 2714. Thus, multiple light sources 2706, 2708, and 2710 can emit corresponding multiple light beams along outbound paths (e.g., paths 2722a, 2722b, and 2722c) into areas 2712a, 2712b, and 2712c. Multiple light beams allow multiple corresponding reflections to travel from the field of view 2712 (or from objects within it) along return paths (e.g., paths 2724a, 2724b, and 2724c). As further shown in Figure 27, each reflection can be directed to the corresponding sensor (e.g., sensors 2716, 2718, and 2720) using a common area 2714 of at least one deflector 2704. As shown in Figure 27, the outbound paths 2722a, 2722b, and 2722c do not coincide, and the return paths 2724a, 2724b, and 2724c do not coincide. In other embodiments, at least two outbound paths and/or at least two return paths may coincide.

[0492] Figure 28 shows another exemplary LIDAR system 2800 having multiple light sources and a common deflector. As shown in Figure 28, light from multiple light sources can be incident on the overlapping area of at least one optical deflector. In addition to or instead of this, light emitted from multiple light sources and reflected back from the scene can be incident on the overlapping area of at least one optical deflector. As shown in Figure 28, at least one processor 2802 of system 2800 controls at least one deflector 2804. At least one deflector 2804 may be positioned at a specific instantaneous position during a scan cycle. At least one processor 2802 can further control multiple light sources (e.g., light sources 2806 and 2808). In the example of Figure 28, light sources 2806 and 28082 may have different wavelengths. In other embodiments, light sources 2806 and 28082 may have different maximum power, duty cycle, pulse timing, pulse length, etc.

[0493] Light sources 2806 and 2808 can be directed toward the field of view 2810. As shown in Figure 28, the field of view 2810 includes a first region 2810a and a second region 2810b. Although shown as non-overlapping in Figure 28, some embodiments may include two or more regions with substantially overlapping areas.

[0494] In the embodiment of Figure 28, at least one deflector 2804 has a common area 2812. Light sources 2806 and 2808 can be aimed at the common area 2812. Thus, multiple light sources 2806 and 28082 can emit corresponding multiple light beams along outbound paths (e.g., paths 2816a and 2816b) into areas 2810a and 2810b. Multiple light beams allow multiple corresponding reflections to travel from the field of view 2810 (or from an object in it) along return paths (e.g., paths 2818a and 2818b). As further shown in Figure 28, each reflection can be guided to a single sensor 2814 using the common area 2812 of at least one deflector 2804. As shown in Figure 28, the outbound paths 2816a and 2816b do not coincide, but the return paths 2818a and 2818b coincide. In other embodiments, at least two outbound paths may coincide, and/or at least two return paths may not coincide.

[0495] In some examples, at least one deflector 2704, 2804, 114, etc., may include a deflector array (e.g., a pivot mirror array such as a pivot piezoelectric mirror array), as shown, for example, in Figure 3B. When such a deflector array is used for both projection and detection (both the transmit and receive paths), arbitrary combinations of individual deflections can be used for transmit, receive, and/or bidirectional use. For example, if the array includes 10 mirrors, which are used by three light sources to illuminate different parts of the FOV, three mirrors may be used for transmission, each corresponding to one light source, and the remaining seven mirrors may be used for reception. Alternatively, one or two mirrors may be used for transmission for all light sources (e.g., by utilizing the overlapping area of those mirrors, an example of which is shown in Figure 27), and all 10 mirrors may be used for reception. Any other combination can also be used. In some embodiments, some or all of the individual deflectors in the deflector array can be synchronized to deflect light to approximately the same angle. In some embodiments, some or all of the individual deflectors in a deflector array can be synchronized to deflect light in different directions while maintaining a fixed angle between them.

[0496] Optical budget allocation in Lidar

[0497] As described herein, one function of a LiDAR system may be to generate a three-dimensional depth map of the environment surrounding the LiDAR system by projecting light into the environment and then collecting and analyzing the light reflections from objects in the environment. In general, the usefulness of a LiDAR system and its depth map can increase with the level of information obtained from the collected light and the resolution of the depth map generated. However, it may not be possible to generate a high-resolution depth map simply by increasing the amount of light energy emitted by the LiDAR system into the environment due to practical constraints. Firstly, eye safety is the most important constraint that may limit the amount of light energy that a LiDAR can output. To ensure eye safety and to comply with applicable regulations, a LiDAR system may be limited to light projection that does not exceed a certain energy density over a specific time period. Furthermore, even when eye safety is not critical, other practical constraints may exist that prohibit the LiDAR from emitting light into the environment without restriction. For example, a LiDAR system may have a finite optical budget and/or computational budget that limits its ability to improve detection resolution simply by increasing the overall light emission to the LiDAR FOV. Conceptually, the optical budget and computational budget can reflect the maximum functionality of the LiDAR system over a specific time period with respect to available optical output power and computational power. Furthermore, a LiDAR system may also be limited by technical constraints such as power constraints, overheating, and light source output.

[0498] However, the depth maps generated by the LIDAR system must not be limited to absolute resolution levels in all areas of the LIDAR FOV. As will be discussed below and throughout various sections of this disclosure, the optical and computational budgets of the LIDAR system may be limited to LIDAR in specific areas of the LIDAR FOV.
It is possible to allocate more resources (e.g., more optical and/or computational budgets) to certain areas of the FOV than to other areas. As a result, it may be possible to generate depth maps with high resolution in some areas (e.g., areas corresponding to regions of interest) and low resolution in other areas (e.g., regions of low interest or regions of no interest). The following descriptions, and many sections of this disclosure, address various situations, conditions, and circumstances in which an uneven allocation of optical and/or computational budgets is desirable. The following descriptions, and the overall description, also provide examples of how available optical and/or computational budgets can be dynamically allocated to help generate depth maps that can provide a high level of information in one or more regions of interest included in the depth map.

[0499] Figure 29 provides a block diagram of the LiDAR system 100, along with various information sources available to the LiDAR system 100 when allocating available optical budgets and/or computational budgets. In some embodiments, the LiDAR system 100 may include at least one processor 118 configured to access optical budgets (or any information that represents at least one aspect of the optical budget, or that can derive or determine the optical budget) stored in memory 2902. The optical budget is associated with at least one light source 112 and defines the amount of light that can be emitted by at least one light source within a given time period. Memory 2902 may be associated with a processing unit 108 of the LiDAR system 100, as shown in Figure 29. However, in some embodiments, memory 2902 may be associated with a host (e.g., a vehicle, a vehicle computer) on which the LiDAR system 100 is deployed. For example, in some embodiments, the memory 2902 is associated with the electronic control unit 2904 of the host vehicle and is accessible by the processor 118 via the data bus 2900. In other embodiments, the memory 2902 can be associated with one or more systems located remotely from the LiDAR system 100 (or its host). For example, in some embodiments, the memory 2902 is associated with a remote server (not shown) and is accessible, for example, via a cloud 2916 (e.g., an internet connection) or using a wireless transceiver 2901.

[0500] The processor 118 may also be configured to receive information indicating platform conditions for the LiDAR system (for example, from any of information sources 2904, 2906, 2908, 2910, 2912, 2914, 2916, 2918, 2920, or from any other suitable information source). Platform conditions for the LiDAR system include any operating parameters, parameter values, observed conditions, instructions, information items, etc., relating to one or more aspects of the LiDAR system, the environment surrounding the LiDAR system, the host on which the LiDAR system is deployed, etc., and may indicate the appropriateness of allocating more light to at least one group or one scan cycle of the LiDAR FOV portion than to another group or another scan cycle of the LiDAR FOV portion during a particular time period.

[0501] The received information may be obtained from one or more external sources of the LiDAR system 100, but information indicating the platform conditions of the LiDAR system may also include information obtained from internal sources of the system 100 (e.g., via one or more components of the system including the light projector 112, deflector 114, detector 116, feedback element, etc.). Based on the received information, the processor 118 can dynamically allocate the optical budget to the field of view of the LiDAR system 100 using two or more operating parameters related to the light source 112 and/or deflector 114, including, for example, scan rate, scan pattern, scan angle, spatial light distribution, and/or temporal light distribution. The processor 118 can further output signals to control the light source 112 and/or deflector 114 so that the light beam can be varied in scanning the field of view of the LiDAR system 100 according to the dynamically allocated optical budget.

[0502] The optical budget can be expressed in terms of any parameter, value, or set of parameters or values relating to the amount of light that can be emitted to the LIDAR FOV over a specific time period (e.g., LIDAR scan cycle units, milliseconds, seconds, or any other arbitrary time period indicative). In some embodiments, the optical budget of a LIDAR system may depend on the function of one or more light sources included in the LIDAR system. For example, the optical budget of LIDAR system 100 can be associated with light source 112 and define the amount of light that light source 112 can emit over a predetermined time period. Defining the amount of light is any parameter or parameter relationship that indicates the amount of light (e.g., power, luminous intensity, luminous flux, intensity, number of photons, number of light pulses, duty cycle, pulse width, pulse amplitude, irradiation duration, etc.) against some measure of time (e.g., microseconds, milliseconds, seconds, minutes, etc.).

[0503] In some examples, the average optical budget of light source 112 may be between approximately 10 milliwatts and 1,000 milliwatts. In addition to or instead of this, the optical budget may refer to the amount of light that can be emitted in a single scan cycle of the LIDAR FOV. For example, the optical budget of LIDAR system 100 may be between 10,000 pulses per light source per scan cycle and 50,000 pulses per light source per scan cycle (e.g., to cover 1,000 to 10,000 beam locations, each associated with one or more pixels). In some embodiments, the optical budget can be expressed as the power available by light source 112 (e.g., from the vehicle or other host in which LIDAR system 100 is deployed). Alternatively, the optical budget can be defined by the amount of light that can be emitted by light source 112 (or any available light source in system 100) in a standard time unit (e.g., milliseconds, seconds, minutes, etc.).

[0504] In some cases, the optical budget may remain fixed. In other cases, the optical budget stored in memory 2902 may be changed and updated. Such changes may be made, for example, based on at least one of the operating parameters of the LIDAR system and the detection information provided by the LIDAR system.

[0505] Furthermore, in some embodiments, the optical budget may correspond to a single LIDAR system having a single light source. In other cases, the optical budget may relate to a single LIDAR system including multiple light sources. In yet another case, the optical budget may apply to multiple LIDAR systems deployed at different locations (e.g., different locations around a vehicle), each including a single or multiple light sources. In any case, the optical budget can define the amount of light that can be emitted from multiple light sources (or multiple LIDAR systems as a whole) that are available for allocation over a given time period. The processor 118 can dynamically allocate the optical budget for a single LIDAR system/light source. In other cases, the processor 118 can dynamically allocate the optical budget related to multiple light sources/LIDAR systems.

[0506] In addition to the optical budget, the LiDAR system 100 (or a combination of multiple LiDAR systems) may have a computational budget that can be allocated. The computational budget generally refers to the processing functions of one or more LiDAR systems over a specific time period. The processing functions may depend on the number of available processors (for example, to control various aspects of the LiDAR system, to receive and process detected reflections, to generate depth maps, to process depth maps, to detect objects and other high-level scene understanding information, and to perform any other functions related to the LiDAR system or group of LiDAR systems). The processing functions may depend not only on the number of available processors but also on other parameters. These other parameters may include, for example, the portion of the processing functions of one or more processors dedicated to specific functions of the LiDAR system (e.g., depth map generation, FOV scanning control, object detection, identification, and/or classification), the processing speed of one or more available processors, the data transfer rate (e.g., via bus 2900), and the number of calculations per unit time that can be performed by one or more available processors.

[0507] The following description includes details regarding the allocation of the optical budget, but the computational budget can be allocated in a similar manner to that described for the optical budget. For example, in some cases, the computational budget relates to the amount of computational resources available for processing point clouds to determine what the LiDAR has detected. In some cases, processing point clouds requires significant or limited computational resources. Therefore, in some cases, it may be desirable to determine that certain areas are of greater interest/importance than others in order to process the relevant point clouds. For example, the area in front of a vehicle may be the most important, especially for a moving vehicle, so much of the available processing power can be dedicated to processing point clouds and generating depth maps. On the other hand, detections performed in the field of view extending from the sides of the vehicle are also important, but in some cases less important than those in front of the vehicle (unless the vehicle changes direction or stops, for example). In such an example, even if grouped detections are detected by LIDAR from reflected signals from a highly reflective object located 130 meters from the host vehicle, the processor 118 can decide to process only the relevant point cloud up to 40 meters (or less than 130 meters) from the vehicle to save on computational costs. (This is because, for example, processing the entire point cloud, including the grouped detections at 130 meters, would be too costly from a computational standpoint. This is especially true when the computational expenditure on the side of the vehicle, as in this example, is not justifiable given the importance of object detection.)

[0508] Computational budgets can be allocated among available LiDAR systems not only to give one LiDAR system more computing power than others by, for example, allocating a larger portion of the computing power of one or more centralized processors to one LiDAR system than to others, but also, in another example, to integrate/network the processors of two or more LiDAR systems, and allocate this integrated processing power so that at least a portion of the processors from one LiDAR system are dedicated to the computing tasks of different LiDAR systems. For example, the processing power from multiple available LiDAR systems can be dedicated to computing tasks related to the area in front of the host vehicle, i.e., areas where high-resolution object detection and depth mapping are desired.

[0509] It is also possible to allocate the computation budget to calculations related to specific LiDAR FOVs so that computation tasks related to one part of the FOV receive more computation budget than computation tasks related to another part of the FOV. Some examples of how to allocate the computation budget include, for example, detection/clustering (point cloud points to object level), fixing the bounding box of an object ("bounding box"), object/object type classification, object tracking (e.g., between frames), and determining object features (e.g., size, orientation, velocity, reflectivity, etc.). The computation budget can be expressed in terms that relate processing capacity to time (e.g., GMAC, Gflops, power, etc.). It should be noted that budget allocation to different parts of the FOV (especially computation budget, but not limited to this) can relate to FOV division in 3D as well as 2D. For example, the calculation budget can be allocated such that, for a given sector of the FOV (e.g., a given 1° × 0.5° sector), 70% of the budget is allocated to processing detections beyond 70m, 30% is allocated to processing detections within a LIDAR range closer than 40m, and no budget is allocated to the range between 40m and 70m.

[0510] Returning to the optical budget, the available optical budget can be allocated to selectively provide more light to one group of LIDAR FOV portions than to another group of LIDAR FOV portions within a specific time period. In this context, a group of LIDAR FOV portions can refer to one or more portions of a particular LIDAR FOV (e.g., one or more pixels, regions, subregions, etc., of a particular LIDAR FOV) or to one or more complete LIDAR FOVs (e.g., when the optical budget can be allocated across multiple LIDAR systems). More light can refer to an increase in luminous flux, an increase in luminous density, an increase in the number of photons, etc., as illustrated in detail above.

[0511] In some cases, the allocation of the optical budget may be achieved by changing the scan rate, scan pattern, scan angle, spatial light distribution (e.g., giving more light to one or more groups of LIDAR FOV portions than to one or more other LIDAR FOV portions), and/or the temporal light distribution associated with a particular LIDAR FOV. The temporal light distribution may include, for example, controlling or otherwise changing the luminous flux or amount applied to a group of LIDAR FOV portions over time so that the total amount of light projected in a first scan cycle is greater than the total amount of light projected in a second subsequent scan cycle. In some cases, the allocation of the optical budget may be achieved by changing two or more of the scan rate, scan pattern, scan angle, spatial light distribution, or temporal light distribution associated with a particular LIDAR FOV or a particular LIDAR FOV portion. Such changes can be applied to two or more LiDAR FOVs, one LiDAR FOV, a portion of a LiDAR FOV (e.g., region of interest), one scan cycle, multiple scan cycles, etc.

[0512] At least part of the dynamic allocation of the optical budget (e.g., modifying or updating the allocation based on feedback or other information received regarding the platform conditions of at least one LIDAR system) may be performed by determining the scan rates of one or more LIDAR systems. For example, at least one processor may be configured to determine the scan rate of at least one of the following: the near-field portion of the LIDAR FOV, the far-field portion of the field of view, the narrow-angle sector of the field of view, and/or the wide-angle sector of the field of view.

[0513] As described above, optical distribution can be achieved, at least in part, by determining the scan pattern for at least one scan cycle of one or more LIDAR systems. The scan pattern can be determined based on the recognition of at least one of the following situation types: driving on a main road, driving off-road, driving in rain, driving in snow, driving in fog, driving in an urban area, driving in a rural area, driving in a tunnel, driving in an area near a predetermined facility, turning left, turning right, crossing a lane, approaching an intersection, and approaching a pedestrian crossing.

[0514] The allocation of the optical budget can be achieved by any suitable processor. In some cases, the processor 118 of the LIDAR system 100 can allocate the optical budget based on information from one or more sources. Alternatively, or in addition to this, the optical budget (e.g., the optical budget associated with a group of LIDAR systems) can be allocated using a processor of another LIDAR system, and/or one or more processors associated with a LIDAR system host (e.g., a vehicle ECU) can be used. It is also possible to allocate the optical budget using any other available processor.

[0515] As noted, optical budgeting can provide more light to one group of LIDAR FOV portions than to others. Such variations in the amount of light applied can be achieved, for example, by changing the optical budgeting ratio (or similar ratio between LIDAR detectors) of optical budgeting to a second light source among multiple light sources. Optical budgeting can also be applied to different LIDAR FOV portions or at different times for different reasons. For example, optical budgeting can target certain portions of the LIDAR FOV or at certain points in a scan cycle to increase resolution, improve detection quality, etc., in a particular FOV portion or time period. In other situations, optical budgeting can target certain FOV portions, specific sub-regions of the FOV, or increase the detection range over a particular time period. In general, optical/power budgeting can be used to achieve various goals when acquiring different frames or different portions of acquired frames. Thus, a LIDAR system can provide a series of useful or high-quality frames for various ROIs, each useful for different reasons. Thus, optical budgets can be spent in a way that is deemed to have a high probability of returning useful information to the host platform (e.g., a vehicle's navigation system).

[0516] With regard to control, any appropriate parameters or information elements can be used in deciding whether or not to allocate and/or how to allocate, and in determining the optical budget. In some embodiments, platform conditions of the LiDAR system can be used as the basis for optical budget allocation. As described above, platform conditions of the LiDAR system refer to any operating parameters, parameter values, observed conditions, commands, information items, etc., relating to one or more aspects of the LiDAR system, the environment surrounding the LiDAR system, the host on which the LiDAR system is deployed, etc., that can demonstrate the appropriateness of allocating more light to at least one group or one scan cycle of the LiDAR FOV portion than to another group or another scan cycle of the LiDAR FOV portion over a specific time period.

[0517] Such platform conditions for a LiDAR system can be determined in various ways and using any suitable information source. In some examples, the platform conditions for a LiDAR system may be determined internally within the LiDAR system. For example, the processor 118 can determine one or more features related to the environment in which the LiDAR system is deployed based on acquired light reflections, reflectance signatures, depth maps, etc. In other examples, the platform conditions (PCLS) for a LiDAR system may be determined based on information received from one or more sources separate from the LiDAR system 100. For example, as shown in Figure 29, the PCLS can be determined based on information from one or more electronic control units 2904 of the host vehicle, one or more temperature sensors 2906, a GPS receiver 2908, a vehicle navigation system 2910, a radar unit 2912, one or more other LiDAR systems 2914, the Internet or other network connection 2916, a camera 2920, or any other suitable source.

[0518] In some examples, information indicating PCLS can establish one or more regions of the LIDAR FOV as regions of interest where a larger proportion of the optical or computational budget is appropriate than other regions (e.g., regions of low interest or regions of no interest). Regions of interest can be identified based on the detected current driving mode of the vehicle in which the LIDAR system is deployed, and can be determined based on one or more outputs from any of the information sources 2904, 2906, 2908, 2910, 2912, 2914, 2916, 2920, or the LIDAR system 100, or any combination thereof. In one example, a region of interest based on the detected current driving mode may include one or more portions of the LIDAR FOV that overlap with the area in which the host vehicle is turning and proceeding (communicated by the navigation system 2910, GPS receiver 2908, etc.). In another example, the region of interest may correspond to one or more portions of the LiDAR FOV in which the LiDAR system 100 detects an object such as another vehicle, pedestrian, or obstacle. Other examples of regions of interest, and examples of how to identify such regions, are included in other sections of this disclosure.

[0519] Information indicating the PLC that can determine the optical distribution (or calculation budget) may include, in particular, at least one of the following: vehicle operating parameters, environmental conditions, driving decisions, vehicle navigation status, or power management mode.

[0520] Examples of vehicle operating parameters or vehicle navigation states that may form the basis for optical allocation (or calculation budget) may include current speed (e.g., from ECU 2904, GPS 2908), current vehicle direction of travel (e.g., from GPS 2908, navigation system 2910), current braking or acceleration state (e.g., from GPS 2908, ECU 2904), and whether the host vehicle is navigating lane crossing situations (e.g., from navigation system 2908, camera 2920, GPS 2908, etc.). Vehicle operating parameters may also relate to the conditions or states of any component related to the vehicle platform on which the LIDAR system 100 is deployed, or to the conditions or states of any component of the LIDAR system 100 itself. Such conditions may include the temperature of at least one component of the LIDAR system, whether a portion of the FOV is obstructed (e.g., by rain, mud, debris, etc.), whether the lens is scratched, whether the deflector 114 is prevented from reaching a specific instantaneous position, and whether there is more internal reflection at one angle than at another. The vehicle's navigation state may also include the host vehicle's position relative to a 3D map, partial map, 2D map, target object, or any combination of map and target object. The map can be pre-stored, received via a communication channel, or generated (e.g., by SLAM).

[0521] Examples of environmental conditions may include at least one of the following: climatic conditions (e.g., rain, snow, fog, etc., determined based on information from camera 2920, cloud 2916, navigation system 2910), lighting conditions (e.g., determined based on information from LIDAR system 100 (ambient light, type of light source, etc.)), environmental temperature (e.g., based on output from temperature sensor 2906), and/or proximity to a predetermined type of facility (e.g., a school, determined based on input from navigation system 2910, GPS 2908, camera 2920, etc.). Additional environmental conditions that may form the basis for optical budget (or computation budget) allocation include: climatic conditions, the location or distribution of detected objects in space (e.g., relative to the LIDAR system 100 and/or host vehicle), detected features of objects in space (e.g., shape, reflectivity, features affecting SNR), type/class of objects (e.g., pedestrians, buildings, vehicles, traffic signal poles), relative position of the sun or other light sources, traffic conditions (e.g., congested or empty main roads), the status of other host vehicle systems (e.g., driving-related sensors or other sensors; in some examples, the LIDAR system 100 can compensate for a malfunctioning camera 2920), conditions of the road itself (e.g., bumps, roughness, inclines/declines, curves, reflectivity), map/GPS-based data (e.g., location and orientation of roads in the scene, location and orientation of buildings in the scene, (LID) Since AR may receive reflections from objects far from a building, which may not be expected, a low-interest area can be established relative to buildings and other obstacles. This may include the ambient temperature of the LIDAR system 100, the ambient temperature of the host vehicle environment, and data analysis from previously collected FOV frames (e.g., point cloud, surface normals, reflectivity, confidence levels, etc.). Generally, the optical/power budget can be allocated based on knowledge of the environment. For example, GPS data, map data, processed LIDAR information from previous frames, data from other sensors on the vehicle, or any other source may indicate the presence of a building within a given range (e.g., 15 m) in a portion of the FOV. While that building may be in a high-interest area (e.g., directly in front of the vehicle), the processor can still allocate relatively low power to this portion of the FOV and extra energy to other parts of the LIDAR FOV, thus avoiding wasting the budget on portions of the FOV that are hidden behind the building (e.g., beyond 15 m) and therefore unreachable regardless of the amount of light that can be allocated in that direction.

[0522] Examples of driving decisions that may form the basis for optical allocation (or calculation budget) may include at least one of the following: rural-related instructions, urban-related instructions, the current speed of the vehicle including the LiDAR system, the next driving procedure, conditional driving procedures (procedures that can only be completed if additional environmental information exists indicating that their execution is safe), driving navigation events, manual driving instructions, and automated driving instructions. Such information can be obtained, for example, based on the output provided by the LiDAR system 100 or 2914, the navigation system 2910, the EVU 2904, the GPS 2908, any combination of their sources, or other potential sources of PCLS indicators.

[0523] Examples of power management modes that may form the basis for optical distribution (or calculation budget) may include at least one of the normal power operation mode and power saving mode indications. Such information may be obtained, for example, from the EVU 2904 and may reflect the amount of power available from the host vehicle. Other indicators of power management modes may be based on the detected conditions of one or more components of the LIDAR system 100 (e.g., whether any component is overheating or at risk of overheating).

[0524] Several examples can further illustrate the collection of PCLS data that may form the basis for optical or computational budget allocation. For example, during operation, the processor 118 may receive inputs indicating the vehicle's current operating environment. For example, the processor 118 may receive inputs including at least one rural-related instruction and at least one urban-related instruction. In another example, the processor 118 may receive inputs including at least one rural-related instruction, at least one urban-related instruction, information related to light conditions, information related to climatic conditions, and information related to vehicle speed.

[0525] In some embodiments, the processor 118 can receive input from decisions made by the processor 118 itself. In such examples, the processor 118 can determine the current driving environment based on information from one or more previous (and/or current) scans of the field of view. For example, the processor may determine that the current driving environment is urban based on the presence of many vehicles and/or buildings very close to the vehicles. In another example, the processor may determine that the current driving environment is rural based on the presence of many trees and/or open land. Alternatively or simultaneously, the processor 118 may determine the current driving environment based on the vehicle's speed and/or map information (which can be stored or received and may include updated traffic information). For example, the processor 118 may determine that the current driving environment is an interstate or highway based on the vehicle maintaining a high speed and/or the vehicle's location matching a known interstate or highway. As another example, the processor 118 can determine that the current driving environment is congested based on the fact that the vehicle frequently stops while maintaining a low speed and/or based on known traffic information.

[0526] Alternatively, or simultaneously, the processor 118 can receive input from the host vehicle's processing unit (e.g., ECU 2904). The central computer can determine the current driving environment using the techniques described above with respect to the processor 118. Similarly, the processor 118 can receive input from remote systems, in addition to or instead of this. For example, the processor 118 can receive climate instructions from a climate server or other source of updated climate information. Similarly, the processor 118 can receive traffic instructions from a traffic server or other source of updated traffic information.

[0527] In some embodiments, the processor 118 can receive input indicating the current driving environment from at least one of the following: GPS, vehicle navigation system, vehicle controller, radar, LiDAR, and camera, as shown in Figure 29. For example, as described above, the processor 118 can derive the current driving environment by using the vehicle location determined by GPS and/or the vehicle navigation system in combination with a map and/or traffic information. In such an example, the processor 118 can determine that the vehicle is located on an interstate highway by combining the vehicle's GPS location with a map, or that the vehicle is in a traffic jam by combining the vehicle's GPS location with traffic information. Similarly, the processor 118 can derive the current driving environment as described above using speed, direction of travel, etc., from the vehicle controller. In addition to or instead of this, the processor 118 can derive the current driving environment using information from radar, LiDAR, and/or camera. For example, the processor 118 can use radar, LiDAR, and/or cameras to identify one or more objects such as fields, trees, buildings, and median strips, and use these identified objects to derive the current driving environment.

[0528] Once an optical budget or computational budget has been allocated and a plan has been made to apply the allocated budget, the processor 118 (or other processing device) can implement this plan. For example, the processor 118 can output signals to control at least one light source 112 and/or optical deflector 114 or any other component that affects the beam of light to the LIDAR FOV (spatially or temporally) so that the beam of light can be varied in scanning the LIDAR FOV according to the dynamically allocated optical budget. In some examples, the application of the allocated optical budget may allocate more beam of light to a particular part (e.g., ROI) of one or more LIDAR FOVs, which may require reducing the beam of light to other areas (e.g., low-interest or non-interest areas). To execute a plan to implement the allocated optical budget, the processor 118 is configured to control at least one optical deflector 114 for scanning the FOV, and during the scan cycle, at least one optical deflector 114 may be positioned at a plurality of different instantaneous positions. Furthermore, the processor 118 can coordinate (for example, synchronize) at least one optical deflector 114 and at least one light source 112 such that when at least one optical deflector is positioned at a specific instantaneous location, a portion of the light beam is deflected by at least one optical deflector from at least one light source to an object in the field of view, and a portion of the light beam reflected from the object is deflected by at least one optical deflector to at least one sensor 116. In some examples, the LIDAR system 100 may include multiple light sources aimed at at least one optical deflector 114, and the processor 118 can be configured to control at least one optical deflector 114 such that when at least one optical deflector 114 is positioned at a specific instantaneous location, light from the multiple light sources is projected onto multiple distinct regions of the LIDAR FOV. Generally, the processor 118 can coordinate at least one light source 112 and at least one optical deflector 114 according to a dynamically allocated optical budget. By applying the allocated optical budget, more light per unit time can be applied to regions of high interest, and less light per unit time to regions of low interest. Furthermore, based on the detection of objects in one or more parts of the LIDAR FOV, the processor 118 can prevent the accumulated energy density of light projected onto a specific area (whether it is a region of interest or a region of low interest) from exceeding the maximum allowable exposure.

[0529] Figure 30A shows a flowchart giving an example of a method 3000 for controlling a LiDAR system based on an allocated budget, according to an embodiment disclosed. For example, in step 3002, the processor 118 (or other available processing device) can receive information indicating one or more platform conditions (PCLS) of the LiDAR system. As described above, these PCLS may include any conditions related to the LiDAR system 100 or the platform host on which it is deployed, where an uneven light distribution may be desirable. In step 3004, the processor 118 can determine system constraints that may partially assist in defining an optical budget or a computation budget (e.g., the optical output capabilities of available light sources, the processing capabilities of available CPUs, etc.). Based on the information obtained in steps 3002 and 3004, the processor 118 can determine an allocated optical budget and/or an allocated computation budget in step 3006. In step 3008, the processor 118 can create a scan plan for applying the allocated budget to the operation of one or more LiDAR systems. In step 3010, the processor 118 can control the light projection for each beam spot (e.g., light projection from a specific instantaneous position of the deflector 114) by controlling the operation of the light source 112 and the deflector 114, for example, based on the allocated budget. For example, it can provide a region of interest with more luminous flux per unit time than is applied to a region of low interest. In step 3012, the processor can detect and process reflected light, for example, based on the output of the detector 116. In step 3014, the processor 118 can determine whether the default application of the allocated optical budget for a particular beam spot has been completed. If completed, the process can return to step 3010 to continue controlling the light projection at another beam spot. If not completed, in step 3016, the processor 118 can determine whether further beam spot projection is permissible (e.g., whether further projection complies with eye safety regulations, whether it exceeds the maximum permissible luminous flux for a particular beam spot, etc.). If further projection is not permissible, the process can return to step 3010 to continue controlling the light projection at another beam spot. If further projection is permitted, in step 3018, the processor 118 can determine whether further projection is necessary at this particular beam spot (for example, whether sufficient data or detection has already been obtained based on previous projections or previously illuminated pixels associated with this particular beam spot). If additional projection is not required, the process can return to step 3010 to continue controlling the light projection at another beam spot. Optionally, the processor 118 may decide to redistribute any remaining unused power allocated to the current beam spot to at least one other beam spot within the same scan cycle. If additional projection is permitted, in step 3020, the processor 118 may perform additional light projection at the particular beam spot, and then return to step 3012 for detection and processing of reflected light.

[0530] Figure 30B shows a flowchart of an exemplary method 3050 for controlling a LiDAR system according to embodiments disclosed herein. Step 3062 may include accessing an optical budget stored in memory. The optical budget is associated with at least one light source and defines the amount of light that can be emitted by at least one light source within a predetermined time period. Step 3064 may include receiving information regarding vehicle operating parameters, including at least one of environmental conditions, driving decisions, and power management modes. Based on the received information, step 3066 may include dynamically allocating the optical budget to the field of view of the LiDAR system based on at least two of the scan rate, scan pattern, scan angle, spatial light distribution, and temporal light distribution. Step 3068 may include outputting a signal to control at least one light source so that the light beam can be varied in scanning the field of view according to the dynamically allocated optical budget.

[0531] In some embodiments, the dynamic allocation of the optical budget based on the spatial light distribution may include projecting more light onto the first portion of the field of view than onto the second portion of the field of view during a single scan cycle. In subsequent scan cycles, the dynamic allocation of the optical budget based on the spatial light distribution may include projecting more light onto the second portion of the field of view than onto the first portion of the field of view during this scan cycle. This method may further include obtaining identification of the first region as a region of interest and identification of the second region as a region of no interest (or low interest). This method may also include determining the presence of objects in the second portion and preventing the stored light energy density in the second portion from exceeding the maximum allowable exposure.

[0532] In another example illustrating how allocated optical budgets can be applied during the operation of a LiDAR system, the processor 118 can be configured to allocate more optical budget (e.g., more luminous flux/FOV) to a specific LiDAR system installed in a vehicle based on a PCLS indicating a failure in another LiDAR system in the same vehicle. Such an allocation can at least partially compensate for the failed LiDAR system. For example, an allocation to a functioning LiDAR system may include emitting pulses in portions of the LiDAR FOV that normally do not transmit pulses at all (or transmit very few pulses) during normal operation. Such an allocation may also include changing deflector parameters to scan a wider FOV, for example.

[0533] To further illustrate this example, Figure 31 shows a schematic diagram of a vehicle with seven LiDAR systems installed at different locations within the vehicle. Each LiDAR device may exhibit different parameters in terms of field of view, range, resolution, accuracy, etc. The installed LiDAR systems may be connected by a bus (e.g., a CAN bus) that provides communication access between these systems, and possibly between other components as shown in Figure 29. During operation, various LiDAR devices can broadcast their operating parameters to each other as part of the function exchange boot process or by on-demand status requests. This information exchange allows the processor of another LiDAR system, such as LiDAR system #7, to recognize that LiDAR system #2 has failed (e.g., based on received error messages, health indicators, etc.). In some cases, a failed device on the bus may not be able to report (e.g., has lost power). In that case, the system that does not report is no longer connected to the shared bus and is assumed to be failed.

[0534] One or more other LiDAR systems can take action to at least partially compensate for a failed LiDAR device. For example, as shown in Figure 31, a failure of LiDAR system #2 may result in a blind spot in the vehicle sensor system. A monitoring layer in the HW or FW (a main controller connected to the bus or a designated master LiDAR) detects that LiDAR #2 is not functioning and designates another LiDAR in the system to compensate for the coverage loss. In this particular example, considering the extended capabilities of LiDAR #7, it was found to be the best option for compensating for the coverage loss. LiDAR #7 is designated to operate in backup mode and expand its field of view to cover the field of view of LiDAR #2. Expanding the scan range of LiDAR #7 may be done at the expense of some of its capabilities, such as a reduction in the entire range, resolution, or frame rate. The updated sensor status with a reduced set of performance parameters can be communicated to the entire system at the vehicle main controller level to compensate for the vehicle behavior. Similar to a narrow spare tire that limits a vehicle's speed to 80 km/h, a vehicle's top speed can be restricted. Compensating for a faulty sensor is ultimately necessary to maintain a minimum level of autonomy so that the vehicle can safely reach a repair location without human intervention.

[0535] Optionally, computing resources can be shared by multiple sensors of two or more types (e.g., LiDAR, camera, ultrasonic sensor, radar) or allocated to processing detection information arriving from different types of sensors. This can be done, for example, by a host computer integrating information from different sensors in a vehicle, such as an autonomous vehicle. The methods and processes disclosed above for allocating a computing budget (e.g., method 3000) can be extended to allocate the computing budget between processing information collected by different types of sensors. Optionally, this can be further extended to allocate the computing budget in various ways between each FOV portion of multiple sensors of different types, and to shift computing resources between different types of detection data based on various parameters such as platform conditions for the vehicle or any system installed in the vehicle. Such parameters may include, for example, vehicle operating parameters, environmental conditions, driving decisions, vehicle navigation status, or power management mode, or any combination of one or more system parameters of one or more detection systems (e.g., LiDAR, radar). Allocation of the computing budget for processing a first sensor of a first type can be based on processing another sensor of a different type.

[0536] For example, if a camera detects an object hanging in one of the ROIs, the allocation processor can allocate more of the LiDAR computation budget to process the detection information from there in the FOV, at the expense of processing LiDAR detection information from other parts of the FOV. In another example, the allocation processor can allocate the computation budget based on detection results and/or platform parameters such that some parts of the FOV (which can, of course, be defined as 3D as well as 2D) are analyzed primarily using detection information from a first type of sensor, and other parts of the FOV are analyzed primarily using detection information from a second type of sensor. In a more advanced allocation scheme, the host (or another processor) can also shift power allocation between different types of sensors by making the necessary modifications, for example, based on one of the previously disclosed parameters and according to one of the considerations described above.

[0537] It should be noted that many of the methods, processes, and techniques of the LIDAR system discussed throughout this disclosure, when considered in conjunction with Method 3000 (and together with the overall consideration of budget allocation), may form part of a broader budget allocation scheme combining any two or more of the disclosed methods, processes, and techniques. Such methods, processes, and techniques may be adapted to various parts of the disclosed budget allocation scheme. For example, some of these methods, processes, and techniques can be used to determine the budget allocation rates for different parts of the FOV. Some of these methods, processes, and techniques can be used to determine the limit rates for budget allocation to different parts of the FOV. Some of these methods, processes, and techniques can be used to utilize the budget allocated to different parts of the FOV, and so on.

[0538] According to some embodiments, the LIDAR system 100 may include the following:
a. A photon emitter assembly (PTX), such as projection unit 102 (or part thereof), for generating pulses of test photons. The pulses are characterized by at least one pulse parameter.
b. A photon receiving and detection assembly (PRX) for receiving reflected photons that have been reflected back from an object. The PRX includes a detector (e.g., detector 116) for detecting the reflected photons and for generating a detected scene signal (e.g., by processor 118). The photon receiving and detection assembly may include a detection unit 106 (or part thereof) and a processing unit 108 (or part thereof).
c. A photon steering assembly (PSY), such as a scan unit 104 (or part thereof), which is functionally associated with both the PTX and PRX to guide the pulse of the inspection photon towards the scene segment being inspected and to return the reflected photon to the PRX.
d. A closed-loop controller (hereinafter also referred to herein as the "controller") implemented by the processing unit 108 (or a part thereof, such as at least one processor 118) for (a) controlling PTX, PRX, and PSY, (b) receiving a detected scene signal from the detector, and (c) updating at least one pulse parameter based at least partially on the detected scene signal.

[0539] According to some embodiments, at least one pulse parameter may be selected from pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, and/or polarization.

[0540] According to some embodiments, the controller may include a situation assessment unit for receiving a detected scene signal and generating a scan/operation plan. The operation plan may include some or all of the determined budget allocation and may also include additional operation decisions (e.g., scan patterns). The situation assessment unit may receive photon steering assembly feedback from the photon steering assembly. The situation assessment unit may receive information stored in memory. Optionally, this information may be selected from laser power budget (or any other form of optical budget), electrical operating characteristics, and/or calibration data. The situation assessment unit may generate a scan/operation plan using the photon steering assembly feedback. The optical budget (e.g., laser power budget) may be derived from constraints such as eye safety regulations, thermal budget, laser degradation over time, and others.

[0541] According to some embodiments, the operation plan may be generated based on (a) a real-time detected scene signal, (b) an intra-frame level scene signal, and (c) an inter-frame level scene signal accumulated and analyzed over two or more frames.

[0542] According to some embodiments, the detector may be a dynamic detector having one or more detector parameters, and a closed-loop controller may update the detector parameters based on the operation plan. The detector parameters may be selected from scan direction, frame rate, sampling rate, ambient light effect, mechanical static and dynamic faults, dynamic gating to reduce parasitic light, dynamic sensitivity, dynamic bias, and/or thermal effect. The PSY may have one or more steering parameters, and a closed-loop controller may update the steering based on the operation plan. The steering parameters may be selected from scan method, power modulation, single-axis or multi-axis method, and synchronization component. Optionally, a situation assessment unit may receive host feedback from a host device and use this host feedback to generate or contribute to an operation plan.

[0543] According to some embodiments, the processor 118 may include situation assessment logic or circuitry, such as situation assessment logic (SAL). The SAL can receive detection scene signals from the detector 116 and information from additional blocks/elements inside or outside the scan unit 104.

[0544] According to some embodiments, the scene signal can be assessed and calculated in a weighted average of local and global cost functions that determine the scan/operation plan, such as the operation plan signal for the scan unit 104, with or without using additional feedback signals such as photon steering assembly feedback, PTX feedback, PRS feedback, and host feedback, as well as information stored in memory 2902 (e.g., which laser parameter budget, which detector parameter budget, and which pixels in the FOV to scan). Thus, the processor 118 can be a closed-loop dynamic controller that receives system feedback and updates the system operation based on this feedback. The scan operation plan can be created, for example, to implement the allocated optical budget or computation budget.

[0545] According to several embodiments, a scan unit 104 can be provided for scanning one or more segments of a scene, also referred to as scene segments. The device includes one or more photon emitter assemblies (PTX), one or more photon receiver and detection assemblies (PRX), a photon steering assembly (PSY), and a context-aware processor adapted to synchronize the operation of the PTX, PRX, and PSY, thereby enabling the device to dynamically perform an active scan of one or more scene segments or regions of a scene during a scan frame. An active scan may include transmitting one or more photon inspection pulses toward and within a scene segment, and, if an inspection pulse collides with a scene element present within the scene segment, measuring the round-trip flight time from when the pulse collides with the element until its reflection returns, in order to estimate the distance and (relative) three-dimensional coordinates of the point on the scene element where the inspection pulse collides. By collecting the coordinates of the set of points on the element using the set of inspection pulses, a three-dimensional point cloud can be generated and used for detecting, registering, and possibly identifying scene elements.

[0546] The processor 118 can be a situation-aware controller and can dynamically adjust the operating modes and operating parameters of the PTX, PRX, and/or PSY based on one or more detected and/or otherwise known scene-related situation parameters. According to some embodiments, the processor 118 can implement an allocated optical or computational budget by generating and/or adjusting an operating plan, such as a scan plan, for scanning a portion of a scene as part of a scan frame intended to scan/cover one or more segments of a scene, based on an understanding of situation parameters such as scene elements present within one or more scene segments. Other situation parameters that may be considered in generating the scan plan may include the location and/or trajectory of the host platform having the device according to the embodiment. Other situation parameters that may be considered in generating the scan plan may include terrain such as road gradient, pitch, and curvature around the host platform having the device according to the embodiment.

[0547] A scan plan may include (a) specifying scene segments within a scene to be actively scanned as part of a scan frame, (b) an inspection pulse setting scheme (PSS) capable of defining the pulse distribution pattern and/or individual pulse characteristics of an inspection pulse set used for scanning at least one of the scene segments, (c) a detection scheme capable of defining the sensitivity or response pattern of the detector, and (d) a steering scheme capable of defining the steering direction, frequency, specifying idle elements in the steering array, and other such actions. In other words, a scan plan can at least partially influence/determine the PTX control signals, steering parameter controls, PRX controls, and/or detector control parameters so that the scan frame is actively scanned based on the scene analysis and allocated optical and/or computational budget.

[0548] The following considerations provide additional examples of controlling one or more scans of the LiDAR FOV based on the determined optical budget and/or computational budget. For example, based on the currently detected or estimated operating environment, the processor 118 can dynamically adjust the instantaneous detection distance by coordinating the control of at least one light source and at least one optical deflector to vary the amount of optical budget applied spatially in the scan of the field of view. For example, the processor 118 can increase the amount of projected light and/or decrease the spatial distribution of light to expand the instantaneous detection distance in a particular region of the FOV (region of interest). As another example, the processor 118 can decrease the amount of projected light and/or increase the spatial distribution of light to reduce the instantaneous detection distance in other regions of the FOV (region of less interest).

[0549] Deflection Sensor Feedback Control in Lidar

[0550] In some embodiments, as described above, the LiDAR system 100 can be incorporated into a vehicle. Due to engine operation and movement on roads and other surfaces, some amount of vibration may occur, and this vibration may interfere with the operation of the LiDAR system 100. For example, vibration may be transmitted to any of the components of the LiDAR system 100 and affect their performance and/or the overall performance of the system. In some examples, vibration of the light source and/or light deflector may cause fluctuations in the direction of light emitted into the LiDAR field of view ("FOV"), reducing light collection from objects in the FOV, making the deflector position and/or the instantaneous FOV position uncertain, and/or making the deflector/sensor coupling inefficient. As a result, the LIDAR FOV may not be illuminated as intended (for example, as illustrated by the difference between the intended FOV 3220 and the actual FOV 3222 in Figure 32A), objects within the LIDAR FOV may not be detected, objects may be detected in the wrong direction, and/or object detection may be at a lower resolution level than desired.

[0551] To address these effects, in some embodiments, the LIDAR system 100 can incorporate a vibration suppression system 3200 (e.g., Figures 32A and 32B). In some examples, the LIDAR system 100 can determine the presence of vibration and take one or more actions to reduce or eliminate the effects of such vibration. The LIDAR system 100 can determine the presence of vibration by any suitable technique. For example, in some examples, vibration can be detected using one or more sensors associated with the vehicle or the LIDAR system 100 itself. Such sensors may include one or more accelerometers, strain gauges, or any other suitable type of sensor. In some examples, vibration can be detected based on feedback received from the deflector 114. In other words, in some examples, the vibration suppression system of the LIDAR system 100 can respond to vibration feedback determined based on mirror position data associated with the deflector 114 (for example, using the mirror position feedback sensor shown in Figure 32C to detect the movement of the deflector 114 caused by vibration, also shown in Figures 62, 65, 67, 76, 77, and 84). The resulting mirror position fluctuations may originate from any vibration source directly or indirectly coupled to the LIDAR system 100. Such vibration sources may include engine operation, wheel rotation on the road surface, and mechanical movement of vehicle components (including movement of components in the LIDAR system 100).

[0552] In addition to being able to address some or all of the effects of vibration on the LIDAR system 100, or instead, the vibration suppression system 3200 of the LIDAR system 100 can also address the effects arising from uncertainty in mirror positioning. For example, when using piezoelectric actuated MEMS mirrors, piezoelectric actuation may involve a certain amount of hysteresis. This means that a particular control voltage does not necessarily produce the desired mirror positioning because the mirror position with respect to the control voltage is ambiguous. Therefore, to address such effects that may exist in any type of installation of the LIDAR system 100 (e.g., on a moving platform such as a vehicle, or on a stationary object such as a building or infrastructure), a position feedback mechanism such as the position feedback mechanism 3256 in Figure 32C (e.g., Figures 62, 65, 67, 76, 77, and 84) may be useful. Furthermore, it should be noted that a sensor 3228 (e.g., Figure 32B) can be used to obtain data indicating the position, orientation, velocity, or acceleration of at least one optical deflector. These determined pieces of information regarding the state of the optical deflector can be determined independently of the reasons for the directional change (e.g., vibration, hysteresis, temperature effects) and can be used for feedback control of the optical deflector to improve the detection accuracy and operability of the LIDAR system 100 by making necessary modifications (for example, in the example given below).

[0553] In a LiDAR system configured to suppress the effects of vibration or uncertainty at the position of the optical deflector, the system 3200 may include at least one processor configured to control at least one light source so as to vary the beam of light from at least one light source during a field scan, to control the positioning of at least one optical deflector so as to deflect light from at least one light source for scanning the field, and to acquire data indicating vibrations of the vehicle in which the LiDAR system is deployed. Based on the acquired data indicating the detected vibrations, at least one processor can determine adjustments to the positioning of at least one optical deflector to compensate for vehicle vibrations. Furthermore, at least one processor can implement the determined adjustments to the positioning of at least one optical deflector to suppress at least some of the effects of vehicle vibrations on the field scan at least one optical deflector.

[0554] Figure 32A shows an exemplary vibration suppression system 3200 and vehicle 3210 on an uneven road surface. The vehicle 3210 may be equipped with various types of sensors for detecting the presence of vibrations and/or features associated with detected vibrations. For example, in some embodiments, the vehicle 3210 may include sensor units and/or vibration sensors deployed at various locations on the vehicle. Such sensors can detect vibrations associated with the vehicle's wheels, engine, body, etc., resulting from the movement of the vehicle on the road or other surface, the operation of the vehicle's engine, the operation of one or more other components of the vehicle, or any other potential vibration sources applied to the vehicle 3210. For example, a sensor unit 3206 including sensor 3216 can be positioned in an area near the vehicle's engine to monitor engine-related vibrations. Furthermore, one or more sensor units 3208 including sensor 3218 can be positioned in an area near or on the wheels of the vehicle to monitor wheel-related vibrations. Furthermore, the vehicle may also be equipped with one or more vibration sensors 3219 positioned near, on the surface of, or inside the LiDAR system 100 to detect vibrations in or around the LiDAR system.

[0555] Sensors 3216, 3218, and/or 3219 may include any type of sensor capable of measuring at least one feature of vibration or the effect of vibration, such as force, acceleration, torque, strain, stress, voltage, optical deflection, etc. Such sensors 3216, 3218, and/or 3219 can be connected directly or indirectly via wired or wireless connection to one or more processors associated with the LiDAR system 100 to transmit information indicating the detected vibration to one or more processors of the LiDAR system.

[0556] In addition to sensors 3216, 3218, and/or 3219, the LIDAR system 100 may be equipped with or configured to have a sensing function for detecting the presence of vibrations and/or one or more features of detected vibrations (e.g., sensor 3228 in Figure 32B). As will be discussed in more detail below, one or more processors associated with the LIDAR system 100 may include programming to enable the detection of vibrations present in, for example, the deflector 114. Such vibrations can be detected, for example, by monitoring the movement of the deflector 114, including movement that is not intentionally given to the deflector 114 as part of a LIDAR FOV scan. For example, Figure 32B shows a vehicle vibration compensation system 3200 together with components of the LIDAR system 100, and further shows a scan unit 104 and a sensor 3228 in contact with the deflector 114. Using the sensor 3228, data indicating the position, orientation, velocity, or acceleration of at least one optical deflector 114 can be obtained. Furthermore, the intended FOV scan 3220 incorporates the instantaneous FOV 3224 and the actual instantaneous FOV 3226 scanned by the deflector 114. Vibrations present in the scan unit 104 cause a discrepancy between the intended instantaneous field of view and the actual instantaneous field of view (3224 and 3226, respectively). Sensor 3228 can detect vibrations affecting the deflector 114. Additionally, while Figure 32B shows a bistatic embodiment, it should be noted that alternative cases may incorporate a monostatic design.

[0557] Returning to Figure 32A, the presence of vibrations in the vehicle 3210 can interfere with the operation of the LIDAR system 100. For example, as the vehicle 3210 moves along the road toward object 3202, the LIDAR system 100 can direct a specific amount of light beam toward object 3202 during the LIDAR FOV scan. As previously mentioned, this light beam directed toward a specific subregion of the LIDAR FOV (where object 3202 is located) can be achieved by the light projection unit 102 supplying light to the light deflector 104, which is positioned at an instantaneous location, so that it projects light toward this subregion of the FOV containing object 3202. However, in the presence of vibrations, the deflector 114 moves, so the light intended to be directed toward the subregion where object 3202 is located may be directed, at least partially, toward a region of the LIDAR FOV that is not intended to receive light. As a result, the LIDAR system 100 may be less able to detect object 3202 and provide sufficient information to generate a proper depth map including the details and location of object 3202. To counteract the effects of this vibration, the processing unit 108 of the LIDAR system 100 can detect vibrations, including one or more characteristics of those vibrations, based on outputs received from the detection unit 106, sensor 116, and/or deflector position monitoring unit. The processing unit 108 can move the deflector 114 to counteract at least some of the movement that has occurred in the deflector 114, the light projection unit 102, the detection unit 106, or any other component of the LIDAR system 100 that affects light projection, focusing, or deflection. For example, in some embodiments, a processing unit 108 including one or more processors 118 monitors the position or orientation of the deflector 114, compares the monitored position to the intended instantaneous position/orientation, and if a difference is found, the processor 118 can move the deflector 114 toward the intended instantaneous position/orientation. Using such a feedback method, the processor 118 can counteract the effects of vibrations that tend to displace the deflector 114 from the intended position or orientation. In some embodiments, the processor 118 can be configured to perform vibration-reducing or counteracting movements on the movable components of the LIDAR system 100 to mitigate the effects of detected vibrations.

[0558] The following section provides further details on the vibration detection and vibration suppression functions of system 3200.

[0559] Embodiments disclosed herein may include a processor 118 (i.e., CPU 3234) configured to determine the instantaneous angular position of at least one optical deflector 114 (i.e., mirror 3236) (e.g., using θ, φ coordinates). The term “instantaneous angular position” refers to the instantaneous position of at least one deflector that deflects light in (and/or from) a given angular direction (e.g., indicated by θ, φ). Such determination may be based on at least one of optical measurements, capacitance measurements, piezoelectric resistance measurements, dielectric constant measurements, and piezoelectric polarization measurements by one or more vibration sensors associated with the vehicle 3210 or the LIDAR system 100 (e.g., sensors associated with the optical deflector 114).

[0560] As discussed above, vibration can be detected using one or more sensors 3216, 3218, and/or 3219. Such sensors may include one or more accelerometers, strain gauges, or any other type of sensor suitable for detecting vibration or at least one feature of vibration. Furthermore, as described above, the LIDAR system 100 can be equipped with a dedicated vibration sensor (i.e., sensor 3228) or can detect vibration using one or more components of the system used for scanning the LIDAR FOV. For example, in some embodiments, vibration can be detected using the deflector 114 and the position feedback sensor 3256 shown in Figure 32C.

[0561] Figure 32C shows an exemplary pivotable mirror configuration including a mirror 3306 that can move along two or more axes (e.g., θ, φ). As shown in Figures 32C to 32G, the mirror 3236 can be included in various configurations, including, for example, rectangular, square, circular, or rounded mirror shapes. The processor 118 can be configured to control at least one deflector 114, including, for example, the position of the mirror 3236, during the scanning of the LiDAR FOV. The processor 118 can also control the movement of the mirror 3236 to provide a desired scan rate, scan pattern, spatial light distribution, temporal light distribution, etc.

[0562] For instantaneous directional control, the steering unit 3232 (i.e., the scan unit 104), including the mirror 3236, may also include an electrically controllable electromechanical driver, such as an actuator driver 3238. The actuator driver 3238 can relay movement or power to an actuator/cantilever/bend section, such as an actuator 3240. The actuator 3240 may be part of a support frame, such as frame 3241, or they may be indirectly connected. Additional actuators, such as actuators 3242, 3244, and 3246, may each be controlled/driven by an additional actuator driver, as shown, and may have a support frame that coincides with (as appropriate) multiple layers 3243, 3245, and 3247. It will be understood that frames 3241, 3243, 3245, and/or 3247 may be a single frame supporting all actuators, or multiple interconnected frames. Furthermore, these frames can be electrically isolated by insulating (Isn) elements or sections (as shown in the figure). Optionally, the actuator 3240 can be connected to the mirror 3236 using a flexible interconnect element or connector, such as a spring 3248, to relay power or motion from the actuator driver 3238 to the mirror 3236. The actuator 3240 may include two or more electrical contacts, such as contacts 3240A, 3240B, 3240C, and 3240D. Optionally, if the frame 3241 and the actuator 3240 are electronically connected, one or more contacts 3240A, 3240B, 3240C, and/or 3240D may be located on the frame 3241 or the actuator 3240. According to some embodiments, the actuator 3240 is made of a doped semiconductor, making it conductive generally between contacts 3240A to 3240D and insulating at insulators 3250 and 3252, thereby enabling the actuator 3240 to be electrically isolated from actuators 3242 and 3246, respectively. Optionally, instead of doping the actuator, the actuator 3240 may include conductive elements that can be bonded to the actuator 3240 or otherwise mechanically or chemically connected. In this case, the insulating elements can be embedded in areas of the actuator 3240 where the conductive elements are not bonded. The actuator 3240 may include a piezoelectric layer, in which case the current flowing through the actuator 3240 triggers a reaction in the piezoelectric section, thereby causing the actuator 3240 to bend controllly.

[0563] For example, a CPU 3234 that can be incorporated into a processing unit 108 can output/relay a desired angular position of the mirror 3236, described by the θ and φ parameters, to a mirror driver 3254. The mirror driver 3254 can be configured to control the movement of the mirror 3236 and can cause an actuator driver 3238 to apply a specific voltage to contacts 3240C and 3240D in order to achieve the specified θ and φ deflection values of the mirror 3236 based on the bending of actuators 3240, 3242, 3244, and 3246. According to some embodiments, the position feedback control circuit is configured to supply power (e.g., voltage or current) to a contact such as contact 3240A (or 3240B), and the other contact, such as 3240B (or optionally 3240A), is connected to a sensor in the position feedback 3256, which can be used to measure one or more electrical parameters of the actuator 3240 to determine the bending of the actuator 3240 and, optionally, the actual deflection of the mirror 3236. Furthermore, it will be understood that by determining the bending of the actuator 3240 and, optionally, the deflection mirror 3236, the CPU 3234 then determines the real-time position of the optical deflector.

[0564] For each of the actuators 3242-3246, additional position feedback similar to position feedback 3256 and additional actuation drivers similar to actuation driver 3238 can be replicated, and the mirror driver 3254 and CPU 3234 can also control these elements so that mirror deflection is controlled in all directions. The actuation drivers, including actuation driver 3238, send signals that cause electromechanical responses in actuators 3240-3246, each of which is then sampled for feedback. The position feedback of the actuators (3240-3246) can function as signals to efficiently focus the mirror driver 3254 to a desired position θ, φ set by the CPU 3234 and to correct the required value based on the detected actual deflection.

[0565] For example, the 3242A/B, 3244A/B, or 3246A/B and position feedback sensors can be used to position the mirror 3236 and obtain feedback, in addition to any other desired operation, such elements can be used to detect vibrations. For example, the processor 118 can monitor feedback from the position feedback sensors to reveal data indicating vehicle vibrations (or vibrations of the LiDAR system). As discussed above, the vehicle vibration compensation system can utilize the measurement reflective optical system data acquired from the deflector. A scan unit 104, or LiDAR system 100, as shown in Figures 3A to 3C, can utilize a piezoelectric actuator micro-electromechanical (MEMS) mirror device to deflect a laser beam scanning the field of view (FOV). The deflection of the mirror 3236 is the result of applying a voltage potential/current to a piezoelectric element stacked on the actuator 3240. The deflection of mirror 3236 is converted into an angular scan pattern, which may not exhibit linear behavior, and actuator 3240 does not convert it into a constant displacement value for a given voltage level. A scan LiDAR system where the FOV dimensions are deterministic and repeatable across various devices is optimally realized using a closed-loop method that provides position feedback and angular deflection feedback from sensor 3256 to mirror driver 3254 and/or CPU 3234. Not only does the reflective optics provide relevant data to the LiDAR system (e.g., reflected light from a specific subregion of the LiDAR FOV used to generate depth maps), but CPU 3234 can also use the measured optical data as a basis for detecting vibrations. For example, if it is determined that a spot of light reflected by mirror 3236 onto sensor 116 is moving relative to the sensor, and especially if this movement matches the frequency and vibration associated with the vibration, the processor 118 can detect the presence of vibrations based on the direction and degree of movement of the focused light beam, and furthermore, one or more characteristics of the vibrations can also be detected.

[0566] Other techniques may also be used by the processor 118 to detect the presence of vibration. Returning to Figure 32C, as discussed above, the position feedback sensor can also be used to measure vibration. For example, the position feedback sensor can detect signals at actuators 3242, 3244, and/or 3246 by contacts 3242A or 3242B, 3244A or 3244B, and/or 3246A or 3246B. The detected signals can be used to reveal the movement of the actuator, which may indicate vibration. In one example of detecting the effect of vibration by monitoring the output of the actuator and/or position feedback sensor, the processor 118 may have moved the mirror 3236 to a specific instantaneous position as part of a scan of the LIDAR FOV. Once the mirror 3236 has been moved to a designated instantaneous position (for example, to guide light to a specific subregion of the Lidar FOV), the processor 118 can expect the mirror to remain in that position for a certain pause time before moving to the next instantaneous position. As discussed in detail above, during the pause time, the mirror 3236 may be expected to remain fixed at the designated instantaneous position or to move at a pace that allows for continuous steering of light to a specific instantaneous FOV. Therefore, if the processor 118 receives a signal indicating that the mirror 3236 has deviated from its expected orientation during the pause time, the processor 118 can determine that the signal indicating movement/deviation during the pause time may be oscillating, especially if the deviation is oscillating, sporadic, random, exceeds a certain frequency, exceeds a certain threshold, etc. Similarly, the processor 118 can also determine that the signal indicating movement/deviation during the instantaneous position scan time indicates an external force applied to the mirror. Optionally, the processor 118 may use the signals indicating movement/deviation during the instantaneous position scan time without determining the cause. If the processor 118 moves the mirror 3236 (for example, between stop times at different instantaneous positions), and the processor 118 observes signals from the position feedback sensor that do not match the signals expected for the prohibited movement, the processor 118 can determine that these mismatched signals are related to the effect of vibration. If the processor 118 moves the mirror 3236 (for example, between stop times at different instantaneous positions), and the processor 118 observes signals from the position feedback sensor that do not match the signals expected for the prohibited movement, the processor 118 may, in response to the signals from the position feedback sensor, transmit a position control signal to at least one actuator of the mirror 3236. It will be apparent to those skilled in the art that such a position control signal can also be transmitted by the processor 118 to any other type of optical deflector 114 with the necessary modifications.

[0567] Determining and/or monitoring the position of the mirror 3236 is useful not only for detecting vibrations but can also be useful for addressing other causes of unintended mirror movement. For example, Figure 32D shows an exemplary actuator-mirror coupling according to some disclosed embodiments. The actuator 3240 consists of several layers and may include a piezoelectric layer 3241, a semiconductor layer 3243, and a base layer 3245. The reflectance of the semiconductor layer 3243 can be measured in an active phase (labeled "Ractive" in the figure) with the mirror deflected to a specific angular position and compared to the reflectance in a stationary state (Rrest). Feedback including Ractive can provide information for measuring/determining the actual mirror deflection angle relative to the expected angle. Based on this information, if there is a difference between the expected angle/orientation/position of the mirror 3236, the actuator 3240 can be controlled to change the angle/orientation/position of the mirror 3236 to match the expected one. The conductivity of the silicon (or other semiconductor) based actuator 3240 may vary depending on the mechanical stress applied to the actuator 3240. When the actuator 3240 is stationary, the conductivity indicated by the two contacts 3240A and 3240B is Rrest. When activated (for example by applying a voltage), the piezoelectric material of layer 3241 exerts force on the actuator 3240, causing it to bend. Furthermore, vibrations applied to the LIDAR system 100 may cause one or more unintended movements of the mirror 3236, which may also bend the actuator 3240. The bending of the actuator 3240 in response to mechanical forces (whether generated by the electrical activation of the piezoelectric layer or induced as a result of vibrations) may change the conductivity Ractive indicated by the two contacts 3240A and 3240B. The difference between Rrest and Ractive can be correlated to an angular deflection value that acts to close the loop, controlled by the mirror drive (e.g., mirror driver 3254 in Figure 32C). This method can be used to dynamically track the actual mirror position. Furthermore, by using this information within the feedback loop, the processor 118 can apply an electrical signal (supply current/voltage) to the actuator 3240 to counteract the motion caused by vibration.

[0568] Figure 32E, which provides a schematic diagram of a two-axis MEMS mirror (3270), Figure 32F, which provides a schematic diagram of a single-axis MEMS mirror (3280), and Figure 32G, which shows a round MEMS mirror (3290), each provide examples of mirror and actuator assemblies that can be used to detect mirror movement caused by vibration and to counteract these movements by an active feedback loop. The mirror and actuator coupling can be configured to have various features according to the requirements of a particular application. For example, in some embodiments, the optical deflector (e.g., a mirror 3236 suspended within an actuator frame) may have a resonant frequency of less than 1000 Hz. Furthermore, the deflector includes a MEMS mirror array, each of which individual mirrors may constitute an optical deflector element. In some embodiments, each optical deflector may include a single MEMS mirror having a width of at least 4.5 mm. In other embodiments, each optical deflector may include a two-dimensional array of mirrors, each having a width of at least 2 mm. As described above, if at least one movement (in particular a movement indicating vibration, but not limited to vibration) is detected by, for example, monitoring one or more indicators of the movement of the mirror 3236, the processor 118 can control various actuators to counteract this movement. Such control can be performed as part of a feedback loop that attempts to reduce or eliminate the difference between the intended/expected position or movement of the mirror 3236 and the observed position/motion/velocity/acceleration of the mirror 3236. The reduction or elimination of the difference between the intended position/orientation/velocity/acceleration of the mirror 3236 and the observed position/orientation/velocity/acceleration of the mirror 3236 can be achieved by driving actuators (e.g., actuators 3240, 3242, 3244, and/or 3246) to move the mirror 3236 in the opposite direction to the movement caused by vibration (or any other force as described above), or to otherwise change the movement characteristics of the mirror 3236. As part of a feedback loop, the position/orientation/velocity/acceleration of the mirror 3236 is continuously monitored, and the mirror 3236 is driven to a intended position/orientation associated with its intended instantaneous position (during the LIDAR FOV scan). This allows the mirror 3236 to be guided to a substantially intended position/orientation regardless of forces acting on it (e.g., forces caused by vibration).

[0569] Alternatively, or in addition to this, the processor 118 may control the position of the mirror 3236 based on the received output of one or more sensors, such as sensors 3216, 3218, and/or 3219. In such an embodiment, the processor 118 may determine adjustments to compensate for observed vibrations. This may include calculating appropriate axis (θ, φ) parameter adjustments to move the mirror 3236 to the intended instantaneous position. In some cases, these adjustments may include moving the deflector 114 with the steering device 3232 to compensate for calculated acceleration, torque, strain, etc., determined based on the output from sensors associated with the vehicle itself.

[0570] In addition to vibration suppression, the LIDAR system 100 can detect and respond to other movements that may be associated with the platform (e.g., a vehicle) on which the LIDAR system 100 is mounted or otherwise associated. For example, a processor (e.g., processor 118, CPU 3234, etc.) can be further configured to collect data indicating the tilt of the vehicle (e.g., Figure 33). Information indicating the tilt of the vehicle may be provided as the output of one or more accelerometers, one or more three-dimensional accelerometers, an inertial measurement unit (IMU), etc. Based on this information, one or more aspects of the LIDAR system 100 can be adjusted. For example, in some embodiments, one or more mechanical actuators can be activated to rotate the LIDAR system 100 (or deflector 114; a light projector assembly including a light projector, one or more light deflectors, and a light sensor; or one or more components including any other components of the LIDAR system 100 that at least partially affect the location of the LIDAR FOV for a particular scene) to counteract changes in the tilt of the vehicle. By counteracting such vehicle tilt, the LIDAR
The FOV can remain substantially fixed relative to the scene (at least for a certain period of time), regardless of changes in the vehicle's tilt, for example. In other cases, the LIDAR FOV may fluctuate relative to the scene, but this fluctuation is smaller than the amount typically associated with a particular change in the vehicle's tilt. For example, if the vehicle is approaching the top of a hill (e.g., a negative curve), the LIDAR system (or one or more of its components) can be moved so that the LIDAR FOV moves downward relative to the scene. Such a movement would reduce the overlap with the sky and increase the overlap with the road. Similarly, if the vehicle is approaching an upward curve in the road (e.g., a positive curve), the LIDAR system (or one or more of its components) can be moved so that the LIDAR FOV moves upward relative to the scene. Such a movement would allow the LIDAR FOV to overlap with areas of the scene that include further parts of the road beyond the upward curve.

[0571] As an example for illustrative purposes, Figure 33 shows two similar scenes in which a vehicle 110 equipped with a LiDAR system 100 is traveling downhill in the direction of a track 3302. In Scene A, the LiDAR system 100 has a fixed field of view 120A with minimum and maximum elevation points such that the track 3302 is not detected. In this case, the LiDAR system 100 does not detect the track 3302 until a later point in time (for example, when the LiDAR FOV moves upward relative to the scene as the vehicle passes a positive curve in the road, thereby overlapping with the area containing the track 3302). However, in Scene B, the LiDAR system 100 has a dynamic field of view 120B that can be positioned relative to the scene by adjusting the aiming direction of one or more of the LiDAR system 100 or its components, as discussed above. In this example, the vehicle's tilt can be detected as it descends a slope, and the dynamic FOV 120B can be adjusted to overlap with a more distant area along the road where the track 3302 is located, rather than overlapping with the lower part of the slope (as in Scene A). As a result, the LIDAR system 100 can detect the track 3302 earlier than without the dynamic FOV function. Clearly, the processor 118 can react to various positions of the vehicle 110, and driving downhill is merely shown as one exemplary situation.

[0572] The processing unit 108 (including, for example, the CPU 3234) can achieve such adjustments in a wide variety of ways. For example, the CPU 3234 can implement a continuous feedback loop from data collected by feedback from various sensors associated with the vehicle to generate changes in the position of the LIDAR system 100. Alternatively, or in addition to this, the deflector 114 (e.g., mirror 3236) can be steered to counteract changes in the vehicle's tilt.

[0573] A method for suppressing vibrations in a LiDAR system used in a vehicle may include controlling at least one light source to vary the beam of light from at least one light source during a field scan. This method may further include controlling the positioning of at least one optical deflector to deflect light from at least one light source for scanning the field. This method acquires data indicating vehicle vibrations. Based on the acquired data, the method adjusts the positioning of at least one optical deflector to compensate for vehicle vibrations. The method further performs the determined adjustments to the positioning of at least one optical deflector, thereby suppressing at least a portion of the effect of vehicle vibrations on the field scan in at least one optical deflector.

[0574] Moving to Figure 34, it will be further understood that a method for suppressing vibrations of a LIDAR configured for use in a vehicle may include controlling at least one light source so that the beam of light from at least one light source can be varied in a scan of the field of view (step 3410). Furthermore, in some embodiments, the optical deflector 114 has a resonant frequency of less than 1000 Hz. In step 3420, the field of view can be scanned by controlling or maneuvering the positioning of the optical deflector to deflect light from at least one light source. In step 3430, data indicating vehicle vibrations can be acquired. Some examples of the data to be acquired are described above. Next, in step 3440, based on the acquired data, an adjustment to the positioning of at least one optical deflector is determined to compensate for vehicle vibrations. Furthermore, step 3450 shows that this method can be implemented by the positioning of at least one optical deflector to suppress or eliminate the effect of vibrations on one or more scans of the LIDAR FOV. Additional embodiments may further include determining the instantaneous angular position of the optical deflector and modifying the instantaneous angular position of the optical deflector to compensate for the difference between the intended or required position and the instantaneous angular position.

[0575] Stable high-energy beam

[0576] In order to promote the use of LIDAR systems by the automotive industry, there is interest in LIDAR systems that exhibit functions similar to some aspects of human vision. For example, human vision provides information that enables a person to perceive a scene in three dimensions (for example, through the parallax given by two eyes viewing the scene from slightly different positions). In road conditions, such functions allow a person to perceive uneven roads, uphill and downhill sections of road, vertical and horizontal swaying, speed bumpers, sharp curves, shoulders, and steep paths (e.g., underground parking lots) as three-dimensional depth. Furthermore, human vision allows a person to determine and/or predict areas within the field of view (or scene) that need to be paid attention to in order to be considered more important than other areas. For example, when driving on a road, areas where pedestrians and vehicles are present may require more attention than areas within the field of view or scene that do not include the skyline or objects that may interfere with, for example, one's own vehicle. Therefore, in response to determining the presence of such an object within a scene (or personal environment), a person can direct more attention to the area where such an object exists. It may be desirable for LIDAR systems to have a similar function.

[0577] In a LiDAR system according to an embodiment of the present disclosure, the system can generate a 3D reconstruction of objects in the environment within the LiDAR system's field of view (FOV). The disclosed LiDAR system may include a "gazing" function that mimics, to some extent, the behavior of human vision. This function shifts the active field of view toward a specific surrounding area by 3 degrees of freedom (3 DOF) up to a maximum of 6 degrees of freedom (6 DOF) according to the environment, road characteristics, and vehicle motion vectors. This function allows the LiDAR system to provide improved detection performance across a large field of view by adaptively dividing the FOV into multiple segments, for example, assigned different levels of quality of service (QoS).

[0578] As described above, the LIDAR system 100 includes, for example, at least one processor 118 in the processing unit 108, which can control at least one light source 112 to increase or decrease the amount of light beam generated for a particular area of the FOV. For example, depending on the level of interest associated with a particular area of the FOV, the amount of light supplied to that area can be increased or decreased in proportion to the level of interest. Parts of the field of view 120 of low interest (for example, the area far from the detected car shown in Figure 5B) can be allocated a low level of light beam or no light beam at all. However, other areas of higher interest (for example, areas where objects are detected, such as the area where the car is detected shown in Figure 5B) can be allocated a higher level of light beam. Such allocation can avoid the expenditure of light energy and detection resources in areas of low interest, while improving resolution and other performance characteristics in areas of higher interest. To generate high or low levels of luminous flux, the light source parameters associated with the first part of the field of view (e.g., pulse timing, pulse length, pulse size, pulse amplitude, pulse frequency, etc.) can be modified so that the luminous flux directed to the first part of the field of view is greater than the luminous flux directed to at least one other part. Alternatively, the processor 118 can modify the light source parameters associated with the first part of the field of view so that the luminous flux directed to the first part of the field of view is less than the luminous flux directed to at least one other part. The difference in luminous flux can also be achieved by changing the deflector parameters (e.g., scan pattern, steering rate) and by changing the synchronization between the light source and the deflector.

[0579] Furthermore, the allocation of light can also be based on a predetermined allocation pattern across the entire FOV. Figure 35A shows an example of an FOV in a plane perpendicular to the sensor's detection surface, for example, in a bird's-eye view of a LIDAR system pointing horizontally. In the example shown in Figure 35A, the FOV is divided into three sectors, but more or fewer sectors can also be implemented. A specific beam of light can be induced in each sector. As a result, each sector may indicate a corresponding signal-to-noise distinction balance and/or a corresponding detection range related to the amount of light supplied to each sector. Of the three sectors shown in Figure 35A, sector II is allocated more beam of light than sector I or sector III. As a result, the LIDAR system can detect the same object in sector II at a greater distance range than in sector I or sector III. Similarly, as shown, sector III is allocated more light than sector I but less than sector II. As a result, sector III can detect objects in a range larger than sector I but smaller than sector II. Of course, other light allocation patterns are also possible. For example, in some embodiments, it is possible to allocate the maximum light beam to sector II and substantially the same amount of light beam to sectors I and III, respectively. Inducing a high level of light beam to the second sector can at least partially compensate for laser signal loss occurring at the distance between the light source and the target of interest. Furthermore, inducing a large amount of light beam to the second sector improves the resolution that the system can provide to that sector, thereby improving the quality of service in that sector and, consequently, in the entire LIDAR system. The improvement in resolution may be an improvement in temporal resolution, an improvement in spatial resolution, or a combination of both.

[0580] Not only can many different levels of light flux be allocated to different sectors of the FOV, but the shape and size of the sectors can also vary considerably. For example, in some embodiments, sector II (e.g., the sector with the maximum light allocation in the embodiment of Figure 35A) can occupy the central region of the FOV, as shown in the explanatory example in Figure 35A. In other words, as described above, the FOV can be divided into multiple sub-regions. The processor 118 can designate any region/sub-region within the FOV as sector II (or any other arbitrary sector). Thus, light allocation by the processor 118 may include supplying a specific level of light flux to one or more sub-regions contained within a particular sector. Furthermore, each sector/sub-sector contains at least one pixel. The systems and methods of this disclosure can collect data on a pixel-by-pixel basis. To scan the field of view (FOV), the processor 118 can control at least one optical deflector (e.g., optical deflector 114 in Figure 1A, deflector 114 and/or deflector 114B in Figure 2A, and/or unidirectional deflector 214 in Figure 2B) to scan the field of view. For example, the processor 118 can scan the field of view by mechanically moving at least one optical deflector. Alternatively, or simultaneously, the processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector to scan the field of view.

[0581] Sectors associated with a LiDAR system may include any appropriate size, any orientation, or different solid angles. For example, in some embodiments, each sector may have a similar size (e.g., they may occupy a similar number of similarly sized FOV subregions, such as a similar number of “pixels” and a similar number of “beam spots”). However, in other cases, sectors may have different sizes. For example, Figure 35C shows different sectors having different sizes and shapes.

[0582] Furthermore, sectors can be positioned within the LiDAR FOV in various different orientations. In the illustrative example shown in Figure 35A, each sector occupies the entire height of the LiDAR FOV, and the LiDAR FOV is vertically divided into three sectors (I, II, and III), each occupying the entire height of the LiDAR FOV. However, this is not necessarily true for all embodiments. Since the processor 118 can assign any FOV sub-region or group of sub-regions to a specific sector, such sectors can correspond to the vertical division of the FOV (Figure 35A), the horizontal division of the FOV, or take on various other shapes or patterns, as will be further examined below with reference to Figure 36. In this specification, for example, a sub-region can be an instantaneous FOV that can be moved within the FOV by changing the instantaneous position of at least one deflector.

[0583] Furthermore, the size, shape, and/or range associated with each sector can be changed in different scan cycles. For example, after acquiring a frame in an FOV scan cycle, the allocation of light to various sectors can be changed, the number of sectors can be changed, the size of sectors can be changed, and/or the relative position of any sector within the FOV can be changed. In the explanatory example shown in Figure 35B, the light allocated to sector III in the later scan of the FOV is different from the amount of light allocated to sector III in the earlier scan of the FOV shown in Figure 35A. Therefore, the scan of sector III in Figure 35B may involve less light flux, and thus a smaller detection range, a lower signal-to-noise ratio, etc., compared to the scan of sector III in Figure 35A. As a result, as shown in Figure 35B, sectors I and III may have the same detection range.

[0584] Furthermore, the position, size, shape, and/or luminous flux associated with different sectors can be varied over multiple scan cycles according to a predetermined pattern based on detection feedback, based on commands or information provided by the host or another external system, or based on any other appropriate grounds. In some embodiments, for example, the relative position of a particular sector can be changed within the FOV over two or more scan cycles of the FOV. As a result, a particular sector, such as sector II, can be made to scan the FOV over multiple scans (e.g., in a raster pattern, sweep motion, etc.). Furthermore, the amount of luminous flux provided to available sectors can be varied from scan to scan. For example, the scan shown in Figure 35B shows a reduction in the amount of light allocated to sector III compared to the scan associated with Figure 35A. As a result, the detection range of sector III may also change (e.g., be reduced). Such changes in light allocation can be performed by the processor according to a predetermined sector scan scheme based on feedback (e.g., sensor output regarding vehicle motion, LIDAR detection of one or more target objects, other sensors/detectors, etc.). Therefore, as shown in Figure 35B, the processor 118 can determine the same detection range for two different sectors within the FOV.

[0585] In one example, referring to Figure 1A, the LIDAR system 100 can be deployed on the vehicle 110, and the processing unit 108 can select a predetermined scan pattern for a specific sector (sector II, etc.) based on the vehicle's driving mode (e.g., pitch, roll, lateral roll, stopped, etc.). As the vehicle moves, the processing unit 108 can move sector II in a sweeping motion with each scan. Such changes in position allow the LIDAR system to effectively detect target objects (including those located within one or more target areas). Furthermore, by sweeping a specific sector across multiple scans of the LIDAR FOV, the system can track one or more target objects across multiple scans. For example, if the vehicle continues to move toward a detected object with each scan, by changing the location of the detected sector relative to the FOV (e.g., moving sector II in a sweeping motion across multiple scans), the LIDAR system can continue to track those objects as the vehicle moves toward one or more target objects.

[0586] Another example for illustrative purposes may include situations in which the vehicle 110 changes direction (turning left or right, making a U-turn, parking, etc.). In such cases, by scanning specific sectors in the FOV over multiple scan cycles, it becomes possible to continuously track a target object that would not have been detected (or, in some cases, not detected at the desired resolution) without scanning specific sectors in the FOV. Such scans can be performed according to a predetermined pattern (e.g., regularly sweeping relative locations in the FOV at a predetermined rate or location, etc.), but the scanning of specific sectors can also be based on any other suitable grounds. For example, in one embodiment, the scan can be performed based on feedback related to the vehicle's movement as described above. As the vehicle moves, sensors on the vehicle (e.g., speed sensors, accelerometers, etc.) can monitor the vehicle's speed and orientation. The location of specific sectors in the FOV can be varied with each scan cycle to at least partially take into account the detected vehicle movement. In one example, the sector location can be changed so that the detected object (e.g., another vehicle, pedestrian, etc.) is tracked at least partially within a specific sector (e.g., sector II in Figure 35A or 35B, or sector IV in Figure 35C) during multiple FOV scan cycles.

[0587] Furthermore, the relative movement of sectors within the FOV can also be based on other types of feedback. For example, if a target object is detected within a specific sector of the FOV (e.g., sector II) and it is determined that the object is moving relative to the LIDAR system, the relative movement of the target object can be taken into account when assigning subsequent locations of that specific sector within the FOV during one or more subsequent scans. For example, as shown in Figure 35C, if a target vehicle is crossing in front of a host vehicle containing a LIDAR system from right to left, the relative location of a specific sector within the FOV (e.g., sector IV) can be swept across multiple scans to track the movement of the target vehicle as it moves from right to left in the FOV. As a result, the location of sector IV may also move from right to left in the FOV across multiple scans of the FOV. The rate at which the sector moves to its new location in the FOV (e.g., the angular change of the sector location per scan) may depend on the observed relative movement characteristics of the target vehicle (e.g., its relative velocity, acceleration, distance from the host vehicle, etc.). Accordingly, Figure 35D shows the FOV divided into sectors corresponding to the sectors in Figure 35C.

[0588] Alternatively, or simultaneously, the relative movement of sectors within the FOV can be a sweep pattern. Similar to the "lighthouse" light projection mode, specific sectors can move across the entire FOV or within a portion of the FOV to detect and track moving objects.

[0589] Alternatively, or simultaneously, the processor 118 may also reduce the dimensions of a specific sector in the center of the FOV frame (e.g., sector II) based on the host vehicle's speed, in order to expand the forward detection range at the expense of a wider scene understanding.

[0590] Sweeping one sector (or more sectors) in the FOV per scan can be performed continuously (for example, if the sector angle change per scan is constant). Alternatively, sweeping or sweeping one or more sectors can be performed discontinuously (for example, if the sector angle change per scan is not constant).

[0591] Furthermore, one or more sectors of the FOV can be designated by the processor 118 to receive little or no light beam. For example, if a dead zone (e.g., where little or no object of interest is present) is detected, or if a nearby object obstructs part of the FOV, the need for additional information from that zone may be low during one or more subsequent scans. Therefore, during one or more subsequent scans of the FOV, low-speed or non-light-beam sectors can be allocated to overlap with that dead zone. In this way, the energy use/energy requirements of the LIDAR system can be reduced. In addition to or instead of this, the light energy that would have been used to allocate to one or more dead zone sectors can be reserved for reallocation to one or more other sectors of greater interest.

[0592] As described above, the FOV can be divided into one or more sectors, each consisting of one or more sub-regions of the FOV. Such sector divisions may include vertical sections of the FOV, horizontal sections of the FOV, or various other patterns. In one embodiment, the FOV can be divided into three sectors (sector I, sector II, and sector III), as shown in Figure 36. In this embodiment, sectors I and III include vertical divisions of the FOV. Sector II, however, is assigned to a sub-region of the FOV completely enclosed by the first and third sectors. The location of sector II may be the same relative position within the FOV with each scan, or it may be moved. In some embodiments, sector II can be swept over multiple consecutive scan cycles such that at least a portion of the second sector is located below the horizontal line or horizontal level in each scan cycle. For example, when driving on an uneven road, if an object (e.g., a pothole, another vehicle, a pedestrian, etc.) is detected on the road, it may be desirable to allocate more light beam to the area of the object to increase the resolution of that area and improve the ability to determine the object's features. Therefore, as described above, the processor 118 can cause sector II to track the object with each scan based on the orientation of the detected vehicle, the relative movement of the object, or any other arbitrary criteria. For this reason, when a vehicle equipped with a LIDAR system is going downhill, sector II can be moved upward with each scan to maintain overlap with the object on the road ahead. Similarly, when the host vehicle is going uphill, the location of sector II in the FOV can be moved downward with each scan. As a result, regardless of the uneven road, the location of sector II in the FOV (e.g., high-light-beam allocation zone) can be moved up and down so that sector II overlaps with the area of the FOV that is substantially below the horizon.

[0593] Figure 37 is a flowchart of an exemplary method 3700 for detecting an object using a LIDAR system. In step 3701, a processor (e.g., processor 118) controls at least one light source (e.g., light source 120) so that the beam of light from at least one light source (e.g., light source 120) can be varied during a scan cycle of a field of view (e.g., field of view 120). The light projected from at least one light source is directed to at least one deflector (e.g., light deflector 114) to scan the field of view. In step 3702, the processor (e.g., processor 118) receives a reflected signal from at least one sensor (e.g., sensor 116) indicating light reflected from an object in the field of view (e.g., field of view 120). In step 3703, the processor (e.g., processor 118) coordinates the beam and scan to generate at least three sectors during a scan cycle. The processor 118 can supply similar or different levels of beam to each of the at least three sectors. In some examples, the luminous flux supplied to the second sector can be greater than the luminous flux supplied to the first and third sectors. Furthermore, the processor 118 may produce different point resolutions for one or more of the available sectors. "Point resolution" refers to the resolution of the point cloud map. Each of the one or more points in the point cloud map corresponds to a location within an object or a location on the surface of an object. That is, if the average space between each point shrinks and the number of points increases, the point resolution can increase. Also, the higher the point resolution, the higher the accuracy of the information (e.g., spatial information, temporal information, etc.).

[0594] In step 3704, the processor (e.g., processor 118) can control at least one light source (e.g., light source 112) such that the luminous flux supplied to the first sector is substantially the same as the luminous flux supplied to the third sector, and the luminous flux level supplied to the second sector is greater than the luminous flux supplied to the first and third sectors. As a result, the detection range associated with the second sector can extend at least 50% further than the detection range associated with the first sector or the detection range associated with the third sector. In step 3705, the processor (e.g., processor 118) can detect an object in the second sector with a point resolution higher than the point resolution given by either the first sector or the third sector, based on input from at least one sensor (e.g., sensor 116). In step 3706, the processor (e.g., processor 118) can detect an object in the second sector using a second point resolution having an average space between each point that is less than approximately 50% of the average space between points in the point resolution associated with the first and third sectors. For example, if more accurate information is desired in the second sector, at least one processor can improve the point resolution by reducing the average space between each point in the point resolution associated with the second sector. In step 3707, the processor (e.g., processor 118) can detect an object in the second sector based on input from at least one sensor (e.g., sensor 116).

[0595] Parallel capture of Lidar frames at different rates

[0596] Figure 38 shows the field of view 3802 of the LIDAR system 3800. The LIDAR system 3800 can operate as described above with reference to the LIDAR system 100. In some embodiments, the LIDAR system 3800 includes one or more processors configured to control one or more light sources so that the light beam can be varied in a number of scans of the field of view 3802. Each scan of the field of view can generate a captured frame. Each frame may contain sensor output information for each region of the field of view where light was guided and collected during the scan. The field of view may include a near-field portion 3804 and a far-field portion 3806. In some examples, the captured frame targets the detection of objects in the near-field portion. In other examples, the captured frame targets the detection of objects in the far-field portion. As will be discussed in more detail below, scans targeting the near-field (and the associated captured frames) may have less light beam and a faster scan rate compared to scans targeting the far-field of view (and the associated captured frames) of the FOV.

[0597] As described above, the processor can control one or more light deflectors to deflect light from the light source for scanning the field of view 3802. For example, the controller 118 can control the movement of one or more deflectors to give a desired scan rate. The processor can implement a near-field scan rate for frames associated with scan cycles corresponding to the near-field portion 3804 of the field of view. The processor 118 can implement a far-field scan rate for frames associated with scan cycles corresponding to the far-field portion 3806 of the field of view.

[0598] During one or more scans to capture one or more corresponding frames, the processor 118 can control the light source to enable detection of objects in the near-field portion 3804 of the field of view. For example, the processor 118 can cause the light source to emit a specific amount of light flux suitable for detecting objects in the near field (e.g., objects at a distance of less than 50 meters, objects at a distance of less than 100 meters, etc.), or to emit light at a specific power level. At other times, the processor 118 can control the light source to emit light to enable detection of objects in the far-field portion 3806. For example, when capturing frames associated with detection in the far-field, the light source can be supplied with a larger amount of light flux, a higher power level of light, etc., to increase LIDAR sensitivity to objects at greater distances, objects with lower reflectivity, etc. It should be noted that such an increase in light flux also enables detection of objects in the near field (including objects with lower reflectivity compared to other objects).

[0599] In some examples, such as when the LIDAR system 3800 is associated with a vehicle, the time it takes to react to a detected object may be shorter when the object is close to the LIDAR system than when the object is farther away. Therefore, in some embodiments, the near-field scan rate can be higher than the scan rate used to acquire frames concentrated in the far-field, where the reaction time is longer. In some embodiments, the near-field scan rate can be at least five times faster than the far-field scan rate. For example, in one exemplary embodiment, the scan rate for the near-field portion 3802 of the field of view is 25 frames per second, and the scan rate for the far-field portion 3804 of the field of view is 5 frames per second. The high scan rate provides increased feedback over a short time period, and the LIDAR system can detect near-field objects while having enough time to react to the detected near-field objects (e.g., by an autonomous or semi-autonomous driving system, driver assistance system, navigation system, etc.). Far-field detection may require more light energy than near-field detection (e.g., as a result of higher luminous flux levels, higher light source power levels, etc.). Adjusting the scan rate for far-field detection (for example, by reducing the FOV scan rate for other scans that focus on near-field detection) offers the advantage of reduced power consumption in the LiDAR system compared to far-field scans at higher scan rates.

[0600] Objects in the near field of view of the LIDAR field of view may include objects located relatively close to the LIDAR system. For example, in some embodiments, near field of view objects may refer to objects located less than 50 meters from the LIDAR system. Similarly, far field of view objects may include objects located at a greater distance than near field of view objects. For example, in some embodiments, far field of view objects may include objects located more than 100 meters from the LIDAR system. In some embodiments, objects in the near field of view of the field of view may be less than 100 meters from the LIDAR system. In some embodiments, objects in the far field of view of the field of view may be more than 50 meters from the LIDAR system. For example, in one exemplary embodiment, the LIDAR system 3800 can detect, in a first frame, a first car parked on a curb, for example 30 meters away, in the near field of view. The LIDAR system 3800 can also detect, in a second frame, the first car parked on the curb and a second car in the far field of view, for example 200 meters away, in the lane ahead. In other words, in some embodiments, the far-field frame can provide near-field information in addition to far-field information so as not to interrupt the rate of information received with respect to the near field.

[0601] In some embodiments, the detection distance of the LIDAR system associated with far-field frames can be extended at least 50% further than the detection distance associated with near-field frames. In some embodiments, the detection distances associated with near-field and far-field frames can be adjusted, for example, by adjusting the luminous flux/light energy level. Far-field detection may require more power than near-field detection because more light may be needed to collect object information from the far-field. As described above, the increased power consumption due to far-field detection can be mitigated by reducing the frame scan rate for frame acquisition associated with far-field detection.

[0602] The processor can control one or more optical deflectors to be positioned at many different instantaneous locations during a scan cycle. In some embodiments, at least one optical deflector 114 and at least one light source 112 can be coordinated so that the optical deflectors are positioned to deflect a portion of the light beam generated by the light source towards an object in the field of view and deflect the reflection from the object towards one or more sensors. In some embodiments, the processor can control the deflectors so that one or more light sources are aimed at a common area of the optical deflectors, thereby projecting light from these light sources onto one or more distinct areas of the field of view.

[0603] In some embodiments, the processor can control one or more light sources to provide a specific spatial light distribution in a given frame and a different spatial light distribution in one or more subsequent frames. Furthermore, the processor can control the light sources to use a specific light distribution scheme for near-field frames and a different light distribution scheme for far-field frames. For example, as shown in Figure 39, light can be emitted to an area in the far-field 3906 located near the horizon according to the far-field light distribution scheme. Conversely, less light can be emitted towards the horizon in the near-field 3904 according to the near-field light distribution scheme. By concentrating the emitted light towards the horizon in the far-field, the LIDAR system can detect objects in the far-field while saving resources that would otherwise be used to scan a large portion of the near-field for object detection. Information regarding the nature, size, or other characteristics of objects in the far-field may not need to be determined or detected until the object to be detected is close to the LIDAR system.

[0604] In some embodiments, the processor can receive information from one or more sensors associated with a frame for detecting one or more objects located in the far field of view. In some embodiments, the processor can receive information from the sensors indicating that in a particular frame, one or more objects are located in the far field of view and/or one or more objects are located in the near field of view.

[0605] In some embodiments, the resolution of acquired frames associated with the far field of view may be lower than the resolution of frames associated with the near field of view. While such resolutions can be expressed in various ways, in one example, the average spacing between points in the near field of view frame may be less than approximately 75% of the average spacing between points in the far field of view frame. It should be noted that similar differentiation schemes between frames can be used with necessary modifications, in which case high-resolution frames are acquired at a first frame rate and low-resolution frames are acquired at a second, higher frame rate.

[0606] In some embodiments, the near-field scan rate and/or far-field scan rate can be controlled according to the driving mode of the vehicle including the LIDAR system. For example, it may be beneficial for a vehicle driving in an urban area to focus more on near-field scanning than for a vehicle driving on rural roads. In other embodiments, the near-field scan rate and/or far-field scan rate can be controlled according to the vehicle speed. For example, at low vehicle speeds, the scan rates associated with one or both of the far-field and near-field scan rates can be reduced. Similarly, at high vehicle speeds, the scan rates associated with one or both of the far-field and near-field scan rates can be increased.

[0607] Figure 40A is a flowchart of an exemplary process 4000 for emitting light from a LiDAR system. In step 4010, the light source emits light into one or more areas of the near-field portion of the field of view. If an object is present in the near-field, in step 4020, the light is reflected by the object and can be detected by one or more sensors of the LiDAR system. The combination of steps 4010 and 4020 can form one scan cycle. In some embodiments, the sequence of steps 4010 and 4020 can be repeated one or more times as part of a single scan cycle.

[0608] At the start of the next scan cycle, in step 4030, light can be emitted from the light source at the far-field scan rate. If an object is present in the far-field, in step 4040, the light is reflected by the object and can be detected by one or more sensors of the LIDAR system. A combination of steps 4030 and 4040 can form another scan cycle. In some embodiments, the sequence of steps 4030 and 4040 can be repeated one or more times as part of a single scan cycle. After steps 4010-4040 are completed, the scan cycle sequence can be executed again from step 4010.

[0609] Figure 40B is a flowchart of another exemplary process for emitting light from the LIDAR system 100. In step 4502, the processor 118 can determine whether the current FOV frame matches the scan rate specified as the far-field scan rate or is otherwise specified as such. If it matches, then in step 4506, the processor 118 can control the light source 112 and deflector 114 according to the far-field illumination scheme. If it does not match, then in step 4504, the processor 118 can control the light source 112 and deflector 114 according to the near-field illumination scheme. In step 4508, reflected light can be acquired. In step 4510, the detected light can be analyzed, and in step 4512, a 3D depth map representation of the FOV can be generated based on the detected light reflection.

[0610] Dynamic operating modes based on the operating environment

[0611] In a vehicle having a LiDAR system according to embodiments of the present disclosure, the driving environment may change throughout the entire driving process. For example, the vehicle may start driving in an urban (or suburban) environment and move to a rural environment during the drive. Other driving environments may include parking lots, traffic congestion, tunnels, intersections, bridges, interstate highways, or main roads. Based on various environmental indicators (or based on direct input to the system), the LiDAR system according to embodiments of the present disclosure can adjust one or more characteristics of the field of view scanning to take the environment into account. For example, the LiDAR system can adjust instantaneous detection distance, spatial resolution, temporal resolution, signal-to-noise ratio, overall light beam distribution of the FOV, frame rate, field of view size, field of view aspect ratio, one or more pulse transmission schemes, etc.

[0612] Accordingly, the systems and methods of this disclosure allow for the adjustment of one or more characteristics of the field of view scan in accordance with the determined driving environment. Figure 41A shows an exemplary method 4100 for changing the detection distance in a LIDAR system. Method 4100 in Figure 41A adjusts the detection distance, but other characteristics as described above may be adjusted in addition to or instead of this. Method 4100 can be performed by at least one processor (for example, the processor 118 of the processing unit 108 of the LIDAR system 100 as shown in Figure 1A, and/or two processors 118 of the processing unit 108 of the LIDAR system as shown in Figure 2A).

[0613] In step 4101, the processor 118 controls at least one light source (e.g., light source 112 in Figure 1A and Figure 2A, field 4203 in Figure 42A, field 4207 in Figure 42B, field 4211 in Figure 42C, field 4215 in Figure 42D, field 4215' in Figure 42E, field 4311 in Figure 43) to vary the luminous flux from at least one light source. For example, the processor 118 may vary the timing of pulses from at least one light source. Alternatively, or simultaneously, the processor 118 may vary the length of pulses from at least one light source. As another example, the processor 118 may, instead or simultaneously, vary the magnitude of pulses from at least one light source (e.g., by changing the length or width, or by otherwise changing the cross-sectional area). In yet another example, the processor 118 may, instead or simultaneously, vary the amplitude and/or frequency of pulses from at least one light source.

[0614] In step 4103, the processor 118 controls at least one optical deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or unidirectional deflector 214 in Figure 2B) to deflect light from at least one light source in order to scan the field of view. For example, the processor 118 may scan the field of view by mechanically moving at least one optical deflector. Alternatively or simultaneously, the processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector to scan the field of view. Alternatively or simultaneously, the processor 118 may induce steering of an optical phased array (OPA) light source by changing the relative amplitude, phase, or other signal characteristics of different emission sources of the OPA. Alternatively or simultaneously, the processor 118 may induce a change in the active light emitter of a vertical cavity surface-emitting laser (VCSEL) array.

[0615] In some embodiments, the field of view (e.g., field of view 4215 in Figure 42D, field of view 4215' in Figure 42E, field of view 4311 in Figure 43) may include multiple parts (e.g., a first part and a second part). For example, these parts may include half, a quarter, or other proportions of the area covered by the field of view. In other examples, these parts may include irregular portions rather than symmetrical and/or any proportion of the area covered by the field of view. In yet another example, these parts may include discontinuous portions of the area covered by the field of view.

[0616] In some embodiments, the processor 118 can control at least one optical deflector so that at least one optical deflector is positioned at a plurality of different instantaneous positions during a single scan cycle (for example, the deflector is controlled to move from one instantaneous position to another instantaneous position or through one instantaneous position during a scan of the Lidar FOV). For example, at least one optical deflector can move continuously or discontinuously from one of the plurality of positions to another during a scan cycle (optionally, additional positions and/or repetitions may also be used).

[0617] In such embodiments, the processor 118 can coordinate at least one optical deflector with at least one light source such that, when at least one optical deflector is positioned at a specific instantaneous location, a portion of the light beam is deflected by at least one optical deflector from at least one light source toward an object in the field of view, and a reflection of a portion of the light beam is deflected from the object toward at least one sensor. Thus, at least one optical deflector can guide a portion of the light beam toward the field of view and receive reflections from the field of view. For example, in the examples shown in Figures 1A, 2B, and 2C, the deflector guides a portion of the light beam toward the field of view and receives reflections from the field of view. In other embodiments, a portion of the light beam from at least one light source can be guided toward the field of view by at least one optical deflector separate from at least one other optical deflector that receives reflections from the field of view. For example, in the example shown in Figure 2A, one deflector guides a portion of the light beam toward the field of view, and another deflector receives reflections from the field of view. In some embodiments, at least one deflector may include a first group of one or more deflectors (for transmission) and a second group of one or more deflectors (for reception). These may be different from each other.

[0618] In some embodiments, at least one light source may include multiple light sources aimed at a common area of at least one optical deflector. In such embodiments, the processor 118 can control at least one optical deflector such that light from the multiple light sources is projected onto multiple distinct regions forming a field of view when at least one optical deflector is positioned at a particular instantaneous location. An example of such an embodiment is shown in Figure 43, which is discussed below.

[0619] In step 4105, the processor 118 receives an input indicating the vehicle's current driving environment. For example, the processor 118 may receive an input including at least one rural-related instruction and an urban-related instruction. As another example, the processor 118 may receive an input including at least one rural-related instruction, an urban-related instruction, information regarding lighting conditions, information regarding weather conditions, and information regarding vehicle speed.

[0620] In some embodiments, the processor 118 can receive input from decisions made by the processor 118 itself. In such examples, the processor 118 can determine the current driving environment based on information from one or more previous (and/or current) scans of the field of view. For example, the processor may determine that the current driving environment is urban based on the presence of many vehicles and/or buildings very close to the vehicles. In another example, the processor may determine that the current driving environment is rural based on the presence of many trees and/or open land. Alternatively or simultaneously, the processor 118 may determine the current driving environment based on the vehicle's speed and/or map information (which can be stored or received and may include updated traffic information). For example, the processor 118 may determine that the current driving environment is an interstate highway or highway based on the vehicle maintaining a high speed and/or the vehicle's location matching a known interstate highway or highway. As another example, the processor 118 can determine that the current driving environment is congested based on the fact that the vehicle frequently stops while maintaining a low speed and/or based on known traffic information.

[0621] Alternatively, or simultaneously, the processor 118 may receive input from a host processing unit, such as a central computer located within the vehicle along with the processor 118. The central computer can determine the current driving environment using the techniques described above with respect to the processor 118. Similarly, the processor 118 may, in addition to or instead, receive input from remote systems. For example, the processor 118 may receive climate instructions from a climate server or other source of updated climate information. Similarly, the processor 118 may receive traffic instructions from a traffic server or other source of updated traffic information.

[0622] In some embodiments, the processor 118 can receive input indicating the current driving environment from at least one of the following: GPS, vehicle navigation system, vehicle controller, radar, LIDAR, and camera. For example, as described above, the processor 118 can derive the current driving environment by using the vehicle location determined by GPS and/or the vehicle navigation system in combination with a map and/or traffic information. In such an example, the processor 118 can determine that the vehicle is located on an interstate highway by combining the vehicle's GPS location with a map, or that the vehicle is in a traffic jam by combining the vehicle's GPS location with traffic information. Similarly, the processor 118 can derive the current driving environment as described above using speed, direction of travel, etc., from the vehicle controller. In addition to or instead of this, the processor 118 can derive the current driving environment using information from radar, LIDAR, and/or camera. For example, the processor 118 can use radar, LiDAR, and/or cameras to identify one or more objects such as fields, trees, buildings, and median strips, and use these identified objects to derive the current driving environment.

[0623] In step 4107, based on the currently detected or estimated driving environment, the processor 118 can dynamically adjust the instantaneous detection distance by coordinating the control of at least one light source and at least one light deflector to vary the amount of projected light and the spatial distribution of light in the field of view scan. For example, the processor 118 can increase the amount of projected light and/or decrease the spatial distribution of light to extend the instantaneous detection distance. As another example, the processor 118 can decrease the amount of projected light and/or increase the spatial distribution of light to reduce the instantaneous detection distance. For example, as shown in the examples of Figures 42D and 42E, the processor 118 can determine when a vehicle has exited a tunnel and coordinate the control of at least one light source and at least one light deflector to increase the light emission in at least one portion of the field of view compared to the light emission used in at least one portion of the field of view when the vehicle was located inside the tunnel.

[0624] The processor 118 can vary the amount of light projected in the field scan by changing the length of pulses from at least one light source, the amplitude and/or frequency of pulses from at least one light source, etc. In addition to or instead of this, the processor 118 can vary the spatial light distribution of light in the field scan by changing the intensity of at least one optical deflector (for example, in embodiments where at least one optical deflector is piezoelectric or thermoelectric), the reflection angle of at least one optical deflector (for example, by expanding or contracting the spread of the light beam from at least one light source), etc.

[0625] The processor 118 can dynamically adjust the instantaneous detection distance based on the current driving environment. For example, the processor 118 can increase the detection distance in rural environments. Since there are fewer objects in rural environments than in urban environments, this can be compensated for by increasing the detection distance. As another example, the processor 118 can decrease the detection distance in traffic congestion. Traffic congestion significantly slows down vehicle speeds and increases the frequency and abruptness of stops, so the importance of detecting distant objects may decrease. Energy not consumed for a wider detection range can simply be saved or used to improve other detection characteristics such as resolution, frame rate, and SNR.

[0626] In step 4107, the processor 118 may, in addition to or instead of the above, adjust other characteristics of the LIDAR system. For example, the processor 118 can dynamically adjust the scan rate by coordinating the control of at least one light source and the control of at least one optical deflector to vary the amount of light projected and the spatial light distribution of light in the field of view scan. In such an example, the processor 118 can increase the scan rate in urban environments. In urban environments, a significant number of other vehicles and pedestrians are moving, so a faster scan rate allows for earlier detection of events such as other vehicles stopping ahead of the vehicle or pedestrians moving onto the road. As another example, the processor 118 can reduce the scan rate in rural environments. In rural environments, there are fewer other vehicles and pedestrians than in urban environments, so the need for a high scan rate may be reduced. Therefore, the processor 118 can determine when a vehicle is located in an urban area and coordinate the control of at least one light source and at least one optical deflector to generate a scan cycle rate increased compared to the scan cycle rate used in non-urban areas.

[0627] In another example, the processor 118 can dynamically adjust the spatial resolution by coordinating the control of at least one light source and at least one light deflector to vary the amount of projected light and the spatial light distribution in the field of view scan. In such an example, the processor 118 can increase the spatial resolution in rainy weather. In urban environments, objects may be present at a higher density than in rural environments, and this density can be compensated for by increasing the spatial resolution. In another example, the processor 118 can reduce the scan rate in tunnels. In tunnels, there are few details other than other vehicles in front of the vehicle, so the need for high resolution may be reduced.

[0628] In yet another example, the processor 118 can dynamically adjust the temporal resolution by coordinating the control of at least one light source and at least one optical deflector to vary the amount of projected light and the spatial light distribution of light in the field of view scan. In such an example, the processor 118 can increase the temporal resolution in urban environments. In urban environments, a significant number of other vehicles and pedestrians are moving, so increasing the temporal resolution allows for more detailed monitoring of the movement of other vehicles and pedestrians. In another example, the processor 118 can decrease the temporal resolution in rural environments. In rural environments, there are fewer other vehicles and pedestrians than in urban environments, so the need for detailed monitoring may be lower.

[0629] In yet another example, the processor 118 can dynamically adjust the signal-to-noise ratio by coordinating the control of at least one light source and at least one optical deflector to vary the amount of projected light and the spatial light distribution of light in the field scan. In such an example, the processor 118 can increase the signal-to-noise ratio in rainy weather. In rainy weather, the amount of reflection in the field increases, which increases ambient noise, so increasing the signal-to-noise ratio can reduce the impact of increased noise. In another example, the processor 118 can decrease the signal-to-noise ratio at night. At night, noise decreases, so the need to acquire a strong signal that can be distinguished from the noise may be reduced.

[0630] In yet another example, the processor 118 can dynamically adjust the field of view by coordinating the control of at least one light source and at least one light deflector to vary the amount of projected light and the spatial light distribution of light in the field of view scan. In such an example, the processor 118 can reduce the field of view in a rural environment. In rural environments, the need for a large field of view may be reduced because there are fewer road lanes. In another example, the processor 118 can enlarge the field of view on an interstate highway. Because interstate highways have many lanes, enlarging the field of view makes it possible to monitor the large number of vehicles present on the interstate highway.

[0631] In yet another example, the processor 118 can dynamically adjust one or more pulse transmission schemes by coordinating the control of at least one light source and at least one optical deflector to vary the amount of projected light and the spatial light distribution of light in the field scan. For example, some schemes are more or less susceptible to noise and/or ambient light. Therefore, the processor 118 can select a pulse transmission scheme that is less susceptible to noise in high-noise environments such as rain or snow, and a pulse transmission scheme that is less susceptible to ambient light in high-ambient-light environments such as urban environments or nighttime.

[0632] In embodiments where the field of view has multiple portions, the processor 118 can dynamically adjust the instantaneous detection distance in a single scan cycle to expand the detection distance in a first portion of the field of view compared to a previous scan cycle, while simultaneously reducing the detection distance in a second portion of the field of view. For example, if a vehicle is in a traffic jam but the other side of the road is not congested, the processor 118 can reduce the detection distance in the field of view portion in front of the vehicle on the side of the road it is traveling on, while expanding the detection distance in another portion of the field of view on the other side of the road adjacent to the vehicle. In such an example, expanding the detection distance in the portion including the other side of the road allows the vehicle to react more quickly if a vehicle on the other side of the road enters its path. Furthermore, reducing the detection distance in the portion including the side it is traveling on prevents unnecessary energy consumption due to the vehicle being stationary in a traffic jam.

[0633] Similarly, the processor 118 can dynamically adjust other characteristics of the scan to increase the characteristics of the first portion of the field of view compared to the previous scan cycle, while decreasing the characteristics of the second portion of the field of view compared to the previous scan cycle. For example, the processor 118 can increase the spatial resolution of the first portion and decrease the spatial resolution of the second portion. In such an example, if the vehicle is located in a tunnel, the processor 118 can increase the spatial resolution of the forward portion and decrease the spatial resolution of the lateral portion. Increasing the spatial resolution can enhance tracking the movement of other vehicles in front of the vehicle, while decreasing the spatial resolution can prevent unnecessary energy consumption in tracking the tunnel walls.

[0634] As yet another example, the processor 118 can increase the temporal resolution of the first portion and decrease the temporal resolution of the second portion. In such an example, if the vehicle is located on an interstate highway, the processor 118 can increase the temporal resolution of the lateral portion and decrease the temporal resolution of the forward portion. Increasing the spatial resolution allows for more detailed tracking of the movement of other vehicles traveling in the opposite direction, preparing for situations where a quick reaction is required if another vehicle enters the lane on the side of the vehicle. Reducing the temporal resolution prevents unnecessary energy consumption in tracking other vehicles traveling ahead of the vehicle.

[0635] In another example, the processor 118 can increase the signal-to-noise ratio of the first portion and decrease the signal-to-noise ratio of the second portion. In such an example, when the vehicle is located in an urban environment, the processor 118 can increase the signal-to-noise ratio of the lateral portion and decrease the signal-to-noise ratio of the forward portion. Increasing the signal-to-noise ratio can compensate for ambient light from road lighting on the sides of the road, and reducing the signal-to-noise ratio can prevent unnecessary energy consumption in tracking other vehicles traveling ahead of the vehicle.

[0636] In yet another example, the processor 118 can enlarge the size of the first portion of the field of view and reduce the size of the second portion of the field of view. In such an example, when the vehicle is located in a rural environment, the processor 118 can enlarge the forward portion of the field of view and reduce the lateral portion of the field of view. Enlarging the field of view makes it possible to see oncoming or preceding vehicles earlier, while reducing the field of view prevents unnecessary energy from being wasted tracking fields or trees to the side of the road.

[0637] In another example, the processor 118 can change the pulse transmission scheme in the first part and also change the pulse transmission scheme in the second part to a different one. In such an example, when the vehicle is located in a rural environment at night, the processor 118 can select a pulse transmission scheme that minimizes noise for the forward part and a pulse transmission scheme that is susceptible to noise for the lateral part. The former scheme can deal with noise from the bright headlights of oncoming vehicles or the taillights of preceding vehicles, while the latter scheme can prevent unnecessary energy consumption in minimizing noise from objects on the sides of the road, which is already minimal.

[0638] Similarly, in some embodiments, the processor 118 can dynamically adjust a first scan cycle rate for detecting objects in the near field of view and a second scan cycle rate for detecting objects in the far field of view by coordinating the control of at least one light source and at least one optical deflector over multiple scan cycles based on the current operating environment. For example, the processor 118 can scan objects in the near field of view at a higher rate than objects in the far field of view. This allows the processor 118 to track the movement of nearby objects more precisely than the movement of distant objects. Alternatively, the processor 118 can scan objects in the near field of view at a lower rate than objects in the far field of view.

[0639] Method 4100 may include additional steps. For example, Method 4100 may further include coordinating the control of at least one light source and at least one optical deflector in at least one scan cycle to dynamically adjust the instantaneous resolution associated with the first portion of the field of view. For example, by spreading the light beam from at least one light source over a larger area, data can be generated over a larger number of pixels, or by compressing it over a smaller area, data can be generated over a smaller number of pixels.

[0640] Method 4100 may further include controlling at least one light source based on an environment type corresponding to a received input indicating the current operating environment. For example, the processor 118 can adjust the characteristics of at least one light source, such as luminous flux or wavelength. In such an example, the processor 118 may select a lower wavelength at night than during the day, or a higher intensity in a rural environment than in an urban environment.

[0641] Method 4100 may further include adjusting the sensitivity mode of at least one sensor based on the current driving environment. For example, the processor 118 may determine when the vehicle is being driven in the rain and adjust the sensitivity mode associated with the output from at least one sensor to dismiss the reflection of raindrops. As discussed in detail above, changes in sensor sensitivity can be achieved by changing sensor parameters (e.g., operating voltage), detection path parameters (e.g., signal amplification level, ADC parameters), or processor parameters (e.g., thresholds or decision rules applied to the processor).

[0642] Figure 41B shows an exemplary method 4100' for changing the detection distance in a LiDAR system. Method 4100' can be performed by at least one processor (for example, the processor 118 of the processing unit 108 of the LiDAR system 100 as shown in Figure 1A, and/or two processors 118 of the processing unit 108 of the LiDAR system as shown in Figure 2A).

[0643] Steps 4101, 4103, and 4105 of method 4100' in Figure 41B are identical to steps 4101, 4103, and 4105 of method 4100 in Figure 41A. Therefore, their descriptions will not be repeated here.

[0644] In step 4107, based on the currently detected or inferred operating environment, the processor 118 can modify the detection operation scheme by coordinating the control of at least one light source and at least one optical deflector. For example, as described above with reference to Figure 17, the processor 118 can modify the operating parameters of at least one sensor and/or the processor 118. For example, modifying the operating parameters in such a case can change the detection sensitivity to the signal level and/or noise level acquired by at least one sensor. For example, the processor 118 can change the sensor sensitivity by changing the post-convolution threshold, as discussed above with reference to Figure 17. However, in addition to or instead of this, it is also possible for the processor 118 to modify other operating parameters of at least one sensor and/or the processor 118.

[0645] Figures 42A to 42E show examples of different driving environments. In the example in Figure 42A, the vehicle 4201 may include a vehicle body and at least one processor located within the vehicle body. The at least one processor can perform the method 4100 of Figure 41 or a variation thereof. In the example in Figure 42A, the vehicle 4201 is operating in an urban environment. Therefore, the LIDAR system of the vehicle 4201 can scan the field of view 4203 using a high frame rate (e.g., 25 frames per second (FPS)), a medium distance (e.g., 100 meters), and a wide horizontal field of view (e.g., 320°, 340°, 360°, etc.), as shown in Figure 42A. This can cope with medium speed, altitude details, nearby objects, and rapid situational changes that may occur in relation to the urban environment.

[0646] In the example in Figure 42B, the vehicle 4205 may include a vehicle body and at least one processor located within the vehicle body. The at least one processor can perform the method 4100 of Figure 41 or a variation thereof. In the example in Figure 42B, the vehicle 4205 is operating in a rural environment. Therefore, the LIDAR system of the vehicle 4205 can scan the field of view 4207 using a moderate frame rate (e.g., 20 frames per second (FPS)), a long distance (e.g., 200 meters), and a moderate horizontal field of view (e.g., 200°, 150°, 120°, etc.), as shown in Figure 42B. In some embodiments, the range can be varied by scan, for example, 100 meters for most scans and 200 meters every five scans. Such settings can accommodate high speeds, low degree of detail, distant objects, and slow situational changes associated with rural environments.

[0647] In the example in Figure 42C, the vehicle 4209 may include a vehicle body and at least one processor located within the vehicle body. The at least one processor may perform the method 4100 of Figure 41 or a variation thereof. In the example in Figure 42C, the vehicle 4209 is driving in traffic congestion. Therefore, the LIDAR system of the vehicle 4209 can scan the field of view 4211 using a moderate frame rate (e.g., 20 frames per second (FPS)), a short distance (e.g., 75 meters), and a moderate horizontal field of view (e.g., 200°, 150°, 120°, etc.), as shown in Figure 42C. Such a setting can handle slow speeds, low degree of detail, nearby objects, and generally slow situational changes associated with traffic congestion.

[0648] In the example of Figure 42D, the vehicle 4213 may include a vehicle body and at least one processor located within the vehicle body. The at least one processor can perform the method 4100 of Figure 41 or a variation thereof. In the example of Figure 42D, the vehicle 4213 is operating in a tunnel. Therefore, the LIDAR system of the vehicle 4213 can scan the field of view 4215 using different characteristics in regions 4217a and 4217c than in region 4217b. In regions 4217a and 4217c, a lower frame rate (e.g., 10 frames per second (FPS)), a short distance (e.g., 75 meters), and lower spatial and/or temporal resolution can be used to address the fact that it is not necessary to track the tunnel walls. On the other hand, in region 4217b, a moderate frame rate (e.g., 20 frames per second (FPS)), a moderate distance (e.g., 100 meters), and moderate spatial and/or temporal resolution can be used to track a sudden stop that may occur in another vehicle preceding vehicle 4213. Alternatively, as shown in Figure 42D, it is possible to scan region 4217b without scanning regions 4217a and 4217c (indicated by "X").

[0649] In the example in Figure 42E, vehicle 4213 has just emerged from a tunnel. Therefore, the vehicle 4213's LiDAR system can scan the field of view 4215' in regions 4217a' and 4217c' using different characteristics than those previously used in regions 4217a and 4217c. For example, the frame rate, detection distance, and/or spatial and/or temporal resolution can be increased to address the need to track objects that may be present in these regions. On the other hand, the frame rate, detection distance, and/or spatial and/or temporal resolution in region 4217b' can be kept the same as those used in region 4217b. Alternatively, at this point, regions 4217a and 4217c may be scanned in addition to region 4217b using either the same characteristics (as shown in Figure 42E) or different characteristics.

[0650] Figure 42F shows vehicles 4201, 4205, and 4209 from Figures 42A, 42B, and 42C, along with their corresponding fields of view 4203, 4207, and 4211, respectively. As shown in Figure 42F, the field of view 4203 of vehicle 4201 in an urban environment has a moderate detection distance and a wide horizontal field of view. As further shown in Figure 42F, the field of view 4207 of vehicle 4205 in a rural environment has a greater detection distance than the field of view 4203 (in the urban environment), but a moderate horizontal field of view compared to the field of view 4203 (in the urban environment). Furthermore, as shown in Figure 42F, the field of view 4211 of vehicle 4209 in traffic congestion has a shorter detection distance than the field of view 4203 (in the urban environment), but a wide horizontal field of view (similar to, for example, the field of view 4203 in the urban environment).

[0651] Additional operating conditions not shown in Figures 42A to 42F may cause one or more characteristics of the LiDAR system to be adjusted. For example, in rainy conditions, a LiDAR system according to an embodiment of the present disclosure can scan the field of view using a high frame rate (e.g., 25 frames per second (FPS)) and high spatial and/or temporal resolution to address large noise and distorted details in each frame.

[0652] Figure 43 shows an exemplary LIDAR system 4300 having multiple light sources aimed at a common area of at least one optical deflector. As shown in Figure 43, light from the multiple light sources can be incident on the overlapping area of at least one optical deflector. In addition to or instead of this, light emitted from the multiple light sources and reflected back from the scene can be incident on the overlapping area of at least one optical deflector. As shown in Figure 43, the system 4300 includes a processing unit (e.g., at least one processor 4301) along with the multiple light sources (e.g., light sources 4303a, 4303b, and 4303c). The multiple light sources 4303a, 4303b, and 4303c can emit corresponding multiple light beams (e.g., light beams 4305a, 4305b, and 4305c).

[0653] In the embodiment of Figure 43, the LIDAR system 4300 includes at least one deflector 4309 having a common area 4307. The at least one deflector 4309 may be positioned, for example, at a specific instantaneous position during a scan cycle. Multiple light sources 4303a, 4303b, and 4303c can be aimed at the common area 4307 and thus can direct multiple corresponding light beams 4305a, 4305b, and 4305c thereto. The common area 4307 can project multiple light beams 4305a, 4305b, and 4305c into a field of view 4311. In the embodiment of Figure 43, the common area 4307 can project multiple light beams 4305a, 4305b, and 4305c into multiple separate regions (e.g., regions 4313a, 4313b, and 4313c) that form a field of view 4311. Multiple light beams 4305a, 4305b, and 4305c produce multiple corresponding reflections 4315a, 4315b, and 4315c from the field of view 4311 (or from objects within the field of view).

[0654] Furthermore, in the example of Figure 43, the scan rates of multiple regions 4313a, 4313b, and 4313c may differ. For example, as shown in Figure 43, the scan rate of region 4313a is slower than that of region 4313b, and the scan rate of region 4313c is...

[0655] In other embodiments, in addition to or instead of the above, the scan characteristics may differ between the multiple regions 4313a, 4313b, and 4313c. For example, the instantaneous detection distance, spatial resolution, temporal resolution, signal-to-noise ratio, field size, one or more pulse transmission schemes, etc., may differ between the multiple regions 4313a, 4313b, and 4313c, either independently or in combination.

[0656] In the example of Figure 43, the light beams 4305a, 4305b, and 4305c, and their corresponding reflections 4315a, 4315b, and 4315c, both enter a common area 4307 of at least one deflector 4309. However, in other embodiments, the light beams 4305a, 4305b, and 4305c may be projected by one or more deflectors different from those that reflect the corresponding reflections 4315a, 4315b, and 4315c.

[0657] As further illustrated in the example in Figure 43, each reflector 4315a, 4315b, and 4315c is directed to at least one corresponding deflector and sensor (e.g., deflectors 4317a, 4317b, and 4317c coupled to sensors 4319a, 4319b, and 4319c). However, in other embodiments, any additional deflectors may be omitted. Furthermore, in other embodiments, two or more reflectors may be directed to a single sensor.

[0658] Lidar detection scheme for lane crossing and direction change

[0659] Lane-crossing turns without a directional control system can present several problems. For example, navigating a vehicle along a route that crosses one or more lanes (e.g., a left turn at an intersection in the US, or a right turn at an intersection in the UK) can be difficult in congested traffic with oncoming vehicles, bicycles, and pedestrians. Human drivers may enter the intersection, wait for an opportunity to accelerate, and perform a dangerous lane-crossing turn. Similar problems may exist for autonomous or semi-autonomous vehicles.

[0660] To assist navigation in other road conditions, including intersections or lane crossings, the LIDAR system 100 may be configured to modify one or more operating characteristics of the system with respect to a particular area of the LIDAR FOV with respect to other areas of the LIDAR FOV. For example, in a LIDAR system according to an embodiment of the present disclosure, the detection range of a particular area of the LIDAR FOV (e.g., the portion of the FOV that overlaps with the crossing lane) may be modified.
The FOV can be extended to one or more other regions. For example, in one instance, if a vehicle (autonomous vehicle or other) is attempting a left turn crossing at least one lane, the detection range associated with one or more regions of the LIDAR FOV overlapping with the crossing lane (e.g., roughly the right half of the FOV corresponding to the right front quarter of the vehicle facing the crossing oncoming lane) may be larger than the detection range of other regions of the FOV (e.g., roughly the left half of the FOV corresponding to the left front quarter of the vehicle not facing the crossing oncoming lane). It should be noted that the LIDAR FOV may include a set of multiple scan regions, which may or may not be continuous. For example, in some embodiments, the LIDAR FOV may consist of multiple parts containing overlapping continuous solid angle values. In other embodiments, the LIDAR FOV is a set of multiple non-overlapping or partially overlapping solid angle ranges, each of which is bisected by axes extending in different directions (as shown in Figure 45). In this way, the LIDAR system 100 can effectively detect oncoming vehicles and generate high-resolution depth maps related to the crossing lanes.

[0661] The system and method of this disclosure can temporarily increase the detection range in the direction opposite to the direction of the vehicle's lane-crossing turn (for example, the detection range related to one or more regions of the FOV in the right half of the FOV if the direction of the lane-crossing turn is left) compared to the detection range in the direction of the lane-crossing turn. Such a modification of the detection range can be achieved, for example, by coordinating the control of at least one light source with the control of at least one light deflector to increase the light beam on the side opposite to the direction of the vehicle's lane-crossing turn, including the far lane the vehicle is attempting to merge into, compared to the rest of the field of view.

[0662] Figure 44 shows an exemplary method 4400 of a LIDAR detection scheme for lane crossing direction changes. Method 4400 can be performed by at least one processor (e.g., processor 118 of processing unit 108 of LIDAR system 100 as shown in Figure 1A, and/or two processors 118 of processing unit 108 of LIDAR system as shown in Figure 2A). In step 4401, processor 118 can control at least one light source so that it can vary the beam of light from at least one light source (e.g., light source 112 in Figure 1A, laser diode 202 of light source 112 in Figure 2A, and/or multiple light sources 102 in Figure 2B) during a scan cycle of the field of view (e.g., field of view 120 in Figures 1A and 2A). For example, processor 118 can vary the timing of pulses from at least one light source. Alternatively or simultaneously, processor 118 can vary the length of pulses from at least one light source. As another example, the processor 118 may, instead of or simultaneously, vary the magnitude of pulses from at least one light source (e.g., by changing the length or width, or by otherwise changing the cross-sectional area). In yet another example, the processor 118 may, instead of or simultaneously, vary the amplitude and/or frequency of pulses from at least one light source.

[0663] Step 4402 may further include the processor 118 controlling at least one deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or unidirectional deflector 214 in Figure 2B) to deflect light from at least one light source in order to scan the field of view. For example, the processor 118 may scan the field of view by mechanically moving at least one optical deflector. Alternatively, or simultaneously, the processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector in order to scan the field of view.

[0664] In some embodiments, a single scan cycle of the field of view may include moving at least one optical deflector so that at least one deflector is positioned at a plurality of different instantaneous positions during the scan cycle (for example, the deflector is controlled to move from or through one instantaneous position to another instantaneous position during the scan of the LIDAR FOV). For example, at least one optical deflector may move continuously or discontinuously from one of the plurality of positions to another during the scan cycle (optionally, additional positions and/or repetitions may also be used).

[0665] In such embodiments, the processor 118 can coordinate at least one optical deflector with at least one light source such that, when at least one optical deflector is positioned at a particular instantaneous location, a light beam is deflected by at least one optical deflector from at least one light source toward the field of view, and reflections from objects in the field of view are deflected by at least one optical deflector toward at least one sensor. Thus, at least one optical deflector can guide a light beam toward the field of view and receive reflections from the field of view. For example, in the examples shown in Figures 1A, 2B, and 2C, the deflector guides a light beam toward the field of view and receives reflections from the field of view. In some embodiments, the reflection may be caused by the light beam guided toward the field of view. In other embodiments, a light beam from at least one light source can be guided toward the field of view by at least one optical deflector separate from at least one other optical deflector that receives reflections from the field of view. For example, in the example shown in Figure 2A, one deflector guides the light beam towards the field of view, while another deflector receives the reflected light from the field of view.

[0666] In step 4403, the processor 118 receives an input indicating that a lane-crossing direction change of the vehicle is likely to occur. An example of the lane-crossing direction change technique is discussed below with reference to Figures 45 and 46.

[0667] In step 4404, in response to an input indicating that a lane crossing turn is likely to occur, the processor 118 coordinates the control of at least one light source with the control of at least one light deflector to increase the light beam on the side opposite to the direction of the vehicle's lane crossing turn (i.e., direction, angle, and region of interest, not part of the vehicle) compared to the rest of the field of view, including the far lane the vehicle is about to merge into, thereby temporarily expanding the detection range on the side opposite to the direction of the vehicle's lane crossing turn beyond the detection range in the direction of the lane crossing turn.

[0668] In some embodiments, the processor 118 can increase the light beam on the side opposite to the direction of the vehicle's lane-crossing turn, including the far lane the vehicle is merging into, by controlling one light source or one deflector, compared to the rest of the field of view. For example, if the deflectors (e.g., the optical deflector 114 in Figure 1A, the deflector 114A and/or the deflector 114B in Figure 2A, and/or the unidirectional deflector 214 in Figure 2B) are continuing to scan according to a fixed scan schedule, the parameters of the light source 112 can be changed to vary the detection range in the region of the LIDAR FOV where an object (e.g., a vehicle) may be present in one or more lanes crossed by the lane-crossing turn. Such regions of the FOV in which the detection range can be expanded (individually or collectively) may be referred to as regions of interest.

[0669] In some embodiments, the detection range within the region of interest can be larger than the detection range within the FOV outside the region of interest. Any suitable detection range ratio can be achieved. In some embodiments, the detection range within the region of interest can be at least twice as large as the detection range outside the region of interest. In a particular example of a situation in which a vehicle is making a lane-crossing turn, the processor 118 can detect a vehicle within range X in the direction corresponding to the region of interest (e.g., a specific portion of the LIDAR FOV overlapping with the crossing lane), and can detect a vehicle only within a range of X/2 or less in the region of the LIDAR FOV that is not the region of interest (e.g., does not overlap with the crossing lane). For example, when an autonomous vehicle is preparing, initiating, and/or performing a left turn, the region of interest can be the right half of the LIDAR FOV encompassing at least a portion of the right side of the vehicle. In some cases, the region of the LIDAR FOV overlapping with the area immediately in front of the vehicle is outside the region of interest. In addition to or instead of this, the LiDAR system FOV can be provided across different segments, each segment covering, for example, a different zone surrounding the vehicle. In such cases, the LiDAR FOV's region of interest may be located on the passenger side of the vehicle, the driver's side, the rear of the vehicle, or in any quarter direction relative to the vehicle (e.g., between the vehicle's longitudinal and transverse axes). The processor 118 can control components of the LiDAR system 100, particularly the light source 112 (including any of its controllable parameters affecting the light output) and the deflector 114, to expand the detection range within the region of interest and to conserve resources for areas of less interest to the LiDAR FOV.

[0670] Method 4400 may include additional steps. For example, Method 4400 may further include controlling the optical deflectors so that at least one optical deflector is positioned at a plurality of different instantaneous locations during a scanning cycle of the field of view. Alternatively, Method 4400 may further include coordinating at least one optical deflector with a light source so that when the optical deflector is positioned at a particular instantaneous location, a portion of the light beam is deflected by the optical deflector from the light source towards an object in the field of view, and a portion of the light beam reflected from the object is deflected by the optical deflector towards at least one sensor. In some embodiments, the LIDAR system may further include a plurality of light sources aimed at at least one optical deflector. The processor can control the optical deflector so that when the optical deflector is positioned at a particular instantaneous location, light from the plurality of light sources is projected onto a plurality of distinct areas in the field of view.

[0671] The processor 118 can determine, based on various sources, that the host vehicle is planning or has initiated a lane-crossing turn. For example, in some embodiments, the processor 118 can receive an input from the vehicle's navigation system indicating that a lane-crossing turn is likely to occur. In other embodiments, the processor 118 can receive an input indicating that a lane-crossing turn is likely to occur from another system in the vehicle (e.g., one or more sensors, linked turn signals, wheel steering direction, GPS sensor, etc.) or from an external system (e.g., one or more autonomous vehicle navigation server systems, mapping systems, etc.). In other embodiments, the processor 118 can determine an input indicating that a lane-crossing turn is likely to occur based on information received from at least one sensor (e.g., sensor 116) configured to detect reflections related to light projected from at least one light source. In other words, the processor 118 can determine that a lane-crossing turn is likely to occur or has begun based on the output of sensor 116 of the LIDAR system 100.

[0672] The processor 118 can be configured to determine one or more features of an object detected in the crossing lane. In some examples, one or more features may include the distance to the detected object and/or the type of object (e.g., car, truck, stationary object, pedestrian, etc.). In some embodiments, the processor 118 can also determine the speed, direction of travel (e.g., by monitoring the location with two or more FOV scans) or any other features associated with the detected object of a moving object detected in the crossing lane (e.g., the far lane). In some embodiments, the processor 118 can monitor the motion characteristics of the host vehicle (e.g., speed, acceleration, position, etc.) and determine whether the host vehicle and the detected object are on a collision course based on the motion characteristics of the host vehicle and the motion characteristics of the detected object (e.g., in the crossing lane). If it is determined that they are on a collision course, the processor 118 can generate a warning (e.g., a horn sound, a visual warning, wireless communication to a controller associated with the detected object, etc.). In other embodiments, the determination of a collision course may include other potential hazards. For example, when the host vehicle is stationary, a warning is issued to the host if, for instance, an approaching vehicle poses a threat to the host vehicle at its current location, or if the host vehicle is expected to move. In some embodiments, the processor 118 can issue warnings (auditory, visual, or other) not only in autonomous mode but also in other modes (e.g., advanced driver assistance system operation, full driver control, etc.).

[0673] In some embodiments, the processor 118 can generate reflectance images related to the LiDAR field of view. The reflectance images may include a fingerprint of the moving object, representing the amount of light reflected from various parts of the detected moving object. For example, when light from the light projector 112 is incident on objects in the environment of the LiDAR system 100, the processor 118 can detect a pattern based on the reflectance characteristics of those objects. For example, the processor 118 can identify a reflectance pattern or fingerprint to determine a type category associated with the detected object (e.g., pedestrian, vehicle, median barrier, etc.). The processor 118 can also determine a subtype associated with the detected object (e.g., whether the detected vehicle is a bus, a small car, or a light van, etc.). All vehicles may exhibit different reflectance fingerprints based on their shape and configuration (e.g., the position and contour around the license plate, the size, shape, spacing, and arrangement of the headlights on the vehicle, etc.).

[0674] In some embodiments, the processor 118 can be configured to determine one or more states of a detected object based on a comparison of the detected reflectance pattern with a predetermined reflectance template. For example, in some embodiments, the processor 118 can determine that a moving object is a vehicle signaling a right turn by comparing one or more reflectance fingerprints acquired with respect to a moving object detected (e.g., over multiple scan cycles) with a plurality of predetermined/stored reflectance templates.

[0675] Furthermore, the processor 118 can be configured to allocate different luminous flux levels to different regions of the LIFAR FOV based on operations detected by the host vehicle. For example, in some embodiments, the processor 118 can apply a different power allocation scheme to right turns than to left turns. For example, depending on the change in luminous flux, the processor 118 can allocate different power to left turns than to right turns. Another example of power budget allocation is further described with reference to Figures 29 to 31.

[0676] In yet another embodiment, the processor 118 can receive an input indicating the current driving environment and apply a different power allocation scheme depending on the determined driving environment. For example, the processor 118 can apply a different power allocation scheme to a lane crossing turn in a rural area than a lane crossing turn in an urban area. For example, in an urban area, pedestrians or cyclists are more likely to approach the host vehicle from the side. Therefore, in an urban area, there may be a greater need to detect objects with higher accuracy in one or more directions relative to the host vehicle (e.g., driver's side, passenger side, rear, etc.) than in a rural area. In contrast, there may be fewer pedestrians and other obstacles in a rural area. Therefore, the need for a high-resolution depth map around the host vehicle may be lower, especially at close ranges to the host vehicle. However, on the other hand, in a rural environment, vehicle speeds tend to be higher than in an urban area due to various factors, including lower traffic volume. As a result, the processor 118 can allocate fewer resources to detecting objects very close to the host vehicle (e.g., approximately 40m, 20m, etc.) and instead allocate more resources to detecting objects at a greater distance.

[0677] In some embodiments, the processor 118 can control at least two light sources and at least two deflectors to enable scanning of a first field of view associated with the right side of the vehicle and a second field of view associated with the left side of the vehicle. In yet another embodiment, the processor 118 can control at least one light source 112 such that the amount of light projected onto the first portion of the field of view, including the road the vehicle is about to merge onto, is greater than the amount of light given to the second portion of the field of view, including buildings adjacent to the road.

[0678] As described above, increasing the luminous flux delivered to specific parts of the LIDAR FOV enhances various functions, particularly detection capabilities, in these regions, and expands the field of view (e.g., field of view 120 in Figures 1A and 2A). As shown in Figure 2C, the main light source 112A can project longer wavelength light to optimize the detection range. Furthermore, as shown in Figure 5A, the time between light pulses may depend on the desired detection range. Specifically, the luminous flux can be increased by delivering the same amount of light pulses over shorter time periods. As shown in Figure 5C, varying the number of pulses or changing the time between pulses is not the only way to adjust the luminous flux. Changes in the luminous flux can also be achieved by other methods. Other methods include, for example, pulse duration, pulse angular dispersion, wavelength, instantaneous power, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, and others.

[0679] As an example, the processor 118 can control at least one light source so as to vary the luminous flux from at least one light source (e.g., light source 112 in Figure 1A, the laser diode 202 of light source 112 in Figure 2A, and/or the multiple light sources 102 in Figure 2B) during a scan cycle of the field of view (e.g., the field of view 120 in Figures 1A and 2A).

[0680] Figure 45 includes Figure 4500, which shows an example of a LiDAR detection scan scheme. As shown, the autonomous vehicle 4510 is traveling on a road from east to west and is approaching a vehicle 4530 traveling from west to east. The autonomous vehicle 4510 may include a LiDAR system that can provide seven fields of view (these may overlap in at least some parts) by projecting light into seven different areas of the environment surrounding the vehicle 4510. In some embodiments, each field of view of the LiDAR system may be associated with a corresponding module such as a light source 112, a deflector 114, a detector 116, and associated optical components (e.g., lenses), although there may be more or fewer components. For simplicity, each of these modules is referred to here as a lens system as a means for identifying different fields of view. Through the lens system, the LiDAR system can receive reflected light from objects in the field of view. The autonomous vehicle 4510 may have a field of view 4521 corresponding to lens system 4511 (similar to field of view 120 in Figure 1A, for example), a field of view 4522 corresponding to lens system 4512, a field of view 4523 corresponding to lens system 4513, a field of view 4524 corresponding to lens system 4514, a field of view 4525 corresponding to lens system 4515, a field of view 4526 corresponding to lens system 4516, and a field of view 4527 corresponding to lens system 4517.

[0681] In one particular example (not shown), the LIDAR system 100 may include four lens systems that "see" ahead of the vehicle. The four lens systems together can cover a 160° field of view using four light sources. Each light source scans a 40° field of view horizontally, for example, via a shared deflector. In some embodiments, the processor 118 can increase the beam from a first light source (corresponding to a portion of the FOV where the crossing lane is located) and simultaneously decrease the beam from a second light source (corresponding to a different direction and a portion of the FOV that does not substantially overlap with the area of interest, such as the crossing lane). Reflections of both signals may be incident on a common light deflector 114 (e.g., a mirror).

[0682] Figure 46A is Figure 4600, which shows an example of a LIDAR detection scheme for lane crossing direction changes. The autonomous vehicle 4510 in Figure 45 may enter an intersection for lane crossing direction changes. In other embodiments, the autonomous vehicle may enter a T-junction, a Y-junction, or any other type of intersection for lane crossing direction changes. The autonomous vehicle 4510 may take a path such as track 4620 for lane crossing direction changes. When the autonomous vehicle 4510 enters an intersection (or even before), the processor 118 may increase the luminous flux level for the portions of the LIDAR field of view represented by FOV portions 4525, 4522, and 4523. As a result, the potential detection range associated with these portions may be larger than that of other portions of the total LIDAR FOV. For example, as shown, portion 4522 may have a larger detection range than portion 4521. FOV portion 4523 may have a larger detection range than 4524. FOV portion 4525 (the portion of the vehicle facing the opposite direction of the crossing lane) may have a larger detection range than FOV portion 4526 and may have a larger detection range than FOV portions 4522 and 4523. In some cases, the detection range of FOV portion 4525 may be more than twice as long as that of the other FOV portions. As shown in Figure 46A, the right-side FOV portion 4525 of the vehicle is in the direction determined to overlap with the region of interest, and therefore can be allocated the most luminous flux and have the largest detection range during a left turn crossing the lane. The detection ranges of the left-side FOV portions 4521, 4526, and 4524 of the vehicle can be kept constant from their default values (for example, as shown in Figure 45 before the lane-crossing direction change shown in Figure 46A). Alternatively, the luminous flux projected onto any of the left-side FOV sections 4521, 4526, and 4524 of the vehicle can be reduced from its default value during lane crossing turns. The FOV section 4527 that captures the area in front of the autonomous vehicle 4510 during the turn-around situation shown in Figure 46A can be assigned a lower luminous flux level than the level normally assigned to the forward FOV section in non-lane crossing situations. For example, in situations such as when the vehicle is traveling at high speed on the road, as shown in Figure 45, a higher luminous flux value can be assigned to the FOV section 4527, which can increase the detection range of the FOV section 4527. In lane crossing situations, where the speed may be lower than in the situation shown in Figure 45, the luminous flux level of the FOV section 4527 can be reduced. Also, in lane crossing situations, the side of the vehicle adjacent to the oncoming lane may represent an area of greater interest (e.g., from the perspective of potential collisions) compared to the direction of travel of the vehicle in the lane crossing situation shown in Figure 46A. Of course, right turns (for example, in the United States and other countries where vehicles travel on the right side of the road) do not involve crossing lanes, so it may not be necessary to increase the luminous flux in the FOV portion of the LIDAR system on the left side of the vehicle. However, in countries like Japan and the United Kingdom, it is possible to increase the luminous flux supplied to the FOV portion on the left side of the vehicle during a right turn that crosses lanes. The processor 118 is equipped with a system that automatically determines the location of the host vehicle (for example, based on the output of the vehicle navigation system, GPS sensors, etc.) and can control the luminous flux supplied to various FOV portions according to driving habits/road configurations at the determined location.

[0683] As further shown in Figure 46A, the processor 118 can detect the vehicle 4610 during lane crossing direction changes by extending the detection range from the FOV portion 4525. For example, the processor 118 can determine whether the detected object is a target object, such as a moving vehicle or a pedestrian. The processor 118 can distinguish between buildings, sidewalks, parked vehicles, pedestrians, and moving vehicles based on at least the reflectance pattern associated with the detected object. Therefore, the processor 118 can conserve resources by allocating a high level of resources to the FOV portion determined to contain such target objects (moving vehicles or pedestrians, etc.) and reducing (or not increasing) resource expenditure to the FOV portion determined to contain buildings, parked vehicles, or other stationary objects.

[0684] In some embodiments, the processor 118 can increase the amount of light supplied to specific sub-regions of the FOV. For example, in the lane-crossing scenario shown in Figure 46A, the moving car 4610 may be determined by the processor 118 to be the object of interest. Therefore, the processor 118 can supply more light to the sub-region of the FOV 4525 that overlaps with the car 4610. The processor 118 can also increase the light level supplied to one or more sub-regions of the FOV in other situations. For example, if an object is not detected within a particular sub-region of the FOV or beyond a certain distance, the processor 118 can allocate increased light to those sub-regions to attempt to detect the object at a greater distance.

[0685] Figure 46B shows another example of a LIDAR detection scheme for lane crossing and direction change. In this example, vehicle 4510 approaches a T-junction, stops, and waits for an opportunity to turn left and enter the far lane. In this situation, there is little risk of encountering an object from behind or to the left of vehicle 4510. Most areas of interest may rather be in front of and to the sides of the vehicle. Therefore, long detection range and/or high-resolution depth mapping are not required for FOVs 4526, 4524, and 4523, and light projection to these areas can be reduced. In some embodiments, the resources made available by the reduction of light projected onto FOVs 4526, 4524, and 4523 can be used for other fields of view. For example, by reducing the resources required to scan FOVs 4526, 4524, and 4523, the light emission power and/or computational resources that would have been used for object detection if FOVs 4526, 4524, and 4523 were larger can be reallocated to other FOVs such as FOVs 4521, 4522, or 4527.

[0686] By making more resources available to enhance detection in critical areas, the LIDAR system 100 can increase the amount of light flux projected onto the FOV 4521 to increase the detection range in the area in front of and to the left of the vehicle 4510. Furthermore, the increase in light flux supplied to the FOV 4521 does not need to be uniform across all areas of the FOV. As shown in Figure 46B, the FOV 4521 can be divided into FOV sub-regions 4680 and 4682. Both sub-regions can receive more light flux than, for example, the FOV 4526, but sub-region 4680 can receive more light flux than sub-region 4682 during one or more scan cycles of the FOV 4526. Such an FOV scan scheme can potentially increase the detection distance in lanes approaching from the left.

[0687] When making a left turn at the T-junction shown in Figure 46B, it may be important to enhance the detection function for vehicles approaching vehicle 4510 from the left. Therefore, as described above, the detection function for vehicles approaching from the left can be enhanced by supplying more light beam to the FOV 4521 and distributing it evenly or unevenly to its small area as described above. However, in the situation shown in Figure 46B, the detection function for traffic approaching vehicle 4510 from the right is even more important. Generally, it is expected that traffic approaching from the left and right will approach vehicle 4510 at similar speeds, but the interaction time with a vehicle approaching from the right may be significantly longer than the interaction time with a vehicle approaching from the left. For example, when making a left turn at the illustrated T-junction, vehicle 4510 may need to determine whether or not a vehicle is approaching from the right, and whether or not there is enough time for vehicle 4510 to pass in front of a vehicle approaching from the left under normal acceleration conditions. Once those vehicles have passed, the interaction between vehicle 4510 and the vehicle approaching from the left may end.

[0688] On the other hand, assuming that vehicle 4510 is turning left at the illustrated T-junction, the interaction between vehicle 4510 and a vehicle approaching from the right may be longer. For example, vehicle 4510 must determine not only whether it has enough time to cross in front of the vehicle approaching from the right, but also whether it has enough time after vehicle 4510 has completed its left turn and during its acceleration period for the vehicle approaching from the right to accelerate to a speed that will allow it to go in front of vehicle 4510 without colliding with the rear of vehicle 4510. Therefore, a larger detection range is required not only to detect a vehicle like vehicle 4690 approaching from the right, but a detection range longer than that required to detect a vehicle approaching from the left may be necessary. Accordingly, in the illustrated example, increased luminous flux levels are assigned to both FOV 4522 and 4525 (FOVs having coverage in the areas in front of and to the right of vehicle 4510) to enhance the detection capabilities in front of and to the right of vehicle 4510. In this case as well, such light amplification does not need to be uniform across all regions of each FOV. Rather, as shown in the figure, sub-region 4675 of FOV 4525 is allocated more light than another sub-region 4676 of FOV 4525. In fact, the amount of light supplied to sub-region 4675 may be significantly greater than the amount supplied to sub-region 4676. As a result, the detection range of sub-region 4675 may be two, three, five, or ten times (or more) larger than the detection range associated with sub-region 4676.

[0689] Similarly, an increased light level can be applied to the FOV 4522. As shown, the FOV 4522 may include three sub-regions 4677, 4678, and 4679. The light flux level applied to these sub-regions gradually increases from sub-region 4677 to sub-region 4679, so that sub-region 4679 provides a larger detection range than sub-region 4678, and sub-region 4678 provides a larger detection range than sub-region 4677. By reallocating available resources (e.g., optical budget) in this way, vehicle 4690 can be detected within a range sufficient to determine whether vehicle 4510 has enough time to merge and accelerate in front of vehicle 4690 approaching from the right.

[0690] Dynamic Illumination Alignment in Main Road Driving

[0691] The LIDAR system 100 can be incorporated into the vehicle (for example, in the vehicle body or in any other suitable location). Furthermore, as described above, the LIDAR system 100 can dynamically control the amount of light beam projected to different parts of the LIDAR FOV. In one example, as will be considered below, the processor 118 may determine or receive an instruction that the vehicle is traveling on a main road (for example, when the vehicle is traveling at high speed and the risk of encountering obstacles such as pedestrians, bicycles, and other vehicles that are normally present in urban environments is low). In response to such an instruction, the processor 118 may allocate the available optical budget so that in one or more scans of the LIDAR FOV, more light beam is directed to the central region of the FOV than to the peripheral region of the FOV. Such a light beam allocation may be suitable for traveling on main roads where it is necessary to increase the detection range in front of the vehicle and there is little need to maintain long-range or high-resolution detection capabilities in the peripheral region of the FOV. When processor 118 determines that the host vehicle has left a main road environment and entered a non-main road or environment (e.g., an urban environment) where the risk of collision with an object crossing may increase, processor 118 can reallocate the optical budget to reallocate the additional light beam applied to the central region of the FOV while driving on the main road to the peripheral region.

[0692] More specifically, in some embodiments, the processor 118 of the LIDAR system 100 can control at least one light source 112 so as to vary the luminous flux from at least one light source during a scan cycle of the FOV. The processor 118 can also control at least one deflector 114 to deflect the light from the light source 112 for scanning the FOV. The FOV can be divided into a central region roughly corresponding to the location of the road on which the vehicle is traveling, a right-side peripheral region roughly corresponding to the area on the right side of the road, and a left-side peripheral region roughly corresponding to the area on the left side of the road. The processor 118 can acquire an input indicating that the vehicle is in a mode corresponding to driving on a main road, and in response to this input, it can coordinate the control of the light source 112 and the control of the light deflector 114 so that more light is directed to the central region than to the right-side and left-side peripheral regions during the FOV scan.

[0693] The processor 118 can receive instructions from any suitable source that the host vehicle is traveling on a main road. In some cases, this information can be obtained by communication with the vehicle navigation system 4740 (Figure 47), by analysis of images from one or more cameras via a GPS receiver and a map server or map application, based on the output from the LIDAR system 100 itself, based on the output from other LIDAR systems, etc. For example, in some embodiments, the navigation system 4730 can incorporate, access, or otherwise receive from a remote server one or more maps that can determine the roadway status as a main road or a non-main road. The processor 118 can receive instructions from the navigation system 4730 directly whether or not the roadway is a main road. In other cases, the processor 118 can determine the roadway status based on one or more indicators associated with the map information used by the navigation system 4730 (e.g., roadway status indicators, speed limits associated with a particular road, etc.). In some examples, the processor 118 can determine the road status as a main road or a secondary road based on a combination of information from two or more sensors or information sources. For example, the processor 118 can receive information from a GPS receiver, speed information from a vehicle sensor (e.g., speedometer), visual information from a camera, and depth map information from one or more LiDAR systems (including the system on which the processor 118 is located) or any other suitable source, in combination with information received from navigation information 4730, and use this combination of information sources to determine whether the road being traveled on is a main road or not. Such auxiliary information sources can transmit information indicating vehicle speed, vehicle position, identified lane markings, identified landmarks, identified road guardrails, direction of traffic flow, road width, lane width, lane configuration, identified traffic signs, identified traffic signals, etc. The processor 118 can use any of this information, individually or in combination, to verify the road status.

[0694] Figure 47 is a schematic diagram of a vehicle traveling in a trunk road environment with the assistance of a LIDAR system according to an exemplary embodiment disclosed. The vehicle 4710 is equipped with a LIDAR system 100 and may also be equipped with a navigation system 4730. As shown, the LIDAR FOV 120 can be divided into a central region 4720, a right peripheral region 4724, and a left peripheral region 4722. The processor 118 can scan the FOV 120 during one or more scan cycles by coordinating the operation of the light projector 112 and the deflector 114 (for example, by moving the deflector continuously or discontinuously to multiple different instantaneous positions at different points in time during the scan cycle). For example, the processor 118 can coordinate the optical deflector 114 and the light source 112 such that, when the optical deflector 114 is positioned at a specific instantaneous location, a portion of the light beam is deflected by the optical deflector from the light source 112 towards an object in the LIDAR FOV, and a portion of the light beam reflected from the object is deflected by the deflector towards at least one sensor 116. In some embodiments, two or more light sources can be used. For example, multiple light sources can be aimed at the deflector 114, and the processor 118 can control the deflector 114 so that when the deflector 114 is positioned at a specific instantaneous location, light from multiple light sources is projected onto separate areas in the LIDAR FOV.

[0695] The available optical budget can be allocated to one or more parts of the LIDAR FOV in any appropriate manner. In some embodiments, for example, if the processor 118 determines that a vehicle is traveling on a main road, the processor 118 can allocate the available optical budget to parts of the LIDAR FOV such that the central region 4720 receives more light than is given to the right peripheral region 4724 or the left peripheral region 4722. For example, the processor 118 can be configured to coordinate the control of the light source 112 and the control of the deflector 114 so that light having a first light beam is directed to the central region 4720 during a first scan cycle. After determining that a vehicle is traveling on a main road, the processor 118 can change the light allocation so that light having a second light beam is directed to the central region 4720 during a second scan cycle. The second light beam is greater than the first light beam. By increasing the luminous flux in the central region of the LiDAR FOV compared to the peripheral region, the detection range in the central region can become larger than that of the peripheral region. In some examples, the processor 118 can control the light source 112 so that the detection distance in the central region is at least twice as large as the detection distances in the right and left peripheral regions.

[0696] Of course, it is also possible to divide the LIDAR FOV into more or fewer regions than three. In addition, any segment may be further divided into multiple sub-regions. Furthermore, the processor 118 can be configured so that more light is directed to one of the sub-regions than to another during the scanning of the field of view.

[0697] Just as the distribution of light may change based on the determination that the host vehicle is traveling on a main road, the processor 118 can reallocate the optical budget based on the determination that the vehicle has left a main road or that the vehicle's environment has otherwise changed from a main road environment. For example, if the vehicle transitions from a road type such as a main road to an urban area, the processor 118 can reallocate the available optical budget. For example, when changing from a main road environment to an urban environment, the processor 118 can reduce the light beam projected onto the central region 4720 (compared to one or more previous scan cycles) and increase the light applied to one or more peripheral regions (compared to one or more previous scans).

[0698] Such allocation of the available optical budget to a selected region of the Lidar Field of View (FOV) can be referred to as defining the spatial light scan pattern. In some embodiments, the processor 118 can determine a spatial light scan pattern associated with multiple scan cycles when it acquires data indicating the type of road the vehicle is traveling on. As described above, the road type may include at least one of urban roads, arterial roads, undivided roads, single-lane roads, multi-lane roads, or roads with lanes for public transport. Furthermore, when data indicating a change in the type of road the vehicle is traveling on (e.g., from an arterial road to an undivided road) is acquired, less light can be directed to the central region and more light to the right and left peripheral regions compared to the light projected in the preceding scan cycle.

[0699] The processor 118 can allocate the available optical budget not only based on the determination of the type of road the host vehicle is traveling on, but also based on detected driving events. For example, Figure 48A illustrates the situation described above. The processor 118 can determine that the vehicle has entered a main road, and as a result of this event, it can allocate the available optical budget to supply more light flux (e.g., a higher optical power level) to the central region compared to the peripheral regions on the right or left.

[0700] In Figure 48B, the processor can determine that a vehicle has entered a closed area where the road is surrounded by buildings on both sides. Such a situation may occur, for example, in an urban environment. Based on the detection of a driving event entering an urban road environment, the processor 118 can allocate more light beam to the left and right peripheral areas than to the central area. Such an allocation of optical budget may be appropriate for urban environments where vehicle speeds are generally slower than those on main roads (meaning that the detection range in front of the vehicle in an urban environment does not need to be as large as it is suitable on main roads). Furthermore, in urban environments, there may be a higher risk of encountering pedestrians that may be present in the areas on both sides of the vehicle (e.g., the sidewalk adjacent to the illustrated building). Therefore, in urban environments, it may be advantageous to improve the detection range and/or resolution capabilities in the peripheral areas of the LIDAR FOV more than in the central area.

[0701] Figure 48C shows another example of a detected driving event. This event can cause a change in the allocation of the available optical budget to a specific area of the LIDAR FOV. For example, as illustrated, another vehicle 102 may be detected in a lane adjacent to the host vehicle, and it may be determined that both vehicle 102 and the host vehicle are moving in the same direction. In such a situation, the processor 118 can determine that the area in front of and to the right of the host vehicle corresponds to the area of interest because it is the area where vehicle 102 is located. To assist in the detection and/or tracking of vehicle 102, the processor 118 may allocate the available optical budget so that the right peripheral area receives the highest level of light beam, the central area receives the next highest level of light beam, and the left peripheral area receives the lowest level of light beam.

[0702] Figure 48D shows yet another example of a detected driving event. This event can cause a change in the allocation of the available optical budget to a particular area of the LIDAR FOV. For example, as illustrated, another vehicle 102 may be detected in a lane adjacent to the host vehicle, and it may be determined that vehicle 102 and the host vehicle are moving in opposite directions. In such a situation, the processor 118 can determine that the area in front of and to the left of the host vehicle corresponds to the area of interest because it is the area where vehicle 102 is located (and where oncoming traffic is expected to be detected). To assist in the detection and/or tracking of vehicle 102, the processor 118 may allocate the available optical budget such that the left peripheral area receives the highest level of light beam, the central area receives the next highest level of light beam, and the right peripheral area receives the lowest level of light beam. Furthermore, as shown in Figure 48D, a specific sub-region of the left peripheral area can be defined, and the processor 118 may supply the highest level of light beam (even within the left peripheral area) to this specific sub-region. In some cases, the defined sub-region in the left peripheral area may overlap with the location of the detected vehicle 102.

[0703] The processor 118 may also allocate or reallocate the available optical budget based on other detected driving events. For example, in some embodiments, detected driving events for which allocating or reallocating the optical budget is appropriate may include at least one of the following: traffic-related events, road-related events, approaches to predetermined facilities, and climate-related events. Based on the detection or indication of any of these types of events, the processor 118 may modify the spatial light scan pattern for each scan cycle. For example, the processor 118 may modify the spatial light scan pattern to direct more light to at least a portion of the right peripheral region than was directed to at least a portion of the right peripheral region in the preceding scan cycle, to direct more light to at least a portion of the left peripheral region than was directed to at least a portion of the left peripheral region in the preceding scan cycle, or to direct more light to at least a portion of the central peripheral region than was directed to at least a portion of the central region in the preceding scan cycle.

[0704] Figure 49 is a flowchart of a method 4900 for operating a LIDAR system according to an embodiment disclosed herein. The method may include controlling at least one light source so that the beam of light from at least one light source can be varied during a scanning cycle of the field of view (step 4910). The method may also include controlling at least one deflector to deflect light from at least one light source for scanning the field of view. The field of view can be divided into a central region roughly corresponding to the main road on which the vehicle is traveling, a right peripheral region roughly corresponding to the area to the right of the main road, and a left peripheral region roughly corresponding to the area to the left of the main road (step 4920). In step 4930, the processor 118 can obtain an input that the vehicle is in a mode corresponding to driving on a main road. In response to this input, in step 4940, the processor 118 can coordinate the control of at least one light source 112 with the control of at least one light deflector so that, during scanning of the field of view encompassing the central region, the right peripheral region, and the left peripheral region, more light is directed to the central region than to the right peripheral region and the left peripheral region.

[0705] Variation of Lidar illumination in response to ambient light level

[0706] LiDAR systems can be used in many different environments with varying levels of ambient light. Furthermore, the level of ambient light can change dramatically within a single scene at any given time. For example, part of the scene may be in shadow, while other parts are illuminated by sunlight or other light sources, and yet another part of the field of view may include ambient light sources such as lamps, headlights, or campfires. Such ambient light generates noise, which can degrade the quality of service (QoS) of the LiDAR system. When LiDAR system 100 operates in the presence of strong ambient light (e.g., bright sunlight or artificial light sources), LiDAR system 100 may experience significant noise from the ambient noise. On the other hand, when LiDAR system 100 operates in an environment with low ambient light, the noise may be less.

[0707] As stated above, systems and methods according to the embodiments disclosed herein can collect light reflection data and allocate luminous flux to the LIDAR FOV pixel by pixel, beam spot by beam spot, or by section (it should be noted that in the following description, any implementation considered with respect to one of the three allocations described above can be implemented for the other two allocations with necessary modifications). In some examples, the amount of luminous flux allocated to a particular section of the LIDAR FOV may depend on the amount of ambient light detected in this particular area of the FOV. In particular, in some cases, the amount of luminous flux allocated to a particular section of the LIDAR FOV (e.g., to a particular pixel) in a given scan cycle may depend on the amount of ambient light detected in this particular area of the FOV in the same scan cycle. In some examples, the amount of light flux allocated to a specific portion of the LiDAR FOV (e.g., a specific pixel) at a given instantaneous position of at least one light deflector 114 may depend on the amount of ambient light detected in that specific region of the FOV while at least one light deflector 114 is maintained at that specific instantaneous position (e.g., without intermittently emitting light to any other part of the FOV between ambient light detection and the emission of allocated light flux). For example, if it is determined that the amount of ambient light in a specific region of the FOV is low, a smaller amount of light flux can be supplied to that region. Conversely, the more ambient light detected in a specific region of the FOV, the greater the amount of light flux supplied to that region. By varying the light allocation to regions of the LiDAR FOV based on the detected ambient light level, it is possible to reduce or eliminate the influence of noise on the operation of the LiDAR system 100.

[0708] In some embodiments, the field of view (e.g., FOV 120 as shown in Figure 50) may include a plurality of portions, each corresponding to a different instantaneous position of the deflector 114. Each of these portions may have any suitable size and/or occupy any suitable portion of the FOV 120.

[0709] At least one processor 118 can receive signals from at least one sensor 116 per pixel (as shown in Figure 50). For example, at least one sensor can detect light collected from a specific portion of the FOV (e.g., pixels A1, B1, C1, A2 of the FOV 120 as shown in Figure 51) for each pixel and generate a signal corresponding to the light collected at each pixel. The signal may indicate multiple sources of light collected from the FOV. For example, one component of the light collected and provided to sensor 116 may include ambient light. Another component of the light collected and provided to sensor 116 may include light from at least one light source 112 that is projected onto a specific portion of the FOV and reflected by one or more objects in that specific portion of the field of view. In some situations (e.g., objects are far away or have low reflectivity), ambient light collected from a specific portion of the LIDAR FOV may account for a larger proportion of the light supplied to sensor 116 than reflected light resulting from LIDAR illumination. In other situations (e.g., when an object is nearby or has high reflectivity), the ambient light collected from a specific portion of the LiDAR FOV may account for a smaller proportion of the light supplied to the sensor 116 than the reflected light. For example, the ambient light in the first portion of the LiDAR FOV (represented by a specific region of the FOV shown in white in Figure 51) may account for a larger proportion of the light collected from the FOV 120 than the reflected light collected from the second portion of the FOV 120 (represented by a specific region of the FOV shown in shaded in Figure 51). It should be noted that the first and second portions of the LiDAR FOV may each contain more or fewer specific regions of the LiDAR FOV than those shown in Figure 51. For example, in some embodiments, the first and/or second portions may each correspond to a single specific region of the FOV 120 or may contain multiple portions (as illustrated).

[0710] The processor 118 can be configured to detect ambient light in a specific part of the LIDAR FOV, independently of detecting projected light reflected from an object, based on the output of the sensor 116. For example, in some embodiments, the processor 118 can sample the output of the sensor 116 at a time when no reflected light is expected. For example, before light is emitted from the projector 112 to a specific part of the FOV, no light has been projected onto that part of the FOV yet, so no reflected light is expected from it. Therefore, the light detected by the sensor 116/processor 118 can be assumed to be ambient light. Similarly, after light has been projected onto a specific part, but after enough time has elapsed that no reflection from the light projection is expected (for example, at a time equivalent to the time of flight of light for a particular light emission relative to the maximum expected range of the LIDAR system), the light collected from that part of the FOV may be due to ambient light. In some embodiments, after light emission by the light projector 112, the sensor 116 can detect reflection of light from the field of view during a first detection period after light emission. Furthermore, the sensor 116 can measure the ambient light level within the field of view during a second detection period after light emission. By monitoring the output of the sensor 116 during this period, the ambient light level in a specific part of the field of view can be determined.

[0711] Optionally, the processor 118 can be configured to determine the amount of ambient light in a particular part of the FOV during the time it takes for light projected onto that part of the FOV to be reflected from one or more objects and received by the sensor 116. For example, in some cases, the projected light is associated with one or more features (e.g., wavelength, modulation pattern, pulse duration, etc.), and these features can be detected and/or distinguished from background ambient light, for example, based on the output of the sensor 116 or one or more other sensors. In some cases, the LIDAR system 100 may include a first sensor configured to detect light reflections from objects in the field of view and a second sensor configured to measure ambient light in the field of view. In other cases, the sensor 116 can detect both reflections from objects and ambient light. By distinguishing ambient light from reflected light in this way, it is possible to determine the amount of ambient light present in a particular part of the FOV. In some cases, an estimate of the ambient light level can be obtained by subtracting the received light, which the processor 118 has determined to be a reflected signal of light emitted by the LIDAR (e.g., based on a matched filter), from the entire received signal.

[0712] In some embodiments, the processor 118 can be configured to identify one or more types of light sources associated with ambient light detected in a particular portion of the LiDAR FOV. For example, after receiving detected light information associated with a particular portion of the FOV, the processor 118 can compare this received information with pre-stored noise level data associated with various sources of ambient light. Based on such a comparison, the processor 118 can identify the types of light sources that may have generated the ambient light. The processor 118 can also use other characteristics of the ambient light (e.g., polarization, fluctuations level) and/or information from multiple pixels (e.g., physical size estimation, distance between light sources) to identify the type of light source. It should be noted that the identification of the type of light source can later be used for object classification (e.g., headlights may indicate that an object is a car or a semi-trailer truck based on the distance between headlights), and vice versa (light sources can be identified using characteristics of an object; for example, light sources high up in a building may be identified as illuminated windows).

[0713] In some embodiments, when an ambient light level is detected in a specific region of the LiDAR FOV, the processor 118 can determine the allocation of light to project onto that specific region of the FOV based on the detected ambient light level. For example, if the detected ambient light level is below a predetermined threshold, the processor 118 can determine that it is not necessary to project additional light onto that specific portion of the FOV. On the other hand, if the detected ambient light level is higher than the predetermined threshold, the processor 118 can determine that additional light should be projected onto that specific portion of the FOV. In such examples, the processor 118 can supply additional light to the specific portion of the FOV.

[0714] As merely an example shown in Figure 51, the first portion of the FOV contains 15 specific regions of the FOV, and the second portion of the FOV contains 25 specific regions. All regions in the second portion were determined to have ambient light levels below a threshold level. Therefore, in these regions, only one light pulse (or any other type of light projection that affects the amount of light flux supplied to a specific FOV region) is assigned to the regions in the second portion of the FOV. On the other hand, each region of the first portion of the FOV was determined to have an ambient light level higher than a predetermined ambient light level threshold. As a result, the processor 118 assigns three light pulses to project onto each region included in the second portion of the FOV. Of course, the concept of light pulses supplied to the regions of the FOV is merely illustrative. It is also possible to use any other type of light projection technique to increase the light flux in the regions of the first portion of the FOV relative to the amount of light supplied to the regions of the second portion of the FOV.

[0715] It should also be noted that, in addition to detecting the ambient light level, other considerations can be used to determine the amount of light flux to be supplied to different parts of the FOV (e.g., to different pixels) during each scan cycle. For example, the processor 118 may decide not to allocate additional light to a part if the detected light source is located within a predetermined region of indifference. The processor 118 may combine the ambient light level information determined for a part of the FOV with information indicating other noise levels for that part, or with any other type of information disclosed herein (from the same scan cycle or otherwise) that can be used to determine the light flux level.

[0716] Furthermore, the amount of light that the processor 118 allocates to a specific area of the FOV may depend on the determined type of light source associated with the ambient light detected in that specific area. For example, if it is determined that the ambient light originates from the sun rather than from an electric lamp, more light can be allocated to that specific area. Of course, the reverse is also true.

[0717] To scan the field of view (e.g., the FOV 120 in Figure 50), at least one processor 118 can coordinate at least one optical deflector with at least one light source such that, when at least one optical deflector is positioned at a particular instantaneous location, the optical beam is deflected by at least one optical deflector from at least one light source toward the field of view, and reflections from objects in the field of view are deflected by at least one optical deflector toward at least one sensor. Thus, at least one optical deflector can guide the optical beam toward the field of view and receive reflections from the field of view. For example, in the examples shown in Figures 1A, 2B, and 2C, the deflector guides the optical beam toward the field of view and receives reflections from the field of view. In some embodiments, the reflection may be caused by the optical beam guided toward the field of view. In other embodiments, the optical beam from at least one light source can be guided toward the field of view by at least one optical deflector separate from at least one other optical deflector that receives reflections from the field of view. For example, in the example shown in Figure 2A, one deflector guides the light beam towards the field of view, while another deflector receives the reflected light from the field of view.

[0718] In some embodiments, the sensitivity level associated with the sensor 116 can be controlled, for example, based on ambient light detected in a specific area of the LiDAR FOV and/or based on whether a light source has been detected (for example, during the same scan cycle, but not necessarily). For example, the processor 118 can be configured to identify the presence of a light source in a specific part of the LiDAR FOV based on information received from the sensor 116. In response to such identification of a light source, the processor 118 can change the sensor sensitivity to light reflection from the specific part of the LiDAR FOV.

[0719] As described above, the greater the ambient light detected within a specific area of the FOV, the greater the amount of light beam directed to that area can be increased to reduce or eliminate the effects of noise. At least one processor 118 can cause at least one light source (e.g., light source 112 in Figure 50) to project more light beam towards a portion of the field of view (e.g., field of view 120 in Figure 50). Projecting more light beam towards a specific portion of the FOV improves the signal-to-noise ratio (or, for example, improves the detection probability in this portion of the FOV as discussed above), and therefore improves the QoS for objects located in this portion of the field of view.

[0720] Furthermore, at least one processor 118 can acquire the identification of at least one distinct region of interest in a portion of the field of view and increase the luminous flux in this region of interest. For example, increasing the luminous flux can be done by increasing the number of light pulses per solid angle, increasing the irradiance to the FOV portion, emitting additional light pulses, increasing the power per pixel, emitting additional photons per unit time, increasing the total energy for a particular time period, emitting additional photons per data point in the generated point cloud model, increasing the total energy per data point in the generated point cloud model, changing the wavelength, increasing the amplitude and/or frequency of the light pulses, or any other feature that increases the luminous flux. In some examples, the region of interest may be a specific region of the LIDAR FOV that is determined to have an ambient light level that matches at least one predetermined feature (e.g., ambient light level, type of light source, nighttime or daytime operation (ambient light source showing headlights at night), etc.).

[0721] Furthermore, it is possible to identify a specific region of interest within a portion of the field of view based on information received from at least one of the following: a GPS unit, a vehicle navigation system, radar, LiDAR, a camera, etc. Based on this identification, at least one processor 118 can cause at least one light source (e.g., light source 112 in Figure 50) to project more light beam onto the portion of the field of view than is projected onto other parts of the field of view. For example, a signal from a camera may indicate that the LiDAR is in a bright environment (e.g., under a light source or in the sun). To compensate for the high level of noise associated with such a light source, at least one processor 118 can cause the light source 112 to project more light beam onto the portion of the LiDAR field of view.

[0722] In order to project more light onto a portion of the FOV, at least one processor 118 can cause at least one light source 112 to project more light pulses. For example, at least one processor 118 can vary the timing of pulses from at least one light source. In order to increase the number of light pulses in a portion, at least one processor can shorten the timing of pulses. As another example, processor 118 can, instead or simultaneously, vary the power level of pulses from at least one light source. In yet another example, processor 118 can, instead or simultaneously, vary the amplitude and/or frequency of pulses from at least one light source. For example, in Figure 51, the system can decide to project one light pulse per pixel by default. Since the first portion of the FOV 120 contains strong ambient light and the received signal may be greater than a predetermined threshold, as described above, three light pulses per pixel can be projected onto the first portion of the FOV instead of one per pixel. In some embodiments, the intensity of light projected onto the first portion of the FOV in Figure 51 can be greater than the intensity of light applied to the second portion of the FOV in Figure 51.

[0723] Furthermore, the processor 118 can be configured to modify the light source parameters associated with the light projector 112 so that, in a single scan cycle, more light pulses per solid angle are projected onto the first portion of the FOV than the number of light pulses per solid angle projected onto the second portion of the FOV in Figure 51. In some examples, without providing a default level of light to a specific area of the LIDAR FOV, the processor 118 can be configured to modify the light source parameters associated with the light projector 112 so that no light is projected onto the second portion of the field of view shown in Figure 51 during at least one scan cycle of the LIDAR FOV. In another example, the processor 118 can be configured to modify the light source parameters so that the light projected onto the first portion has a different wavelength than the light projected onto the second portion.

[0724] Figure 52 shows an exemplary method 5200 for detecting an object using a LIDAR system. In step 5201, as described above, the processor 118 can control at least one light source (e.g., light source 112 in Figure 1A, the laser diode 202 of light source 112 in Figure 2A, and/or multiple light sources 102 in Figure 2B) so that the beam can be varied in scanning a field of view (e.g., field of view 120 in Figures 1A and 2A). In step 5202, at least one processor 118 controls at least one optical deflector (e.g., optical deflector 114 in Figure 1A, deflector 114A and/or deflector 114B in Figure 2A, and/or unidirectional deflector 114 in Figure 2B) to scan the field of view. For example, the processor 118 can scan the field of view by mechanically moving at least one optical deflector. Alternatively, or simultaneously, the processor 118 may induce a piezoelectric or thermoelectric change in at least one deflector to scan the field of view. In some embodiments, a single scan cycle of the field of view may include moving at least one deflector so that at least one optical deflector is instantaneously positioned at multiple locations during the scan cycle. For example, at least one optical deflector may be moved from one of multiple locations to another during the scan cycle, rather than being moved in a continuous sweep (optionally, additional locations and/or repetitions may also be used).

[0725] In step 5203, at least one processor 118 receives a signal from at least one sensor (e.g., the detection unit 106 in Figure 51) for each pixel. For example, the signal may indicate at least one of ambient light and light from at least one light source reflected by an object in the field of view. In step 5204, at least one processor 118 can identify from the received information the type of light source that generates ambient noise in a portion of the field of view and determine the level of light beam to allocate to at least one light source (e.g., light source 112 in Figure 1A, the laser diode 202 of light source 112 in Figure 2A, and/or the multiple light sources 102 in Figure 2B). In step 5205, as described above, at least one processor 118 can cause at least one light source (e.g., light source 112 in Figure 1A, the laser diode 202 of light source 112 in Figure 2A, and/or the multiple light sources 102 in Figure 2B) to project more light beam onto a portion of the field of view (field of view 120 in Figures 1A and 2A). For example, at least one processor 118 can project more light beam onto a portion of the field of view before moving to another portion of the field of view (for example, while at least one deflector is still in the same instantaneous position).

[0726] Temperature-based control in Lidar

[0727] In a LiDAR system, it may be important to manage the temperature of the components to prevent damage to the system components themselves and for safety reasons. Furthermore, since some components of a LiDAR system may exhibit suboptimal performance when the temperature range is exceeded, it may be important to modify the operation of the LiDAR system to optimize the performance provided under such suboptimal conditions. In some embodiments, a LiDAR system may be used in a vehicle. A LiDAR system 100 may include one or more light sources 1120 for projecting light toward the FOV 120 in order to illuminate one or more objects within the FOV 120 in the vehicle environment. A LiDAR system 100 may include one or more processors 118 for varying the luminous flux in scanning one or more parts of the FOV by controlling the light sources 112. During scanning of the FOV 120, heat may be radiated from one or more LiDAR system components. System components may include one or more light sources, deflectors, sensors, processors, and/or other LiDAR system components. Furthermore, heat may radiate from the vehicle on which the LIDAR system is installed (especially if the LIDAR system is installed in a hot and/or poorly ventilated location, such as under the vehicle's hood). Heat can also be generated from climatic conditions or other ambient conditions (e.g., operation in a warehouse). It should also be noted that the LIDAR system 100 may be highly sensitive to low temperatures resulting from similar causes.

[0728] The processor 118 can receive information via one or more temperature sensors indicating that the temperature of one or more components exceeds a threshold temperature. The threshold temperature can be dynamically determined by the processor or a statically preset value. The threshold temperature can be a temperature above which there is a risk of overheating of the component or system. In some embodiments, the threshold temperature can be a constant percentage of the temperature above which there is a risk of overheating of the system or system components. For example, the threshold temperature may be 80% of the temperature above which there is a risk of overheating of the system or system components. If the system or components overheat, this can damage the LIDAR system, create a fire hazard, and/or impair system function. If the detected temperature of one or more system components matches and/or exceeds the threshold, the processor 118 can change the illumination ratio between the two parts of the FOV 120 so that less light is sent to the FOV 120 during one or more subsequent scan cycles.

[0729] As described above, the temperature threshold can be dynamically determined and changed based on the state of various components of the LIDAR system 100, the state of the vehicle in which it is installed, and other parameters. It should also be noted that the temperature threshold may be related to the measured temperature, but may also be related to other temperature-based parameters (e.g., temperature changes over time, multiple temperatures measured at different locations, etc.). In implementations where the processor 118 implements such complex temperature thresholds and/or time-dependent temperature thresholds, the processor 118 can be considered to implement temperature determination rules. The processor 118 can be configured to measure excessively high temperatures and/or temperature rise rates by managing high temperatures in the LIDAR system by determining when the temperature threshold has been exceeded. The processor 118 can also be configured to measure excessively low temperatures and/or excessively fast temperature drop rates by managing low temperatures in the LIDAR system by determining when the temperature threshold has been exceeded.

[0730] Figure 53 is a schematic diagram of a LiDAR system according to an embodiment of the present disclosure. For example, as shown, the LiDAR system 5300 may include a light source 5310, a sensor assembly 5340, and a temperature sensing unit 5380. In some embodiments, the LiDAR system 5300 may include all or some of the components of the LiDAR system 100 and the temperature sensing unit 5380. In some embodiments, the LiDAR system 100 may include the temperature sensing unit 5380. The temperature sensing unit 5380 can be configured to detect the temperature of individual system components and/or the overall temperature of the LiDAR system 5300.

[0731] In some embodiments, the LiDAR system 5300 may include a control system 5300 comprising a sensor interface 5350, a light source controller 5360, and a temperature processor 5370. The temperature processor 5370 can receive temperature information from an optional temperature detector 5380 (or from another source such as an external temperature sensor, from the host, etc.) and process this received information to determine whether the temperature of one or more system components or the LiDAR system 5300 exceeds a threshold temperature. If the threshold temperature is exceeded (for example, during a particular scan cycle), the temperature processor 5370 can communicate with the light source controller 5360 to change the illumination ratio between two parts of the FOV so that less light is delivered to the FOV during subsequent scan cycles, thereby lowering the temperature of one or more system components and/or the LiDAR system 5300. In some embodiments, the LiDAR system 5300 may include a cooling component for cooling system components whose temperature matches or exceeds the threshold.

[0732] In some embodiments, the processor 118 can be configured to control one or more optical deflectors 114 so that one or more optical deflectors 114 are positioned at one or more different instantaneous locations during a scan cycle of the FOV. The processor 118 can also coordinate one or more optical deflectors 114 and one or more light sources 112 so that when an optical deflector 114 is positioned at a particular instantaneous location, a portion of the light beam is deflected by the deflector 114 from the light sources 112 toward an object in the FOV 120. The reflection of a portion of the light beam from an object can be deflected by one or more deflectors 114 toward one or more sensors 116. In some embodiments, one or more light sources 112 can be aimed toward one or more optical deflectors 114. The processor 118 can control one or more optical deflectors 114 so that when an optical deflector 114 is positioned at a particular instantaneous location, light from one or more light sources 112 is projected toward several distinct regions in the FOV 120.

[0733] In some embodiments, the processor 118 can determine the spatial light distribution in a single scan cycle of the FOV based on temperature information and change the illumination ratio so that more light is projected onto the first portion of the FOV than onto the second portion of the FOV. In some embodiments, the processor 118 can coordinate the control of one or more light sources 112 and one or more light deflectors 114 based on temperature information so that the same amount of light projected onto the portion of the FOV in one or more subsequent scan cycles as was projected in a preceding scan cycle. In some embodiments, the processor 118 can coordinate the control of one or more light sources 112 and one or more light deflectors 114 based on temperature information so that more light pulses are projected onto the first portion of the FOV than onto the second portion of the FOV. In some embodiments, the processor 118 can coordinate the control of one or more light sources 112 and one or more light deflectors 114 in one or more scan cycles based on temperature information so that the scan cycle rate for illuminating the first portion of the FOV and the scan cycle rate for illuminating the second portion of the FOV are dynamically adjusted. The temperature of one or more system components can be reduced by adjusting the scan cycle rate. In some embodiments, the processor 118 can restore the settings of the LiDAR system 100 once the temperature of the system or one or more components returns to a value below a threshold temperature.

[0734] In some embodiments, the processor 118 can identify a region of interest in a portion of the FOV and, based on temperature information, can change the illumination ratio between that portion of the FOV and another portion of the FOV to direct more light to the portion of the FOV containing the region of interest during one or more subsequent scan cycles. In some embodiments, the processor 118 can identify a region of non-interest in a portion of the FOV. Based on the determination of the region of interest and temperature information, it is also possible to allocate more light flux to the region of interest than to the less-interested region of the LIDAR FOV, or to not allocate any light flux to the determined region of interest at all. As a result, illumination of the region of interest (e.g., using different scan rates, power levels, light intensity, light flux, pulse duration, number of pulses, etc.) may increase local heating of one or more components of the LIDAR system 100 (e.g., deflector 114) in the optical path of the projected light. Therefore, temperature control by the light allocation techniques described herein may become even more important in certain parts of the system components when the region of interest is illuminated.

[0735] In some embodiments, the processor 118 may receive information that the temperature associated with one or more components (or one or more parts of those components) of the LiDAR system 100 exceeds a threshold temperature. The processor 118 may receive temperature information from a temperature sensing component, such as a temperature sensor 5380, via a wired or wireless connection. In some embodiments, the processor 118 may receive information from a vehicle controller, which may include one or more additional processors, that the temperature of one or more system components exceeds a threshold temperature.

[0736] In some embodiments, the temperature information received by the processor 118 may include information regarding the temperature of the environment outside the vehicle, information regarding the heat of the vehicle's engine, information regarding the temperature of one or more light sources 112, and/or information regarding the temperature of one or more processors, including the processor 118.

[0737] In some embodiments, the processor 118 can determine a threshold temperature value based on information regarding the ambient temperature around the vehicle. For example, the threshold temperature can be lowered on hot days to take into account external heat from the sun or the air surrounding the LiDAR system 100.

[0738] In some embodiments, the processor 118 can reduce the resolution in a portion of the FOV if the temperature of one or more system components exceeds a temperature threshold. The resolution can be reduced in areas of the FOV with less illumination, but the detection distance may be the same as the detection distance achieved with the original resolution. In some embodiments, a greater detection distance is achieved using a resolution lower than the original resolution.

[0739] Figure 54 is an exemplary flowchart of an exemplary process 5400 for detecting the temperature of one or more components of a LiDAR system 100. In step 5402, the LiDAR system 100 starts a scan cycle at an initial scan rate. In some embodiments, step 5402 can be performed at predetermined intervals rather than during each scan cycle. In other embodiments, step 5402 can be performed outside of a scan cycle, for example, when the LiDAR system is turned on or activated. In step 5404, the processor 118 can receive information from one or more temperature sensors 5380 indicating the temperature of one or more system components. In step 5404, the processor 118 can determine whether the temperature of the system components matches or exceeds a threshold temperature. If the temperature does not match the threshold temperature, there is no risk of overheating (or freezing) of one or more components, and the LiDAR system 100 can perform subsequent scan cycles without taking measures to change the temperature of the components. If the temperature matches or exceeds a threshold temperature, the processor 118 can function to suppress (or reduce) the component temperature by adjusting the LIDAR system 100 to reduce the overall light emitted to the FOV. In some embodiments, the processor 118 may function to prevent the component temperature from reaching the threshold temperature. In some embodiments, the processor 118 may adjust the irradiation ratio between portions of the FOV during subsequent scan cycles. Based on which components are at risk of overheating, the processor 118 can use various thermal reduction techniques. In some embodiments, the processor 118 may reduce the scan rate in subsequent scan cycles. In some embodiments, the processor 118 may reduce the resolution in subsequent scan cycles. In other embodiments, the processor 118 can control cooling components to cool components that are overheating or at risk of overheating. In other embodiments, the processor 118 can control heating components to heat components that are frozen or at risk of freezing.

[0740] Figure 55 is an exemplary flowchart of an exemplary process 5500 for determining and detecting the threshold temperature of one or more components of a LiDAR system 100 in a vehicle. In step 5502, the LiDAR system 100 starts a process to determine the component temperature. The processor 118 can receive temperature information from one or more sensors, such as a temperature sensor 5380, indicating the temperature of one or more system components, the vehicle engine, and/or the environment around the vehicle. In step 5504, the processor 118 can receive information from one or more of the temperature sensors. In step 5506, the processor 118 can determine an appropriate threshold temperature for the system component based on the vehicle engine temperature, the temperature inside the vehicle, and/or the temperature of the environment around the vehicle, using an algorithm or other calculation method. The calculated threshold temperature may also take into account the temperature at which the component begins to run in an unoptimal manner. In step 5508, the processor 118 can determine whether the component temperature is equal to or greater than the calculated threshold temperature. If the temperature does not match the threshold, the processor 118 may take no action until the temperature is next evaluated. Process 5500 may run at predetermined time intervals, independently of the scan cycle of the LiDAR system 100. The calculated threshold temperature can be calculated during each scan cycle or repeatedly at different intervals. For example, the threshold may be calculated every 5 minutes, every 30 minutes, every 2 hours, etc.

[0741] If the component temperature is above the threshold temperature, the processor 118 may, in step 5510, modify the scene projection scheme to reduce the light emitted to the FOV. As described above, the processor 118 can reduce the component temperature in numerous ways. The process 5500 can then be restarted at the next predetermined interval. In some embodiments, the component performance can be restored when the component temperature is below the threshold temperature by reversing the changes made in step 5510.

[0742] MEMS mirror and operating technique

[0743] Figure 56 shows exemplary embodiments of a scanning device (e.g., a deflector 114, hereinafter referred to as "scanning device 8202") and a processing device (e.g., a processor 118, hereinafter referred to as "controller 8204"). According to this disclosure, the controller 8204 is local and may be contained within the scanning device 8202. The controller 8204 may include at least one hardware component, one or more integrated circuits, one or more FPGAs, one or more ASICs, one or more hardware accelerators, etc. A central processing unit (CPU) and an operating driver are some examples of the controller 8204.

[0744] As shown in Figure 56, the mirror configuration may include a mirror 8206 that can move along two or more axes (θ, φ). The mirror 8206 can be associated with an electrically controllable electromechanical driver, such as an actuation driver 8208. The actuation driver 8208 can relay movement or power to an actuator/cantilever/bend section, such as an actuator 8210. The actuator 8210 may be part of a support frame, such as a frame 8211. Additional actuators, such as actuators 8212, 8214, and 8216, may each be controlled/driven by an additional actuation driver, as shown, and may (as appropriate) have support frames 8213, 8215, and 8217. It will be understood that the frames 8211, 8213, 8215, and/or 8217 may be a single frame supporting all actuators or multiple interconnected frames. Furthermore, these frames may be electrically isolated by insulating elements or sections. Optionally, the actuator 8210 can be connected to the mirror 8206 using a flexible interconnect element or connector (interconnection) such as the spring 8218, thereby relaying power or motion from the actuating driver 8208 to the spring 8218.

[0745] The actuator 8210 may include two or more electrical contacts such as contacts 8210A, 8210B, 8210C, and 8210D. Optionally, if the frame 8211 or the actuator 8210 is electronically connected, one or more contacts 8210A, 8210B, 8210C, and/or 8210D can be located on the frame 8211 or the actuator 8210. According to some embodiments, by making the actuator 8210 a doped semiconductor, sections of the actuator 8210 (excluding the insulating piezoelectric layer) are generally conductive between contacts 8210A to 8210D and insulating between insulation 8220 and 8222, thereby enabling the actuator 8210 to be electrically isolated from actuators 8212 and 8216, respectively. Optionally, instead of doping the actuator, the actuator 8210 may include conductive elements that can be bonded to the actuator 8210 or otherwise mechanically or chemically connected. In this case, insulating elements can be present in areas of the actuator 8210 where the conductive elements are not bonded. The actuator 8210 may include a piezoelectric layer, in which case the current flowing through the actuator 8210 triggers a reaction in the piezoelectric section, thereby causing the actuator 8210 to bend controllly.

[0746] According to some embodiments, the controller 8204 can output/relay a desired angular position described by the θ, φ parameters to the mirror driver 8224. The mirror driver 8224 can be configured to control the movement of the mirror 8206 and can cause the actuator driver 8224 to apply a specific voltage amplitude to contacts 8210C and 8210D in order to achieve a specified required value for the θ, φ deflection of the mirror 8206 based on the bending of actuators 8210, 8212, 8214, and 8216. Furthermore, the position feedback control circuit is configured to supply power (voltage or current) to contacts such as contact 8210A or 8210B, and the other contact (8210B or 8210A, respectively) is connected to a sensor in the position feedback 8226, which can be used to measure one or more electrical parameters of actuator 8210 to determine the bending of actuator 8210 and, as appropriate, to determine the actual deflection of the mirror 8206. As shown in the figure, each of the actuators 8212-216 can have additional position feedback similar to the position feedback 8226 and additional actuation drivers similar to the actuation driver 8208 replicated. The mirror driver 8224 and controller 8204 can also control these elements so that mirror deflection is controlled in all directions.

[0747] The actuator driver, including actuator driver 8208, sends a signal that causes an electromechanical response in actuators 8210-216, each of which is then sampled for feedback. Feedback regarding the position of actuators (8210-8216) can function as a signal to efficiently focus the mirror driver 8224 to a desired position θ, φ set by controller 8204 and to correct the requested value based on the actual detected value. According to some embodiments, a scanning device or LIDAR can utilize a piezoelectric actuator micro-electromechanical (MEMS) mirror device to deflect a laser beam scanning the field of view. The deflection of mirror 8206 is the result of applying a voltage potential to a piezoelectric element stacked on actuator 8210. The deflection of mirror 8206 is converted into an angular scan pattern, which may not exhibit linear behavior, and actuator 8210 does not convert it into a constant displacement value for a given voltage level. A scanning LiDAR system (e.g., LiDAR system 100) with deterministic and repeatable field dimensions across various devices is optimally realized using a closed-loop method that provides positional feedback and angular deflection feedback from sensor 8226 to mirror driver 8224 and/or controller 8204.

[0748] In some embodiments, the position feedback and sensor 8226 can also be used as a reliability feedback module. According to some embodiments, multiple elements may include semiconductor or conductive elements or layers, so that actuators 8201-8216 may include at least partially semiconductor elements, and each of the springs 8218, 8226, 8228, and 8230 may include semiconductors, and for this reason the mirror 8206 may also include semiconductors. Power (current and/or voltage) is supplied to the first actuator contacts via the position feedback 8226, and the position feedback 8226 can detect appropriate signals in actuators 8212, 8214, and/or 8216 via contacts 8214A or 8214B and/or 8261A or 8261B. Some of the following figures show MEMS mirrors, actuators, and interconnections. The number of interconnects, the shape of the interconnects, the number of actuators, the shape of the actuators, the shape of the MEMS mirrors, and the spatial relationships between any of the MEMS mirrors, actuators, and interconnects may differ from those shown in the following diagram.

Interconnection
[0749] Figure 57 shows four L-shaped interconnects 9021, 9022, 9023, and 9024 connected between a circular MEMS mirror 9002 and four actuators 9011, 9012, 9013, and 9014. Each L-shaped interconnect (e.g., 9021) includes a first segment 90212 and a second segment 90211. The first and second segments are mechanically connected to each other. In Figure 57, the first and second segments are perpendicular to each other. In Figure 57, the second segment of each L-shaped interconnect is connected to the outer circumference of the actuator, and the first segment of each L-shaped interconnect is connected to the outer circumference of the MEMS mirror. The second segment is perpendicular to the outer circumference of the first actuator. The first segment is perpendicular to the outer circumference of the MEMS mirror and/or may be directed towards the center of the MEMS mirror when the MEMS mirror is in the idle position. The MEMS mirror is in the idle position when no bending electric field is applied to all of the actuators coupled to the MEMS mirror.

[0750] In one embodiment, the use of an L-shaped interconnect provides excellent durability and stress relief. The use of an L-shaped interconnect facilitates seamless movement around two mutually perpendicular axes of rotation (see the dashed line labeled AOR near interconnect 9024). As a result, the bending and unbending of the actuator does not place excessive stress on the L-shaped interconnect. Furthermore, since the L-shaped interconnect is relatively compact and can have a small volume, the mechanical load on the attenuator is reduced, which can help increase the scan amplitude of the MEMS mirror. It should be noted that the different segments of the interconnect may be oriented at angles different by 90 degrees relative to each other (and/or relative to the MEMS mirror and/or actuator). These angles may be substantially equal to 90 degrees (substantially meaning a deviation not exceeding 5%, 10%, 15%, or 20%, etc.). Furthermore, it should be noted that the L-shaped interconnect may be replaced by an interconnect containing a single segment or two or more pairs of segments. An interconnection having two or more segments may include segments that are equal to each other and/or segments that are different to each other. These segments may differ in shape, size, cross-section, or any other parameter. The interconnection may also include linear segments and/or nonlinear segments. The interconnection can be connected to MEMS mirrors and/or actuators in any way.

[0751] Figure 58 shows four interconnections 9021', 9022', 9023', and 9024' connected between a circular MEMS mirror 9002 and four actuators 9011, 9012, 9013, and 9014. The first and second segments of each interconnection are connected by a joint. For example, interconnection 9021' includes a first segment 90212, a second segment 90211, and a joint 90213 connected to the first and second segments to facilitate relative movement between the first and second interconnections. The joint may be a ball joint or any other type of joint.

[0752] Figure 59 shows ten non-limiting examples of interconnections. Interconnections 90215, 90216, 90217, 90218, and 90219 do not include fittings. Interconnections 90215', 90216', 90217', 90218', and 90219' include at least one fitting. Furthermore, Figure 59 shows interconnections including linear segments, non-linear segments, one segment, two segments, and even nine segments. Interconnections can include any number of segments, have segments of any shape, and include zero to many fittings.

Response to mechanical vibrations
[0753] The scan unit (e.g., scan unit 104) may include MEMS mirrors, actuators, interconnects, and other structural elements of the LiDAR system. The scan unit 104 may be subjected to mechanical vibrations propagating along different directions. For example, a LiDAR system installed in a vehicle may be subjected to different vibrations (from different directions) when the vehicle moves from one point to another. If all actuators have the same structure and dimensions, the unit's response to some frequencies can be very large (high Q factor). By introducing some asymmetry between actuators, the scan unit 104 can respond to more frequencies, but the response can be made milder (low Q factor).

[0754] Figure 60 shows a first pair of actuators 9011 and 9013 facing each other. These are shorter (by a difference of L9040) than the actuators of the second pair of actuators 9012 and 9014. Actuators 9012 and 9014 face each other and are oriented relative to actuators 9011 and 9013. Figure 60 also shows L-shaped interconnects 9021, 9022, 9023, and 9024, as well as a circular MEMS mirror 9002. The resonant frequency of this unit can be outside the frequency range of mechanical vibrations. The resonant frequency of the unit can be at least twice as high as the maximum frequency of its particular frequency range. The resonant frequency of the unit is between 400 Hz and 1 kHz.

[0755] Figure 61A shows the actuators 9011, 9012, 9013, and 9014, the interconnects 9021, 9022, 9023, and 9024, and the frame 9015 surrounding the MEMS mirror 9002. Actuators 9011, 9012, 9013, and 9014 are connected to the frame 9050 at bases 9071, 9072, 9073, and 9074, respectively. In one embodiment, the width of the base can be any percentage of the total length of the actuator (e.g., less than 50%). Furthermore, the base can be positioned at any distance from the connection point of the actuator to the interconnect. For example, the base can be positioned near the end of the actuator facing the end of the connector connected to the interconnect.

[0756] Figure 61B shows a frame 9550 surrounding actuators 9511, 9512, 9513, and 9514 connected via shafts 9590 to a MEMS mirror 9002 positioned on a plane different from the plane of the frame 9550, according to an example of the subject disclosed herein. The interconnections between actuators 9511, 9512, 9513, and 9514 and shafts 9590 are not shown for simplification. These interconnections may, but may not, have similar shapes and features to those discussed above with respect to interconnections 9021, 9022, 9023, and 9024. As illustrated in Figure 61A, the MEMS mirror 9002 can be actuated by actuators positioned on a plane different from the plane of the MEMS mirror 9002. The movement of actuators 9511, 9512, 9513, and 9514 is transmitted to the MEMS mirror 9002 via a shaft 9590, one end of which is connected to the actuator and the other end of which is connected to the base surface of the MEMS mirror 9002. It should be noted that the shaft 9590 may be replaced with any type of rigid connector. Referring to the actuators that move the shaft (which can, of course, be any other number and not necessarily four as shown), it should be noted that these actuators may be actuated using any type of actuation technique, such as piezoelectric, electrostatic, electromagnetic, or electromechanical actuation, including any actuation methods considered in this disclosure. It should be noted that the MEMS mirror may perform actuation on different planes for one-dimensional (1D) or two-dimensional (2D) scanning.

[0757] The disclosed position of the operating assembly of the MEMS mirror 9002 on different surfaces behind the reflective surface makes it possible to generate a reflector array (e.g., reflector array 312) containing multiple reflectors positioned very close together. This increases the usable surface area of the reflector array and reduces the amount of undesirable reflections (reflections from parts of the reflector assembly that are not the mirror surface). Furthermore, by positioning the moving actuator behind the MEMS mirror 9002 (and away from the optical transmission path in the system), the amount of photons reflected in unintended directions from the moving actuator is reduced, and therefore the noise level in the system is reduced. The MEMS mirror 9002 and the operating surface (including the actuator and frame 9550) can be manufactured on two different wafers and interconnected in various ways, including those known in the art.

Monitoring of MEMS mirrors using variable capacitors
[0758] In accordance with this disclosure, the orientation of a MEMS mirror can be estimated by monitoring the bending of an actuator connected to the MEMS mirror (via interconnection). For example, the LIDAR system 100 may include one or more variable capacitors. There may be one variable capacitor per actuator, two or more variable capacitors per actuator, and/or fewer variable capacitors than actuators. For each variable capacitor, the capacitance of the variable capacitor represents the spatial relationship between the frame and the actuator. The capacitance of the variable capacitor may be a function of the overlap area between one or more plates of the variable capacitor connected to the frame and one or more other plates of the variable capacitor connected to the actuator, in particular to the outer circumference of the actuator facing the frame.

[0759] Figure 62 shows the actuators 9011, 9012, 9013, and 9014, the interconnects 9021, 9022, 9023, and 9024, and the frame 9050 surrounding the MEMS mirror 9002. Figure 62 also shows a variable capacitor 9061 formed between the frame 9050 and the actuator 9011. The variable capacitor 9061 includes a first plate 90612 of a plurality of plates connected to the actuator and a plurality of second plates 90611 connected to the frame. It may be advantageous to have at least three variable capacitors between at least three actuators and the frame. For simplicity of explanation, only a single variable capacitor is shown. The variable capacitor can be placed at any location along the outer circumference of the actuator and at any distance from the outer circumference of the actuator connected to the interconnect. Furthermore, the location of the variable capacitor can be determined based on the shape and size of the plates of the variable capacitor, as well as the amount of bending that may occur due to various parts of the actuator. For example, positioning the variable capacitor near the base may reduce the change in the overlapping area between the first and second plates, while positioning the variable capacitor near the connection point for the interconnection may eliminate the overlap between the first and second plates altogether.

[0760] Figure 62 (from left to right) shows the state in which the first and second plates (90611 and 90612) are completely overlapping, then the state in which they are almost overlapping (overlapping area 9068) (when the actuator begins to bend), and then the state in which they are slightly overlapping (small overlapping area 9068) (when the actuator continues to bend). The first plate 90612 is coupled to each other in parallel. The second plate 90611 is coupled to each other in parallel. The first and second plates are coupled to a capacitance sensor 9065 configured to detect the capacitance of the variable capacitors. The controller of the LIDAR system can estimate the orientation of the MEMS mirror based on the capacitance of one or more variable capacitors.

[0761] Figure 63 shows the actuators 9011, 9012, 9013, and 9014, the interconnects 9021, 9022, 9023, and 9024, and the frame 9050 surrounding the MEMS mirror 9002. Figure 63 also shows electrodes 9081, 9082, 9083, and 9084 connected to the actuators 9011, 9012, 9013, and 9014. These electrodes can be connected to any part of the actuator. It is also possible to connect a single actuator to multiple electrodes. The electrodes typically extend along most of the actuator.

Monitoring of MEMS mirrors using dummy piezoelectric elements
[0762] According to the Disclosure, an electrode provided can transmit an electrical signal for bending an actuator and/or for detecting the bending of an actuator. By using an actuator that includes a dummy element, the bending of the actuator can be monitored. The dummy element may be a dummy electrode and a dummy piezoelectric element. The dummy piezoelectric element is mechanically coupled to a piezoelectric element to which a bending electric field is applied. The piezoelectric element is bent. This bending causes the dummy piezoelectric element to bend. The bending of the dummy piezoelectric element can be measured by an electrode coupled to the dummy piezoelectric element.

[0763] Figure 64 shows the actuators 9011, 9012, 9013, and 9014, the interconnects 9021, 9022, 9023, and 9024, and the frame 9050 surrounding the MEMS mirror 9002. Figure 64 also shows electrodes 9081, 9082, 9083, and 9084 connected to the piezoelectric elements 9111, 9112, 9113, and 9114 of the actuators 9011, 9012, 9013, and 9014. Electrodes 9081, 9082, 9083, and 9084 are used to transmit bending control signals. Figure 64 also shows electrodes 9091, 9092, 9093, and 9094 connected to dummy piezoelectric elements 9011', 9012', 9013', and 9014' of actuators 9011, 9012, 9013, and 9014. Electrodes 9081, 9082, 9083, 9084, 9091, 9092, 9093, and 9094 typically cover most of the piezoelectric elements. It should be noted that each piezoelectric element is positioned between electrode pairs, and that Figure 64 shows only the external electrodes. Internal electrodes positioned between the actuator substrate (or body) and the piezoelectric elements are not shown.

[0764] Figure 65 is a cross-sectional view of the actuator 9011, the feedback sensor 9142, and the steering source signal 9140. The actuator 9011 may include a substrate (or body) layer 9121, an internal electrode 9081', an internal dummy electrode 9091', a piezoelectric element 9111, a dummy piezoelectric element 9111', an external electrode 9081, and an external dummy electrode 9091. The steering signal sensor 9140 transmits steering signals SS1 9151 and SS2 9152 to the external electrode 9081 and the internal electrode 9121 to bend the actuator 9011. The feedback sensor 9142 detects the bending of the blunt piezoelectric element 9111', which measures the electric field between the internal dummy electrode 9091' and the external dummy electrode 9091. It should be noted that only one steering signal may be provided.

[0765] Figure 66 shows that actuators 9011, 9012, 9013, and 9014 can be formed from four main layers. These four layers are: external electrode layers (9124, 9134, 9144, and 9154), piezoelectric layers (9123, 9133, 9143, and 9153), internal electrode layers (9122, 9132, 9142, and 9152), and substrate (or body) layers (9121, 9131, 9141, and 9151).

[0766] Monitoring of MEMS mirrors by measuring changes in dielectric constant

[0767] According to this disclosure, bending of an actuator can change the dielectric constant of a piezoelectric element. Therefore, the actuator can be monitored by measuring the change in the dielectric constant of the piezoelectric element. The actuator can be supplied with an electric field induced by one or more control signals from a control signal source. One or more control signals are supplied to one or more electrodes of a LiDAR system, such as a pair of electrodes located on both sides of the piezoelectric element. One control signal, both control signals, and/or the difference between the control signals have an alternating bias component and a steering component. Bending of the body responds to the steering component. In some embodiments, the frequency of the alternating bias component may be greater than the maximum frequency of the steering component (e.g., at least 10 times), and the amplitude of the alternating bias component may be any degree smaller than the amplitude of the steering component, for example, by more than 1/100th. For example, the steering component may be in the range of tens of volts, and the alternating bias component may be in the range of tens to hundreds of millivolts. Therefore, the sensors of the LiDAR system 100 can be configured to detect changes in the dielectric constant of the actuator caused by bending of the actuator.

[0768] Figure 67 shows an actuator including an external electrode layer 9124, a piezoelectric layer 9123, an internal electrode layer 9122, and a substrate layer 9121. A steering signal source 9140 transmits a control signal SS1 9151 to the external electrode layer 9124 and a control signal SS2 9152 to the internal electrode layer 9122. At least one of the control signals SS1 9151 and SS2 9152, or the difference between these control signals, includes an alternating bias component and a steering component. A feedback sensor 9124 is coupled to the external electrode layer 9124 and the internal electrode layer 9122 and can detect (directly or indirectly) a change in the dielectric constant of the piezoelectric layer 9123. The feedback sensor 9124 may be, for example, a current amplitude sensor, or a combination of a current amplitude sensor and a phase shift sensor. The LIDAR sensor may include a controller that can be configured to receive information (from the feedback sensor 9142) regarding changes in dielectric constant to determine the orientation of the MEMS mirror. Figure 67 also shows a steering signal source 9140, which includes an initial signal source 9141 that outputs steering components (9161 and 9164) of the control signals SS1 9151 and SS2 9152. These steering components are mixed (by mixers 9163 and 9165) with alternating bias components (generated by oscillators 9162 and 9165) to generate the control signals SS1 9151 and SS2 9152. The actuator can be monitored by sensing the actuator's resistance.

[0769] Figure 68 shows two electrodes 9211 and 9212 positioned at the two opposite ends of actuator 9011 and used to measure the actuator's resistance. Electrode 9135 is used to bend the actuator. Electrodes 9211, 9212, and 9135 are electrically coupled to three conductors 9201, 9202, and 9203.

[0770] Figure 69 shows stress relief apertures 9220 formed on actuator 9011. The stress relief apertures in Figure 69 are curved and substantially parallel to each other. The number of stress relief apertures may differ from four. Their elongated holes may have arbitrary shapes or sizes and may differ from each other. In some of the figures above, the piezoelectric elements were positioned above the substrate. It should be noted that the piezoelectric elements may be positioned below the substrate. It is also possible to position multiple piezoelectric elements above and below the substrate.

[0771] Figure 70 shows the actuator 9012, which includes seven main layers: an external electrode layer 9124, a piezoelectric layer 9123, an internal electrode layer 9122, a substrate (or body) layer 9121, an additional internal electrode layer 9129, an additional piezoelectric layer 9128, and an additional external electrode layer 9127. The external electrode layer 9124, the piezoelectric layer 9123, and the internal electrode layer 9122 are positioned above the substrate layer 9121. The additional internal electrode layer 9129, the additional piezoelectric layer 9128, and the additional external electrode layer 9127 are positioned below the substrate layer 9121. The additional piezoelectric layer 9128 may be the same as the piezoelectric layer 9123, or may differ from the piezoelectric layer 9123 in at least one of its characteristics, such as size or shape. Specifically, any of the electrode layers may be identical or different from each other. The additional piezoelectric layers 9128 and 9123 can be controlled independently or dependently to each other. The additional piezoelectric layer 9128 can also be used to bend the actuator downwards, and the piezoelectric layer 9123 can be used to bend the actuator upwards. The additional piezoelectric layer 9128 can be used as a dummy piezoelectric sensor (for monitoring the actuator) when piezoelectric layer 9123 is activated to bend the actuator. In one example, piezoelectric layer 9122 can be used as a dummy piezoelectric sensor (for monitoring the actuator) when piezoelectric layer 9128 is activated to bend the actuator.

[0772] Figure 71 shows, from top to bottom, (i) the idle state of the mirror 9002, (ii) a downwardly bent actuator lowering the outer circumference of the MEMS mirror 9002, and (iii) an upwardly bent actuator raising the outer circumference of the MEMS mirror 9002. The MEMS mirror 9002 is coupled to the actuators via the interconnect 9300. The MEMS mirror 9002 may include a thin reflective surface reinforced by reinforcing elements.

[0773] Figures 72 and 73 show the back of the frame 9050 and the MEMS mirror 9002. For simplicity of explanation, the actuator is not shown. The reinforcing element 9003 includes a concentric ring and radial portions. Any configuration and shape of the reinforcing element is possible.

[0774] The orientation of the MEMS mirror 9002 can be monitored by illuminating the back of the MEMS mirror 9002. It may be advantageous to illuminate at least one area of the MEMS mirror and detect reflected light at at least three locations. The orientation of the MEMS mirror can be monitored by illuminating the back of the MEMS mirror 9002. It may be advantageous to illuminate at least one area on the back of the MEMS mirror and detect reflected light at at least three locations. It should be noted that the LIDAR system 100 may include a dedicated light source for illuminating the back of the MEMS mirror. The dedicated light source (e.g., an LED) can be positioned behind the mirror (i.e., away from the main reflection sensor used for deflecting light from at least one light source 112). Alternatively, the LIDAR system 100 may include an optical system for directing light to the back of the mirror. In some examples, light guided to the back of the MEMS mirror (e.g., light from a dedicated light source) is confined to the back area of the mirror and prevented from reaching the primary reflecting side of the MEMS mirror. Processing of the back-side sensor signals can be performed by processor 118, but it can also be done by dedicated circuitry integrated into a chip located within the mirror casing. This processing may include comparing reflected signals to different back-side sensors (e.g., 9231, 9232, 9233), subtracting such signals, and normalizing such signals. Such signal processing can be based on information collected during the calibration phase.

[0775] Figure 74 shows the illumination area 9030 and three sensors 9231, 9232, and 9233. These sensors are positioned below the MEMS mirror and are arranged to detect reflected light (dashed line) in three different directions, thereby enabling detection of the orientation of the MEMS mirror. The illumination area can be positioned anywhere on the back of the MEMS mirror and can have any shape and size. In embodiments, the MEMS mirror may not be parallel to the window of the Lidar system. The MEMS mirror receives light that has passed through the window of the Lidar system, deflects the reflective mirror to produce deflected light, which can pass through the window and reach other components of the Lidar system (such as light sensors). Some of the deflected light may be reflected backward (by the window) towards the MEMS mirror, frame, or actuator. However, if the MEMS mirror and the window are parallel to each other, the light may be repeatedly reflected by the MEMS mirror and the window, resulting in undesirable optical artifacts. These optical artifacts can be attenuated, or even prevented, by providing a window that is not parallel to the MEMS mirror, or by ensuring that the optical axes of the MEMS mirror and the window are not parallel to each other. If either the MEMS mirror or the window is curved or has multiple sections that are not oriented relative to each other, it may be advantageous for no part of the MEMS mirror to be parallel to any part of the window. The angle between the window and the MEMS mirror can be set so that the window does not reflect light towards the MEMS mirror, whether the MEMS mirror is in an idle position or moved by one of the actuators.

[0776] It should be noted that illumination of the back of a MEMS mirror can be performed when the back surface of the mirror is substantially uniformly reflective (e.g., a flat back surface without reinforcing ribs). However, this is not always the case, and the back surface of a mirror may be designed to reflect light in a non-uniform pattern. Patterned reflective behavior of the back surface of a mirror can be achieved in various ways, such as the geometric shape of the surface (e.g., protrusions, intrusions), surface texture, and different materials (e.g., silicon, silicon oxide, metal). Optionally, a MEMS mirror may include a patterned back surface having a reflectivity pattern on at least a portion of the back surface of the mirror, providing patterned reflection of back-side illumination (e.g., from a back-side-only light source as described above) onto back-side sensors (e.g., 9231, 9232, 9233). Optionally, the patterned back surface may, but is not necessarily, include a portion of an optional reinforcing element 9003 located on the back surface of the MEMS mirror. For example, the reinforcing element 9003 can be used to generate an angled shadow (or deflect light to a different angle) above the sensor 9231, etc. This means that the reflection on the sensor changes from shadow to bright as the mirror moves.

[0777] Optionally, the processing of the output of the backside sensors (9231, 9232, 9233, etc.) can take into account the backside reflectance pattern (e.g., resulting from the pattern of reinforcing ribs). Thus, this processing can use the patterning resulting from the backside surface pattern as part of the feedback to be processed. Optionally, the backside mirror feedback options considered herein allow for the use of a backside reflectance pattern that can be processed by data from a backside sensor positioned very close to the mirror (compared to implementing uniform reflectance), thus enabling miniaturization of the MEMS assembly and improving its packaging. For example, the backside pattern can be designed so that the reflectance pattern includes a sharp transition between dark and bright reflections. A sharp transition means that even a small change in the angle/position of the MEMS mirror causes a large change in the light reflected by a detector located at a close distance. Furthermore, the reflectance pattern can also be associated with a reflectance gradient rather than sharp edges (i.e., light or shadow). This embodiment offers linearity from a first option of sharp edges, thus facilitating post-processing, and furthermore, it can accommodate a wider angular range and, in some cases, is less sensitive to assembly tolerances.

MEMS mirror not parallel to the LIDAR system window
[0778] In accordance with this disclosure, a MEMS mirror receives light passing through a window of a LiDAR system, deflects a reflective mirror to produce deflected light, which can pass through the window to reach other components of the LiDAR system 100 (such as a light sensor). Some of the deflected light may be reflected backward (by the window) toward the MEMS mirror, frame, or actuator. If the MEMS mirror and the window are parallel to each other, the light may be repeatedly reflected by the MEMS mirror and the window, resulting in undesirable optical artifacts. These optical artifacts can be attenuated, or even prevented, by providing a window that is not parallel to the MEMS mirror, or by ensuring that the optical axis of the MEMS mirror and the optical axis of the window are not parallel to each other. If either the MEMS mirror or the window is curved or has multiple sections that are not oriented toward each other, it may be advantageous that no part of the MEMS mirror is parallel to any part of the window. The angle between the window and the MEMS mirror can be set so that the window does not reflect light towards the MEMS mirror, whether the MEMS mirror is in an idle position or is moved by one of the actuators.

[0779] Figure 75 shows a housing 9320 including a window 9322. The housing houses a MEMS mirror 9002. The housing 9320 can be a sealed housing that can be manufactured using wafer-level packaging or any other technique. The housing 9320 includes a base 9310. The base 9310 may be transparent or opaque. A transparent base may be useful if the back side of the MEMS mirror 9002 is monitored by illumination. Light 9601 passes through the window 9322 and enters the MEMS mirror 9002. The MEMS mirror 9002 deflects the light, giving deflected light 9602. Some of the deflected light passes through the window 9322, but another portion 9603 is reflected by the mirror 9322 towards the housing 9320. Therefore, portion 9603 may not be reflected towards the MEMS mirror 9002.

[0780] Figure 76 shows the housing 9320 including the upper section. The upper section includes the mirror 9320 and two side walls 9321 and 9323. The middle section of the housing can be formed from the exterior of an integrated circuit (but not limited to a frame 9050, etc.) including various layers (9121 and 9122, etc.). The integrated circuit may include a MEMS mirror 9002 (having an upper reflective surface 9004, various intermediate elements of layers 9121 and 9122, and reinforcing elements 9003), interconnects 9022 and 9021, and actuators 9012 and 9014. A bonding layer 9301 can be positioned between the integrated circuit and the base 9310.

[0781] Figure 77 shows the housing 9320, including a transparent base. For simplicity of explanation, this figure shows the illumination unit 9243, the beam splitter 9263, and the sensor 9253. The illumination unit 9243 and the light sensor 9253 are located outside the housing.

[0782] Figure 78 shows an anti-reflective layer 9380 positioned on the actuator and interconnects. Figure 79 shows an anti-reflective layer 9380 positioned on the actuator, frame, and interconnects. Figure 80 shows an anti-reflective layer 9380 positioned on the frame. Any of the above-described anti-reflective layers may be replaced with one or more anti-reflective elements different from the layers. The anti-reflective elements may be parallel to the window, oriented toward the window, etc.

[0783] Figure 81 shows a housing with a window parallel to the MEMS window. The housing includes a transparent base. For simplicity of explanation, this figure shows the illumination unit 9243, the beam splitter 9263, and the sensor 9253. The illumination unit 9243 and the light sensor 9253 are located outside the housing. The MEMS mirror can be of any shape and size. For example, the MEMS mirror may be rectangular.

[0784] Figures 82 and 83 show a rectangular MEMS mirror 9402, two actuators 9404 and 9407, two interconnects 9403 and 9406, electrodes 9410 and 9413, and a rectangular frame. The rectangular frame includes an upper section 9504, a lower section 9408, and two insulating sections 9411 and 9422 connected between the upper and lower sections of the frame. In Figure 82, actuators 9404 and 9407 are parallel to each other, facing both sides of the MEMS mirror, and connected to opposing sections of the frame. In Figure 83, actuators 9404 and 9407 are parallel to each other, facing both sides of the MEMS mirror, and connected to the same side of the frame.

[0785] Figure 84 shows a rectangular MEMS mirror 9402, four actuators 9404, 9407, 9424, 9427, four interconnects 9403, 9406, 9423, and 9436, four electrodes 9410, 9413, 9440, and 9443, and a rectangular frame. The rectangular frame includes an upper section 9504, a lower section 9408, and two insulating sections 9411 and 9422 connected between the upper and lower sections of the frame. The four actuators face four facets of the MEMS mirror 9402, each connected to a different facet of the frame. Although Figures 56 to 84 show a single MEMS mirror, the LIDAR system 100 can also include an array of multiple MEMS mirrors. Any number of MEMS mirrors can be monitored to provide feedback used to control any of the multiple MEMS mirrors. For example, if there are N MEMS mirrors, which is more than any number between 1 and N, the MEMS mirrors can be monitored to provide feedback that can be used to monitor any number of the N MEMS mirrors.

[0786] In one embodiment, the LIDAR system 100 may include a window for receiving light, a micro-electromechanical (MEMS) mirror for deflecting light and providing deflected light, a frame, an actuator, and an interconnection element that can be mechanically connected between the actuator and the MEMS mirror. Each actuator may include a body and a piezoelectric element. The piezoelectric element may be configured to bend the body and move the MEMS mirror when an electric field is applied. When the MEMS mirror is positioned in an idle position, it may be oriented relative to the window. The light may be reflected light that may be within at least one segment of the field of view of the LIDAR system. The light may be transmitted light from the light source of the LIDAR system. During a first period, the light is transmitted light from the light source of the LIDAR system, and during a second period, the light is reflected light that may be within at least one segment of the field of view of the LIDAR system.

[0787] In another embodiment, the LIDAR system 100 may include at least one anti-reflective element that can be positioned between the window and the frame. The anti-reflective element may be oriented relative to the window. The orientation angle between the MEMS mirror and the window may range from 20 to 70 degrees. The window may be shaped and positioned such that any portion of the deflected light is prevented from being reflected towards the MEMS mirror. The MEMS mirror may be oriented relative to the window even when moved by at least one of the actuators. One of the interconnection elements may include a first segment that can be connected to the MEMS mirror and a second segment that can be connected to the actuator. The first and second segments may be mechanically coupled to each other.

[0788] In a related embodiment, the first segment may be oriented substantially 90 degrees to the second segment. The first segment may be connected to the outer circumference of the MEMS mirror and oriented substantially 90 degrees to the outer circumference of the MEMS mirror. When the MEMS mirror is positioned in the idle position, the first segment may face the center of the MEMS mirror. The second segment may be connected to the outer circumference of the actuator and oriented substantially 90 degrees to the outer circumference of the actuator. The longitudinal axis of the second segment may be substantially parallel to the longitudinal axis of the actuator. When the MEMS mirror is positioned in the idle position, the first and second segments may be arranged in an L-shape. The interconnection element may include at least one additional segment that can be mechanically coupled between the first and second segments. The first and second segments may differ in length from one another. The first and second segments may differ in width from one another. The first and second segments may differ in cross-sectional shape from one another. When the MEMS mirror is positioned in the idle position, the first and second segments can be positioned on the same plane as the MEMS mirror. The first and second segments can also be positioned on the same plane as the actuator.

[0789] In another embodiment, the LIDAR system 100 may include a MEMS mirror that may have an elliptical shape (for example, the MEMS mirror may be circular). The actuator may include at least three independently controlled actuators. Each pair of actuators and interconnection elements may be directly connected between the frame and the MEMS mirror. The MEMS mirror is operable to pivot about two axes of rotation.

[0790] In related embodiments, the actuator may include at least four independently controlled actuators. The longitudinal axis of the MEMS mirror may correspond to the longitudinal axis of the light beam. The longitudinal axis of the MEMS mirror may correspond to the longitudinal axis of the detector array of the LIDAR system. The actuator may include a first pair of actuators facing each other along a first direction and a second pair of actuators facing each other along a second direction. The first pair of actuators may be different from the second pair of actuators. The window, MEMS mirror, frame, and actuators may form a unit. This unit may respond differently to mechanical vibrations propagating along a first direction and mechanical vibrations propagating along a second direction. When the first pair of actuators is idle, it may have a substantially different length than the length of the second pair of actuators when they are idle. When the first pair of actuators is idle, it may have a substantially different shape than the shape of the second pair of actuators when they are idle. During operation, the LIDAR system may be subjected to mechanical vibrations having a specific frequency range. The unit's resonant frequency may be outside this specific frequency range. The unit's resonant frequency may be at least twice as high as the maximum frequency within the specific frequency range. The unit's resonant frequency may be between 400 Hz and 1 kHz. An actuator may include a piezoelectric element positioned below the actuator's body, and another actuator may include a piezoelectric element positioned above the body of the other piezoelectric element. The LIDAR system may further include a controller that can be configured to receive indications of the state of additional piezoelectric elements from sensors. The controller may be configured to control the actuators based on the indications of the state of the additional piezoelectric elements. The controller may be configured to determine the orientation of the MEMS mirror based on the indications of the state of the additional piezoelectric elements.

[0791] In another embodiment, the LIDAR system 100 may include a variable capacitor and a sensor. The capacitance of the variable capacitor represents the spatial relationship between the frame and one of the actuators. The sensor may be configured to detect the capacitance of the variable capacitor.

[0792] In related embodiments, the variable capacitor may include a first plate connectable to an actuator and a second plate connectable to a frame. The spatial relationship between the frame and the actuator determines the overlap between the first and second plates. The variable capacitor may include a plurality of first plates connectable to the actuator and a plurality of second plates connectable to the frame. The actuator may have a first end mechanically connectable to the frame and a second end opposite to the first end, mechanically connectable to an interconnection element. The distance between the variable capacitor and the first end may be greater than the distance between the variable capacitor and the second end. The actuator may have a first end mechanically connectable to the frame and a second end opposite to the first end, mechanically connectable to an interconnection element. The distance between the variable capacitor and the first end may be less than the distance between the variable capacitor and the second end.

[0793] In another embodiment, the LIDAR system 100 may include a controller configured to receive an instruction for the capacitance of a variable capacitor and to determine the orientation of the MEMS mirror based on the capacitance of the variable capacitor. The piezoelectric element may be configured to bend its body and move the MEMS mirror when an electric field induced by a control signal from a control signal source is applied. The control signal may be supplied to the electrodes of the LIDAR system.

[0794] The control signal may have an alternating bias component and a steering component. Bending of the main body may respond to the steering component. The frequency of the alternating bias component is greater than the maximum frequency of the steering component. The sensor may be configured to detect a change in the dielectric constant of the actuator caused by bending of the actuator.

[0795] In related embodiments, the sensor may be a current amplitude sensor. Alternatively, the sensor may be a current amplitude sensor and a phase shift sensor. The amplitude of the alternating bias component may be at least 1/100th smaller than the amplitude of the steering component. The LIDAR system may further include a controller that can be configured to receive information about changes in dielectric constant and to determine the orientation of the MEMS mirror. A window may be attached to the housing. The housing may be a sealed housing that accommodates the MEMS mirror, frame, and actuator. The housing may include a transparent region positioned below the MEMS mirror. The LIDAR system may further include at least one optical sensor and at least one light source, the at least one light source may be configured to transmit at least one light beam through the transparent region to the back of the MEMS mirror. At least one optical sensor may be configured to receive light from the back of the MEMS mirror. The LIDAR system may include a controller that can be configured to determine the orientation of the MEMS mirror based on information from at least one optical sensor. Different parts of the housing may be formed by wafer-level packaging. The frame may be attached to the integrated circuit forming the lower region of the enclosure. One of the interconnecting elements may include multiple segments that can be mechanically connected to each other by at least one joint. The joint may be a ball joint. Alternatively, the joint may be a MEMS joint.

[0796] The foregoing description is presented for illustrative purposes only. It is not exhaustive and is not limited to the exact forms or embodiments disclosed. Modifications and adaptations will be apparent to those skilled in the art from a review of this specification and the implementation of the disclosed embodiments. Furthermore, while the embodiments of the disclosed embodiments are described as being stored in memory, those skilled in the art will recognize that these embodiments can also be stored in other forms of computer-readable media, such as hard disks or CD-ROMs, or other forms of RAM or ROM, USB media, DVDs, Blu-rays, or other optical drive media.

[0797] Computer programs based on the descriptions and disclosed methods are within the skill of an experienced developer. Various programs or program modules can be generated using any technique known to those skilled in the art, or designed in association with existing software. For example, program sections or program modules can be designed in or by the .NET framework, the .NET Compact framework (and related languages such as Visual Basic and C), Java, C++, Objective-C, HTML, HTML/AJAX combinations, XML, or HTML including Java applets.

[0798] Furthermore, while exemplary embodiments have been described herein, the scope of any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., aspects across various embodiments), adaptations, and/or variations will be recognized by those skilled in the art based on this disclosure. The limitations in the claims shall be interpreted broadly based on the language used in the claims and shall not be limited to the examples described herein or during the examination of this application. These examples shall be interpreted non-exclusively. Furthermore, the steps of the disclosed methods may be modified at will, including by changing the order of the steps and/or by inserting or deleting steps. Accordingly, this specification and the examples are to be considered merely illustrative, and the true scope and spirit are intended to be shown by the entire scope of the following claims and equivalents.

Claims (44)

At least one processor,
Control at least one light source so that the beam can be varied in a scan of a field of view including the first and second parts.
Each pixel receives a signal from at least one sensor, the signal representing at least one of the following: ambient light, light from the at least one light source reflected by an object in the field of view, and noise associated with the at least one sensor.
To estimate the noise in at least a portion of the signal related to the first portion of the field of view,
Based on the estimation of noise in the first portion of the field of view, the sensor sensitivity to reflections related to the first portion of the field of view is changed.
To estimate the noise in at least a portion of the signal related to the second portion of the field of view,
Based on the estimation of noise in the second portion of the field of view, the sensor sensitivity for reflections related to the second portion of the field of view is changed, and the changed sensor sensitivity for reflections related to the second portion is different from the changed sensor sensitivity for reflections related to the first portion.
A LiDAR system comprising at least one processor configured as follows. The LIDAR system according to claim 1, wherein the at least one processor is further configured to control at least one optical deflector for scanning the field of view, and a single scan cycle includes moving the at least one optical deflector such that the field of view is positioned at a plurality of different instantaneous positions during the scan cycle. The LIDAR system according to claim 2, wherein the at least one processor is configured to coordinate the at least one optical deflector and the at least one light source such that, when the at least one optical deflector is positioned at a specific instantaneous location, the optical beam is deflected by the at least one optical deflector from the at least one light source towards the field of view, and reflections from objects in the field of view are deflected by the at least one optical deflector towards the at least one sensor. The LIDAR system according to claim 3, wherein the at least one processor is further configured to modify the sensor sensitivity to reflections associated with the portion of the field of view corresponding to a single instantaneous position of the at least one optical deflector. The LIDAR system according to claim 3, wherein the at least one processor is further configured to modify the sensor sensitivity to reflections associated with portions of the field of view corresponding to a plurality of instantaneous positions of the at least one optical deflector. The LIDAR system according to claim 3, wherein the at least one processor is further configured to change the sensor sensitivity to a first reflection associated with the first portion received in a first scan cycle, and to change the sensor sensitivity to a second reflection associated with the second portion received in a second scan cycle. The LIDAR system according to claim 3, wherein the at least one processor is further configured to modify the sensor sensitivity to a first reflection associated with the first portion and a second reflection associated with the second portion, and the first and second reflections are received in a single scan cycle. The LIDAR system according to claim 3, wherein the at least one processor is further configured to estimate noise in each portion based on reflections associated with a single portion of the at least one optical deflector. The LIDAR system according to claim 1, wherein the at least one processor is further configured to estimate noise in the signals associated with the particular portion of the field of view based on a comparison of signals associated with the particular portion of the field of view received in at least one preceding scan cycle. The LIDAR system according to claim 1, wherein the at least one processor is further configured to modify the sensor sensitivity to reflections associated with a third portion of the field of view, distinct from the first and second portions, based on the estimation of noise in the first portion. The LIDAR system according to claim 10, wherein the at least one processor is further configured to modify the sensor sensitivity to reflections associated with the third portion based on the estimation of noise in both the first and second portions. The LIDAR system according to claim 1, wherein the at least one processor is further configured to increase the amount of light projected toward the first portion to the amount of light projected toward the second portion if the estimated noise in the first portion is greater than the estimated noise in the second portion. The LIDAR system according to claim 1, wherein the sensor sensitivity is a signal threshold, and the at least one processor is further configured to increase the signal threshold for the first portion above the signal threshold for the second portion if the noise estimate in the first portion is greater than the noise estimate in the second portion. The LIDAR system according to claim 13, wherein the at least one processor is further configured to detect an external light source at a first distance in the first portion, and to vary the sensor sensitivity to reflections associated with the first and second portions to enable the detection of an object at a second distance greater than the first distance in the second portion. The LIDAR system according to claim 1, wherein the at least one processor is further configured to individually change the sensor sensitivity to reflections associated with the first and second portions such that, for the same amount of light projected toward the first and second portions, the detection distance associated with the first portion is greater than the detection distance associated with the second portion. The LIDAR system according to claim 1, wherein the at least one processor is further configured to individually change the sensor sensitivity to reflections associated with the first and second portions such that, for the same amount of light projected toward the first and second portions, the resolution associated with the first portion is greater than the resolution associated with the second portion. The LIDAR system according to claim 1, wherein the signal from at least one sensor further includes noise generated by an amplifying electronic device. The LIDAR system according to claim 1, wherein the noise includes at least one of background noise, amplified noise, or ambient noise. The LIDAR system according to claim 18, wherein the at least one processor is further configured to individually modify the sensor sensitivity to reflections associated with the first and second portions, and to detect an object in at least one of the first and second portions based on the modified sensor sensitivity. The LIDAR system according to claim 1, wherein the signal includes a combination of reflection measurement and the noise. The LIDAR system according to claim 20, wherein the at least one processor is further configured to determine the distance to an object in at least one of the first and second portions based on the signal. The LIDAR system according to claim 20, wherein the at least one processor is further configured to individually change the sensor sensitivity to reflections associated with the first and second portions such that the detection distance associated with the first portion is greater than the detection distance associated with the second portion. The LIDAR system according to claim 1, wherein the at least one processor is further configured to detect the sun in the first portion and to enable object detection in the second portion by differently changing the sensor sensitivity to reflections associated with the first and second portions. The LIDAR system according to claim 1, wherein the at least one processor is further configured to detect vehicle headlights in the first portion and to enable object detection in the second portion by differently changing the sensor sensitivity to reflections associated with the first portion and the second portion. Estimating noise in at least a portion of the signal associated with the first portion of the field of view includes estimating the first noise level originating from the first portion of the field of view, such that changing the sensor sensitivity to reflections associated with the first portion of the field of view is based on the estimation of the first noise level.
The LiDAR system according to claim 1, wherein estimating noise in at least a portion of the signal associated with the second portion of the field of view includes estimating the second noise level originating from the first portion of the field of view, such that changing the sensor sensitivity to reflections associated with the second portion of the field of view is based on the estimation of the second noise level. The LIDAR system according to claim 1, wherein the noise associated with the at least one sensor includes noise based on reflections associated with a single position of the at least one light source. The LIDAR system according to claim 1, wherein changing the sensor sensitivity to reflections associated with the first portion of the field of view is based on a signal threshold for the first portion of the field of view relative to a signal threshold for the second portion of the field of view. The LIDAR system according to claim 1, wherein the noise is generated in at least partially in response to the ambient light received by the at least one sensor. A method for changing the sensor sensitivity in a LIDAR system,
Controlling at least one light source so that the light beam can be varied in a scan of a field of view including the first and second parts,
The method involves receiving a signal from at least one sensor for each pixel, wherein the signal represents at least one of ambient light and light from the at least one light source reflected by an object in the field of view.
To estimate noise in at least a portion of the signal related to the first portion of the field of view,
Based on the estimation of noise in the first portion of the field of view, the sensor sensitivity to reflections related to the first portion of the field of view is changed.
To estimate noise in at least a portion of the signal related to the second portion of the field of view,
Based on the estimation of noise in the second portion of the field of view, the sensor sensitivity for reflections related to the second portion of the field of view is changed, wherein the changed sensor sensitivity for reflections related to the second portion is different from the changed sensor sensitivity for reflections related to the first portion.
A method that includes this. The method according to claim 29, wherein the signal includes a combination of reflection measurement and noise. The method according to claim 30, further comprising estimating noise in the signals associated with the particular portion of the field of view based on a comparison of signals associated with the particular portion of the field of view received in at least one preceding scan cycle. The method according to claim 30, further comprising changing the sensor sensitivity to reflections related to the third portion of the field of view based on the estimation of noise in at least one of the first and second portions. The method according to claim 30, further comprising modifying the light source parameters associated with the first portion such that the luminous flux induced in the first portion is greater than the luminous flux induced in at least one other portion of the field of view. The method according to claim 30, further comprising increasing the amount of light projected toward the first portion to the amount of light projected toward the second portion if the estimated noise in the first portion is greater than the estimated noise in the second portion. The method according to claim 29, further comprising estimating noise in the signals associated with the particular portion of the field of view based on a comparison of signals associated with the particular portion of the field of view received in at least one preceding scan cycle. The method according to claim 29, further comprising modifying the sensor sensitivity to reflections related to the third portion of the field of view based on the estimation of noise in at least one of the first and second portions. The method according to claim 29, further comprising modifying the light source parameters associated with the first portion such that the luminous flux induced in the first portion is greater than the luminous flux induced in at least one other portion of the field of view. The method according to claim 29, further comprising increasing the amount of light projected toward the first portion to the amount of light projected toward the second portion if the estimated noise in the first portion is greater than the estimated noise in the second portion. A non-temporary computer-readable storage medium storing instructions, wherein, when executed by at least one processor, the instructions cause the at least one processor to execute a method for changing the sensor sensitivity in a LIDAR system, and the method is
Controlling at least one light source so that the light beam can be varied in a scan of a field of view including the first and second parts,
The method involves receiving a signal from at least one sensor for each pixel, wherein the signal represents at least one of ambient light and light from the at least one light source reflected by an object in the field of view.
To estimate noise in at least a portion of the signal related to the first portion of the field of view,
Based on the estimation of noise in the first portion of the field of view, the sensor sensitivity to reflections related to the first portion of the field of view is changed.
To estimate noise in at least a portion of the signal related to the second portion of the field of view,
Based on the estimation of noise in the second portion of the field of view, the sensor sensitivity for reflections related to the second portion of the field of view is changed, wherein the changed sensor sensitivity for reflections related to the second portion is different from the changed sensor sensitivity for reflections related to the first portion.
Non-temporary computer-readable storage media, including [specific type of storage medium]. The aforementioned method,
Controlling at least one optical deflector to scan the field of view, wherein a single scan cycle moves the at least one optical deflector such that the field of view is positioned at a plurality of different instantaneous positions during the scan cycle.
When the at least one optical deflector is positioned at a specific instantaneous location, the at least one optical deflector and the at least one light source are coordinated such that the light beam is deflected by the at least one optical deflector from the at least one light source towards the field of view, and reflections from objects in the field of view are deflected by the at least one optical deflector towards the at least one sensor.
To change the sensor sensitivity to reflections associated with the portion of the field of view corresponding to a single instantaneous position of the at least one optical deflector,
A non-temporary computer-readable storage medium according to claim 39, further comprising: The aforementioned method,
Controlling at least one optical deflector to scan the field of view, wherein a single scan cycle moves the at least one optical deflector such that the field of view is positioned at a plurality of different instantaneous positions during the scan cycle.
When the at least one optical deflector is positioned at a specific instantaneous location, the at least one optical deflector and the at least one light source are coordinated such that the light beam is deflected by the at least one optical deflector from the at least one light source towards the field of view, and reflections from objects in the field of view are deflected by the at least one optical deflector towards the at least one sensor.
The sensor sensitivity to a first reflection associated with the first part and a second reflection associated with the second part is changed such that the first and second reflections are received in a single scan cycle.
A non-temporary computer-readable storage medium according to claim 39, further comprising: The aforementioned method,
Controlling at least one optical deflector to scan the field of view, wherein a single scan cycle moves the at least one optical deflector such that the field of view is positioned at a plurality of different instantaneous positions during the scan cycle.
When the at least one optical deflector is positioned at a specific instantaneous location, the at least one optical deflector and the at least one light source are coordinated such that the light beam is deflected by the at least one optical deflector from the at least one light source towards the field of view, and reflections from objects in the field of view are deflected by the at least one optical deflector towards the at least one sensor.
Estimating noise in each part based on reflections associated with a single part of the at least one optical deflector,
A non-temporary computer-readable storage medium according to claim 39, further comprising: The non-temporary computer-readable storage medium according to claim 39, wherein the signal includes a combination of reflection measurement and the noise. At least one processor,
Control at least one light source so that the beam can be varied in a scan of a field of view including the first and second parts.
With respect to the first portion of the field of view, a plurality of first signals are received from at least one sensor for each pixel, the plurality of first signals indicate received light from the at least one light source reflected by an object in the field of view, and the plurality of first signals include noise generated from an ambient light source in the first portion of the field of view.
The noise level generated from the ambient light source in the first portion of the field of view is estimated.
Based on the estimated noise level, set the sensitivity level of the at least one sensor to reflections related to the first portion of the field of view, which is greater than the sensitivity of the at least one sensor to reflections related to the second portion of the field of view.
A LiDAR system comprising at least one processor configured as follows.