Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
  • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
  • G01S17/06—Systems determining position data of a target
  • G01S17/42—Simultaneous measurement of distance and other co-ordinates
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
  • G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
  • G01S7/495—Counter-measures or counter-counter-measures using electronic or electro-optical means
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
  • G01S7/497—Means for monitoring or calibrating

Definitions

• This disclosure relates to a light detection device and a ranging system.
• TOF range-finding systems that measure the distance to an object (target) based on ToF (Time of Flight) are commonly known.
• TOF is generally divided into direct TOF (dTOF) and indirect TOF (iTOF).
• Direct ToF emits laser light from a laser light source as pulsed light through a lens, and detects photons reflected from the object on which each pulse of light is irradiated using a light-receiving element called a SPAD (Single Photon Avalanche Diode).
• SPAD Single Photon Avalanche Diode
• the carriers thus generated are converted into electrical signal pulses using avalanche multiplication, and this is input into a TDC (Time to Digital Converter) to measure the arrival time of the reflected light and calculate the distance to the object.
• TDC Time to Digital Converter
• multiple light detection devices may be used for monitoring. In such cases, it is possible to eliminate blind spots in the monitoring range by partially overlapping the monitoring areas.
• the present disclosure provides an optical detection device and a ranging system that can suppress deterioration in ranging accuracy.
• a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal; a control unit capable of executing at least one of a process of detecting a time lag between the light emission timing and a generation timing of the detection signal, a process of detecting a deviation of coordinates of the plurality of pixels, and a process of setting the light emission timing;
• An optical detection device comprising:
• control unit may detect the lag between the light emission timing and the generation timing of the detection signal based on the measured time difference between the light emission timing and the generation timing of the detection signal generated in response to the reflected light from the object placed at a predetermined position from the laser light source.
• the control unit may detect, as the deviation time, the difference between the time obtained by dividing the sum of a first distance from the laser light source to the position and a second distance from the position to the pixel array unit by the speed of light and the measured time difference.
• the control unit may calibrate the light emission timing or the detection signal generation timing based on the time lag.
• the signal processing unit may correct the distance value based on the deviation time.
• a mirror for changing the direction of irradiation of the laser light is further provided.
• the object is the mirror.
• the control unit may control the orientation of the mirror so that the laser light is incident on the pixel array unit.
• the plurality of pixels are arranged in a matrix,
• the control unit When executing the coordinate deviation detection process, the control unit: A laser light of a predetermined shape may be irradiated onto the object from a second laser light source different from the laser light source, and the deviation of the coordinates may be detected based on a first position of the reflected light of the laser light of the predetermined shape with respect to the plurality of pixels.
• the control unit may receive the reflected light of the laser light of the predetermined shape in a second pixel array unit different from the pixel array unit, and detect the deviation of the coordinates based on the difference between a second position of the reflected light of the laser light of the predetermined shape for multiple pixels in the second pixel array unit and the first position.
• the signal processing unit is capable of generating a two-dimensional distance image based on detection signals of the plurality of pixels;
• the control unit may change coordinates of the two-dimensional range image based on the deviation of the coordinates.
• the laser light of a predetermined shape is reflected as a rectangular light pattern along the row direction of the pixel array section, the pixel array unit is capable of partially driving each of a plurality of rectangular regions each having a first side in the row direction and a second side in a column direction perpendicular to the row direction; the control unit drives a portion of the pixel array unit for each rectangular region when detecting the reflected light;
• the area with the highest amount of received light may be selected from the plurality of areas, and the first position may be detected based on the area with the highest amount of received light.
• the control unit may select the region with the highest amount of light received from the plurality of regions based on the detection signal, then partially drive the pixel array unit for each of the plurality of rectangular second regions with the second side being shorter, select the second region with the highest amount of light received from the plurality of second regions, and detect the first position based on the second region with the highest amount of light received.
• the control unit is In a range of the pixel array section limited based on the second region with the highest amount of light received, the pixel array section may be partially driven for each of a plurality of rectangular third regions with the second side shorter, the third region with the highest amount of light received may be selected from the plurality of third regions, and the first position may be detected based on the third region with the highest amount of light received.
• the control unit is When pixels generating a detection signal equal to or greater than a predetermined value within the region are discontinuous in the row direction, it may be detected that a rotational misalignment has occurred between the irradiation optical system of the second laser light source and the light receiving optical system of the pixel array section.
• the control unit executes the light emission timing setting process, Irradiating the target with laser light at a predetermined interval from a second laser light source different from the laser light source;
• the light emission timing of the laser light source may be set based on a generation timing of the detection signal.
• the control unit may set the light emission timing of the laser light source based on an intermediate point in the generation timing of the detection signals that are successive in time series.
• a distance measuring system including a plurality of light detection devices having overlapping irradiation ranges of laser light
• Each of the plurality of photodetection devices includes: a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal, At least one of the plurality of light detection devices further comprises: A distance measurement system is provided having a control unit capable of performing at least one of a process of detecting the time difference between the light emission timing and the timing of generating the detection signal, a process of detecting a difference in the coordinates of the multiple pixels, and a process of setting the light emission timing.
• a ranging system comprising: Each of the plurality of photodetection devices includes: a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal,
• the overall control unit includes: A distance measuring system is provided that is capable of performing at least one of a process of detecting a time difference between the light emission timing and the generation timing of the detection signal, a process of detecting a difference in the coordinates of the multiple pixels, and a process of setting the light emission timing.
• FIG. 1 is a block diagram showing an example of a schematic configuration of a distance measuring system according to an embodiment of the disclosure.
• FIG. 1 is a diagram showing an example of a distance measurement system installed in a vehicle.
• FIG. 2 is a block diagram showing a detailed configuration example of a distance measuring device.
• 1A and 1B are diagrams illustrating examples of the configuration of a light-emitting device.
• FIG. 1 illustrates an example of a flash-type optical system.
• FIG. 2 is a diagram showing a schematic configuration of an optical system of the distance measuring system according to the embodiment.
• FIG. 4 is a diagram showing an example of a schematic configuration of the pixel array shown in FIG. 3 .
• FIG. 4 is a block diagram showing a configuration related to a calibration process of an overall control unit.
• FIG. 1 is a diagram showing an example of a distance measurement system installed in a vehicle.
• FIG. 2 is a block diagram showing a detailed configuration example of a distance measuring device.
• FIG. 1 is a diagram illustrating a typical operation of a scanning optical system.
• FIG. 10 is a diagram showing an example of a histogram during normal operation in FIG. 9 .
• FIG. 4 is a diagram illustrating the measurement distance of an object.
• FIG. 13 is a diagram showing a typical example of detection of a shift between light emission timing and measurement start timing.
• FIG. 11 is a diagram showing an example of a histogram in a first mode.
• 1A and 1B are diagrams illustrating a case where a deviation occurs in the optical axis of a light emitting device.
• 6A to 6C are diagrams illustrating an example of detection processing in a second mode by a second detection unit.
• FIG. 4 is a diagram showing coordinate correction data.
• FIG. 11A and 11B are diagrams showing an example in which a rotational deviation occurs in the optical axis of the optical system between distance measuring devices.
• FIG. 13 is a diagram illustrating a detection example of a third mode. 6 is a flowchart showing an example of a detection process according to the embodiment.
• FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system.
• FIG. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.
• Fig. 1 is a block diagram showing an example of a schematic configuration of a distance measuring system 1 according to an embodiment of the present disclosure.
• the distance measuring system 1 is a system capable of monitoring a plurality of monitoring areas B5a, B5, and B5b, and includes a plurality of photodetectors 5a, 5, and 5b, and an overall control unit 30.
• Each of the photodetectors 5a, 5, and 5b includes a light emitting device 10a, 10, and 10b, and a distance measuring device 20a, 20, and 20b.
• the distance measuring system 1 is a sensor system including a light source and a ToF sensor, and each of the photodetectors 5a, 5, and 5b is configured to emit laser light within the range of the monitoring areas B5a, B5, and B5b, and to detect reflected light reflected by an object.
• the target object may be one or more objects present within the angle of view of the distance measurement system 1. Details of the light emitting devices 10a, 10, 10b and the distance measurement devices 20a, 20, 20b will be described later.
• dTOF direct TOF
• iTOF indirect TOF
• an example of three multiple light detection devices 5a, 5, 5b is described, but this is not limited to this.
• the multiple light detection devices 5a, 5, 5b may be two or more, and the distance measurement system 1 may have 5, 8, 12, etc. light detection devices.
• FIG. 2 is a diagram showing an example in which the distance measurement system 1 is mounted on a vehicle 700.
• the multiple monitoring areas B5a, B5, B5b (see FIG. 1) are provided with overlapping monitoring area areas B55a, B55b, etc. This ensures that no blind spots occur in the multiple monitoring areas B5a, B5, B5b (see FIG. 1). In other words, it becomes possible to continuously construct two-dimensional distance images generated by the multiple light detection devices 5a, 5, 5b, respectively.
• FIG. 3 is a block diagram showing a detailed configuration example of the distance measuring device 20.
• the light emitting devices 10a and 10b have a configuration equivalent to the light emitting device 10
• the distance measuring devices 20a and 20b have a configuration equivalent to the distance measuring device 20.
• the explanation of the light emitting devices 10a and 10b and the distance measuring devices 20a and 20b may be omitted by explaining the light emitting device 10 and the distance measuring device 20.
• FIG. 4 is a diagram showing an example of the configuration of the light-emitting device 10.
• the light-emitting device 10 is configured to emit laser light (irradiation light) L0 toward the target object 50 under control of the overall control unit 30.
• the light-emitting device 10 emits laser light L0 at a predetermined emission cycle by performing a light-emitting operation that alternates between emitting and not emitting light under instructions from the overall control unit 30.
• the light emitting device 10 is, for example, a laser array unit, and has a number of light emitting elements 101 arranged two-dimensionally along the light emitting surface.
• the light emitting device 10 is capable of irradiating the object 50 with laser light. This irradiating light is generated by the light emitted by the number of light emitting elements 101, and is irradiated in a predetermined direction.
• a vertical cavity surface emitting laser (VCSEL) can be used for each light emitting element 101.
• VCSEL vertical cavity surface emitting laser
• the light emitting device 10 according to this embodiment corresponds to a laser light source.
• the light receiving optical system 40 is configured to include a lens that forms an image on the light receiving surface of the distance measuring device 20. Photons emitted from the light emitting device 10 and reflected by the object 50 enter this light receiving optical system 40 as reflected light pulse L1. Note that the reflected light pulse L1 is sometimes referred to as reflected light.
• the distance measuring device 20 includes a pixel array 100, a distance measuring processing unit 110, a distance measuring control unit 120, a drive circuit 130, a light emission timing control unit 140, a control unit 150, a clock generating unit 160, and an output unit 170.
• the distance measuring device 20 is configured to detect reflected light pulses L1 based on instructions from the overall control unit 30. The distance measuring device 20 then generates a distance image based on the detection results, and outputs image data of the generated distance image from the output unit 170 as distance information D1.
• the pixel array 100, the ranging processing unit 110, the ranging control unit 120, the drive circuit 130, the light emission timing control unit 140, the control unit 150, the clock generation unit 160, and the output unit 170 can be arranged on a single semiconductor chip.
• the ranging device 20 may be configured by stacking a first semiconductor chip and a second semiconductor chip. In this case, for example, a configuration in which a part of the pixel array 100 (the photoelectric conversion unit 1001) is arranged on the first semiconductor chip, and other parts included in the ranging device are arranged on the second semiconductor chip is conceivable.
• the overall control unit 30 controls the operation of the entire distance measuring system 1 according to, for example, a pre-installed program.
• the overall control unit 30 can also perform calibration processing such as the control timing and monitoring area of the multiple light detection devices 5a, 5, and 5b, as described below.
• the overall control unit 30 can also execute control according to an external control signal supplied from the outside.
• the control unit 150 controls the operation of the entire distance measuring device 20 according to instructions from the overall control unit 30. Details of the calibration processing will be described later.
• the overall control unit 30 is provided outside the distance measuring device 20, but this is not limited to this.
• the overall control unit 30 and the control unit 150 can be configured in the same element.
• the control unit 150 can have the processing function of the overall control unit 30.
• the clock generation unit 160 generates one or more clock signals used in the distance measurement device 20 based on a reference clock signal supplied from the outside.
• the light emission timing control unit 140 generates a light emission control signal indicating the light emission timing according to a light emission trigger signal supplied from the overall control unit 30.
• the light emission control signal is supplied to the light emitting device 10 and also to the distance measurement processing unit 110.
• the distance measurement control unit 120 controls the operation of the distance measurement processing unit 110 based on instructions from the control unit 150, thereby causing the distance measurement processing unit 110 to generate distance information based on detection signals output from each pixel 1000 of the pixel array 100.
• the pixel array 100 includes a plurality of pixels 1000 arranged in a matrix.
• the pixels 1000 are configured to detect light and generate a detection signal PLS that corresponds to the amount of light detected. Details will be described later with reference to FIG. 7.
• the reflected light L1 is detected using all or a part of the pixel array 100.
• the area used in the pixel array 100 may be a rectangle that is long in a direction perpendicular to the scanning direction (up and down in the drawing, also referred to as the vertical direction hereinafter), which is the same as the image of the reflected light L1 formed on the pixel array 100 when the entire laser light L0 is reflected as the reflected light L1.
• it may be a rectangle that is long in a direction parallel to the scanning direction (left and right in the drawing, also referred to as the horizontal direction hereinafter).
• this is not limited to this, and various modifications may be made, such as a larger or smaller area than the image of the reflected light L1 formed on the pixel array 100.
• the pixel array 100 according to this embodiment corresponds to the pixel array section.
• the drive circuit 130 includes a shift register, an address decoder, etc., and drives each pixel 1000 of the pixel array 100, either all at once or on a column-by-column basis.
• the drive circuit 130 includes at least a circuit that applies a quench voltage V_QCH, described below, to each pixel 1000 in a selected column in the pixel array 100, and a circuit that applies a selection control voltage V_SEL, described below, to each pixel 1000 in the selected column.
• the drive circuit 130 applies the selection control voltage V_SEL to the pixel drive line LD corresponding to the column to be read out, thereby selecting the pixel 1000 to be used to detect the incidence of photons on a column-by-column basis.
• the detection signal output from the pixel array 100 is supplied to the ranging processing unit 110.
• the ranging processing unit 110 includes a TDC unit 111, a histogram generating unit 112, and a signal processing unit 113.
• the detection signal PLS read out from each pixel 1000 is supplied to the TDC unit 111.
• the detection signal is read out at a predetermined sampling period for each pixel column in the pixel array 100, for example, and supplied to the TDC unit 111.
• the TDC unit 111 measures the time difference from a reference timing (for example, the timing when a light emission control signal is input from the light emission timing control unit 140) to the input of the detection signal PLS supplied from the pixel array 100, and generates digital information indicating the measured time difference. In other words, based on the light emission control signal and the detection signal PLS, the TDC unit 111 generates time information indicating the flight time from when light is emitted from the light source unit 10 to when the light is reflected by the object 50 and enters each pixel 1000.
• a reference timing for example, the timing when a light emission control signal is input from the light emission timing control unit 140
• the histogram generating unit 112 generates a histogram based on the time information generated by the TDC unit 111.
• the histogram generating unit 112 counts the time information based on the unit time d set by the distance measurement control unit 120, and generates a histogram.
• the unit time d may be, for example, a time width assigned to one bin in the histogram.
• the unit time d may be, for example, the same time width as the sampling period for reading out a detection signal from each pixel 1000 of the pixel array 100.
• the signal processing unit 113 performs a predetermined calculation process based on the histogram data generated by the histogram generation unit, and calculates, for example, distance information. For example, the signal processing unit 113 creates a curve approximation of the histogram based on the histogram data. The signal processing unit 113 detects the peak of the curve to which this histogram is approximated, and can obtain the distance D to the object 50 based on the detected peak. The signal processing unit 113 can also generate a two-dimensional distance image using the distance information generated based on the histogram of each pixel 1000.
• the distance information and distance image output from the distance measurement processing unit 110 are supplied to the output unit 170.
• the output unit 170 is also called an interface unit, and outputs the distance information and distance image supplied from the distance measurement processing unit to the outside as output data.
• MIPI Mobile Industry Processor Interface
• the overall control unit 30 is configured to control the operation of the distance measurement system 1 by supplying control signals to the light emitting devices 10 and the distance measurement devices 20 and controlling their operation. Note that, as described above, when the overall control unit 30 is configured within the control unit 150, the control unit 150 is configured to control the operation of the distance measurement system 1 by supplying control signals to each of the light emitting devices 10, 10a, 10b and the distance measurement devices 20, 20a, 20b and controlling their operation.
• FIG. 5 is a diagram showing an example of a flash-type optical system.
• the angle of view of the distance measurement system 1 can be fixed, so-called a flash-type optical system.
• the light source 11, the lens 17, the condenser lens 18, and the pixel array 100 are provided.
• the laser light emitted from the light source 11 is converted into a light beam B10 having a necessary and sufficient spread angle through the lens 17, and is irradiated over the entire distance measurement range AR.
• light emission control is possible for each light-emitting element 101 (see FIG. 4). For example, light emission control is also possible in units of one row or one column.
• the flash-type distance measurement system 1 which can measure the entire distance measurement range with a single flash of light, does not require a drive unit 16, half mirror 13, or polygon mirror 14 for scanning the distance measurement range, and therefore has the advantage that the optical system can be made smaller than the scan-type distance measurement system 1 described below.
• FIG. 6 is a diagram showing a schematic configuration of an optical system of the distance measuring system 1 according to this embodiment.
• Fig. 6 shows an example of a so-called scanning type optical system that scans the angle of view of the distance measuring device 20 in the horizontal direction.
• the distance measurement system 1 includes an optical system including a light source 11, a lens 12, a half mirror 13, a polygon mirror 14, a light receiving lens 15, and a pixel array 100.
• the light source 11, the lens 12, the half mirror 13, and the polygon mirror 14 are included in the light emitting device 10 in FIG. 1, for example.
• the light receiving lens 15 is included in the light receiving optical system 40 in FIG. 1.
• the half mirror 13 and the polygon mirror 14 may be shared by the light source unit 10 and the light receiving optical system 40.
• the laser light L0 emitted from the light source 11 is converted by the lens 12 into a rectangular parallel light beam B10 whose cross-sectional intensity spectrum is long in the vertical direction, and then enters the half mirror 13.
• the half mirror 13 reflects a portion of the incident laser light L0.
• the laser light L0 reflected by the half mirror 13 enters the polygon mirror 14.
• the polygon mirror 14 is vibrated in the horizontal direction with a predetermined rotation axis as the vibration center by the drive unit 16, which operates based on control from the overall control unit 30, for example.
• the drive unit 16 can be a MEMS (Micro Electro Mechanical System), a micromotor, or the like.
• the laser light L0 reflected by the polygon mirror 14 is reflected by an object 50 present within the distance measurement range AR and enters the polygon mirror 14 as reflected light L1.
• a portion of the reflected light L1 that enters the polygon mirror 14 passes through the half mirror 13 and enters the light receiving lens 15, thereby forming an image on a specific area in the pixel array 100.
• the specific area may be the entire pixel array 100 or a part of it.
• FIG. 7 is a diagram showing an example of a schematic configuration of the pixel array 100 shown in Fig. 3.
• the pixel array 100 has a plurality of pixels 1000 arranged in rows and columns, and each pixel 1000 includes a photoelectric conversion unit 1001, a quench resistor 1002, a selection transistor 1003, and an inverter 1004.
• the quench resistor 1002 may be formed of a PMOS transistor.
• the photoelectric conversion unit 1001 converts the incident light into an electrical signal by photoelectric conversion and outputs it.
• the photoelectric conversion unit 1001 converts the incident photons into an electrical signal by photoelectric conversion and outputs a pulse according to the incident photons.
• a single photon avalanche diode (SPAD) is used as the photoelectric conversion unit 1001.
• the SPAD has a characteristic that when a large negative voltage that causes avalanche multiplication is applied to the cathode, the electrons generated in response to the incidence of one photon cause avalanche multiplication, causing a large current to flow. By utilizing this characteristic of the SPAD, the incidence of one photon can be detected with high sensitivity.
• the photoelectric conversion unit 1001 corresponds to one specific example of the "photoelectric conversion unit" in this disclosure.
• the photoelectric conversion unit 1001 has a cathode connected to the drain of the quench resistor 1002, and an anode connected to a voltage source of a negative voltage (-Vop) corresponding to the voltage Vbd, which is the breakdown voltage of the photoelectric conversion unit 1001.
• the source of the quench resistor 1002 is connected to a power supply voltage Ve.
• a quench voltage V_QCH is input to the gate of the quench resistor 1002.
• the quench resistor 1002 is a current source that outputs a current corresponding to the power supply voltage Ve and the quench voltage V_QCH from its drain.
• avalanche multiplication begins and a current flows from the cathode to the anode, causing a voltage drop in the photoelectric conversion unit 1001.
• this voltage drop causes the voltage between the cathode and anode of the photoelectric conversion unit 1001 to drop to voltage Vop, avalanche multiplication is stopped (quenching operation).
• the photoelectric conversion unit 1001 is then charged by a current (recharge current) from the quench resistor 1002, which is a current source, and the state of the photoelectric conversion unit 1001 returns to the state it was in before the photon was incident (recharge operation).
• the voltage Vca extracted from the connection point between the drain of the quench resistor 1002 and the cathode of the photoelectric conversion unit 1001 is input to the inverter 1004.
• the inverter 1004 performs a threshold determination on the input voltage Vca based on the threshold voltage Vth, and inverts the output signal Vinv every time the voltage Vca exceeds the threshold voltage Vth in the positive or negative direction.
• the inverter 1004 inverts the signal Vinv at the first timing when the voltage Vca crosses the threshold voltage Vth in the voltage drop caused by avalanche multiplication in response to the incidence of photons on the photoelectric conversion unit 1001.
• the photoelectric conversion unit 1001 is charged by a recharge operation, and the voltage Vca rises.
• the inverter 1004 again inverts the signal Vinv at the second timing when this rising voltage Vca crosses the threshold voltage Vth.
• the width in the time direction between the first timing and the second timing becomes the output pulse in response to the incidence of photons on the photoelectric conversion unit 1001. This output pulse corresponds to the detection signal PLS described in FIG. 1.
• the selection transistor 1003 is, for example, an NMOS transistor, whose drain is connected to the connection point between the drain of the quench resistor 1002 and the cathode of the photoelectric conversion unit 1001, and whose source is connected to a voltage Vg.
• the voltage Vg may be a GND voltage (0 V) or a negative voltage.
• the gate of the selection transistor 1003 is connected to the drive circuit 130, and when a selection control voltage V_SEL from the drive circuit 130 is applied to the gate via the pixel drive line LD, the selection transistor 1003 changes from an off state to an on state.
• the output state of the pixel 1000 operates, for example, as follows. During the period when the selection transistor 1003 is in the off state (disconnected period), the power supply voltage Ve is supplied to the cathode of the photoelectric conversion unit 1001, and therefore, as described above, when a photon is incident on the photoelectric conversion unit 1001, a voltage drop occurs and an output pulse is output from the pixel 1000.
• the pixel 1000 in this state is hereinafter referred to as an active pixel 1200.
• a voltage Vg is applied to the cathode of the photoelectric conversion unit 1001.
• the photoelectric conversion unit 1001 is in a state where no voltage exceeding the breakdown voltage is applied, and even if a photon is incident on the photoelectric conversion unit 1001, no output pulse is output from the pixel 1000.
• the pixel 1000 in this state is hereinafter referred to as an inactive pixel.
• the number of pixels 1000 used to create one histogram may be more than one.
• a set of multiple pixels 1000 used to create one histogram is referred to as a macro pixel 1100 (also called a pixel unit).
• This macro pixel 1100 is composed of, for example, m ⁇ n pixels 1000 (m and n are integers of 2 or more).
• m and n are integers of 2 or more.
• the depth image may be image data in which the value of each pixel is distance information determined based on a histogram.
• [Calibration process] 8 is a block diagram showing a configuration related to the calibration process of the overall control unit 30.
• the overall control unit 30 has a storage unit 300, a first detection unit 302, a second detection unit 304, a third detection unit 306, and a calibration processing unit 308.
• the calibration process of the overall control unit 30 has three inspection modes.
• the first mode is a mode for detecting a clock shift between the light emitting device 10 and the distance measuring device 20.
• the second mode is a mode for detecting a position shift of the multiple distance measuring devices 20, 20a, and 20b.
• the third mode is a mode for detecting and setting the distance measuring timing of the multiple light detecting devices 5, 5a, and 5b.
• each control unit 150 of the distance measuring devices 20, 20a, 20b can be configured to have a memory unit 300, a first detection unit 302, a second detection unit 304, a third detection unit 306, and a calibration processing unit 308.
• the calibration process of each control unit 150 has three inspection modes.
• the first mode is a mode for detecting a clock deviation between the light emitting device 10 and the distance measuring device 20.
• the second mode is a mode for detecting a position deviation of the multiple distance measuring devices 20, 20a, 20b.
• the third mode is a mode for detecting and setting the distance measuring timing of the multiple light detection devices 5, 5a, 5b.
• the storage unit 300 stores a program that executes the detection process. It also stores data for the calibration process or correction process detected in the first to third modes.
• FIG. 9 is a diagram showing a schematic diagram of normal operation of a scanning optical system.
• laser light (irradiated light) L0 is irradiated from the laser light emitting element 101 (see FIG. 4) of the light emitting device 10 toward the object 50, and a light pulse (reflected light pulse L1) reflected by the object 50 enters the pixel array 100 of the distance measuring device 20.
• the light emitting device 10 and the distance measuring device 20 may be controlled according to independent clocks.
• the clock on the light emitting side is 62.5 MHz
• the clock on the light receiving side has a sampling period of 1 GHz.
• FIG. 10 is a diagram showing an example of a histogram during normal operation in FIG. 9.
• the horizontal axis indicates the elapsed time from when the laser light emitting element 101 emits light until the pixel array 100 receives a photon.
• the vertical axis indicates the number of times the pixel array 100 receives a photon.
• the detection signal PLS output from the pixel array 100 corresponds to the measurement time measured by the TDC unit 111.
• the TDC unit 111 measures the time difference from a reference timing (for example, the timing when a light emission control signal is input from the light emission timing control unit 140) until the detection signal PLS supplied from the pixel array 100 is input, and measures the measured time difference.
• This histogram can be generated, for example, by the histogram generation unit 112 (see FIG. 3) based on the measured time difference.
• Histogram ht is a histogram when there is no misalignment between the internal clock on the light-emitting side and the internal clock on the light-receiving side. In other words, this is a case where the light-emitting timing of the laser light-emitting element 101 (see Figure 4) and the measurement start timing of the pixel array 100 match according to the reference timing described above. Histogram hn is an example where the internal clock on the light-receiving side is delayed relative to the internal clock on the light-emitting side. In other words, this is a case where the measurement start timing is delayed compared to the light-emitting timing.
• Histogram hf is an example where the internal clock on the light-receiving side is advanced relative to the internal clock on the light-emitting side. In other words, this is a case where the measurement start timing is advanced compared to the light-emitting timing.
• FIG. 11 is a diagram showing a schematic representation of the measured distance of the object 50.
• 50hn shows a schematic representation of the measured distance of the object 50 when histogram hn is used.
• 50ht shows a schematic representation of the measured distance of the object 50 when histogram ht is used.
• 50hf shows a schematic representation of the measured distance of the object 50 when histogram hf is used.
• the measurement start timing of the pixel array 100 will be delayed with respect to the emission timing of the laser light emitting element 101 (see Figure 4).
• the object 50 is measured as an object 50hn that is closer than the object 50ht (true value).
• the measurement start timing of the pixel array 100 will be ahead of the light emission timing of the laser light emitting element 101 (see FIG. 4).
• the object 50 is measured as an object 50hf that is farther away than the object 50ht.
• the first detection unit 302 detects and calibrates such a deviation between the light emission timing and the measurement start timing.
• FIG. 12 is a diagram showing a schematic example of detection of a deviation between the light emission timing and the measurement start timing.
• the first detection unit 302 controls the orientation of the polygon mirror 14 to a position where a light pulse (reflected light pulse L1) resulting from reflection of the laser light (illumination light) L0 by the polygon mirror 14 directly enters the pixel array 100 of the distance measuring device 20.
• the first detection unit 302 controls the orientation of the polygon mirror 14 to a position where the laser light (illumination light) L0 is reflected within the housing.
• the polygon mirror 14 in this embodiment corresponds to a mirror.
• FIG. 13 is a diagram showing an example histogram in the first mode.
• the horizontal axis indicates the elapsed time from when the laser light emitting element 101 emits light until the pixel array 100 receives a photon.
• the vertical axis indicates the number of times the pixel array 100 receives a photon.
• it indicates the time difference t10 from a reference timing (for example, the timing when a light emission control signal is input from the light emission timing control unit 140) until the detection signal PLS supplied from the pixel array 100 is input to the TDC 111 (see FIG. 3).
• This histogram h10 can be generated, for example, by the histogram generation unit 112 (see FIG. 3) based on the measured time difference.
• the distance K1 from the laser light-emitting element 101 to the polygon mirror 14 and the distance K2 from the polygon mirror 14 to the pixel array 100 are known from design values, etc. Therefore, the difference (t10-t10true) between the time t10true obtained by dividing the distance K1+K2 by the high speed C and the measured time difference t10 corresponds to the timing deviation time.
• t10-t10true is negative, it indicates that the measurement start timing of the pixel array 100 is delayed relative to the emission timing of the laser light-emitting element 101 (see FIG. 4).
• t10-t10true is positive, it indicates that the measurement start timing of the pixel array 100 is advanced relative to the emission timing of the laser light-emitting element 101 (see FIG. 4).
• the first detection unit 302 stores the time difference (t10-t10true) in the storage unit 300.
• the calibration processing unit 308, for example, adds the time difference t10-t10true to the measurement start timing to calibrate the measurement deviation.
• the calibration processing unit 308 adds the time difference t10-t10true to the measurement start timing to calibrate the measurement start timing so that the time difference (t10-t10true) becomes zero.
• the calibration processing unit 308 may cause the histogram generation unit 112 (see FIG. 3) to make corrections based on the time difference t10-t10true when generating a histogram.
• the calibration processing unit 308 may calibrate the light emission timing so that the time difference (t10-t10true) becomes zero.
• the object 50 In the case of a flash-type optical system (see FIG. 5), it is possible to place the object 50 at a known distance during the inspection in the first mode and calculate the time difference (t10-t10true). In other words, the time t10true is calculated by doubling the known distance and dividing by the high speed C. The same applies between the distance measuring devices 20a and 20b and the object 50 placed at a known distance. In this way, it is possible to detect and correct the difference between the emission timing and measurement timing of the distance measuring devices 20, 20a, and 20b.
• the second detection unit 304 executes the detection process in the second mode. Note that, although the following describes a case where the optical axis of the light-emitting device 10, 10a is misaligned, the same process can be performed when the optical axis of the light-emitting device 10, 10b is misaligned.
• FIG. 14 is a diagram showing a case where the optical axes of the light-emitting devices 10 and 10a are misaligned.
• FIG. 14(a) is a diagram showing the light-emitting devices 10 and 10a and the object 50.
• FIG. 14(b) is a diagram showing the overlapping area B55a (see FIG. 2) where the light-emitting devices 10 and 10a overlap.
• FIG. 14(b) shows a case where there is no misalignment, and the object 50 becomes the same object 50, 50a in the overlapping area B55a when a distance image is generated.
• FIG. 14(c) is a diagram showing the overlapping area B55a (see FIG.
• FIG. 15 is a diagram for explaining an example of detection processing in the second mode by the second detection unit 304.
• FIGS. 15(a) to 15(c) are diagrams showing the light emission pattern 14p on the polygon mirror 14. For example, they show a state in which a specific light emitting element 101 is emitting light and the polygon mirror is rotating from time ts to tf.
• 15(d) to 15(f) are schematic diagrams showing the pixel array 100 of the distance measuring device 20, the pixel array 100a of the distance measuring device 20a, and the light receiving state of the light emitting pattern 14p.
• Range A10 in FIG. 15(d) shows the driving area of the pixel array 100a.
• the second detection unit 304 executes a control process to narrow the driving area of the pixel array 100a to driving areas A102 and A104 as the inspection time elapses after capturing the position of the light emitting pattern 14p.
• the driving range A100 of the pixel array 100 is moved from top to bottom or bottom to top in the Y-axis direction.
• the first driving range d1 (entire range in the y-direction) of the pixel array 100 is driven in a time series manner.
• the range of the area in which the histogram generating unit 112 (see FIG. 3) generates a histogram also corresponds to the driving range A100.
• the second detection unit 304 obtains the sum of the detection signals PLS for each pixel 1000 in the pixel array 100 from the histogram generating unit 112, and detects the Y1 coordinate of the driving range A100 that shows the maximum value.
• the driving range A100 of the pixel array 100 is narrowed to driving range A102 and moved from top to bottom or bottom to top.
• the driving range d2 in the Y-axis direction is set to a range narrower than the first driving range d1, based on the Y1 coordinate of the driving range A100 that shows the maximum value. That is, the second detection unit 304 moves the driving range A102 from top to bottom or bottom to top within the second driving range d2 in the Y-axis direction.
• the range of the area in which the histogram generation unit 112 (see FIG. 3) generates a histogram also corresponds to the driving range A102.
• the second detection unit 304 obtains the sum of the detection signals PLS for each pixel 1000 in the pixel array 100 from the histogram generation unit 112, and detects the Y2 coordinate of the driving range A102 that shows the maximum value.
• the driving range A100 of the pixel array 100 is narrowed to driving range A104, and moved from top to bottom or bottom to top.
• the driving range d3 in the Y-axis direction is set to a range narrower than the second driving range d3, based on the Y2 coordinate of the driving range A102 that shows the maximum value.
• the second detection unit 304 moves the driving range A104 from top to bottom or bottom to top within the second driving range d3 in the Y-axis direction.
• the driving range A104 corresponds to the width of the light-emitting pattern 14p.
• the range of the area in which the histogram generating unit 112 (see FIG. 3) generates a histogram also corresponds to the driving range A102.
• the histogram generating unit 112 generates a histogram for each pixel 1000 (see FIG. 7).
• the second detection unit 304 obtains the sum of the detection signals PLS for each pixel 1000 in the pixel array 100 from the histogram generating unit 112, and detects the range corresponding to the pixel 1000 that exceeds the threshold as the detection area P100a. In this way, by narrowing the detection area to A100, A102, and A104, the processing time such as the generation time of the histogram is shortened, and the detection area P100a can be detected more quickly.
• a similar inspection can be performed by emitting light from a predetermined row of light-emitting elements 101 (see FIG. 4) in the light-emitting device 10.
• FIG. 16 is a diagram showing a schematic of coordinate correction data.
• FIG. 16(a) is a diagram showing a case where there is no deviation in coordinates between the pixel array 100 of the distance measuring device 20 and the pixel array 100a of the distance measuring device 20a.
• FIG. 16(b) is a diagram showing a case where there is deviation in coordinates in the x direction between the pixel array 100 of the distance measuring device 20 and the pixel array 100a of the distance measuring device 20a.
• FIG. 16(c) is a diagram showing a case where there is deviation in coordinates in the y direction between the pixel array 100 of the distance measuring device 20 and the pixel array 100a of the distance measuring device 20a.
• the second detection unit 304 stores the difference dx in the storage unit 300.
• the calibration processing unit 308 causes the signal processing unit 113 to generate a new coordinate based on the difference dx for the x coordinate of the pixel array 100a or the x coordinate of the pixel array 100. For example, the calibration processing unit 308 causes the signal processing unit 113 to correspond the position obtained by adding the difference dx to the X coordinate of the pixel array 100 to the x coordinate of the pixel array 100a.
• the calibration processing unit 308 causes the signal processing unit 113 to generate a new coordinate based on the difference dy for the y coordinate of pixel array 100a or the y coordinate of pixel array 100. For example, the calibration processing unit 308 causes the signal processing unit 113 to make the position obtained by adding the difference dy to the y coordinate of pixel array 100 correspond to the y coordinate of pixel array 100a.
• FIG. 17 is a diagram showing an example in which rotational deviation such as yaw, roll, or pitch occurs in the optical system between the distance measuring device 20 and the distance measuring device 20a.
• rotational deviation such as yaw, roll, or pitch occurs in the optical system between the distance measuring device 20 and the distance measuring device 20a.
• the detection area P100a becomes discontinuous.
• the drive area is expanded with the detection area P100a in FIG. 15(f) at the center, and the detection area P100 ⁇ is redetected.
• the second detection unit 304 determines that a rotational deviation has occurred when the detection area P100 ⁇ is discontinuous in the row direction of the pixel array 100a.
• the second detection unit 304 calculates values indicating the degree of rotational deviation such as yaw, roll, and pitch from the pattern of the detection area P100 ⁇ , and stores the values in the memory unit 300.
• Such patterns are determined by the degree of rotational deviation such as yaw, roll, and pitch. Therefore, for example, the second detection unit 304 can correct the X and Y coordinates of the pixel array 100a according to the detected pattern.
• the calibration processing unit 308 causes the signal processing unit 113 to carry out such correction processing. That is, it corrects the coordinates of at least one of the distance images of the distance measuring device 20 and the distance measuring device 20a. This makes it possible to match the positions of both objects 50, 50a, as shown in FIG. 14(b). As described above, the same is true between the distance measuring device 20 and the distance measuring device 20b. It is possible to detect the detection area P100a and detection area P100 ⁇ on the distance measuring device 20 side in the same way as the distance measuring device 20a. Alternatively, when the object 50 for inspection is placed at a predetermined position during measurement in the second mode, the position of the detection area P100a on the distance measuring device 20 side can be calculated without measurement, and therefore may be omitted.
• FIG. 18 is a diagram showing a schematic example of detection in the third mode.
• FIG. 18(a) is a diagram showing the light emission signal P10 of the light emitting device 10.
• the horizontal axis indicates time, and the vertical axis indicates the high level value of the light emission signal P10.
• the light emitting device 10 emits pulsed light in sequence.
• the light emission signal P10 has an interval of 1 us.
• Figure 18(b) is a diagram showing the detection signal PLS of the distance measuring device 20a.
• the horizontal axis indicates time, and the vertical axis indicates the high level value of the detection signal P20a. In other words, when the detection signal P20a is at a high level, the distance measuring device 20a receives reflected light.
• FIG. 18(c) is a diagram showing the light emission signal P10a after adjustment of the light emitting device 10a in the third mode.
• the horizontal axis indicates time, and the vertical axis indicates the high level value of the light emission signal P10a0. In other words, when the light emission signal P10a is at a high level, the light emitting device 10a sequentially emits pulsed light.
• the third detection unit 306 causes the light emitting device 10 to emit pulsed light in accordance with the light emission signal P10. As shown in FIG. 18(b), the third detection unit 306 causes the distance measuring device 20a to generate a detection signal P20a in response to the pulsed light emission of the light emitting device 10.
• the third detection unit 306 sets the light emission pattern of the light emitting device 10a so that the measurements of the distance measuring device 20 and the distance measuring device 20a do not interfere with each other. That is, the third detection unit 306 sets the light emission timing of the light emission signal p10a of the light emitting device 10a, for example, in the middle of the detection signal P20a. In FIG. 18(b), the interval obtained by dividing the time between the detection signals P20a in half corresponds to the time tp10a. The third detection unit 306 stores the time tp10a in the storage unit 300.
• the calibration processing unit 308 then controls the time difference between the light emission timing of the light emitting device 10 and the light emitting device 10a as the time tp10a. Note that inspection and setting processing in the third mode between the light detecting device 5 and the light detecting device 5b are also possible in the same way.
• the third mode can be executed after the detection process in the second mode and the calibration process, allowing for more accurate detection processing. Furthermore, in the detection process in the third mode, the object 50 to be inspected can be placed in the center of the overlapping areas B55a, B55b (see Figure 2) and inspected. This makes it possible to more efficiently suppress interference in the distance measuring process, using the overlapping areas B55a, B55b (see Figure 2) as a reference.
• FIG. 19 is a flowchart showing an example of the detection process according to this embodiment.
• the first detection unit 302 detects the difference between the emission timing of the laser emission element 101 (see FIG. 4) and the measurement start timing of the pixel array 100, and the calibration processing unit 308 executes the calibration process (step S100).
• the second detection unit 304 detects the positional deviation of the multiple distance measuring devices 20, 20a, 20b. Then, when the calibration processing unit 308 generates a distance image for the signal processing unit 113, it causes the signal processing unit 113 to perform coordinate transformation based on the positional deviation (step S102).
• the third detection unit 306 performs an inspection to obtain the detection timing of the distance measuring devices 20a and 20b with respect to the light emitting device 10. Then, the calibration processing unit 308 performs a process to set the light emission timing for the light emitting devices 10a and 10b. (Step S104).
• the overall control unit 30 has three inspection modes. In the first mode, it detects clock deviations between the light emitting devices 10, 10a, 10b and the distance measuring devices 20, 10a, 20b. In the second mode, it detects position deviations between the multiple distance measuring devices 20, 20a, 20b. In the third mode, it detects the light emission timing of the light emitting device 20 using the distance measuring devices 20a, 20b. Then, the calibration processing unit 308 calibrates the clock deviations, corrects the position deviations, and sets the light emission timing of the light emitting devices 10, 10a, 10b. In this way, the timing deviations and position deviations between the multiple light detecting devices 5, 5a, 5b are calibrated, and a decrease in distance measurement accuracy is suppressed.
• the technology according to the present disclosure can be applied to various products.
• the technology according to the present disclosure may be realized as a part mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor).
• FIG. 20 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology disclosed herein can be applied.
• the vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010.
• the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside vehicle information detection unit 7400, an inside vehicle information detection unit 7500, and an integrated control unit 7600.
• the communication network 7010 connecting these multiple control units may be, for example, an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).
• CAN Controller Area Network
• LIN Local Interconnect Network
• LAN Local Area Network
• FlexRay registered trademark
• Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various control target parts.
• Each control unit includes a network I/F for communicating with other control units via a communication network 7010, and a communication I/F for communicating with parts or sensors inside and outside the vehicle by wired or wireless communication.
• the functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I/F 7660, an audio/image output unit 7670, an in-vehicle network I/F 7680, and a storage unit 7690.
• Other control units also include a microcomputer, a communication I/F, a storage unit, and the like.
• the drive system control unit 7100 controls the operation of parts related to the drive system of the vehicle according to various programs.
• the drive system control unit 7100 functions as a control unit for a drive force generating unit for generating the drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking unit for generating the braking force of the vehicle.
• the drive system control unit 7100 may also function as a control unit for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.
• the drive system control unit 7100 is connected to a vehicle state detection unit 7110.
• the vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the rotation speed of the wheels, etc.
• the drive system control unit 7100 performs arithmetic processing using the signal input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering unit, the brake unit, etc.
• the body system control unit 7200 controls the operation of various parts installed in the vehicle body according to various programs.
• the body system control unit 7200 functions as a control unit for a keyless entry system, a smart key system, a power window section, or various lamps such as headlamps, tail lamps, brake lamps, turn signals, and fog lamps.
• radio waves or signals from various switches transmitted from a portable device that replaces a key can be input to the body system control unit 7200.
• the body system control unit 7200 accepts the input of these radio waves or signals and controls the door lock section, power window section, lamps, etc. of the vehicle.
• the battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, according to various programs. For example, information such as the battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from the battery section equipped with the secondary battery 7310. The battery control unit 7300 performs calculations using these signals, and controls the temperature regulation of the secondary battery 7310 or the cooling section equipped in the battery section.
• the outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000.
• the imaging unit 7410 and the outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400.
• the imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras.
• the outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., around the vehicle equipped with the vehicle control system 7000.
• the environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunlight, and a snow sensor that detects snowfall.
• the surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar unit, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) unit.
• the imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or unit, or may be provided as a unit in which multiple sensors or units are integrated.
• FIG. 21 shows an example of the installation positions of the imaging unit 7410 and the outside vehicle information detection unit 7420.
• the imaging units 7910, 7912, 7914, 7916, 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle cabin of the vehicle 7900.
• the imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900.
• the imaging units 7912, 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900.
• the imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900.
• the imaging unit 7918, which is installed on the top of the windshield inside the vehicle is primarily used to detect preceding vehicles, pedestrians, obstacles, traffic signals, traffic signs, lanes, etc.
• FIG. 21 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916.
• Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose
• imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively
• imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door.
• image data captured by the imaging units 7910, 7912, 7914, and 7916 are superimposed to obtain an overhead image of the vehicle 7900.
• External information detection units 7920, 7922, 7924, 7926, 7928, 7930 provided on the front, rear, sides, corners, and upper part of the windshield inside the vehicle 7900 may be, for example, ultrasonic sensors or radar units.
• External information detection units 7920, 7926, 7930 provided on the front nose, rear bumper, back door, and upper part of the windshield inside the vehicle 7900 may be, for example, LIDAR units. These external information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.
• the outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle, and receives the captured image data.
• the outside-vehicle information detection unit 7400 also receives detection information from the connected outside-vehicle information detection unit 7420. If the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar unit, or a LIDAR unit, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves, and receives information on the received reflected waves.
• the outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface, based on the received information.
• the outside-vehicle information detection unit 7400 may perform environmental recognition processing for recognizing rainfall, fog, road surface conditions, etc., based on the received information.
• the outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.
• the outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface based on the received image data.
• the outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410.
• the outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different imaging units 7410.
• the in-vehicle information detection unit 7500 detects information inside the vehicle.
• a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500.
• the driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound inside the vehicle.
• the biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or a driver gripping the steering wheel.
• the in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off.
• the in-vehicle information detection unit 7500 may perform processing such as noise canceling on the collected sound signal.
• the integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs.
• the integrated control unit 7600 is connected to an input unit 7800.
• the input unit 7800 is realized by a unit that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600.
• the input unit 7800 may be, for example, a remote control unit using infrared or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000.
• PDA Personal Digital Assistant
• the input unit 7800 may be, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable unit worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by a passenger or the like using the input unit 7800 and outputs the signal to the integrated control unit 7600. The passenger or the like operates the input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.
• the storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc.
• the storage unit 7690 may also be realized by a magnetic storage device such as a HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device, etc.
• the general-purpose communication I/F 7620 is a general-purpose communication I/F that mediates communication between various devices present in the external environment 7750.
• the general-purpose communication I/F 7620 may implement cellular communication protocols such as GSM (registered trademark) (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-A (LTE-Advanced), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark).
• GSM Global System of Mobile communications
• WiMAX registered trademark
• LTE registered trademark
• LTE-A Long Term Evolution
• Bluetooth registered trademark
• the general-purpose communication I/F 7620 may connect to devices (e.g., application servers or control servers) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point.
• the general-purpose communication I/F 7620 may be connected to a terminal located near the vehicle (e.g., a driver's, pedestrian's, or store's terminal, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.
• the dedicated communication I/F 7630 is a communication I/F that supports a communication protocol developed for use in vehicles.
• the dedicated communication I/F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of the lower layer IEEE 802.11p and the higher layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol.
• WAVE Wireless Access in Vehicle Environment
• DSRC Dedicated Short Range Communications
• the dedicated communication I/F 7630 performs V2X communication, which is a concept that includes one or more of, for example, vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.
• the positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites) and generates position information including the latitude, longitude, and altitude of the vehicle.
• GNSS Global Navigation Satellite System
• GPS Global Positioning System
• the positioning unit 7640 may determine the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.
• the beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations installed on the road, and acquires information such as the current location, congestion, road closures, and travel time.
• the functions of the beacon receiver 7650 may be included in the dedicated communication I/F 7630 described above.
• the in-vehicle device I/F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle.
• the in-vehicle device I/F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB).
• the in-vehicle device I/F 7660 may also establish a wired connection such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown.
• USB Universal Serial Bus
• HDMI registered trademark
• MHL Mobile High-Definition Link
• the in-vehicle device 7760 may include, for example, at least one of a mobile device or wearable device owned by the passenger, or an information device carried into or attached to the vehicle.
• the in-vehicle device 7760 may also include a navigation unit that searches for a route to an arbitrary destination.
• the in-vehicle device I/F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.
• the in-vehicle network I/F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010.
• the in-vehicle network I/F 7680 transmits and receives signals in accordance with a specific protocol supported by the communication network 7010.
• the microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680.
• the microcomputer 7610 may calculate the control target value of the driving force generating unit, the steering mechanism, or the braking unit based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100.
• the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.
• ADAS Advanced Driver Assistance System
• the microcomputer 7610 may control the driving force generating unit, steering mechanism, braking unit, etc. based on the acquired information about the surroundings of the vehicle, thereby performing cooperative control for the purpose of automatic driving, which allows the vehicle to travel autonomously without relying on the driver's operation.
• the microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and generate local map information including information about the surroundings of the vehicle's current position, based on information acquired via at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle equipment I/F 7660, and the in-vehicle network I/F 7680.
• the microcomputer 7610 may also predict dangers such as vehicle collisions, the approach of pedestrians, or entry into closed roads, based on the acquired information, and generate warning signals.
• the warning signals may be, for example, signals for generating warning sounds or turning on warning lights.
• the audio/image output unit 7670 transmits at least one output signal of audio and image to an output unit capable of visually or audibly notifying the vehicle occupants or the outside of the vehicle of information.
• an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output units.
• the display unit 7720 may include, for example, at least one of an on-board display and a head-up display.
• the display unit 7720 may have an AR (Augmented Reality) display function.
• the output unit may be other units such as headphones, a wearable device such as a glasses-type display worn by the occupant, a projector, or a lamp, in addition to these units.
• the display unit visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, graphs, etc. Furthermore, if the output unit is an audio output unit, the audio output unit converts an audio signal consisting of reproduced voice data or acoustic data, etc., into an analog signal and outputs it audibly.
• control units connected via the communication network 7010 may be integrated into one control unit.
• each control unit may be composed of multiple control units.
• the vehicle control system 7000 may include another control unit not shown.
• some or all of the functions performed by any of the control units may be provided by the other control units.
• a predetermined calculation process may be performed by any of the control units.
• a sensor or part connected to any of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.
• a computer program for implementing each function of the distance measuring system 1 according to this embodiment described with reference to FIG. 1 can be implemented in any of the control units, etc.
• a computer-readable recording medium on which such a computer program is stored can also be provided.
• the recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc.
• the above computer program may be distributed, for example, via a network, without using a recording medium.
• the distance measurement system 1 according to this embodiment described using FIG. 1 can be applied to the external information detection unit 7400 and the external information detection section 7420 of the application example shown in FIG. 20.
• This technology can be configured as follows:
• a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal; a control unit capable of executing at least one of a process of detecting a time lag between the light emission timing and a generation timing of the detection signal, a process of detecting a deviation of coordinates of the plurality of pixels, and a process of setting the light emission timing;
• a light detection device comprising:
• control unit detects the difference between the time obtained by dividing the sum of a first distance from the laser light source to the position and a second distance from the position to the pixel array unit by the speed of light and the measured time difference as the deviation time.
• the signal processing unit corrects the distance value based on the deviation time.
• a mirror for changing the direction of irradiation of the laser light is further provided.
• the object is the mirror,
• the plurality of pixels are arranged in a matrix
• the control unit When executing the coordinate deviation detection process, the control unit: The optical detection device described in (1), wherein a laser light of a predetermined shape is irradiated onto the object from a second laser light source different from the laser light source, and the deviation of the coordinates is detected based on a first position of the reflected light of the laser light of the predetermined shape with respect to the multiple pixels.
• control unit receives the reflected light of the laser light of the predetermined shape in a second pixel array unit different from the pixel array unit, and detects the coordinate deviation based on the difference between a second position of the reflected light of the laser light of the predetermined shape for multiple pixels of the second pixel array unit and the first position.
• the signal processing unit is capable of generating a two-dimensional distance image based on detection signals of the plurality of pixels;
• the laser light of a predetermined shape is reflected as a rectangular light pattern along the row direction of the pixel array section, the pixel array unit is capable of partially driving each of a plurality of rectangular regions each having a first side in the row direction and a second side in a column direction perpendicular to the row direction; the control unit drives a portion of the pixel array unit for each rectangular region when detecting the reflected light; a region having a highest light receiving amount selected from the plurality of regions, and detecting the first position based on the region having the highest light receiving amount;
• An optical detection device according to (9).
• control unit selects the region with the highest amount of light received from the multiple regions based on the detection signal, then partially drives the pixel array unit for each of the multiple rectangular second regions with the second side shorter, selects the second region with the highest amount of light received from the multiple second regions, and detects the first position based on the second region with the highest amount of light received.
• the control unit is The photodetection device of claim 11, further comprising: a step of partially driving the pixel array section for each of a plurality of rectangular third regions having a shorter second side in a range of the pixel array section limited based on the second region having the highest amount of light received; selecting the third region having the highest amount of light received from the plurality of third regions; and detecting the first position based on the third region having the highest amount of light received.
• the control unit is The optical detection device described in (10) detects that a rotational misalignment occurs between the irradiation optical system of the second laser light source and the receiving optical system of the pixel array unit when pixels in the region that generate a detection signal equal to or greater than a predetermined value are discontinuous in the row direction.
• control unit executes the light emission timing setting process, Irradiating the target with laser light at a predetermined interval from a second laser light source different from the laser light source; setting the light emission timing of the laser light source based on the generation timing of the detection signal; An optical detection device as described in (1).
• a distance measuring system including a plurality of light detection devices having overlapping irradiation ranges of laser light
• Each of the plurality of photodetection devices includes: a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal, At least one of the plurality of light detection devices further comprises: A distance measuring system having a control unit capable of performing at least one of a process of detecting a time difference between the light emission timing and the generation timing of the detection signal, a process of detecting a difference in the coordinates of the multiple pixels, and a process of setting the light emission timing.
• a ranging system comprising: Each of the plurality of photodetection devices includes: a laser light source that irradiates a target object with a laser light based on a light emission timing; a pixel array unit having a plurality of pixels each capable of generating a detection signal corresponding to an amount of light reflected from the object; a signal processing unit that generates a distance value to the object based on information about a difference between the light emission timing and the generation timing of the detection signal,
• the overall control unit includes: a distance measuring system capable of executing at least one of a process of detecting a time difference between the light emission timing and a generation timing of the detection signal, a process of detecting a difference in coordinates of the plurality of pixels, and a process of setting the light emission timing.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• General Physics & Mathematics (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Electromagnetism (AREA)
• Optical Radar Systems And Details Thereof (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=92842010

Family Applications (1)

Country Status (2)

Cited By (1)

Citations (9)

• 2024
  • 2024-03-06 JP JP2025508296A patent/JPWO2024195531A1/ja active Pending
  • 2024-03-06 WO PCT/JP2024/008430 patent/WO2024195531A1/ja not_active Ceased

Patent Citations (9)

Also Published As

Similar Documents

Legal Events