WO2021015208A1 - アクティブセンサ、ゲーティングカメラ、自動車、車両用灯具 - Google Patents

アクティブセンサ、ゲーティングカメラ、自動車、車両用灯具 Download PDF

Info

Publication number
WO2021015208A1
WO2021015208A1 PCT/JP2020/028325 JP2020028325W WO2021015208A1 WO 2021015208 A1 WO2021015208 A1 WO 2021015208A1 JP 2020028325 W JP2020028325 W JP 2020028325W WO 2021015208 A1 WO2021015208 A1 WO 2021015208A1
Authority
WO
WIPO (PCT)
Prior art keywords
wavelength
sensor
subject
probe light
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/028325
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
▲高▼橋　昌之
晃志 伊多波
大騎 加藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koito Manufacturing Co Ltd
Original Assignee
Koito Manufacturing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koito Manufacturing Co Ltd filed Critical Koito Manufacturing Co Ltd
Priority to JP2021534052A priority Critical patent/JPWO2021015208A1/ja
Publication of WO2021015208A1 publication Critical patent/WO2021015208A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481—Constructional features, e.g. arrangements of optical elements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/484—Transmitters

Definitions

  • the present invention relates to an active sensor.
  • An object identification system that senses the position and type of objects existing around the vehicle is used for automatic driving and automatic control of the light distribution of headlamps.
  • the object identification system includes a sensor and an arithmetic processing unit that analyzes the output of the sensor.
  • the sensor is selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter-wave radar, ultrasonic sonar, etc. in consideration of application, required accuracy, and cost.
  • Depth information cannot be obtained from a general monocular camera. Therefore, when a plurality of objects located at different distances overlap, it is difficult to separate them.
  • the TOF camera is known as a camera that can obtain depth information.
  • a TOF (TimeOfFlight) camera emits infrared light with a light emitting device, measures the flight time until the reflected light returns to the image sensor, and obtains an image obtained by converting the flight time into distance information. is there.
  • the applicant has proposed a sensor (hereinafter referred to as a gating camera in the present specification) as an alternative to the TOF camera (Patent Documents 1 and 2).
  • the gating camera divides the shooting range into a plurality of ranges, changes the exposure timing and the exposure time for each range, and captures images a plurality of times. As a result, images are obtained for each target range, and each image contains only objects included in the corresponding range.
  • the present inventor initially considered using near infrared light having a wavelength of around 800 nm as the probe light of the gating camera.
  • Image sensors such as general CMOS sensors and CCD cameras have a sensitivity of 800 nm, so images can be acquired with a simple configuration.
  • 800 nm near infrared rays are also included in daytime sunlight. Therefore, when 800 nm probe light is used, the use of the gating camera is limited to nighttime.
  • LiDAR can be used for daytime photography even if near infrared rays or visible light is used as the probe light. This is due to the fact that it is possible to scan and photograph a spot of laser light having a higher intensity than ambient light, instead of photographing the entire object at once.
  • An aspect of the present invention has been made in such a situation, and one of its exemplary purposes is to provide an active sensor that can be used day and night.
  • Another aspect of the present invention has been made in view of the above problem, and one of its exemplary purposes is to address at least one problem due to the dependence of the brightness of the subject on the position of the depth direction in the sliced image.
  • the solution is to provide a gating camera.
  • the active sensor includes a lighting device that irradiates an infrared probe light, a sensor, and a controller that controls the irradiation of the probe light by the lighting device and the detection timing by the sensor.
  • the wavelength of the probe light is included in the wavelength range of the dip included in the spectrum of sunlight on the surface of the earth.
  • One aspect of the present invention relates to a gating camera that divides a plurality of ranges in the depth direction and generates a plurality of slice images corresponding to the plurality of ranges.
  • the gating camera straddles an illumination device that irradiates probe light, an image sensor, a controller that controls the irradiation of probe light by the illumination device, and the exposure timing by the image sensor, and two adjacent slice images.
  • the subject is set as a correction target, and an image processing unit for correcting the subject to be corrected is provided.
  • an active sensor that can be used day and night. Further, according to an aspect, at least one of the problems caused by the dependence of the brightness of the subject on the position in the depth direction in the sliced image can be solved.
  • FIG. It is a block diagram of the object identification system including the gating camera which concerns on Embodiment 1.
  • FIG. It is a figure explaining the operation of a gating camera.
  • 3 (a) and 3 (b) are diagrams illustrating slice images obtained by a gating camera. It is a figure which shows the spectrum of sunlight.
  • 5 (a) and 5 (b) are diagrams showing a configuration example of a camera.
  • 8 (a) and 8 (b) are diagrams for explaining an image obtained by a gating camera. It is a figure which shows the sensitivity characteristic in the depth direction of a gating camera.
  • 10 (a) and 10 (b) are diagrams illustrating slice images obtained by a gating camera.
  • 11 (a) to 11 (d) are diagrams for explaining the correction methods 1 to 3.
  • 12 (a) to 12 (d) are diagrams for explaining the correction method 4. It is a figure which shows the lamp for the vehicle which built in the gating camera. It is a block diagram which shows the lighting equipment for a vehicle provided with an object detection system.
  • the active sensor includes a lighting device that irradiates an infrared probe light, a sensor, and a controller that controls the irradiation of the probe light by the lighting device and the detection timing by the sensor.
  • the wavelength of the probe light is included in the wavelength range of the dip included in the spectrum of sunlight on the surface of the earth.
  • the light emitted by sunlight contains spectral components ranging from ultraviolet rays of 300 nm to around 3000 nm, but some of the spectral components are absorbed and scattered by substances in the atmosphere, so they do not reach the surface of the earth. Therefore, the spectrum of sunlight observed on the surface of the earth has a dip at a specific wavelength. In one embodiment, by using the wavelength of this dip as probe light, sensing can be performed even in the daytime without being affected by ambient light.
  • the sensor may be an imaging device, and the active sensor may be a gating camera that divides the depth direction into a plurality of ranges and generates a plurality of slice images corresponding to the plurality of ranges.
  • the sensor may include an image sensor, a bandpass filter that transmits the wavelength of the probe light, and a wavelength conversion element that converts the transmitted light of the bandpass filter into a wavelength having high sensitivity of the image sensor. This eliminates the need to match the sensitivity wavelength of the image sensor with the wavelength of the probe light. As a result, there are many choices when selecting the light source of the image sensor and the lighting device from commercially available products, or when designing them independently, the degree of freedom in design can be significantly increased. Since the total detection sensitivity of the active sensor is determined by the product of the wavelength conversion efficiency and the light receiving sensitivity of the image sensor, it is possible to increase the total detection sensitivity by optimally designing each of the wavelength conversion element and the image sensor. Become.
  • the wavelength of the probe light may be the absorption wavelength of water molecules in the vicinity of 940 nm.
  • the wavelength of the probe light may be the absorption wavelength of water molecules in the vicinity of 1120 nm.
  • the gating camera straddles an illumination device that irradiates probe light, an image sensor, a controller that controls the irradiation of probe light by the illumination device, and the exposure timing by the image sensor, and two adjacent slice images.
  • an illumination device that irradiates probe light
  • an image sensor that controls the irradiation of probe light by the illumination device, and the exposure timing by the image sensor, and two adjacent slice images.
  • the subject is set as a correction target, and an image processing unit for correcting the subject to be corrected is provided.
  • the pixel values of the subjects to be corrected for each of the two adjacent sliced images may be combined.
  • a predetermined coefficient may be multiplied by the pixel value of the subject to be corrected.
  • FIG. 1 is a block diagram of an object identification system 10 including the gating camera 20 according to the first embodiment.
  • This object identification system 10 is mounted on a vehicle such as an automobile or a motorcycle, and determines the type (also referred to as a category or class) of an object OBJ existing around the vehicle.
  • the object identification system 10 mainly includes a gating camera 20 and an arithmetic processing unit 40.
  • the gating camera 20 divides a plurality of N (N ⁇ 2) ranges RNG 1 to RNG N in the depth direction and performs imaging. Adjacent ranges may overlap in the depth direction at their boundaries.
  • the gating camera 20 includes a lighting device 22, a camera 24, and a controller 26.
  • the lighting device 22 irradiates the probe light L1 to the front of the vehicle in synchronization with the projection timing signal S1 given from the controller 26.
  • the probe light L1 is preferably infrared light.
  • the camera 24 is configured to be capable of exposure control synchronized with the shooting timing signal S2 given from the controller 26 and to be able to generate a slice image IMG.
  • the camera 24 has sensitivity to the same wavelength as the probe light L1, and photographs the reflected light (return light) L2 reflected by the object OBJ.
  • the controller 26 holds a predetermined light projection timing and exposure timing for each range RNG.
  • the controller 26 When a certain range RNG i is photographed, the controller 26 generates a projection timing signal S1 and an imaging timing signal S2 based on the projection timing and the exposure timing corresponding to the range, and performs imaging.
  • the gating camera 20 can generate a plurality of slice images IMG 1 to IMG N corresponding to a plurality of ranges RNG 1 to RNG N , and the i-th slice image IMG i is included in the corresponding range RNG i . Only the objects to be captured will be captured.
  • FIG. 2 is a diagram illustrating the operation of the gating camera 20.
  • FIG. 2 shows a state when the i-th range RNG i is measured.
  • the lighting device 22 emits light during the light emission period ⁇ 1 between the times t 0 and t 1 in synchronization with the light projection timing signal S1.
  • a diagram of rays with time on the horizontal axis and distance on the vertical axis is shown. From gating camera 20, the distance d MINi up before the boundary of the range RNG i, the distance to the far side of the boundary of the range RNG i and d MAXi.
  • T MINI 2 ⁇ d MINI / c Is. c is the speed of light.
  • TMAXi 2 ⁇ d MAXi / c Is.
  • the controller 26 may repeat the above-mentioned exposure operation a plurality of times in a predetermined period ⁇ 2 .
  • the slice image actually taken includes an object included in a narrow region (indicated by p in FIG. 2) in front of the distance d MINI .
  • the image is reflected with a relatively lower exposure than the object included in the range RNG i .
  • an object included in a narrow region (indicated by q in FIG. 2) behind the distance d MAXi is also reflected with a relatively low exposure.
  • FIG. 3 (a) and 3 (b) are diagrams illustrating slice images obtained by the gating camera 20.
  • the object to the range RNG 2 (pedestrian) OBJ 2 are present, there is an object (vehicle) OBJ 3 to the range RNG 3.
  • FIG. 3 (b) shows a plurality of slice images IMG 1 to IMG 3 obtained in the situation of FIG. 3 (a).
  • the image sensor is exposed only by the reflected light from the range RNG 1, so that no object image is captured in the slice image IMG 1 .
  • the image sensor When the slice image IMG 2 is photographed, the image sensor is exposed only by the reflected light from the range RNG 2, so that only the object image OBJ 2 is captured in the slice image IMG 2 . Similarly, when the slice image IMG 3 is photographed, the image sensor is exposed only by the reflected light from the range RNG 3, so that only the object image OBJ 3 is captured in the slice image IMG 3 . In this way, according to the gating camera 20, it is possible to separate and shoot an object for each range.
  • the arithmetic processing unit 40 is configured to be able to identify the type of the object based on the plurality of slice images IMG 1 to IMG N corresponding to the plurality of ranges RNG 1 to RNG N obtained by the gating camera 20.
  • the arithmetic processing unit 40 includes a classifier 42 that is implemented based on a prediction model generated by machine learning.
  • the algorithm of the classifier 42 is not particularly limited, but YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), R-CNN (Region-based Convolutional Neural Network), SPPnet (Spatial Pyramid Pooling), Faster R-CNN. , DSSD (Deconvolution -SSD), Mask R-CNN, etc. can be adopted, or algorithms developed in the future can be adopted.
  • the arithmetic processing unit 40 can be implemented by combining a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer, and a software program executed by the processor (hardware).
  • a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer
  • the arithmetic processing unit 40 may be a combination of a plurality of processors. Alternatively, the arithmetic processing unit 40 may be configured only by hardware.
  • FIG. 4 is a diagram showing the spectrum of sunlight.
  • FIG. 4 shows a spectrum (i) outside the atmosphere and a spectrum (ii) near the surface of the earth. Outside the atmosphere, a blackbody radiation spectrum with a temperature of about 5800 K is observed, and its wavelength range extends from the ultraviolet of 300 nm to around 3000 nm. Some of the spectral components of sunlight do not reach the surface of the earth because they are absorbed and scattered by substances in the atmosphere. Therefore, the spectrum (ii) of sunlight observed on the surface of the earth has dips ( ⁇ 1, ⁇ 2, ⁇ 3 ...) At specific wavelengths.
  • All of ⁇ 1, ⁇ 2, and ⁇ 3 correspond to the absorption wavelength of water, and ⁇ 1 ⁇ 940 nm, ⁇ 2 ⁇ 1120 nm, and ⁇ 3 ⁇ 1820 nm.
  • the wavelength ⁇ p of the probe light L1 is included in any of the wavelength ranges ⁇ 1, ⁇ 2, and ⁇ 3 of the dip included in the spectrum of sunlight on the ground surface.
  • the camera 24 of FIG. 5A includes a lens 50, a bandpass filter 52, and an image sensor 54.
  • the lens 50 forms an image on the imaging surface of the image sensor 54.
  • the bandpass filter 52 transmits the wavelength of the probe light ⁇ p and removes other wavelengths.
  • the image sensor 54 has sensitivity to the wavelength ⁇ p and images the light transmitted through the bandpass filter 52.
  • CMOS sensors and CCD cameras are generally equipped with an infrared filter (IR), and can generate an image when the wavelength ⁇ p of the probe light L1 is infrared. Can not. Therefore, when adopting the configuration of FIG. 5A, it is necessary to adopt an image sensor from which the infrared filter has been removed.
  • IR infrared filter
  • the image sensor 54 has no sensitivity to the wavelength ⁇ p, for example, a wavelength ⁇ c different from ⁇ p.
  • ⁇ c is not particularly limited, but may be a visible region.
  • the camera 24 further includes a wavelength conversion element 56 in front of the image sensor 54, and the wavelength conversion element 56 converts the light transmitted through the bandpass filter 52 into the wavelength ⁇ c.
  • the material of the wavelength conversion element 56 is not particularly limited, and may be selected based on the relationship between ⁇ p and ⁇ c. When ⁇ c> ⁇ p, the up-conversion element is selected.
  • the total detection sensitivity is determined by the product of the wavelength conversion efficiency and the light receiving sensitivity of the image sensor. Therefore, by optimally designing each of the wavelength conversion element and the image sensor, In some cases, the total detection sensitivity can be increased as compared with the camera 24 of FIG. 5A.
  • the above is the configuration of the gating camera 20.
  • the gating camera 20 by matching the wavelength of the probe light L1 with a wavelength that is not easily affected by sunlight, sensing can be performed even in the daytime without being affected by ambient light.
  • FIG. 6 is a block diagram of the gating camera 20 according to the second embodiment.
  • the gating camera 20 performs imaging by dividing into a plurality of N (N ⁇ 2) ranges RNG 1 to RNG N in the depth direction.
  • the gating camera 20 includes a lighting device 22, an image sensor 24, a controller 26, and an image processing unit 28.
  • the lighting device 22 irradiates the probe light L1 to the front of the vehicle in synchronization with the projection timing signal S1 given from the controller 26.
  • the probe light L1 is preferably infrared light, but is not limited to this, and may be visible light having a predetermined wavelength.
  • the image sensor 24 is configured to be capable of exposure control synchronized with the shooting timing signal S2 given from the controller 26 and to be able to generate a slice image IMG.
  • the image sensor 24 has sensitivity to the same wavelength as the probe light L1, and photographs the reflected light (return light) L2 reflected by the object OBJ.
  • the controller 26 holds a predetermined light projection timing and exposure timing for each range RNG.
  • the controller 26 When a certain range RNG i is photographed, the controller 26 generates a projection timing signal S1 and an imaging timing signal S2 based on the projection timing and the exposure timing corresponding to the range, and performs imaging.
  • the gating camera 20 can generate a plurality of slice images IMG 1 to IMG N corresponding to a plurality of ranges RNG 1 to RNG N.
  • the i-th slice image IMG i shows an object included in the corresponding range RNG i .
  • FIG. 7 is a diagram illustrating the operation of the gating camera 20.
  • FIG. 7 shows a state when the i-th range RNG i is measured.
  • the lighting device 22 emits light during the light emission period ⁇ 1 between the times t 0 and t 1 in synchronization with the light projection timing signal S1.
  • a diagram of rays with time on the horizontal axis and distance on the vertical axis is shown. From gating camera 20, the distance d MINi up before the boundary of the range RNG i, the distance to the far side of the boundary of the range RNG i and d MAXi.
  • T MINI 2 ⁇ d MINI / c Is. c is the speed of light.
  • TMAXi 2 ⁇ d MAXi / c Is.
  • a plurality of exposures may be repeated (multiple exposure).
  • the controller 26 may repeat the above-mentioned exposure operation a plurality of times in a predetermined period ⁇ 2 .
  • 8 (a) and 8 (b) are diagrams for explaining an image obtained by the gating camera 20.
  • the object (pedestrian) on the range RNG 1 OBJ 1 is present, there is an object (vehicle) OBJ 3 to the range RNG 3.
  • 8 (b) shows a plurality of slice images IMG 1 to IMG 3 obtained in the situation of FIG. 8 (a).
  • the image sensor is exposed only by the reflected light from the range RNG 1, so that the slice image IMG 1 shows the object image OBJ 1 of the pedestrian OBJ 1 .
  • the image sensor When capturing the sliced image IMG 2 , the image sensor is exposed by the reflected light from the range RNG 2 , and therefore no object image is captured in the sliced image IMG 2 .
  • the image sensor is exposed by the reflected light from the range RNG 3, so that only the object image OBJ 3 is captured in the slice image IMG 3 .
  • the gating camera 20 it is possible to separate and shoot an object for each range.
  • the above is the basic operation of the gating camera 20.
  • FIG. 9 is a diagram showing the sensitivity characteristics of the gating camera 20 in the depth direction.
  • the horizontal axis represents the position of the object in the depth direction, and the vertical axis represents the detection intensity (that is, the pixel value) when an object having a certain reflectance is assumed.
  • FIG. 10A is a diagram showing an exemplary shooting scene.
  • d MINi the i th range RNG i is, i-1 th range RNG i-1 consistent with d MAXi-1 '
  • d MINi the i th range RNG i matches a d MAXi-1 of the (i-1) -th range RNG i-1
  • i th range RNG i D MAXi may match d MINi + 1 in the i + 1th range RNG i + 1 .
  • the shooting range RNG i ' contains three objects OBJ 1 ⁇ OBJ 3.
  • OBJ 1 is a pedestrian facing forward, existing in the front range p i.
  • the OBJ 2 is a front-facing vehicle and exists in the highly sensitive range RNG i .
  • OBJ 3 is a pedestrian facing horizontal, present in the range q i sensitivity lower rear side.
  • FIG. 10B is a diagram showing slice images IMG i-1 , IMG i , and IMG i + 1 obtained in the shooting scene of FIG. 10A. Here, it is assumed that the reflectances of all objects are equal. In FIG. 10B, dense hatching indicates bright pixels and coarse hatching indicates dark pixels.
  • the object OBJ 2 appears bright in the slice image IMG i corresponding to the range RNG i .
  • the objects OBJ 1 and OBJ 3 in the foreground and the back are darkened.
  • the object OBJ 2 is emphasized, and the objects OBJ 1 and OBJ 3 are difficult to identify. Further, since the objects OBJ 1 and OBJ 3 are included in both of the two slice images, there arises a problem that their positions in the depth direction are unclear. These are due to the fact that the brightness of the subject in the sliced image has a dependence on the position in the depth direction.
  • the gating camera 20 has a function of correcting the slice image IMG in order to solve such a problem.
  • the image processing unit 28 sets the subject as a correction target, corrects the correction target subject, and outputs the corrected slice image IMG_CORR. ..
  • the object OBJ 1 is included in the two adjacent slice images IMG i-1 and IMG i , it is a subject to be corrected.
  • the object OBJ 3 is included in the two adjacent slice images IMG i and IMG i + 1 , it is a subject to be corrected.
  • the object OBJ 2 is not a correction target because it is reflected in only one slice image IMG i .
  • the image processing unit 28 selects a plurality of slice images IMG 1 to IMG N as inspection targets in order, detects a subject included in the slice image to be inspected, and the detected subject is a slice image before and after the slice image to be inspected. It may be determined whether or not it is included in.
  • the above is the configuration of the gating camera 20.
  • the subject is located near the boundary between the two adjacent ranges corresponding to the two adjacent slice images. Therefore, when such a subject is detected, at least one of the problems caused by the dependence of the brightness of the subject on the position in the depth direction in the sliced image can be solved by correcting the subject.
  • 11 (a) to 11 (d) are diagrams for explaining the correction methods 1 to 3.
  • the correction method 1 will be described with reference to FIGS. 11A and 11B.
  • the correction target is the i-th slice image IMG i
  • FIG. 11A shows the slice image IMG i to be corrected and the slice images IMG i-1 and IMG i + 1 before and after the correction target.
  • FIG. 11B shows the corrected i-th slice image IMG_CORR i obtained by the correction method 1.
  • the objects OBJ 1 , OBJ 2 , and OBJ 3 are rectangles of 2 ⁇ 2 pixels, 3 ⁇ 2 pixels, and 1 ⁇ 2 pixels, respectively, and pixel values are shown in the pixels.
  • the pixel values of the subjects to be corrected of the two adjacent sliced image IMGs including the object OBJ k are combined.
  • the pixels for each pixel constituting the object OBJ k of the slice image IMG i to be corrected The pixel value of the previous slice image IMG i-1 is combined with the value.
  • the pixel value u'of the subject OBJ 1 of the sliced image IMG_CORR i after correction can be the sum (simple addition) of the pixel values u and v of the subject OBJ 1 in each of the IMG i and IMG i-1 .
  • u' u + v
  • Weighted addition may be used instead of simple addition.
  • ⁇ and ⁇ are weighting coefficients.
  • u' ⁇ u + ⁇ v
  • the pixel value u'of the subject OBJ 1 of the corrected slice image IMG_CORR i is the combined value (sum) of the pixel values u and v of the subject OBJ 1 in each of the IMG i and IMG i-1 . ..
  • the pixel value of the subject OBJ 3 of the corrected slice image IMG_CORR i becomes zero.
  • the pixel value is maintained.
  • the subject when the same subject is included in two consecutive sliced images, the subject is included only in the front (closer to the camera) range and in the back (farther from the camera) range. Is treated as not included.
  • the subject OBJ 1 that appears dark in the original slice image IMG i can be corrected brightly. Therefore, when the corrected slice image is displayed on the display, the driver can easily recognize the subject.
  • the identification rate can be improved when object recognition is performed by arithmetic processing.
  • each subject is included in only one sliced image after correction, it becomes easy to specify the position of each subject.
  • correction method 2 In the correction method 2, contrary to the correction method 1, when the same subject is included in two consecutive sliced images, the subject is included only in the range on the back side (far side from the camera) and is included in the front side (camera). It is not included in the range (closer to).
  • Figure 11 (c) is a diagram showing a slice image IMG_CORR i 'after correction obtained by the correction method 2.
  • the pixel value u of the subject included in a certain slice image IMG i is corrected by using the pixel value w of the slice image on the back side of the pixel value u.
  • Weighted addition may be used instead of simple addition.
  • ⁇ and ⁇ are weighting coefficients.
  • u' ⁇ u + ⁇ w
  • the pixel value u'of the subject OBJ 3 of the corrected slice image IMG_CORR i is a composite value (simple sum) of the pixel values u and w of the subject OBJ 3 in each of the IMG i and IMG i-1. There is.
  • correction method 3 when the same subject is included in two consecutive sliced images, the subject is included in both the back side (far side from the camera) range and the front side (closer side to the camera) range. It shall be.
  • FIG. 11D is a diagram showing a corrected slice image IMG_CORR i ”obtained by the correction method 3.
  • the pixel values of the slice image IMG i to be corrected and the subject (OBJ 1 in this example) included in the slice image IMG i-1 in front of the slice image IMG i are corrected as follows.
  • u' f (u, v)
  • the subjects OBJ 1 and OBJ 3 that appear dark in the original slice image IMG i can be corrected brightly. Therefore, when the corrected slice image is displayed on the display, the driver can easily recognize the subject.
  • the identification rate can be improved when object recognition is performed by arithmetic processing.
  • (Correction method 4) 12 (a) to 12 (d) are diagrams for explaining the correction method 4.
  • a predetermined coefficient is multiplied by the pixel value u of the subject to be corrected.
  • FIG. 13 is a diagram showing a vehicle lamp 200 having a built-in gating camera 20.
  • the vehicle lamp 200 includes a housing 210, an outer lens 220, a high beam and low beam lamp unit 230H / 230L, and a gating camera 20.
  • the lamp unit 230H / 230L and the gating camera 20 are housed in the housing 210.
  • a part of the gating camera 20, for example, the camera 24 may be installed outside the vehicle lamp 200, for example, behind the rearview mirror.
  • FIG. 14 is a block diagram showing a vehicle lamp 200 including the object identification system 10.
  • the vehicle lamp 200 constitutes a lamp system 310 together with the vehicle side ECU 304.
  • the vehicle lamp 200 includes a light source 202, a lighting circuit 204, and an optical system 206. Further, the vehicle lamp 200 is provided with an object identification system 10.
  • the object identification system 10 includes a gating camera 20 and an arithmetic processing unit 40.
  • the arithmetic processing unit 40 is configured to be able to identify the type of the object based on the plurality of slice images IMG 1 to IMG N corresponding to the plurality of ranges RNG 1 to RNG N obtained by the gating camera 20.
  • the arithmetic processing unit 40 includes a classifier implemented based on a prediction model generated by machine learning.
  • the algorithm of the classifier is not particularly limited, but YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), R-CNN (Region-based Convolutional Neural Network), SPPnet (Spatial Pyramid Pooling), Faster R-CNN, DSSD (Deconvolution -SSD), Mask R-CNN, etc. can be adopted, or algorithms developed in the future can be adopted.
  • the arithmetic processing unit 40 can be implemented by combining a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer, and a software program executed by the processor (hardware).
  • a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer
  • the arithmetic processing unit 40 may be a combination of a plurality of processors. Alternatively, the arithmetic processing unit 40 may be configured only by hardware.
  • the information about the object OBJ detected by the arithmetic processing unit 40 may be used for the light distribution control of the vehicle lamp 200.
  • the lamp side ECU 208 generates an appropriate light distribution pattern based on the information regarding the type of the object OBJ generated by the arithmetic processing unit 40 and its position.
  • the lighting circuit 204 and the optical system 206 operate so as to obtain the light distribution pattern generated by the lamp side ECU 208.
  • the information about the object OBJ detected by the arithmetic processing unit 40 may be transmitted to the vehicle side ECU 304.
  • the vehicle-side ECU may perform automatic driving based on this information.
  • Modification example 1 In the configuration of the camera 24 of FIG. 5B, an infrared camera may be used as the image sensor 54, and the sensitivity wavelength ⁇ c of the image sensor 54 may be longer than the wavelength of the probe light ⁇ c.
  • the down conversion element may be selected as the wavelength conversion element 56.
  • Modification 2 Although the gating camera has been described as the active sensor in the first embodiment, the application of the present invention is not limited to this.
  • the present invention can be applied to a normal active infrared sensor or a TOF camera that collectively captures all ranges without dividing in the depth direction.
  • the present invention can also be applied to a quantum radar (quantum camera) based on correlation calculation using the theory of computer ghost imaging.
  • a quantum radar quantum camera
  • a single pixel photodetector is used instead of the camera 24.
  • the present invention relates to an active sensor.
  • Object identification system 20 Gating camera 22 Lighting device 24 Image sensor 26 Controller S1 Flooding timing signal S2 Shooting timing signal 40 Arithmetic processing device 42 Classifier 50 Lens 52 Band pass filter 54 Image sensor 56 Wavelength conversion element 200 Vehicle lighting equipment 202 Light source 204 Lighting circuit 206 Optical system 310 Lighting system 304 Vehicle side ECU

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Mechanical Engineering (AREA)
  • Studio Devices (AREA)
PCT/JP2020/028325 2019-07-22 2020-07-21 アクティブセンサ、ゲーティングカメラ、自動車、車両用灯具 Ceased WO2021015208A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021534052A JPWO2021015208A1 (https=) 2019-07-22 2020-07-21

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2019134744 2019-07-22
JP2019-134744 2019-07-22
JP2019-147318 2019-08-09
JP2019147318 2019-08-09

Publications (1)

Publication Number Publication Date
WO2021015208A1 true WO2021015208A1 (ja) 2021-01-28

Family

ID=74194203

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/028325 Ceased WO2021015208A1 (ja) 2019-07-22 2020-07-21 アクティブセンサ、ゲーティングカメラ、自動車、車両用灯具

Country Status (2)

Country Link
JP (1) JPWO2021015208A1 (https=)
WO (1) WO2021015208A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023116142A (ja) * 2022-02-09 2023-08-22 株式会社小糸製作所 測定装置
US12483626B2 (en) 2023-05-04 2025-11-25 The Hong Kong University Of Science And Technology Sensor system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11508359A (ja) * 1995-06-22 1999-07-21 ３ディブイ・システムズ・リミテッド 改善された光学測距カメラ
JP2007316036A (ja) * 2006-05-29 2007-12-06 Honda Motor Co Ltd 車両の乗員検知装置
JP2013108835A (ja) * 2011-11-21 2013-06-06 Mitsubishi Heavy Ind Ltd 監視装置
US20150192676A1 (en) * 2014-01-03 2015-07-09 Princeton Lightwave, Inc. LiDAR System Comprising A Single-Photon Detector
US20180372873A1 (en) * 2017-06-21 2018-12-27 Analog Value Ltd. LIDAR system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11508359A (ja) * 1995-06-22 1999-07-21 ３ディブイ・システムズ・リミテッド 改善された光学測距カメラ
JP2007316036A (ja) * 2006-05-29 2007-12-06 Honda Motor Co Ltd 車両の乗員検知装置
JP2013108835A (ja) * 2011-11-21 2013-06-06 Mitsubishi Heavy Ind Ltd 監視装置
US20150192676A1 (en) * 2014-01-03 2015-07-09 Princeton Lightwave, Inc. LiDAR System Comprising A Single-Photon Detector
US20180372873A1 (en) * 2017-06-21 2018-12-27 Analog Value Ltd. LIDAR system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023116142A (ja) * 2022-02-09 2023-08-22 株式会社小糸製作所 測定装置
JP7854304B2 (ja) 2022-02-09 2026-05-01 株式会社小糸製作所 測定装置
US12483626B2 (en) 2023-05-04 2025-11-25 The Hong Kong University Of Science And Technology Sensor system

Also Published As

Publication number Publication date
JPWO2021015208A1 (https=) 2021-01-28

Similar Documents

Publication Publication Date Title
CN113557172B (zh) 选通照相机、汽车、车辆用灯具、物体识别系统、运算处理装置、物体识别方法、图像显示系统、检查方法、摄像装置、图像处理装置
JP7594539B2 (ja) ゲーティングカメラ、自動車、車両用灯具、画像処理装置、画像処理方法
TWI801637B (zh) 用於相機之紅外光預閃光
JP2019508717A5 (ja) 自動被写体検出および視覚システムの照明不変画像センサの方法および装置
KR102480618B1 (ko) 낮은 광 및 높은 광 레벨 이미징을 위한 단일 옵틱
CN113767304B (zh) 成像装置、车辆用灯具、汽车、成像方法
JP7472136B2 (ja) イメージング装置およびその照明装置、車両、車両用灯具
JP7653436B2 (ja) ゲーティングカメラ、車両用センシングシステム、車両用灯具
WO2016031666A1 (ja) 視線検出装置
WO2021015208A1 (ja) アクティブセンサ、ゲーティングカメラ、自動車、車両用灯具
JP7695230B2 (ja) ゲーティングカメラ、車両用センシングシステム、車両用灯具
US11303817B2 (en) Active sensor, object identification system, vehicle and vehicle lamp
JP6336337B2 (ja) 撮像装置及びその制御方法、プログラム、記憶媒体
JP7746300B2 (ja) ゲーティングカメラ、車両用センシングシステム、車両用灯具
WO2021193645A1 (ja) ゲーティングカメラ、センシングシステム、車両用灯具
JP7804684B2 (ja) ゲーティングカメラ、車両用センシングシステム、車両用灯具
JP7474759B2 (ja) 車両用灯具
CN117795376A (zh) 门控照相机、车辆用传感系统、车辆用灯具
JP7080724B2 (ja) 配光制御装置、投光システム及び配光制御方法
WO2021172478A1 (ja) センサ、自動車および周囲環境のセンシング方法
WO2023085403A1 (ja) センシングシステム
WO2023224077A1 (ja) ＴｏＦカメラ、車両用センシングシステム、および車両用灯具
CN118284825A (zh) 有源传感器、物体识别系统、车辆用灯具

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20844117

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021534052

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20844117

Country of ref document: EP

Kind code of ref document: A1