WO2021256223A1 - 物体検知システム及び物体検知方法 - Google Patents

物体検知システム及び物体検知方法 Download PDF

Info

Publication number
WO2021256223A1
WO2021256223A1 PCT/JP2021/020469 JP2021020469W WO2021256223A1 WO 2021256223 A1 WO2021256223 A1 WO 2021256223A1 JP 2021020469 W JP2021020469 W JP 2021020469W WO 2021256223 A1 WO2021256223 A1 WO 2021256223A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
object information
optical sensor
unit
generation unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/020469
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
佑亮 湯浅
繁 齋藤
信三 香山
裕 廣瀬
明弘 小田川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2022532459A priority Critical patent/JP7503750B2/ja
Priority to CN202180034295.2A priority patent/CN115552280A/zh
Publication of WO2021256223A1 publication Critical patent/WO2021256223A1/ja
Priority to US17/982,104 priority patent/US20230053841A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G01S17/18—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves wherein range gates are used
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50—Systems of measurement based on relative movement of target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/66—Tracking systems using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • G01S7/4865—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak

Definitions

  • the present disclosure relates to an object detection system and an object detection method, and more particularly to an object detection system and an object detection method that process information about a distance to an object.
  • Patent Document 1 discloses an image monitoring device that detects intruding objects and moving objects in the imaging field of view in a series of image data captured by the image pickup device and records a series of image data including those objects.
  • This image monitoring device includes an imaging unit that captures an image of a surveillance area and inputs quantized image data, an input image storage means that stores the image data, and a reference image storage means that stores a background image of the surveillance area.
  • a moving object is detected by comparing the difference calculation means that outputs the difference image between the input image and the reference image with the object position one frame before the difference image, and at the same time, the pixels in the area excluding the moving object are calculated by the value of the input image.
  • It is provided with an animal body detecting means for updating and a display means for displaying an input image and notifying the detection result of a moving object.
  • Patent Document 2 discloses an information processing device that continues high-precision tracking.
  • This information processing device has an acquisition unit that acquires information in which a vertical position, a horizontal position, and a depth position of an object are associated with each other at a plurality of time points, and an acquisition unit previously acquired by the acquisition unit.
  • the prediction unit Based on the position of the predetermined object in the information, the prediction unit that predicts the position of the predetermined object in the information acquired this time by the acquisition unit, and the position of the predetermined object from the information this time.
  • a plurality of objects satisfying the predetermined conditions according to the above conditions are extracted, and among the plurality of objects in the present information, based on the degree of similarity between each image of the plurality of objects and the image of the predetermined object. It is provided with an extraction unit for extracting the same object as the predetermined object in the previous information.
  • the object of the present disclosure is to provide an object detection system and an object detection method for detecting an object at high speed.
  • the object detection system in one embodiment of the present disclosure includes a light emitting unit that emits light, an optical sensor that receives the reflected light reflected in the range in which the light can be measured in the target space, and a light sensor. It includes a light emitting unit, a control unit that controls the optical sensor, and a signal processing unit that processes information indicated by an electric signal generated by the optical sensor, and the control unit divides the distance-measurable range.
  • the distance section signal which is a signal from the pixel that received the light among the plurality of pixels included in the optical sensor, is output from the optical sensor for each of the plurality of distance sections.
  • the light emitting unit and the optical sensor are controlled, and the signal processing unit has a plurality of generation units capable of parallel operation, and the plurality of distance sections are based on the distance section signal output from the optical sensor.
  • the object information generation unit that generates object information indicating the characteristics of the object detected by the optical sensor, and the object corresponding to each of the plurality of distance sections generated by the object information generation unit.
  • the object information generation unit includes a storage unit for storing object information and an output unit for outputting the object information corresponding to each of the plurality of distance sections, and the object information generation unit has the same for each of the plurality of distance sections.
  • the object information is generated by comparing the past object information stored in the storage unit with the current feature of the object detected by the optical sensor.
  • the object detection method in one embodiment of the present disclosure includes a light emitting unit that emits light and an optical sensor that receives the reflected light reflected in the range in which the light can be measured in the target space.
  • a method for detecting an object by an object detection system including the control step of controlling the light emitting unit and the optical sensor, and a signal processing step of processing information indicated by an electric signal generated by the optical sensor.
  • the control step for each of the plurality of distance sections configured by dividing the distance-measurable range, the distance section signal is a signal from the pixel that received the light among the plurality of pixels included in the optical sensor.
  • an object information generation sub-step for generating object information indicating the characteristics of the object detected by the optical sensor, and the plurality of objects generated in the object information generation sub-step.
  • the object includes a storage sub-step for storing the object information corresponding to each of the distance sections of the above and an output sub-step for outputting the object information corresponding to each of the plurality of distance sections.
  • the information generation substep the past object information stored in the storage unit is compared with the characteristics of the current object detected by the optical sensor for each of the plurality of distance sections. Generate object information.
  • the object detection system and the object detection method in the present disclosure can detect an object at high speed.
  • FIG. 1 is a diagram showing a configuration of an object detection system according to an embodiment.
  • FIG. 2 is a diagram showing an outline of a method of measuring a distance to an object by an object detection system according to an embodiment.
  • FIG. 3A is a diagram showing a configuration of an information processing system included in the object detection system according to the embodiment.
  • FIG. 3B is a timing chart showing the processing of the information processing system included in the object detection system according to the present embodiment.
  • FIG. 4 is a flowchart showing a flow of processing performed by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 5 is a diagram illustrating an example of distance interval image generation processing by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 6 is a diagram illustrating a speed generation process by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 7 is a diagram illustrating an example of object information that can be generated by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 8 is a diagram illustrating an example of changing the distance measurement setting by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 9A is a diagram illustrating an example of an image displayed by the presentation unit of the object detection system according to the embodiment.
  • FIG. 9B is a diagram illustrating correction of the center coordinates of an object using a luminance image by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 9C is a diagram illustrating a method of calculating the depth distance of an object using distance section signals of a plurality of distance sections by the object information generation unit of the object detection system according to the embodiment.
  • FIG. 10 is a timing chart showing an example of the order of processing in the object detection system according to the embodiment.
  • FIG. 11 is a diagram illustrating an example of distance measurement by the object information generation unit in the modified example of the embodiment.
  • FIG. 1 is a diagram showing a configuration of an object detection system 200 according to an embodiment. Note that FIG. 1 also shows an external device 5 connected to the object detection system 200 via a communication path. As shown in FIG. 1, the object detection system 200 includes an information processing system 100, a light emitting unit 1, an optical sensor 2, and a presentation unit 4. The object detection system 200 is a system that directly detects an object for each of a plurality of distance sections by using a TOF (Time of Flight) method.
  • the external device 5 is a storage device such as a semiconductor memory, a computer device, a display, or the like.
  • the light emitting unit 1 includes a light source for emitting measurement light to an object under the control of the control unit 101a.
  • the measurement light is pulsed light.
  • the measurement light has a single wavelength, the pulse width is relatively short, and the peak intensity is relatively high.
  • the wavelength of the measured light has low human visual sensitivity and is affected by the disturbance light from sunlight. It is preferable that the wavelength range is in the near-infrared band, which is difficult to receive.
  • the light source is composed of, for example, a laser diode, and outputs a pulse laser.
  • the intensity of the pulsed laser output by the light source meets the class 1 or class 2 standard of the safety standard for laser products (JIS C 6802).
  • the light source is not limited to the above configuration, and may be a light emitting diode (LED: Light Emitting Diode), a surface emitting laser (VCSEL: Vertical Cavity Surface Emitting Laser), a halogen lamp, or the like. Further, the measurement light may be in a wavelength range different from the near infrared band.
  • the optical sensor 2 is a sensor that receives the reflected light reflected by the measurement light within the range that can be measured in the target space, and includes a pixel portion including a plurality of pixels. An avalanche photodiode is arranged in each pixel. Other photodetection elements may be arranged in each pixel. Each pixel is configured to be able to switch between an exposed state that receives reflected light and a non-exposed state that does not receive reflected light under the control of the control unit 101a. The optical sensor 2 outputs a charge based on the reflected light received by each pixel in the exposed state.
  • the information processing system 100 includes a control unit 101a that controls the light emitting unit 1 and the optical sensor 2, and a signal processing unit 101b that processes information indicated by an electric signal generated by the optical sensor 2.
  • the distance section signal which is a signal from the pixel that received the light among the plurality of pixels included in the optical sensor 2
  • the light emitting unit 1 and the optical sensor 2 are controlled (that is, the control step is executed) so as to be output from the optical sensor 2.
  • the signal processing unit 101b processes the information indicated by the electric signal generated by the optical sensor 2 (that is, executes the signal processing step). Therefore, the signal processing unit 101b generates object information indicating the characteristics of the object detected by the optical sensor 2 in each of the plurality of distance sections, based on the distance interval signal output from the optical sensor 2. That is, the object information generation unit 102 having a plurality of generation units (first generation unit to fifth generation unit) capable of parallel processing (which executes the object information generation substep), and a plurality of output from the optical sensor 2.
  • Each of the composite image generation unit 104 that generates a composite image (that is, executes the composite image generation substep) from the plurality of distance section signals corresponding to the distance sections, and the plurality of distance sections generated by the object information generation unit 102.
  • the storage unit 103 that stores the object information corresponding to (that is, executes the storage substep), and the output unit 105 that outputs the object information and the composite image corresponding to each of the plurality of distance sections to the external device 5.
  • the object information generation unit 102 compares the past object information stored in the storage unit 103 with the characteristics of the current object detected by the optical sensor 2 for each of the plurality of distance sections. Generate object information.
  • each distance section For each distance section, light reception consisting of light emission of measurement light and exposure operation of each pixel of the optical sensor 2 is performed one or more times, and each pixel receives light for the number of times the light is received in the light reception operation. Outputs the corresponding electrical signal.
  • the number of times of light receiving operation is not particularly limited, but may be about 50 times in one example.
  • FIG. 2 is a diagram showing an outline of a method for measuring a distance to an object by the object detection system 200 according to the embodiment.
  • the object detection system 200 measures the distance to the object by using the light reflected by the object for the measurement light emitted from the light emitting unit 1.
  • the object detection system 200 can be used, for example, as an in-vehicle object detection system mounted on an automobile to detect an obstacle, a surveillance camera for detecting an object or a person, a security camera, or the like.
  • the object detection system 200 measures the distance to an object existing in the range-measurable range FR of the target space.
  • the range-finding range FR is determined according to the time (set time) from when the light emitting unit 1 emits the measured light to when the optical sensor 2 finally performs the exposure operation under the control of the control unit 101a.
  • the distance-measurable range FR is not particularly limited, but as an example, it is several tens of centimeters to several tens of meters.
  • the range-finding range FR may be fixed or variable. Here, it is assumed that the setting can be made variably.
  • the object information generation unit 102 determines the presence / absence of an object in each of one or more (here, five as an example) distance sections R1 to R5 included in the range-finding range FR. judge. Further, the object information generation unit 102 generates object information which is information about the feature amount of the object for the distance section where it is determined that the object exists.
  • the plurality of distance sections R1 to R5 are sections in which the distance-measurable range FR is cut out according to the difference in the elapsed time from the time when the light emitting unit 1 emits the measurement light. That is, the distance-measurable range FR is composed of a plurality of distance sections R1 to R5.
  • the plurality of distance sections R1 to R5 have the same length.
  • each of the plurality of distance sections R1 to R5 is several cm to several m.
  • the plurality of distance sections R1 to R5 do not necessarily have to have the same length, and the number of distance sections is not particularly limited.
  • the number of distance intervals can typically be selected from 1 to 15.
  • the interval between the distance sections is not particularly limited.
  • a distance between a certain distance section and an adjacent distance section may be set to be several meters so that the distance is not measured, or a part of some distance sections may be set to overlap. ..
  • the control unit 101a is, for example, at a time point after the lapse of time corresponding to twice the distance to the latest point of the target distance section among the plurality of distance sections R1 to R5 after the light emitting unit 1 emits the measurement light. Then, the light emitting unit 1 and the optical sensor 2 are controlled so as to start the exposure of the pixels by the optical sensor 2. Further, the control unit 101a uses light so as to end the exposure of the pixels by the optical sensor 2 (end of the exposure operation) at the time when the time corresponding to twice the distance to the farthest point in this distance section elapses. Control the sensor 2.
  • the optical axis of the object detection system 200 of the object among the plurality of pixels of the optical sensor 2 Light is received by the pixels in the region corresponding to the position in the plane perpendicular to.
  • the object information generation unit 102 can obtain information about the presence / absence of the object and the two-dimensional position of the object in the target distance section. Further, the object information generation unit 102 assigns a value of "1" or "0" to each of the plurality of pixels depending on whether or not light is received, so that the object can be an object in a target distance section. It is possible to generate a binary image (distance interval image) showing a two-dimensional position where is present.
  • control unit 101a may perform light emission of the measurement light and light reception including the exposure operation of each pixel of the optical sensor 2 a plurality of times in the measurement in each distance section.
  • the object information generation unit 102 may determine that the object is at the position corresponding to the pixel when the number of times the light is received in each pixel (the number of times of light reception) exceeds the threshold value.
  • the object information generation unit 102 can determine the presence or absence of the object in each distance section and acquire the object information. Will be.
  • an object exists in each of the plurality of distance sections R1 to R5.
  • a tree exists as an object in the distance section R1
  • an electric pole exists as an object in the distance section R2
  • a person exists as an object in the distance section R3
  • a tree exists as an object in the distance section R4.
  • a fence exists as an object in the distance section R5.
  • the distance from the object detection system 200 to the distance section R1 is D0
  • the lengths of the distance sections R1 to R5 are D1 to D5, respectively.
  • the distance from the object detection system 200 to the farthest point in the distance section R1 corresponds to D0 + D1. Further, D0 is 0 m as an example. Further, the width of the depth of the distance-measurable range FR is represented by D0 + D1 + D2 + D3 + D4 + D5.
  • the object detection system 200 when determining the presence or absence of an object in the distance section R1, in the object detection system 200, the time (2 ⁇ (2 ⁇ ) after the light emitting unit 1 emits the measurement light under the control of the control unit 101a.
  • D0 + D1) / c) has elapsed, the exposure of the optical sensor 2 is terminated.
  • c is the speed of light.
  • the object information generation unit 102 obtains the distance section image Im1 shown in FIG. 2 as an image showing the object existing in the distance section R1 for the distance section R1.
  • the object information generation unit 102 obtains the distance section images Im3 to Im5 shown in FIG. 2, respectively.
  • a part of the tree that is an object existing in the distance section R4 is actually hidden by a person who is an object existing in the distance section R3 closer to the object detection system 200.
  • the actual shape of the tree is shown in the distance section image Im4. The same applies to other distance section images.
  • the composite image generation unit 104 further synthesizes the plurality of distance section images Im1 to Im5 obtained for the plurality of distance sections R1 to R5, and as an example of the composite image, the distance image Im100 for the distance-measurable range FR. To generate. Specifically, the composite image generation unit 104 assigns different weights to the pixels in the region corresponding to the object among the plurality of distance section images Im1 to Im5 for each of the distance sections R1 to R5, and a plurality of them. The distance section images Im1 to Im5 are superimposed. As a result, for example, the distance image Im100 shown in FIG. 2 is generated.
  • the distance image Im100 is an example of a composite image generated by the composite image generation unit 104, and is an image in which a plurality of distance interval images, which are binary images, are weighted and composited.
  • a plurality of distance interval images which are binary images
  • it is not always necessary to give different weights for each distance interval R1 to R5 it is not always necessary to give different weights for each distance interval R1 to R5, and the composition may be performed with the same weight, or by ORing at the same pixel position. There may be.
  • the composite image generation unit 104 generates a luminance image as a composite image in addition to generating the distance image Im100. That is, the composite image generation unit 104 further adds an electric signal obtained by executing one or more exposure operations for each of the plurality of distance sections R1 to R5 for each pixel. As a result, for example, a luminance image showing the luminance of each pixel in 8 bits is generated.
  • the luminance image is another example of the composite image generated by the composite image generation unit 104, and is an image composed of information indicating the luminance of each pixel.
  • the object detection system 200 of the present embodiment it is possible to generate a distance section image Im1 to Im5, a distance image Im100, and a luminance image by the operation as described above.
  • the object detection system 200 does not necessarily have to generate the distance section images Im1 to Im5, and may generate information (signal) that can generate the distance section images Im1 to Im5. For example, an image having information on the number of times of light reception for each pixel may be generated as "information that can generate distance interval images Im1 to Im5". The same applies to the distance image Im100 and the luminance image.
  • the information processing system 100 includes a control unit 101a, a signal processing unit 101b, an output unit 105, and a presentation unit 4.
  • the control unit 101a and the signal processing unit 101b can be realized by, for example, a computer system including one or more processors and one or more memories. That is, one or more processors execute one or more programs stored in one or more memories, thereby functioning as the control unit 101a and the signal processing unit 101b.
  • the program is pre-recorded in the memory here, it may be recorded and provided through a telecommunication circuit such as the Internet or a non-temporary storage medium such as a memory card.
  • the control unit 101a is configured to control the light emitting unit 1 and the optical sensor 2.
  • control unit 101a controls the timing of outputting the measured light from the light source of the light emitting unit 1 (light emitting timing), the pulse width of the measured light output from the light source of the light emitting unit 1, and the like.
  • control unit 101a controls the timing (exposure timing) of exposing each pixel of the optical sensor 2 to the exposure state, the exposure period, the reading timing of the electric signal, and the like.
  • the control unit 101a controls, for example, the light emission timing of the light emitting unit 1 and each operation timing of the light sensor 2 based on the timing stored internally.
  • the control unit 101a sequentially measures the distances of the plurality of distance sections R1 to R5 constituting the distance-measurable range FR. That is, the control unit 101a first emits light from the light emitting unit 1 and exposes the light sensor 2 to the distance section R1 closest to the object detection system 200, so that the distance section signal Si1 relating to the distance section R1 is transmitted from the optical sensor 2. To generate. Next, the control unit 101a emits light from the light emitting unit 1 and exposes the light sensor for the distance section R2 second closest to the object detection system 200, so that the distance section signal Si2 relating to the distance section R2 is transmitted from the optical sensor 2. To generate. The control unit 101a also sequentially generates distance section signals Si3 to Si5 from the optical sensor 2 for the distance sections R3 to R5. The control unit 101a repeatedly causes the optical sensor 2 to generate such distance interval signals Si1 to Si5.
  • the signal processing unit 101b receives the electric signal output from the optical sensor 2.
  • the electric signal includes any of the distance section signals Si1 to Si5.
  • the electric signal received by the signal processing unit 101b is processed by the signal processing unit 101b.
  • FIG. 3A is a diagram showing a configuration of an information processing system 100 included in the object detection system 200 according to the embodiment.
  • the information processing system 100 includes a control unit 101a and a signal processing unit 101b (object information generation unit 102, synthetic image generation unit 104, storage unit 103, and output unit 105).
  • the object information generation unit 102 targets the object information, which is information about the feature amount of the object existing in each of the plurality of distance sections R1 to R5, among the electric signals generated by the optical sensor 2. Generated based on the distance interval signal associated with the interval.
  • the object information generation unit 102 includes, for example, a generation unit (first generation unit 102a to fifth generation unit 102e) capable of parallel operation according to the number of distance sections (five here).
  • the first generation unit 102a receives the distance section signal Si1 from the optical sensor 2.
  • the first generation unit 102a generates object information regarding an object existing in the distance section R1 based on the distance section signal Si1 which is an electric signal related to the distance section R1.
  • the second generation unit 102b generates object information regarding an object existing in the distance section R2 based on the distance section signal Si2 which is an electric signal related to the distance section R2.
  • the third generation unit 102c generates object information regarding an object existing in the distance section R3 based on the distance section signal Si3 which is an electric signal related to the distance section R3.
  • the fourth generation unit 102d generates object information regarding an object existing in the distance section R4 based on the distance section signal Si4 which is an electric signal related to the distance section R4.
  • the fifth generation unit 102e generates object information regarding an object existing in the distance section R5 based on the distance section signal Si5 which is an electric signal related to the distance section R5.
  • a plurality of distance section signals Si1 to Si5 are input to the object information generation unit 102 through different routes, and different elements in the object information generation unit 102 ( It is assumed that the processing is performed by the first generation unit 102a to the fifth generation unit 102e). However, not limited to this, the plurality of distance section signals Si1 to Si5 may be input to the object information generation unit 102 by the same route and processed by the same element.
  • FIG. 3B is a timing chart showing the processing of the information processing system 100 included in the object detection system 200 according to the present embodiment.
  • a timing chart showing an example of parallel operation of the first generation unit 102a to the fifth generation unit 102e is shown.
  • “distance section” indicates an arrangement of five subframes (distance sections R1 to R5) constituting each frame
  • "light emission” indicates the timing at which the light emitting unit 1 emits the measured light, and "exposure”.
  • the first generation unit 102a to the fifth generation unit 102e generate object information, respectively. Indicates the processing period to be performed.
  • an example is shown in which object information is generated for each pair of light emission and exposure.
  • the generation of the object information in the distance section R1 is started by the first generation unit 102a after the light emission and the exposure for the distance section R1 are performed. Subsequently, in the subframe of the distance section R2, after the light emission and the exposure for the distance section R2 are performed, the generation of the object information in the distance section R2 by the second generation unit 102b is started, and so on. , The distance section R3, the distance section R4, and the distance section R5 are sequentially processed in the subframe.
  • Each of the first generation unit 102a to the fifth generation unit 102e does not wait for the input of the signal (distance section signal) to the other generation unit and the completion of the processing in the other generation unit, and the signal (distance section signal). Starts processing as soon as is entered. That is, the first generation unit 102a to the fifth generation unit 102e operate in parallel. As a result, the generation of the object information corresponding to each of the five distance sections is speeded up by the first generation unit 102a to the fifth generation unit 102e operating in parallel.
  • some of the processes of the first generation unit 102a to the fifth generation unit 102e overlap in time, but whether or not they overlap in time depends on the processing load. It does not necessarily have to overlap in time. For example, depending on the processing load, the processing of the first generation unit 102a to the fifth generation unit 102e may be completed within the subframe of the corresponding distance section.
  • FIG. 4 is a flowchart showing a flow of processing performed by the object information generation unit 102 of the object detection system 200 in the present embodiment.
  • the third generation unit 102c of the object information generation unit 102 first receives the distance section signal Si3 related to the target distance section R3 among the plurality of distance sections R1 to R5 from the optical sensor 2, and receives the distance section signal.
  • a distance interval image generation process is performed on Si3 using the reference image Im101 that has been acquired in advance and stored in the storage unit 103 (S1 in FIG. 4).
  • FIG. 5 is a diagram illustrating an example of a distance interval image generation process performed by the object information generation unit 102 of the object detection system 200 in the present embodiment.
  • the reference image Im101 (upper part of FIG. 5) is a distance image acquired in advance by the object detection system 200 and stored in the storage unit 103 (upper part of FIG. 5).
  • the distance image Im102 (lower part of FIG. 5) shows the distance section signals Si1 to Si5 input to the object information generation unit 102.
  • the third generation unit 102c assumes that the equivalent object exists at the same distance as the object included in the reference image Im101 among the received distance interval signal Si3 in the distance interval image generation process.
  • the position information is rewritten to the information that the object does not exist.
  • the unit 102c When the above-mentioned distance section image generation process is performed, the unit 102c generates a signal derived from a person as the distance section image Im3 (generates an image in which "1" is stored in the human area). As the other distance section images Im1, Im2, Im4, and Im5, signals derived from the respective objects are not generated (images in which "0" is stored in all regions are generated). In this way, the object information generation unit 102 shows the difference between the distance section signal related to the distance section corresponding to the reference image Im101 stored in the storage unit 103 and the reference image Im101. A distance interval image is generated as one of the object information.
  • the method of the reference image Im101 and the distance interval image generation processing is not particularly limited to such processing.
  • the reference image Im101 may be immutable or may be updated during the operation of the object detection system 200.
  • the reference image Im101 is constantly updated as the distance section signal Si3 one frame before, and the third generation unit 102c calculates the optical flow in the distance section image generation process, and the object included in the reference image Im101 is It may be determined how much the object has moved in the current distance interval signal Si3, and when the movement amount of the object exceeds the threshold value, it may be determined that the object exists and a distance interval image may be generated.
  • the distance interval image generated by the above process is a binary image in which a value of "1" is assigned to the pixels in the area where the object exists and a value of "0" is assigned to the pixels in the area where the object does not exist. ..
  • the third generation unit 102c may perform noise filter processing in general image processing.
  • the third generation unit 102c may apply a morphology operation or a median filter (S2 in FIG. 4). This may reduce noise and shorten subsequent processing time.
  • the third generation unit 102c may encode the distance interval image by a method capable of reducing the amount of data. For example, it may be compressed using run-length coding.
  • the third generation unit 102c performs labeling processing (S3 in FIG. 4).
  • labeling processing when the area to which "1" is assigned is connected by pixels adjacent to each other, it is determined that this block of connected pixels is one object, and a different label is given to each object. .. If the pixel to which "1" is assigned does not exist, it is determined that the object does not exist in the target distance section.
  • the third generation unit 102c performs a feature amount generation process (S4 in FIG. 4).
  • the feature amount generation process the feature amount of the object is generated based on the region of consecutive pixels considered to correspond to one object.
  • the type of feature amount is not particularly limited, but here, as an example, a given label, information indicating a distance interval, an object area, a circumference length, a first-order moment, a center of gravity, and a position of the center of gravity in the world coordinate system are used.
  • the world coordinate system is a three-dimensional Cartesian coordinate system in a virtual space corresponding to the target space.
  • the information indicating the position of the center of gravity of the object in the world coordinate system is an example of the object position information regarding the position of the object in the three-dimensional space.
  • the third generation unit 102c performs an object filter process after the feature amount generation process (S5 in FIG. 4).
  • the feature amount of each object is referred to, and the objects other than the objects satisfying the specified conditions are deleted.
  • the specified conditions are not particularly limited, but for example, the area (number of pixels) of the object is 100 pixels or more.
  • the third generation unit 102c performs object link processing using the past object information stored in the storage unit 103 (S6 in FIG. 4).
  • object link processing a similarity is defined in which the value increases as the current object and the past object become similar to each other for a plurality of distance sections, and the similarity is the highest among the past objects. It is determined that the object to be enlarged is the same object as itself, and the object is linked. At this time, the label of the past object to be linked is added to the feature amount of the current object as a linker so that the same past object to be linked can be searched later.
  • the ones for which the similarity is calculated and selected as the candidate to be linked are typically the objects one frame before, but the objects two or more frames before may be the candidates.
  • the definition of similarity is not particularly limited, and can be defined as a function in which the distance between the centers of gravity, the primary moment, and the area contribute. If there is no past object to be linked, a specific value indicating that there is no object to be linked is added as one of the features. If there is no past object to be linked, for example, the label of the current object itself may be added to the feature amount as a linker.
  • the third generation unit 102c executes a speed generation process for generating the moving speed of the object (S7 in FIG. 4).
  • the third generation unit 102c can trace back to the past object linked to the object.
  • the linker that goes back to the past object is stored in the past object that goes back, it is possible to go back to the time when the object first appears in the range-finding range FR.
  • the velocity generation process for example, by going back to the same object N seconds ago and referring to the object position information at each time, the movement distance is calculated from the movement trajectory of the center of gravity up to the present in the world coordinate system. Then, divide by the elapsed time (N seconds) to calculate the speed, and add it as one of the feature quantities.
  • the object information generation unit 102 uses the distance section signals of the plurality of distance sections to obtain object position information regarding the position of the object in the three-dimensional space.
  • the moving speed of the generated object is calculated by using the object position information of the past object which is the same as the current object.
  • the definition of how far back to calculate the speed is not particularly limited, and it may be defined as before N frames.
  • the frame rate at which the third generation unit 102c generates the distance interval image is variable, if it is defined as N frames before, the number of frames to be traced back is fixed when calculating the speed, and the frame rate is set to the frame rate. It is possible to reduce the influence of the calculation error of the center of gravity position due to the independent noise.
  • the method of calculating the speed is not particularly limited, and for example, for simplification of the calculation, it is calculated from the linear distance between the world coordinate system position of the center of gravity N seconds ago and the world coordinate system position of the current center of gravity. Is also good.
  • FIG. 6 is a diagram illustrating an example of a method of generating a moving direction of an object by the object information generation unit 102 (here, the third generation unit 102c) of the object detection system 200 in the present embodiment.
  • the method for estimating the moving direction is not particularly limited, but here, a method using an arc approximation of the locus of the object in the world coordinate system is adopted. For example, when using the movement path of an object for N seconds, the set of the centers of gravity of the object from N seconds before to the present in the world coordinate system is used.
  • the direction of movement of the object is the direction away from the position of the center of gravity N seconds before the object along the tangent line at the current center of gravity of the arc 71 centered on the center of rotation and passing through the position of the current center of gravity.
  • the object information generation unit 102 (here, the third generation unit 102c) approximates the locus of the same object as the current object in the past with a curve, so that the moving speed of the object Is calculated.
  • the feature amount to be used does not necessarily have to be the center of gravity, and other feature amounts related to the position of the object in the world coordinate system, such as the upper left point of the circumscribing rectangle given to the object. But it may be.
  • the third generation unit 102c sets the velocity and the moving direction of the object as the velocity vector 72 of the object, and adds it as one of the feature quantities.
  • the third generation unit 102c executes the movement destination prediction process from the speed of the object (S8 in FIG. 4). By executing the velocity generation process, the third generation unit 102c can estimate the position where the object will move in the near future.
  • the method of moving destination prediction processing is not particularly limited, but for example, when predicting a moving destination after N seconds, it is predicted that the distance obtained by multiplying the speed by N is linearly moved in the moving direction.
  • the predicted position of the destination where the destination prediction process is executed is added as one of the feature quantities. In this way, the object information generation unit 102 (here, the third generation unit 102c) generates a future predicted position of the object from the moving speed of the object.
  • FIG. 7 is a diagram showing an example of object information that can be generated by the object information generation unit 102 (here, the third generation unit 102c) of the object detection system 200 in the present embodiment.
  • the object information shown in this figure includes various feature quantities (center coordinates, area, aspect ratio) related to the two objects O1 and O2 detected at time t and the two objects O3 and O4 detected at time t + 1. , Speed, Linker) is included.
  • the linker which is one of the feature quantities of the object O3 detected at time t + 1, refers to the object O2 detected at time t, so that the object O2 and the object O3 detected at different times are the same. It can be seen that it is determined to be an object.
  • the linker which is one of the feature quantities of the object O4 detected at time t + 1, points to the object O1 detected at time t, the object O1 and the object O4 detected at different times are the same object. It can be seen that it is judged to be a thing.
  • the object information includes tracking information of the same object.
  • FIG. 8 is a diagram illustrating an example of changing the distance measurement setting by the object information generation unit 102 of the object detection system 200 in the present embodiment. That is, this figure is a diagram illustrating an example of a method of changing the setting stored in the control unit 101a based on the object information.
  • the object information generation unit 102 extracts a distance section including the predicted position 81 of the movement destination, and measures only three distance sections including the distance sections before and after the distance section.
  • the setting stored in the control unit 101a is changed. For example, in the example of FIG. 2, when the predicted position 81 of the destination of the person detected in the distance section R3 is the distance section R4 ((a) in FIG. 8), the control unit 101a is changed by the above setting.
  • the light emission timing and the exposure timing are controlled so that the distance sections R1 and R2 are ignored and only the distance sections R3 to R5 are measured ((b) in FIG. 8).
  • the range to be measured is changed to the distance sections R2 to R4.
  • the object information generation unit 102 generates the object information only for the changed distance section. In this way, the control unit 101a changes the control signal to the light emitting unit 1 and the optical sensor 2 so that the number of distance sections and the target distance section for generating the object information change.
  • control unit 101a has a distance section signal corresponding to a distance section that does not include the predicted position of the object among the plurality of distance sections (in FIG. 8B, the distance section signals of the distance sections R1 and R2). ) Is not output from the optical sensor 2, and the control signal to the light emitting unit 1 and the optical sensor 2 is changed.
  • the feature amount used to change the range to be measured is not particularly limited, and for example, a method of measuring a distance section including the current center of gravity position and a distance section before and after the current center of gravity may be used. Further, the number of distance sections to be measured after changing the range to be measured is not particularly limited.
  • the output unit 105 outputs the feature amount of the object to the presentation unit 4 or the external device 5 as the object information (S9 in FIG. 4).
  • the output unit 105 completes the processing of the object information generation unit 102 (more specifically, each of the first generation unit 102a to the fifth generation unit 102e) without waiting for the end of the measurement in all the distance sections.
  • the generated object information is sequentially output.
  • the output unit 105 may output not only the object information but also, for example, a luminance image, a distance image, or a distance interval image.
  • the output unit 105 may output information in the form of a wireless signal.
  • the presentation unit 4 visualizes the information output from the output unit 105.
  • the presentation unit 4 may include a two-dimensional display such as a liquid crystal display or an organic EL display.
  • the presentation unit 4 may include a three-dimensional display for displaying a distance image in three dimensions.
  • FIG. 9A is an example of the image 91 displayed by the presentation unit 4 of the object detection system 200 in the present embodiment.
  • a rectangle (detection frame) indicating that the detection has been detected is displayed on the vehicle, which is a moving object reflected in the screen, and the depth distance (“Deepth27.0 m”) and speed (“Speed45.”) Included in the object information. 9km / h ”) and the direction of the velocity vector (arrows in the figure) are shown.
  • FIG. 9B is a diagram illustrating correction of the center coordinates of the object using the luminance image by the object information generation unit 102.
  • FIG. 9B (a) shows an example of an object frame (“detection frame”) detected by the object information generation unit 102 at time t
  • FIG. 9B (b) is a composite image generation unit at time t.
  • An example of the brightness image generated in 104 is shown, and FIG.
  • 9B (c) shows an example of a frame (“detection frame”) of the same object detected by the object information generation unit 102 at time t + 1.
  • (D) of 9B shows an example of correction of the center coordinates of the object by the object information generation unit 102 at time t + 1.
  • the object information generation unit 102 specified a rectangle surrounding the detected object as a detection frame in a certain distance section image, and detected the center of the detection frame. Specify as the center coordinates of the object (“object center”).
  • the object information generation unit 102 detects a rectangle surrounding the same object as the object detected at time t in the distance section image or another distance section image. It is specified as a frame, and the center of the detection frame is specified as the provisional center coordinates (“object center”) of the object.
  • the object information generation unit 102 calculates the coordinate shift amount of the object in the distance interval image at time t + 1 by using the brightness image of the object at time t as a template (that is, the reference image), and the calculated coordinate shift amount.
  • the provisional center coordinates are shifted in the opposite direction by the amount of.
  • the center coordinates of the object are corrected by using the high-precision luminance image, so that the center coordinates of the object can be specified with high accuracy as compared with the case of using only the distance interval image.
  • the center coordinates of the object thus identified are used to calculate the depth distance, velocity and velocity vector of the object.
  • the feature amount to be corrected is not limited to the center coordinates of the object.
  • the position of the circumscribed rectangle of the object itself, the position of a specific point such as the upper right corner point of the circumscribed rectangle, or the position of the silhouette of the object may be corrected.
  • the object information generation unit 102 is one of the object information by using the distance section signals (or distance section images) of a plurality of distance sections when calculating the depth distance of the detected object. Correct the depth distance of the object (that is, calculate the depth distance with high accuracy).
  • FIG. 9C is a diagram illustrating a method of calculating the depth distance of an object by using the distance section signals of a plurality of distance sections by the object information generation unit 102.
  • an object composed of a point cloud detected in each of the five distance sections is shown in the detection frame. For example, 8 white circles are points detected in a distance section of 1.5 m, 15 black circles are points detected in a distance section of 3.0 m, and 2 triangles are 4.
  • the object information generation unit 102 weights the distance of each point cloud by the number of points as the depth distance of the object in the detection frame, that is, the average distance, that is, (8 ⁇ 1.5 m + 15 ⁇ 3). It is specified as 0.0 m + 2 ⁇ 4.5 m + 1 ⁇ 21.0 m + 1 ⁇ 22.5 m) / (8 + 15 + 2 + 1 + 1) ⁇ 4.05 m.
  • the distance of the object is calculated using the distance section signals of a plurality of distance sections, so that the distance is more accurate and realistic considering the depth of the object as compared with the case of using only one distance section signal.
  • the plurality of distance section signals used may be a plurality of distance section images after processing by the object information generation unit 102.
  • the object information may be corrected by the composite image generation unit 104 by using a part or all of the distance image generated as the composite image.
  • the calculation method for the depth distance of an object is not particularly limited.
  • the weighted average may be calculated so that the weight corresponding to the distance interval in which the most points are detected is the heaviest.
  • a weighted average in which the weight is multiplied by 1/2 as the distance interval is 1.5 m from the center of 3.0 m where the most points are detected, that is, ( 8 ⁇ 1.5m / 2 + 15 ⁇ 3.0m + 2 ⁇ 4.5m / 2 + 1 ⁇ 21.0m / 4096 + 1 ⁇ 22.5m / 8192) / (8 + 15 + 2 + 1 + 1) ⁇ 2.06m is specified as the depth distance of the object.
  • the weighted average method may work effectively for accurate depth distance calculation when noise is mixed in the point cloud corresponding to the object.
  • the points detected in the distance sections of 21 m and 22.5 m may be noise. It is plausible to specify the depth distance between 1.5m and 3.0m.
  • the generation of the object information including the time-dependent feature amount and the tracking of the object regarding the object existing in each of one or more distance sections Is generated based on the distance interval signal associated with the target distance interval.
  • FIG. 10 is a timing chart showing an example of the order of processing in the object detection system 200 according to the present embodiment. That is, in this figure, the light receiving operation by the optical sensor 2 (“measurement”), the object information generation operation by the first generation unit 102a to the fifth generation unit 102e (“information generation”), and the object by the output unit 105. The outline of the time relationship between the output operation (“data output”) and the distance image generation operation (“image composition”) by the composite image generation unit 104 is shown. More specifically, in FIG. 10, the “measurement” stage indicates the distance section in which the optical sensor 2 measures the distance in the distance sections R1 to R5.
  • the “information generation” stage indicates the timing at which the object information generation unit 102 processes the distance section signal to generate the object information among the distance section signals Si1 to Si5.
  • the “data output” stage indicates the timing of the object information output by the output unit 105.
  • the “image composition” stage indicates the timing at which the composite image generation unit 104 generates the distance image Im102. Further, in the figure, the start point of the arrow in each stage indicates the start time point of the process, and the end point of the arrow indicates the end point point of the process.
  • each of the distance section signals is processed by each of the first generation unit 102a to the fifth generation unit 102e without waiting for the processing of the other distance interval signals, and the object information is generated. , It is possible to shorten the processing time.
  • the processes by the first generation unit 102a to the fifth generation unit 102e are performed in order (that is, in different time zones), but when the load of "information generation" is large, the first is The processing times of the 1 generation unit 102a to the 5th generation unit 102e may overlap (that is, parallel processing may be performed).
  • the object detection system 200 of the present embodiment since the object detection system 200 generates the object information and tracks the object, the distance image is output to the external device 5 and the target is output by the external device 5. It is possible to significantly reduce the amount of information data output to the external device 5 as compared with the case of generating the object information and tracking the object. It is also possible to improve the processing speed by reducing the amount of data to be output.
  • the object information generation unit 102 When at least one of the object information corresponding to each of the plurality of distance sections satisfies a predetermined condition, the object information generation unit 102 further generates the object information or stores the object information in the storage unit 103. The storage is stopped, or the output unit 105 stops the output of the object information.
  • the object information generation unit 102 determines whether or not the detected object is a person by matching the outer shape pattern between the detected object and the model indicating the human pattern, and is not a person. When it is determined that the object information is not important, further generation of the object information or storage of the object information in the storage unit 103 is stopped. Alternatively, the output unit 105 stops the output of the object information. As a result, an object detection system that generates detailed object information by focusing on people as detection targets is realized.
  • the light emitting unit 1 that emits light
  • the light sensor 2 that receives the reflected light reflected by the light within the range that can be measured in the target space
  • the light emitting unit 1 that receives the reflected light reflected by the light within the range that can be measured in the target space
  • the control unit 101a for controlling the sensor 2 and the signal processing unit 101b for processing the information indicated by the electric signal generated by the optical sensor 2 are provided, and the control unit 101a is configured by dividing the distance-measurable range. For each of the plurality of distance sections, the light emitting unit 1 and the optical sensor so that the distance section signal, which is a signal from the pixel that received the light among the plurality of pixels included in the optical sensor 2, is output from the optical sensor 2.
  • the signal processing unit 101b has a plurality of generation units (first generation unit 102a to fifth generation unit 102e) capable of parallel operation, and is based on a distance interval signal output from the optical sensor 2.
  • the object information generation unit 102 that generates the object information indicating the characteristics of the object detected by the optical sensor 2 and the plurality of distance sections generated by the object information generation unit 102 for each of the plurality of distance sections.
  • a storage unit 103 for storing object information corresponding to each and an output unit 105 for outputting object information corresponding to each of a plurality of distance sections are provided, and the object information generation unit 102 includes a plurality of distance sections. For each, the object information is generated by comparing the past object information stored in the storage unit 103 with the characteristics of the current object detected by the optical sensor 2.
  • the object information generation unit 102 has a plurality of generation units (first generation unit 102a to fifth generation unit 102e) capable of parallel operation, and is detected by the optical sensor 2 for a plurality of distance sections. Since the object information indicating the characteristics of the object is generated, the object detection system 200 for detecting the object at high speed is realized.
  • the object detection system 200 includes a composite image generation unit 104 that generates a composite image from a plurality of distance section signals corresponding to a plurality of distance sections output from the optical sensor 2, and the output unit 105 is a composite image generation unit.
  • the composite image generated in 104 is output.
  • the storage unit 103 stores the reference image corresponding to at least one of the plurality of distance sections
  • the object information generation unit 102 stores the distance related to the distance section corresponding to the reference image stored in the storage unit 103.
  • Object information is generated by comparing the section signal with the reference image. As a result, object information indicating the same or different points as the reference image is generated. Therefore, for example, by using a past image as the reference image, it is possible to immediately know only the changed portion.
  • the object information generation unit 102 corrects the object information by using the composite image.
  • the object information corresponding to each of the distance sections is corrected by using the composite image having the overall information over the plurality of distance sections, so that the accuracy of the object information corresponding to each distance section is improved.
  • the accuracy of the center coordinates of the detected object is improved.
  • the object information generation unit 102 corrects the object information by using the distance section signals of a plurality of distance sections. As a result, the object information corresponding to each of the distance sections is corrected by using the distance section signals of the plurality of distance sections, so that the object information having a three-dimensional shape existing over the plurality of distance sections is obtained.
  • the accuracy of is improved. For example, the accuracy of the depth distance to an object having a three-dimensional shape is improved.
  • the object information generation unit 102 generates object position information regarding the position of the object in the three-dimensional space by using the distance interval signals of a plurality of distance sections, and the same past object as the current object.
  • the moving speed of the object is calculated by using the object position information of the object. As a result, the moving speed of the object in the three-dimensional space can be obtained.
  • the object information generation unit 102 calculates the moving speed of the object by approximating the trajectory of the same object as the current object in the past with a curve. As a result, the moving speed of the object is calculated with higher accuracy than the linear approximation.
  • the object information generation unit 102 generates the future predicted position 81 of the object as the object information from the moving speed. As a result, the future predicted position 81 of the object is known in advance.
  • control unit 101a changes the control signals to the light emitting unit 1 and the optical sensor 2 so that the number of distance sections, the distance width of the distance sections, or the target distance section for generating the object information changes.
  • control unit 101a sends the light emitting unit 1 and the optical sensor 2 so that the distance section signal corresponding to the distance section not including the predicted position 81 of the object is output from the optical sensor 2 among the plurality of distance sections. Change the control signal.
  • the object information generation unit 102 further generates the object information or stores the object information in the storage unit 103.
  • the storage is stopped, or the output unit 105 stops the output of the object information.
  • the object detection method is an object detection system 200 including a light emitting unit 1 that emits light and an optical sensor 2 that receives reflected light reflected in a range that can be measured in the target space.
  • a control step for controlling the light emitting unit 1 and the optical sensor 2 and a signal processing step for processing information indicated by an electric signal generated by the optical sensor 2 are included in the object detection method.
  • the distance section signal which is a signal from the pixel that received the light among the plurality of pixels included in the optical sensor 2, is output from the optical sensor 2.
  • the light emitting unit 1 and the optical sensor 2 are controlled so as to be output from the optical sensor 2 by a plurality of generation units (first generation unit 102a to fifth generation unit 102e) capable of parallel operation.
  • An object information generation substep that generates object information indicating the characteristics of an object detected by the optical sensor 2 in a plurality of distance sections based on a distance section signal, and a plurality of distances output from the optical sensor 2.
  • the storage unit 103 stores the composite image generation sub-step for generating a composite image from a plurality of distance section signals corresponding to the section and the object information corresponding to each of the plurality of distance sections generated in the object information generation sub-step.
  • the storage substep includes an object information corresponding to each of the plurality of distance sections and an output substep for outputting a composite image.
  • the storage unit 103 stores each of the plurality of distance sections.
  • the object information is generated by comparing the stored past object information with the characteristics of the current object detected by the optical sensor 2.
  • the object information generation substep a plurality of generation units (first generation unit 102a to fifth generation unit 102e) capable of parallel operation, based on the distance section signals output from the optical sensor 2, a plurality of generation units. Since the object information indicating the characteristics of the object detected by the optical sensor 2 is generated in the distance section, the object detection method for detecting the object at high speed is realized.
  • the program according to one aspect is a program for causing one or more processors to execute the above information processing method.
  • the program may be recorded and provided on a computer-readable medium.
  • the object information generation unit 102 reduces the distance interval to be measured as in the above embodiment when changing the setting stored in the control unit 101a by using the feature amount of the object.
  • the distance may be measured around the position of the center of gravity of the object or the predicted position 81 of the moving destination by reducing the width of the distance section.
  • FIG. 11 is a diagram illustrating an example of distance measurement by the object information generation unit 102 in the modified example of the embodiment. For example, as shown in FIG. 11A, when a person is detected in the distance section R3 in the distance sections R1 to R5, as shown in FIG. 11B, from the next frame.
  • a group of sections obtained by dividing the three distance sections R2 to R4 in the immediately preceding frame into five may be newly set as the distance sections R1 to R5 and the distance may be measured.
  • the control unit 101a changes the control signal to the light emitting unit 1 and the optical sensor 2 so that the distance width of the distance section changes. More specifically, the control unit 101a reduces the distance-measurable range to the distance width including the predicted position of the object, and shortens the distance width of each of the plurality of distance sections.
  • the control signal to the optical sensor 2 is changed. In the above modification, the resolution of the measurement distance is improved, and the accuracy of the object information may be improved.
  • the object information generation unit 102 may detect an object by expanding the range-finding range FR at regular time intervals. According to the above modification, it is possible to find an object that appears at a distant position from an object that has already been detected.
  • the object detection system 200 may generate a distance section signal by an indirect TOF method instead of the direct TOF as in the embodiment.
  • the object information generation unit 102 may include an inter-section information generation unit.
  • the inter-section information generation unit generates object information for each of a plurality of different distance section signals, and then compares the object information generated in different distance sections to determine whether or not they indicate the same object. If it is determined that they are the same object, the object information is regenerated as one object and output to the storage unit 103 and the output unit 105.
  • the object detection system 200 and the object detection method of the present disclosure have been described above based on the embodiments and modifications, but the present disclosure is not limited to these embodiments and modifications. As long as it does not deviate from the gist of the present disclosure, various modifications that can be conceived by those skilled in the art are applied to the present embodiment and the modified examples, and other forms constructed by combining some components in the embodiments and the modified examples. Is also included within the scope of this disclosure.
  • the present disclosure discloses an object detection system that detects an object for each of a plurality of distance sections, particularly an object detection system that detects an object at high speed, for example, an in-vehicle object detection system that is mounted on an automobile and detects an obstacle, an object.
  • an object detection system that detects an object at high speed
  • it can be used as a surveillance camera, a security camera, or the like for detecting a person or the like.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Measurement Of Optical Distance (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2021/020469 2020-06-19 2021-05-28 物体検知システム及び物体検知方法 Ceased WO2021256223A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022532459A JP7503750B2 (ja) 2020-06-19 2021-05-28 物体検知システム及び物体検知方法
CN202180034295.2A CN115552280A (zh) 2020-06-19 2021-05-28 物体检测系统以及物体检测方法
US17/982,104 US20230053841A1 (en) 2020-06-19 2022-11-07 Object detection system and object detection method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-106125 2020-06-19
JP2020106125 2020-06-19

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/982,104 Continuation US20230053841A1 (en) 2020-06-19 2022-11-07 Object detection system and object detection method

Publications (1)

Publication Number Publication Date
WO2021256223A1 true WO2021256223A1 (ja) 2021-12-23

Family

ID=79267889

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/020469 Ceased WO2021256223A1 (ja) 2020-06-19 2021-05-28 物体検知システム及び物体検知方法

Country Status (4)

Country Link
US (1) US20230053841A1 (https=)
JP (1) JP7503750B2 (https=)
CN (1) CN115552280A (https=)
WO (1) WO2021256223A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024204561A1 (ja) * 2023-03-30 2024-10-03 ヌヴォトンテクノロジージャパン株式会社 物体検知装置、物体検知システム及び物体検知方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024202462A1 (ja) * 2023-03-30 2024-10-03 ヌヴォトンテクノロジージャパン株式会社 センシング方法、プログラムおよびセンシング装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002228750A (ja) * 2001-02-05 2002-08-14 Mitsubishi Heavy Ind Ltd レーザレーダ装置及びこれを用いる撮像方法
JP2010008093A (ja) * 2008-06-24 2010-01-14 Toshiba Corp 赤外線撮像装置および赤外線撮像方法
JP2010145255A (ja) * 2008-12-19 2010-07-01 Calsonic Kansei Corp 車両用距離画像データ生成装置及び方法
US8988662B1 (en) * 2012-10-01 2015-03-24 Rawles Llc Time-of-flight calculations using a shared light source

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4854698A (en) * 1987-10-05 1989-08-08 Robotic Vision Systems, Inc. 3-D measurement via multiple gating
JP2006322795A (ja) * 2005-05-18 2006-11-30 Olympus Corp 画像処理装置、画像処理方法および画像処理プログラム
JP2006318062A (ja) * 2005-05-10 2006-11-24 Olympus Corp 画像処理装置、画像処理方法、および画像処理用プログラム
US9217415B2 (en) * 2011-10-14 2015-12-22 Vestas Wind Systems A/S Estimation of wind properties using a light detection and ranging device
CN102736085B (zh) * 2012-06-21 2014-03-12 中国科学院半导体研究所 图像寻的激光成像测距方法及装置
US10249025B2 (en) * 2015-07-06 2019-04-02 The United States Of America As Represented By The Secretary Of The Navy Atmospheric channel characterization system and method using target image information
CN107088071B (zh) * 2016-02-17 2021-10-15 松下知识产权经营株式会社 生物体信息检测装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002228750A (ja) * 2001-02-05 2002-08-14 Mitsubishi Heavy Ind Ltd レーザレーダ装置及びこれを用いる撮像方法
JP2010008093A (ja) * 2008-06-24 2010-01-14 Toshiba Corp 赤外線撮像装置および赤外線撮像方法
JP2010145255A (ja) * 2008-12-19 2010-07-01 Calsonic Kansei Corp 車両用距離画像データ生成装置及び方法
US8988662B1 (en) * 2012-10-01 2015-03-24 Rawles Llc Time-of-flight calculations using a shared light source

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024204561A1 (ja) * 2023-03-30 2024-10-03 ヌヴォトンテクノロジージャパン株式会社 物体検知装置、物体検知システム及び物体検知方法

Also Published As

Publication number Publication date
CN115552280A (zh) 2022-12-30
JPWO2021256223A1 (https=) 2021-12-23
JP7503750B2 (ja) 2024-06-21
US20230053841A1 (en) 2023-02-23

Similar Documents

Publication Publication Date Title
CN110084895B (zh) 对点云数据进行标注的方法和设备
EP3872688B1 (en) Obstacle identification method and device, storage medium, and electronic device
US11257237B2 (en) Optimized exposure control for improved depth mapping
US10345447B1 (en) Dynamic vision sensor to direct lidar scanning
US11568654B2 (en) Object recognition method and object recognition device performing the same
CN107305126B (zh) 记录介质、环境地图的制作系统和方法及更新系统和方法
US11620734B2 (en) Machine-learned depth dealiasing
US12523772B2 (en) Ranging device
CN113160327A (zh) 一种点云补全的实现方法和系统
JP6876445B2 (ja) データ圧縮装置、制御方法、プログラム及び記憶媒体
US20230290153A1 (en) End-to-end systems and methods for streaming 3d detection and forecasting from lidar point clouds
US10812745B2 (en) Bit depth reduction of image pixels
EP4170305A1 (en) System and method for simultaneous online lidar intensity calibration and road marking change detection
US20240183983A1 (en) Systems and methods for pose determination of a mobile subject
WO2021256223A1 (ja) 物体検知システム及び物体検知方法
KR20230079999A (ko) 교통 수신호 인식 시스템 및 방법
CN113450385A (zh) 一种夜间工作工程机械视觉跟踪方法、装置及存储介质
US20220404499A1 (en) Distance measurement apparatus
EP4334675B1 (en) Systems and methods for structured light depth computation using single photon avalanche diodes
US20200213499A1 (en) Active sensor, object identification system, vehicle and vehicle lamp
CN119942142A (zh) 一种基于红外图的透明物体检测方法、系统、设备及存储介质
Khalil et al. Licanext: Incorporating sequential range residuals for additional advancement in joint perception and motion prediction
US20240077615A1 (en) Systems and methods for convolutional high resolution lidar imaging
WO2020105225A1 (ja) 機械学習方法、学習済みモデル、制御プログラム、および物体検知システム
WO2021199225A1 (ja) 情報処理システム、センサシステム、情報処理方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21826700

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022532459

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21826700

Country of ref document: EP

Kind code of ref document: A1