WO2020098297A1

WO2020098297A1 - Method and system for measuring distance to leading vehicle

Info

Publication number: WO2020098297A1
Application number: PCT/CN2019/095980
Authority: WO
Inventors: 陆璐; 梅鵾; 钱浩然; 谢畅; 王恒; 孙谷飞
Original assignee: 众安信息技术服务有限公司
Priority date: 2018-11-15
Filing date: 2019-07-15
Publication date: 2020-05-22
Also published as: SG11202010955SA; JP2021508387A; CN109541583B; JP6851505B2; CN109541583A

Abstract

A method and a system for measuring the distance to a leading vehicle. The method for measuring the distance to a leading vehicle comprises: acquiring an RGB image and a depth image at a front viewing angle of a vehicle (S101); setting a target area frame in the RGB image (S102); extracting size data corresponding to a plurality of vehicles in the RGB image (S103); for each of the plurality of vehicles, according to the size data of the vehicle, determining a constraint frame of the vehicle in the RGB image (S104), calculating an overlapping ratio of the constraint frame of the vehicle to the target area frame (S105), calculating a normalized distance between the vehicle and the target area frame on the basis of the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image (S106); and according to the overlapping ratio and normalized distance corresponding to each of the plurality of vehicles, determining a target leading vehicle from the plurality of vehicles, and obtaining a vehicle distance between the target leading vehicle and the vehicle according to the depth image (S107).

Description

Method and system for detecting distance in front of vehicle

Technical field

The present disclosure relates to the technical field of distance detection, and in particular to a method and system for distance detection of a preceding vehicle.

Background technique

With the development of economy, the number of cars has increased year by year, and safe driving has become a problem that cannot be ignored. After statistical analysis, the proportion of car rear-end accidents in the total traffic accidents remains high, and in order to reduce the occurrence of car rear-end accidents The distance detection and early warning technology came into being. The principle of detection and early warning is that when the vehicle distance is too close and touches the threshold, the driver will be reminded of the collision or automatically take braking measures to reduce the occurrence of rear-end collisions.

Commonly used vehicle distance detection and early warning technologies include two types. The first is a visual solution based on a common camera, which first performs target recognition, and then obtains the distance of the vehicle in front and warns according to the monocular or binocular ranging algorithm; The millimeter wave radar sends electromagnetic waves to the front area and receives echoes to measure the distance, speed and angle of the object in front of it, to get the distance of the vehicle in front and to give early warning. These two early warning technologies have their own advantages and disadvantages. The solution based on the ordinary camera has a lower cost, can accurately identify the position of the front car in the field of view, and provides more semantic information, but the ranging distance and ranging of the visual solution The accuracy is far inferior to millimeter wave radar. In contrast, although millimeter-wave radar has high ranging accuracy, its field of view is relatively narrow, and it cannot return any semantic information. It is difficult to accurately identify the position of the preceding vehicle in two-dimensional space.

Summary of the invention

A brief overview of the present disclosure is given below in order to provide a basic understanding of some aspects of the present disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to be used to determine critical or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

The purpose of the present disclosure is to provide a method and system for detecting the distance between a vehicle in front and capable of ensuring both ranging accuracy and positioning accuracy.

In order to achieve the above objective, an aspect of the present disclosure provides a method for detecting a distance ahead of a vehicle, including:

Obtain RGB images and depth images of the front view of the vehicle;

Setting a target area frame in the RGB image;

Extract size data corresponding to multiple vehicles in the RGB image;

For each vehicle in the plurality of vehicles:

Determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle;

Calculating the overlapping rate of the constraint frame of the vehicle and the target area frame;

Based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image, calculating the normalization of the vehicle and the target area frame Distance

According to the overlap rate and the normalized distance corresponding to each of the plurality of vehicles, determine the target vehicle in front of the plurality of vehicles, and obtain the target vehicle in front of the vehicle and the own vehicle according to the depth image Distance.

In some embodiments, the acquiring RGB images and depth images of the front angle of the vehicle includes:

Collect the RGB image and the depth image at the same time using a depth camera installed in the vehicle; or

The RGB image is collected using a 2D camera installed in the vehicle, and the depth image is collected using a millimeter wave radar / distance sensor installed in the vehicle.

In some embodiments, the determining the constraint frame of the vehicle in the RGB image according to the size data of the vehicle includes:

Construct a virtual coordinate system in the RGB image;

Extract the coordinates of the upper left corner of the vehicle in the RGB image based on the virtual coordinate system, and draw a rectangular constraint frame including the vehicle in the RGB image according to the size data of the vehicle.

In some embodiments, the calculating the overlapping rate of the constraint frame of the vehicle and the target area frame includes:

Use the cross-combination formula

Calculating the intersection ratio IOU of the constraint frame of the vehicle and the target area frame as the overlapping rate,

Wherein, Car represents the bounding box area of the vehicle, and ROI represents the area of the target area box.

In some embodiments, the vehicle is calculated based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image. The normalized distance of the target area frame includes:

Use the normalized distance formula

Calculating the normalized distance between the vehicle and the target area frame,

Where Car.x represents the horizontal coordinate value of the center point of the constraint frame, Car.y represents the vertical coordinate value of the center point of the constraint frame, and Target.x represents the predetermined target point of the target area frame The abscissa value, Target.y represents the ordinate value of the target point, Car.width represents the width data of the vehicle, and Car.height represents the height data of the vehicle.

In some embodiments, the target area frame is a trapezoidal area frame directly in front of the vehicle selected from the RGB image, and the target point is a center point of the target area frame.

In some embodiments, the target vehicle ahead is determined from the plurality of vehicles according to the overlapping rate and normalized distance corresponding to each vehicle of the plurality of vehicles, and the front The distance between the target vehicle and the vehicle includes:

Selecting vehicles from the plurality of vehicles whose overlap rate is greater than a first overlap rate threshold or the normalized distance is less than a distance threshold as a first target vehicle set;

Selecting vehicles from the first target vehicle set with the overlap rate greater than a second overlap rate threshold as a second target vehicle set, wherein the second overlap rate threshold is less than the first overlap rate threshold;

Selecting the vehicle with the largest overlap rate from the second target vehicle set as the preceding vehicle target vehicle;

Extract the vehicle distance corresponding to the preceding vehicle target vehicle from the depth image as the vehicle distance between the preceding vehicle target vehicle and the own vehicle.

In some embodiments, the preceding vehicle distance detection method may further include:

Perform the median filtering, abnormal window detection processing and Kalman filtering on the distance between the target vehicle in front of the vehicle and the vehicle in a predetermined length of time window centered on the current time to obtain the optimized target vehicle in front of the vehicle Distance

Wherein, the abnormal window detection process detects an abnormal window in the predetermined length of time window, and replaces the abnormal window with a vehicle distance fitting result calculated based on the vehicle distance values in time windows before and after the abnormal window Car distance value.

Another aspect of the present disclosure provides a front vehicle distance detection system, including:

The image acquisition unit is configured to acquire RGB images and depth images of the front angle of the vehicle;

A target area frame setting unit configured to preset a target area frame in the RGB image;

A size data extraction unit configured to extract size data corresponding to multiple vehicles in the RGB image;

A constraint frame determination unit configured to determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle for each of the plurality of vehicles;

An overlapping rate calculation unit configured to calculate the overlapping rate of the constraint frame of the vehicle and the target area frame for each of the plurality of vehicles;

The distance calculation unit is configured to, for each vehicle of the plurality of vehicles, based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the target area frame in the The position in the RGB image, calculating the normalized distance between the vehicle and the target area frame;

The preceding vehicle target vehicle determining unit is configured to determine the preceding vehicle target vehicle from the plurality of vehicles according to the overlap rate and the normalized distance corresponding to each vehicle of the plurality of vehicles, and according to the depth image Obtain the distance between the preceding target vehicle and the own vehicle.

In some embodiments, the image acquisition unit is configured to:

In some embodiments, the constraint box determination unit is configured to:

Construct a virtual coordinate system in the RGB image;

In some embodiments, the overlapping rate calculation unit is configured to:

Use the cross-combination formula

In some embodiments, the distance calculation unit is configured to:

Use the normalized distance formula

In some embodiments, the preceding vehicle target vehicle determination unit is configured to:

In some embodiments, the preceding vehicle distance detection system may further include:

A motion filtering unit configured to sequentially perform median filtering, anomalous window detection processing, and Kalman filtering on the distance between the preceding target vehicle and the own vehicle in a predetermined length of time window centered on the current time , To get the optimized distance between the target vehicle in front and the vehicle,

Yet another aspect of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the processor is caused to perform the following processing:

Obtain RGB images and depth images of the front view of the vehicle;

Setting a target area frame in the RGB image;

Extract size data corresponding to multiple vehicles in the RGB image;

For each vehicle in the plurality of vehicles:

According to one or more embodiments of the present disclosure, a preceding vehicle distance detection method and system capable of simultaneously ensuring ranging accuracy and positioning accuracy can be realized.

BRIEF DESCRIPTION

The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments and descriptions of the present disclosure are used to explain the present disclosure and do not constitute an undue limitation on the present disclosure. In the drawings:

FIG. 1 is a schematic flowchart of a method for detecting a distance ahead of a vehicle according to an embodiment of the present disclosure;

2 is an exemplary structural block diagram of a preceding vehicle distance detection system according to an embodiment of the present disclosure;

3 is a block diagram of an exemplary configuration of a computing device that can implement embodiments of the present disclosure.

detailed description

To make the above objectives, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, but not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

FIG. 1 is a schematic flowchart of a method for detecting a distance ahead of a vehicle according to an embodiment of the present disclosure.

As shown in FIG. 1, in step S101, an RGB image and a depth image of the front angle of the host vehicle are acquired.

In some embodiments, the RGB image and the depth image of the front view of the vehicle can be collected in real time by an image acquisition unit installed in the vehicle (for example, the front of the vehicle). The RGB image may include all vehicles in the current perspective. In addition, the depth image can extract the distance information between each vehicle and the own vehicle in the RGB image.

In some embodiments, a depth camera installed in the vehicle may be used to simultaneously acquire RGB images and depth images of the front view angle. In other embodiments, a 2D camera mounted on the vehicle can be used to collect RGB images in the front view, and a millimeter wave radar / distance sensor mounted on the front of the vehicle can be used to collect depth images in the front view.

In the specific implementation process, if the depth camera is selected, the RGB image and the depth image of the front angle of the vehicle collected have been automatically matched, that is, the distance of each vehicle can be directly extracted from the depth image without the need for a depth camera Match and adjust it again during installation; if you choose the combination of 2D camera and millimeter wave radar / distance sensor, you need to match and correct the 2D camera and millimeter wave radar / distance sensor when installing the RGB image and The depth image can match the correspondence. Exemplarily, the depth camera may be an Intel RealSense active infrared stereo depth camera D435.

In step S102, a target area frame is set in the RGB image.

The target area box indicates the area where the vehicle in front may appear. In some embodiments, the target area frame may be set directly in front of the vehicle. In some embodiments, the target area frame may be a trapezoidal area frame. In some embodiments, a fixed point may be selected as a target point in the target area frame, and the target point may represent a desired position where the preceding vehicle appears. In some embodiments, the target point may be the center point of the target area frame.

In some embodiments, the trapezoidal area directly in front of the depth camera or 2D camera can be selected from the RGB image as the target area frame; the center point of the selected target area frame is defined as the target point. Of course, in the actual operation process, the settings of the target area frame and the target point can also be fine-tuned according to the experience of the engineer. For example, when the depth camera is installed on the front left of the vehicle, the target area frame and target point can be set to the left of the center of the RGB image.

In step S103, size data corresponding to a plurality of vehicles in the RGB image is extracted.

In some embodiments, a vehicle detection algorithm may be used to identify all vehicles in the RGB image, and correspondingly extract the size data of each vehicle. In some embodiments, the size data of the vehicle may include width data, height data, and the like of the vehicle.

In some embodiments, a pre-trained vehicle detection model may be used to obtain all vehicles in the RGB image and identify the size data of each vehicle in the RGB image. Among them, the vehicle detection model can use, for example, a target detection algorithm (such as Faster RCNN, SSD, YOLO, etc.), which is obtained by training using the COCO data set.

In step S104, for each of the plurality of vehicles, the constraint frame of the vehicle is determined in the RGB image based on the size data of the vehicle.

In some embodiments, a virtual coordinate system can be constructed in the RGB image. For example, a virtual coordinate system can be constructed in an RGB image with a depth camera or a 2D camera as the origin. Then, extract the coordinates of the upper left corner of each vehicle in the RGB image based on the virtual coordinate system, and draw the rectangular constraint frame including the vehicle in the RGB image according to the size data of the corresponding vehicle obtained.

In the specific implementation, since the coordinates of the upper left corner of the vehicle can be obtained, and the size data (such as width and height data) of the vehicle is also known, based on the upper left corner of the vehicle, the width of the vehicle in the RGB image is used as the rectangular constraint frame The width of the car, with the height of the vehicle in the image as the length of the rectangular constrained frame, can quickly draw the rectangular constrained frame corresponding to the vehicle in the RGB image.

It should be understood that in the above method, the coordinates of other points of the vehicle in the RGB image (for example, the upper right corner point, the lower left corner point, the lower right corner point, etc.) may also be extracted to draw the constraint frame. In addition, the shape of the constraint frame is not limited to a rectangle, but may be any shape designed according to actual needs.

In step S105, for each of a plurality of vehicles, the overlapping rate of the constraint frame of the vehicle and the target area frame is calculated.

In some embodiments, the intersection ratio of the vehicle and the target area may be calculated based on the bounding frame area of the vehicle and the target area frame area as the overlap ratio.

Specifically, you can use the cross-combination formula

Calculate the IOU of the bound frame of each vehicle and the target area frame, where Car represents the area of the bound frame of the vehicle and ROI represents the area of the target area frame.

In step S106, for each of the plurality of vehicles, based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image, calculate the vehicle and the The normalized distance of the target area frame.

In some embodiments, a normalized distance formula may be used

To calculate the normalized distance between the vehicle and the target area frame, where Car.x represents the abscissa value of the center point of the constraint frame, Car.y represents the ordinate value of the center point of the constraint frame, Target.x Represents the abscissa value of the target, Target.y represents the ordinate value of the target, Car.width represents the width data of the vehicle, and Car.height represents the height data of the vehicle.

In step S107, according to the overlap rate and the normalized distance corresponding to each of the plurality of vehicles, a preceding vehicle target vehicle is determined from the plurality of vehicles, and the preceding vehicle target vehicle is obtained according to the depth image The distance from the car.

In some embodiments, an unsupervised front vehicle screening algorithm may be used to lock the target vehicle in front of the vehicle based on the overlap rate and normalized distance corresponding to each vehicle.

The unsupervised preceding vehicle screening algorithm is as follows:

Count the overlap rate and normalized distance of multiple vehicles in RGB images respectively;

Select vehicles from multiple vehicles whose overlap rate is greater than the first overlap rate threshold or normalized distance is less than the distance threshold as the first target vehicle set;

Screening vehicles from the first target vehicle set with an overlap rate greater than the second overlap rate threshold as the second target vehicle set, where the second overlap rate threshold is less than the first overlap rate threshold;

The vehicle distance corresponding to the preceding vehicle target vehicle is extracted from the depth image as the vehicle distance between the preceding vehicle target vehicle and the own vehicle.

The first overlap rate threshold, the second overlap rate threshold, and the distance threshold can be arbitrarily selected according to actual experience, and the disclosure does not limit the selection of the threshold.

It should be noted that, in addition to using the unsupervised front-vehicle screening algorithm to locate the preceding vehicle target vehicle, the foregoing embodiment may also use a neural network front-vehicle screening algorithm to locate the preceding vehicle target vehicle. The specific method is:

The labeled training sample set S = {xi, yi} is generated by pre-training to filter the target vehicle in front, where xi represents the vehicle characteristics, including three dimensional vectors, which are the overlap rate (intersection and merge ratio IOU), normalization The distance Dist_norm and the distance to the own vehicle, yi represents the screening result of the target vehicle in front, and yi will automatically output the recognition result based on the three-dimensional vector values in xi. For example, when the output of yi is 1, it indicates that the vehicle is a target vehicle in front, and when the output of yi is 0, it indicates that the vehicle is not a target vehicle in front. The classifier can select different neural network frameworks to train the sample set, and the neural network frameworks can be AlexNet, VGG, etc.

In the method for detecting the distance to the preceding vehicle provided in this embodiment, a combination of RGB image and depth image is adopted, which can accurately locate the position of each vehicle in the RGB image and the distance from the vehicle, and can simultaneously ensure the ranging accuracy and positioning Accuracy. In addition, based on the overlap rate of each vehicle and the normalized distance, an unsupervised front vehicle screening algorithm is used to lock the target vehicle in front of the vehicle, and the distance between the target vehicle in front is combined with the depth image to achieve accurate and fast detection of the distance in front of the vehicle.

In the embodiment of the present disclosure, the distance between the target vehicle in front of the vehicle and the own vehicle can also be optimized.

In some embodiments, median distance filtering, anomaly window detection processing, and Kalman filtering may be sequentially performed on the distance between the target vehicle in front of the vehicle and the vehicle in a predetermined length of time window centered on the current time to obtain the optimized The distance between the target vehicle in front and the own vehicle.

Since the accuracy of the target detection algorithm is not 100%, it will cause a certain false detection rate due to some isolated noise in the frame jump before and after, and the isolated noise can be smoothed out by median filtering. Among them, the median filter is a commonly used time-series filtering algorithm. The isolated noise caused by the false detection of the target detection algorithm is similar to salt and pepper noise, showing the characteristics of pulses. The median filter can be used to remove it, that is, at the current time as the center In a time window with a length of Tn, after sorting the distance of the target vehicle in front of the vehicle, the median is selected as the filtered distance at the current time. For example, it is generally better to set Tn to 5.

Screening errors due to interference of adjacent lanes and other factors occur from time to time. Existing filtering methods for the above interference factors through threshold comparison methods have a large probability of false screening. It is found through experiments that the false screening appears in the detection results as an abnormal window with a maintenance time of Te (different from the noise in the median filter). This abnormal window is also called an isolated window. The values before and after the isolated window are approximately continuous. The abnormal window detection is used to fit the distance between the target vehicle in front of and behind the isolated window, and the value in the isolated window is replaced with the result of the distance fitting to achieve accurate filtering of the above interference factors. For example, a third-order polynomial function fitting can be used to fit the target vehicle distance between the front and back of the isolated window, and the interpolation calculation result of the function is used as the distance fitting result instead of the value in the isolated window.

After median filtering and abnormal window detection, visual errors caused by RGB image recognition have been basically eliminated, and measurement errors caused by depth image measurement can be eliminated by using Kalman filtering. Kalman filtering is an optimized autoregressive data processing algorithm. In a dynamic system where the state is approximately linear and the measurement result is disturbed by Gaussian noise, the regression data processing algorithm is known when both the state transition equation and the measurement variance are known. Filtering the measured values can be used in the fields of robot navigation, control, sensor data fusion, radar missile tracking, and computer graphics processing. The state transition equation of Kalman filter is as follows:

X (k) = AX (k-1) + BU (k) + W (k)

X (k) represents the state variable of the system at time k, which is the distance between the target vehicle in front of the vehicle and the target vehicle speed in this embodiment; A is the state transition matrix, which can be obtained by using the first-order constant speed model Out; BU (k) means external control items; W (k) is the state change caused by other unknown interference, in the absence of more information, it can be replaced by Gaussian noise with known variance, the larger the variance is set , Representing less confidence in the equation of state, that is, the higher the randomness of the movement of the distance ahead.

During specific implementation, the time window maintained by false detection and missed detection is about 1-15 frames, showing a peak-noise-like property. In this embodiment, the median distance between the target vehicle in front of the vehicle and the vehicle is sequentially filtered and The abnormal window detection process can remove the false detection and missed detection noise, and improve the accuracy of the screening of the target vehicle in front. In addition, Kalman filtering is performed on the distance between the target vehicle in front and the vehicle to obtain the optimized distance between the target vehicle in front and the vehicle, which can ensure the measurement accuracy of the distance between vehicles.

FIG. 2 is an exemplary structural block diagram of a preceding vehicle distance detection system according to an embodiment of the present disclosure.

In some embodiments, the system 200 may include a processing circuit 201. The processing circuit 201 of the system 200 provides various functions of the system 200. In some embodiments, the processing circuit 201 of the system 200 may be configured to perform the preceding vehicle distance detection method described above with reference to FIG. 1.

The processing circuit 201 may refer to various implementations of digital circuitry, analog circuitry, or mixed-signal (combination of analog and digital) circuitry that performs functions in a computing system. The processing circuit may include, for example, a circuit such as an integrated circuit (IC), an application specific integrated circuit (ASIC), a part or circuit of a separate processor core, an entire processor core, a separate processor, such as a field programmable gate array (FPGA) Programmable hardware devices, and / or systems that include multiple processors.

In some embodiments, the processing circuit 201 may include an image acquisition unit 202, a target area setting unit 203, a size data extraction unit 204, a constraint frame determination unit 205, an overlap ratio calculation unit 206, a distance calculation unit 207, a preceding vehicle target vehicle Determination unit 208.

The image acquisition unit 202 is configured to acquire an RGB image and a depth image of the front angle of the vehicle; the target area frame setting unit 203 is configured to preset the target area frame in the RGB image; the size data extraction unit 204 is configured to extract the RGB image The size data corresponding to the plurality of vehicles in; the constraint frame determination unit 205 is configured to determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle for each vehicle in the plurality of vehicles; The overlap rate calculation unit 206 is configured to calculate the overlap rate of the constraint frame of the vehicle and the target area frame for each of the plurality of vehicles; the distance calculation unit 207 is configured to target the plurality of vehicles For each vehicle in, based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image, calculate the relationship between the vehicle and the target area frame Normalized distance; the preceding vehicle target vehicle determining unit 208 is configured to determine the preceding vehicle target vehicle from the plurality of vehicles according to the overlap rate and the normalized distance corresponding to each vehicle of the plurality of vehicles, and The distance between the preceding target vehicle and the own vehicle is obtained from the depth image. The above units 202 to 208 may be configured to execute steps S101 to S107 in the preceding vehicle distance detection method shown in FIG. 1, respectively.

In some embodiments, the system 200 may also include memory (not shown). The memory of the system 200 may store the information generated by the processing circuit 201 and the programs and data used for the operation of the system 200. The memory may be volatile memory and / or non-volatile memory. For example, the memory may include, but is not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read only memory (ROM), and flash memory. In addition, the system 200 may be implemented at the chip level, or may also be implemented at the device level by including other external components.

In some embodiments, the system 200 may include a motion filtering unit (not shown) configured to sequentially median the distance between the target vehicle in front and the own vehicle in a predetermined length of time window centered on the current time Filtering, anomaly window detection processing and Kalman filtering to obtain an optimized distance between the target vehicle in front and the own vehicle, wherein the anomaly window detection processing detects an anomalous window in the predetermined length of time window, using The vehicle distance fitting result calculated by the vehicle distance value in the time window before and after the abnormal window replaces the vehicle distance value in the abnormal window.

It should be understood that the above units are only logical modules divided according to the specific functions they implement, and are not intended to limit specific implementations. In actual implementation, the above units may be implemented as independent physical entities, or may be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.).

The preceding vehicle distance detection system provided by the embodiment of the present disclosure and the preceding vehicle distance detection method provided by the embodiments of the present disclosure belong to the same inventive concept, and can execute the preceding vehicle distance detection method provided by any embodiment of the present disclosure, and have The corresponding functional modules and beneficial effects of the detection method. For technical details that are not described in detail in this embodiment, reference may be made to the preceding vehicle distance detection method provided in the embodiments of the present disclosure, and details are not described herein again.

The computing device 300 is an example of a hardware device to which the above-mentioned aspects of the present disclosure can be applied. The computing device 300 may be any machine configured to perform processing and / or calculations. The computing device 300 may be, but not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal data assistant (PDA), smart phone, in-vehicle computer, or combination thereof.

As shown in FIG. 3, the computing device 300 may include one or more elements that may connect or communicate with the bus 302 via one or more interfaces. The bus 302 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, etc. The computing device 300 may include, for example, one or more processors 304, one or more input devices 306, and one or more output devices 308. The one or more processors 304 may be any kind of processors, and may include, but are not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). The processor 304 may correspond to, for example, the processing circuit 201 in FIG. 2 and is configured to implement the functions of the units of the preceding vehicle distance detection system of the present disclosure. The input device 306 may be any type of input device capable of inputting information to a computing device, and may include, but is not limited to, a mouse, keyboard, touch screen, microphone, and / or remote controller. The output device 308 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer.

The computing device 300 may also include or be connected to a non-transitory storage device 314, which may be any non-transitory storage device that can implement data storage, and may include but is not limited to disk drives, optical Storage devices, solid-state memory, floppy disks, flexible disks, hard disks, magnetic tape, or any other magnetic media, compact disks, or any other optical media, cache memory, and / or any other memory chips or modules, and / or computers can read data from them , Instructions and / or any other medium of code. The computing device 300 may also include random access memory (RAM) 310 and read-only memory (ROM) 312. The ROM 312 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 310 may provide volatile data storage and store instructions related to the operation of the computing device 300. The computing device 300 may also include a network / bus interface 316 coupled to the data link 318. The network / bus interface 316 may be any kind of device or system capable of enabling communication with external devices and / or networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as Bluetooth ^(TM) device, an 802.11 device, WiFi equipment, WiMax, cellular communication facilities, etc.).

In addition, one or more embodiments of the present disclosure may be implemented as follows.

Scheme 1: A vehicle distance detection device, including:

One or more processors;

A memory on which computer-executable instructions are stored, which when executed by the one or more processors cause the one or more processors to:

Obtain RGB images and depth images of the front view of the vehicle;

Setting a target area frame in the RGB image;

Extract size data corresponding to multiple vehicles in the RGB image;

For each vehicle in the plurality of vehicles:

Solution 2: In the preceding vehicle distance detection device of Solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

Solution 3: In the preceding vehicle distance detection device of solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

Construct a virtual coordinate system in the RGB image;

Solution 4: In the preceding vehicle distance detection device of solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

Use the cross-combination formula

Solution 5: In the preceding vehicle distance detection device of solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

Use the normalized distance formula

Scheme 6: In the preceding vehicle distance detection device of Scheme 5, the target area frame is a trapezoidal area frame directly in front of the vehicle selected from the RGB image, and the target point is a center point of the target area frame .

Solution 7: In the preceding vehicle distance detection device of Solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

Solution 8: In the preceding vehicle distance detection device of solution 1, the computer-executable instructions, when executed by the one or more processors, cause the one or more processors to:

All the above optional technical solutions may be combined in any combination to form optional embodiments of the present disclosure, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed method, device, and system may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.

It should be noted that, in the description of the present disclosure, the terms “first”, “second”, etc. are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or order. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

The various aspects, implementations, specific implementations or features of the foregoing examples can be used individually or in any combination. The various aspects of the foregoing embodiments may be implemented by software, hardware, or a combination of hardware and software.

For example, the aforementioned embodiments may be embodied as computer-readable codes on a computer-readable medium. The computer-readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of computer-readable media include read-only memory, random access memory, CD-ROM, DVD, magnetic tape, hard disk drives, solid-state drives, and optical data storage devices. The computer-readable medium can also be distributed among network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion.

For example, the foregoing embodiments may take the form of hardware circuits. Hardware circuits may include combined logic circuits, clock storage devices (such as floppy disks, flip-flops, latches, etc.), finite state machines, memories such as static random access memory or embedded dynamic random access memory, custom-designed circuits, Any combination of programmable logic arrays, etc.

The above are only specific implementations of the present disclosure, but the scope of protection of the present disclosure is not limited to this, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present disclosure. It should be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

A method for detecting the distance ahead of a vehicle is characterized by comprising:

Obtain RGB images and depth images of the front view of the vehicle;

Setting a target area frame in the RGB image;

Extract size data corresponding to multiple vehicles in the RGB image;

For each vehicle in the plurality of vehicles:

Determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle;

Calculating the overlapping rate of the constraint frame of the vehicle and the target area frame;

Based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image, calculating the normalization of the vehicle and the target area frame Distance

According to the overlap rate and the normalized distance corresponding to each of the plurality of vehicles, determine the target vehicle in front of the plurality of vehicles, and obtain the target vehicle in front of the vehicle and the own vehicle according to the depth image Distance.
The method according to claim 1, wherein the acquiring the RGB image and the depth image of the front angle of the vehicle include:

Collect the RGB image and the depth image at the same time using a depth camera installed in the vehicle; or

The RGB image is collected using a 2D camera installed in the vehicle, and the depth image is collected using a millimeter wave radar / distance sensor installed in the vehicle.
The method according to claim 1, wherein the determining the constraint frame of the vehicle in the RGB image according to the size data of the vehicle comprises:

Construct a virtual coordinate system in the RGB image;

Extract the coordinates of the upper left corner of the vehicle in the RGB image based on the virtual coordinate system, and draw a rectangular constraint frame including the vehicle in the RGB image according to the size data of the vehicle.
The method according to claim 1, wherein the calculating the overlapping rate of the constraint frame of the vehicle and the target area frame comprises:

Use the cross-combination formula
Calculating the intersection ratio IOU of the constraint frame of the vehicle and the target area frame as the overlapping rate,

Wherein, Car represents the bounding box area of the vehicle, and ROI represents the area of the target area box.
The method according to claim 1, characterized in that, based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image and the target area frame in the RGB image For the location, calculating the normalized distance between the vehicle and the target area frame includes:

Use the normalized distance formula
Calculating the normalized distance between the vehicle and the target area frame,

Where Car.x represents the horizontal coordinate value of the center point of the constraint frame, Car.y represents the vertical coordinate value of the center point of the constraint frame, and Target.x represents the predetermined target point of the target area frame The abscissa value, Target.y represents the ordinate value of the target point, Car.width represents the width data of the vehicle, and Car.height represents the height data of the vehicle.
The method according to claim 5, wherein the target area frame is a trapezoidal area frame directly in front of the vehicle selected from the RGB image, and the target point is a center point of the target area frame.
The method according to claim 1, wherein the target vehicle in front of the vehicle is determined from the plurality of vehicles according to the overlap rate and the normalized distance corresponding to each vehicle in the plurality of vehicles, and Obtaining the distance between the preceding target vehicle and the own vehicle according to the depth image includes:

Selecting vehicles from the plurality of vehicles whose overlap rate is greater than a first overlap rate threshold or the normalized distance is less than a distance threshold as a first target vehicle set;

Selecting vehicles from the first target vehicle set with the overlap rate greater than a second overlap rate threshold as a second target vehicle set, wherein the second overlap rate threshold is less than the first overlap rate threshold;

Selecting the vehicle with the largest overlap rate from the second target vehicle set as the preceding vehicle target vehicle;

Extract the vehicle distance corresponding to the preceding vehicle target vehicle from the depth image as the vehicle distance between the preceding vehicle target vehicle and the own vehicle.
The method according to claim 1, wherein the method further comprises:

Perform the median filtering, abnormal window detection processing and Kalman filtering on the distance between the target vehicle in front of the vehicle and the vehicle in a predetermined length of time window centered on the current time to obtain the optimized target vehicle in front of the vehicle Distance

Wherein, the abnormal window detection process detects an abnormal window in the predetermined length of time window, and replaces the abnormal window with a vehicle distance fitting result calculated based on the vehicle distance values in time windows before and after the abnormal window Car distance value.
A vehicle distance detection system is characterized by including:

The image acquisition unit is configured to acquire RGB images and depth images of the front angle of the vehicle;

A target area frame setting unit configured to preset a target area frame in the RGB image;

A size data extraction unit configured to extract size data corresponding to multiple vehicles in the RGB image;

A constraint frame determination unit configured to determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle for each of the plurality of vehicles;

An overlapping rate calculation unit configured to calculate the overlapping rate of the constraint frame of the vehicle and the target area frame for each of the plurality of vehicles;

The distance calculation unit is configured to, for each vehicle of the plurality of vehicles, based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the target area frame in the The position in the RGB image, calculating the normalized distance between the vehicle and the target area frame;

The preceding vehicle target vehicle determining unit is configured to determine the preceding vehicle target vehicle from the plurality of vehicles according to the overlap rate and the normalized distance corresponding to each vehicle of the plurality of vehicles, and according to the depth image Obtain the distance between the preceding target vehicle and the own vehicle.
The system according to claim 9, wherein the image acquisition unit is configured to:

Collect the RGB image and the depth image at the same time using a depth camera installed in the vehicle; or

The RGB image is collected using a 2D camera installed in the vehicle, and the depth image is collected using a millimeter wave radar / distance sensor installed in the vehicle.
The system according to claim 9, wherein the constraint frame determination unit is configured to:

Construct a virtual coordinate system in the RGB image;

Extract the coordinates of the upper left corner of the vehicle in the RGB image based on the virtual coordinate system, and draw a rectangular constraint frame including the vehicle in the RGB image according to the size data of the vehicle.
The system according to claim 9, wherein the overlapping rate calculation unit is configured to:

Use the cross-combination formula
Calculating the intersection ratio IOU of the constraint frame of the vehicle and the target area frame as the overlapping rate,

Wherein, Car represents the bounding box area of the vehicle, and ROI represents the area of the target area box.
The system according to claim 9, wherein the distance calculation unit is configured to:

Use the normalized distance formula
Calculating the normalized distance between the vehicle and the target area frame,

Where Car.x represents the horizontal coordinate value of the center point of the constraint frame, Car.y represents the vertical coordinate value of the center point of the constraint frame, and Target.x represents the predetermined target point of the target area frame The abscissa value, Target.y represents the ordinate value of the target point, Car.width represents the width data of the vehicle, and Car.height represents the height data of the vehicle.
The system according to claim 13, wherein the target area frame is a trapezoidal area frame directly in front of the vehicle selected from the RGB image, and the target point is a center point of the target area frame.
The system according to claim 9, wherein the preceding vehicle target vehicle determination unit is configured to:

Selecting vehicles from the plurality of vehicles whose overlap rate is greater than a first overlap rate threshold or the normalized distance is less than a distance threshold as a first target vehicle set;

Selecting vehicles from the first target vehicle set with the overlap rate greater than a second overlap rate threshold as a second target vehicle set, wherein the second overlap rate threshold is less than the first overlap rate threshold;

Selecting the vehicle with the largest overlap rate from the second target vehicle set as the preceding vehicle target vehicle;

Extract the vehicle distance corresponding to the preceding vehicle target vehicle from the depth image as the vehicle distance between the preceding vehicle target vehicle and the own vehicle.
The system according to claim 9, wherein the system further comprises:

A motion filtering unit configured to sequentially perform median filtering, anomalous window detection processing, and Kalman filtering on the distance between the preceding target vehicle and the own vehicle in a predetermined length of time window centered on the current time , To get the optimized distance between the target vehicle in front and the vehicle,

Wherein, the abnormal window detection process detects an abnormal window in the predetermined length of time window, and replaces the abnormal window with a vehicle distance fitting result calculated based on the vehicle distance values in time windows before and after the abnormal window Car distance value.
A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the processor is caused to perform the following processing:

Obtain RGB images and depth images of the front view of the vehicle;

Setting a target area frame in the RGB image;

Extract size data corresponding to multiple vehicles in the RGB image;

For each vehicle in the plurality of vehicles:

Determine the constraint frame of the vehicle in the RGB image according to the size data of the vehicle;

Calculating the overlapping rate of the constraint frame of the vehicle and the target area frame;

Based on the size data of the vehicle, the position of the constraint frame of the vehicle in the RGB image, and the position of the target area frame in the RGB image, calculating the normalization of the vehicle and the target area frame Distance

According to the overlap rate and the normalized distance corresponding to each of the plurality of vehicles, determine the target vehicle in front of the plurality of vehicles, and obtain the target vehicle in front of the vehicle and the own vehicle according to the depth image Distance.