WO2021012254A1

WO2021012254A1 - Target detection method, system, and mobile platform

Info

Publication number: WO2021012254A1
Application number: PCT/CN2019/097680
Authority: WO
Inventors: 刘寒颖; 邱凡; 李星河
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-07-25
Filing date: 2019-07-25
Publication date: 2021-01-28
Also published as: CN111712731A

Abstract

A target detection method, a system, and a mobile platform. The method comprises: acquiring a target to be processed detected by a millimeter-wave radar within a preset region; acquiring target auxiliary information detected by an auxiliary sensor within the preset region; and using the target auxiliary information to optimize said target, and obtaining the resulting detected target. The invention employs an auxiliary sensor to optimize a target detected by a millimeter-wave radar, thereby solving the problem in which a millimeter-wave radar has a low resolution and is less sensitive to weak reflectors, and increasing the accuracy of a target detection result.

Description

Target detection method, system and movable platform

Technical field

The embodiments of the present application relate to target detection technology, and in particular to a target detection method, system and movable platform.

Background technique

With the development of science and technology, smart driving cars have gradually become familiar to people. In practical applications, smart driving cars can use the on-board sensing system to obtain the surrounding environment of the vehicle, and feed back the obtained road, vehicle position and obstacle information to the smart driving unit. The smart driving unit receives the road condition information and calculates the control signal according to the road condition information. , And send the control signal to the vehicle control unit, and the vehicle control unit controls the vehicle to drive safely and reliably on the road. Target detection is of great significance to intelligent driving and affects its planning and navigation strategies. Target detection can be based on millimeter wave radar. Specifically, millimeter wave radar transmits electromagnetic waves and receives reflected echoes of targets to provide a basis for driving safety judgments.

However, the resolution capability of millimeter-wave radar is generally low. Taking detection of vehicles as an example, a car may only return several millimeter-wave radar point data, which cannot accurately describe the contour and shape of the vehicle; for example, for weak reflectors (such as Pedestrians, bicycles, etc.) are not sensitive and easily lead to missed inspections. Therefore, there may be some problems with target detection results based only on millimeter wave radar.

Summary of the invention

The embodiments of the present application provide a target detection method, system, and movable platform to solve the problem of target detection results based only on millimeter wave radar.

In the first aspect, an embodiment of the present application provides a target detection method, including:

Obtain the target to be processed detected by the millimeter wave radar in the preset area;

Acquiring target auxiliary information detected by the auxiliary sensor in the preset area;

The target auxiliary information is used to optimize the target to be processed to obtain a detection target.

In the second aspect, an embodiment of the present application provides a target detection system, including a millimeter wave radar, an auxiliary sensor, a memory, and a processor, where:

The memory is used to store program instructions;

The processor is configured to execute the program instructions, and when the program instructions are executed, the processor executes the following steps:

In a third aspect, an embodiment of the present application provides a movable platform, including: a movable platform body, a millimeter wave radar, an auxiliary sensor, and a target detection system; the millimeter wave radar and the auxiliary sensor are arranged on the movable platform On the body, the movable platform body and the target detection system are connected wirelessly or wiredly;

The target detection system is used to obtain the target to be processed detected by the millimeter-wave radar in the preset area; to obtain the target auxiliary information detected by the auxiliary sensor in the preset area; Optimize to obtain the detection target.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer-executable instructions stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the above first aspect and the first aspect are implemented. In terms of various possible designs, the target detection method described.

The target detection method, system and movable platform provided in this embodiment, the method obtains the target to be processed detected by the millimeter wave radar in the preset area, obtains the target assistance information detected by the auxiliary sensor in the preset area, and uses the target assistance The information optimizes the above-mentioned target to be processed to obtain the detection target, that is, the auxiliary sensor is used to optimize the target detected by the millimeter wave radar, which overcomes the problems of low resolution of the millimeter wave radar and insensitivity to weak reflectors, and improves the target The accuracy of the test results.

Description of the drawings

The drawings here are incorporated into the specification and constitute a part of the specification, show embodiments that conform to the application, and are used together with the specification to explain the principle of the application.

FIG. 1 is a schematic diagram of the architecture of a target detection system provided by an embodiment of the application;

FIG. 2 is a schematic flowchart of a target detection method provided by an embodiment of the application;

FIG. 3 is a schematic diagram of the multipath effect provided by an embodiment of the application;

4 is a schematic flowchart of another target detection method provided by an embodiment of the application;

5 is a schematic diagram of the hardware structure of a target detection device provided by an embodiment of the application;

6 is a schematic diagram of the hardware structure of another target detection device provided by an embodiment of the application;

FIG. 7 is a schematic diagram of the hardware structure of the target detection system provided by an embodiment of the application;

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the application.

Through the above drawings, the specific embodiments of the present application have been shown, which will be described in more detail below. These drawings and text description are not intended to limit the scope of the concept of the present application in any way, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments.

Detailed ways

Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

Target detection is of great significance to intelligent driving and affects its planning and navigation strategies. With the rapid development of deep learning today, the monocular camera sensor has been relatively mature for two-dimensional image target detection, but the estimation accuracy of depth is still inaccurate, and it is greatly affected by weather and light; binocular cameras can be achieved through matching Ranging, but also affected by weather and ambient light; laser sensors can achieve accurate ranging of targets, but the range is limited, it is difficult to detect targets 100m away stably, and the current cost is very high, and it also works in poor weather. Affected. Relatively speaking, millimeter-wave radar has a short wavelength and good penetrability. It is hardly affected by weather and light. It has a long measurement distance and can also measure target speed.

In order to solve this technical problem, this embodiment provides a target detection method, which uses auxiliary sensors to optimize the targets detected by the millimeter wave radar, and overcomes the problems of low resolution of the millimeter wave radar and insensitivity to weak reflectors. , Improve the accuracy of target detection results.

FIG. 1 is a schematic diagram of the architecture of a target detection system provided by an embodiment of the application. As shown in FIG. 1, the system provided by this embodiment includes a processing system 101. In some embodiments, the processing system 101 can obtain the detection data of the millimeter wave radar, and then perform processing to detect the target to be processed in the preset area, or obtain the sensing data of the auxiliary sensor, and then perform processing to detect the target The target auxiliary information of the preset area may also use the target auxiliary information to optimize the target to be processed to obtain a detection target. In other embodiments, the millimeter-wave radar can have preliminary data processing functions. For example, the millimeter-wave radar has terminal computing capabilities, so that the millimeter-wave radar can perform preliminary data processing on its detection data and detect the waiting area in the preset area. Process the target, and send the processing detection result to the processing system 101 for subsequent further calculation; the auxiliary sensor may also have a preliminary data processing function, so that the auxiliary sensor can perform preliminary data processing on its sensing data and detect The target auxiliary information of the preset area can be sent to the processing system 101 for subsequent further calculations. Here, the processing system 101 may be a computing platform for a vehicle, an unmanned aerial vehicle processor, or the like. This embodiment does not particularly limit the implementation of the processing system 101, as long as the processing system 101 can obtain the information detected by the millimeter wave radar and the auxiliary sensor, and can use the information detected by the auxiliary sensor to compare the information detected by the millimeter wave radar. Just optimize. The millimeter-wave radar and auxiliary sensors can be installed according to the actual situation. For example, taking the target vehicle in motion to detect other vehicles as an example, the millimeter-wave radar and auxiliary sensors can be installed at the designated positions of the target vehicle, and the Vehicle detection, where the target vehicle is any vehicle that needs vehicle detection.

The technical solutions of the present application and how the technical solutions of the present application solve the above-mentioned technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the drawings.

FIG. 2 is a schematic flowchart of a target detection method provided by an embodiment of this application. The execution subject of this embodiment may be the processing system in the embodiment shown in FIG. 1. As shown in FIG. 2, the method includes:

S201: Acquire a target to be processed detected by the millimeter wave radar in a preset area.

Among them, the millimeter wave radar is a radar that works in the millimeter wave band (millimeter wave) detection. Generally, millimeter waves refer to waves in the frequency domain of 30 to 300 GHz (wavelengths of 1 to 10 mm). The wavelength of millimeter wave is between microwave and centimeter wave, so millimeter wave radar has some advantages of microwave radar and photoelectric radar. The above-mentioned preset area may be one or more areas that require target detection, which may be preset by relevant personnel. For example, taking the detection of a vehicle as an example, set the area in front of the vehicle body with millimeter wave radar as the preset area. , The front area can be determined according to the performance parameters of the millimeter wave radar.

Optionally, after the acquisition of the target to be processed detected by the millimeter wave radar in the preset area, the target to be processed may also be denoised. Exemplarily, performing denoising processing on the target to be processed includes: acquiring the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar; Targets outside the preset angle range, targets whose duration is lower than the preset time threshold, targets whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or the radial relative velocity difference between adjacent frames is greater than Targets with preset speed thresholds are filtered out.

S202: Acquire target auxiliary information detected by the auxiliary sensor in the preset area.

Optionally, the auxiliary sensor includes at least one of the following:

Lidar, vision sensor.

Among them, lidar is a radar system that emits a laser beam to detect the position and speed of the target. Its working principle is to transmit a detection signal (laser beam) to the target, and then reflect the received signal from the target (target return). Wave) is compared with the transmitted signal, and after proper processing, relevant information about the target can be obtained, such as target distance, azimuth, height, speed, attitude, and even shape, etc., so as to detect surrounding obstacles or moving objects. , Tracking and identification. Vision sensor refers to an instrument that uses optical elements and imaging devices to obtain image information of the external environment. Image resolution is usually used to describe the performance of the vision sensor.

Optionally, the auxiliary sensor is a lidar, and the target auxiliary information is a target point cloud.

Optionally, after acquiring the target point cloud detected by the lidar in the preset area, the method further includes:

Performing ground segmentation processing on the target point cloud;

The top view of the point cloud is obtained according to the result of the ground segmentation processing.

Optionally, the performing ground segmentation processing on the target point cloud includes:

Perform ground segmentation processing on the target point cloud according to a preset three-dimensional grid.

Here, the three-dimensional grid (voxel) can be divided according to the three-dimensional coordinates (x, y, z) to create an index. Then, perform ground segmentation processing on the target point cloud to obtain ground area and non-ground area. In an implementation manner, it can be determined whether the maximum height difference of all target point clouds falling in the same two-dimensional coordinate (x, y) grid exceeds the threshold. If it exceeds, it is determined that the grid is an obstacle, otherwise it is the ground, so as to realize the ground segmentation of the target point cloud. Among them, the above threshold can be set according to actual conditions. The above-mentioned ground segmentation method has high parallelism and can be accelerated by a graphics processor (Graphics Processing Unit, GPU for short). In another embodiment, the density of the target point cloud in the three-dimensional grid can also be counted, and then the density distribution can be further analyzed on the two-dimensional coordinates (x, y) to obtain the ground area and the non-ground area. Of course, other methods can also be used to achieve the ground segmentation processing of the target point cloud, which is not limited in the present invention.

Since the detection results of the millimeter wave radar need to be optimized later, that is, the millimeter wave radar data is fused on a two-dimensional plane, the obstacle part of the three-dimensional laser point cloud needs to be projected under the top view to obtain a top view of the point cloud. Further, the connected domains are calculated by the depth-first traversal method, and each connected domain is labeled to indicate different regions.

Optionally, before the ground segmentation processing is performed on the target point cloud, the method further includes:

Denoising and/or removing obstacles in the air is performed on the target point cloud.

Optionally, the performing denoising and/or removing obstacles in the air on the target point cloud includes:

Obtaining the three-dimensional mesh density of the target point cloud according to the density histogram of the target point cloud;

Determine noise points and/or air obstacles in the target point cloud according to the three-dimensional grid density;

According to the determined noise points and/or aerial obstacles, the target point cloud is denoised and/or aerial obstacles removed.

Exemplarily, the density histogram method is used to count the three-dimensional grid density of the point cloud, and then the density characteristics of each grid are analyzed to determine whether the point cloud in the grid is noise or an obstacle in the sky. Among them, the point cloud noise is isolated in three-dimensional space, and the density of aerial obstacles and ground point clouds at different heights of the same two-dimensional grid will form double peaks or multi-peaks. These characteristics can pass the point cloud density To describe. The above method has high parallelism and can be implemented through GPU acceleration.

Optionally, the auxiliary sensor may also be a visual sensor. For example, the binocular vision sensor can be used to obtain the depth information of the surrounding environment, and the change of the binocular vision sensor itself and the change of the frame before and after the image can be combined to detect the position and speed of the target. Specifically, the binocular vision sensor uses two cameras to capture the same environment at the same time to obtain two images. Based on the parallax between the two images and the positional relationship between the two cameras themselves, the depth information of each pixel in the image can be obtained. ; Then according to the camera projection transformation, the position of the target represented by each pixel can be obtained. Based on the movement of the binocular vision sensor itself and the changes in the front and back frames of the image, combined with image recognition, the movement information of a specific target can be obtained. Similarly, although the monocular vision sensor does not have a positional relationship between the two cameras, it can similarly obtain the position and speed of the target based on the position change caused by its motion and the change of the frame before and after the image. Therefore, the visual sensor can also obtain three-dimensional information of the surrounding environment.

Since the sensing data of the vision sensor, that is, the image, is formed by pixels, the three-dimensional information it obtains is also based on the pixels. Therefore, when the auxiliary sensor is a vision sensor, the target auxiliary information can also be similar to the target point cloud Information, so that subsequent processing can be performed similarly, or other methods can be used based on the characteristics of the visual sensor.

S203. Use the target auxiliary information to optimize the target to be processed to obtain a detection target.

Here, fusion of laser point clouds optimizes millimeter-wave radar data, eliminates false targets, and obtains corresponding detection targets.

Optionally, the method further includes:

Use the point cloud top view to optimize the target to be processed to obtain the detection target.

Optionally, the using the point cloud top view to optimize the target to be processed to obtain the detection target includes:

Using the point cloud top view to perform multipath target determination on the target to be processed;

Multi-path target filtering is performed on the target to be processed according to the determination result to obtain the detection target.

Among them, the multipath effect is the phenomenon that the millimeter wave produces a mirror image false target due to multiple reflections. The principle is shown in Figure 3. It can be seen from Figure 3 that false targets usually appear outside the static area, so by judging whether the candidate false target exists outside the static area, it can be determined whether it is a false target. Here, take vehicle detection as an example. First, the static target after radar processing can be projected under the top view of the point cloud, and the nearest neighbor is marked which connected domain belongs to, then the marked connected domain is regarded as a static area. Then, it traverses targets with similar speeds in the multipath effect target set, and draws a ray toward the origin of the vehicle. Assuming that the ray passes through the static area, it is considered that the candidate target is indeed a false target. Further, in order to ensure the stability of the determination result, binary Bayesian filtering is performed on the determination result.

Using the point cloud top view to determine noise points in the target to be processed;

Perform noise point filtering on the target to be processed according to the noise point to obtain the detection target.

Exemplarily, each millimeter wave radar target can be traversed, and the point cloud top view is used to determine whether it is within the laser enhancement range. If it is included, judge whether there is a connected area of laser point cloud around the radar target. If not, consider the reflection point as noise and filter it out. If it is nearby, find the nearest connected block as the association and record its label.

Optionally, before optimizing the target to be processed by using the point cloud top view to obtain the detection target, the method further includes:

Perform filtering processing on the target to be processed, and perform target association processing on the filtered target to be processed.

Here, filtering the target to be processed is mainly for the following purposes: smoothing the target state; obtaining the target state and covariance matrix to facilitate tracking and fusion. Resolve the correspondence between the target at the previous moment and the target at this moment in the multi-target tracking scenario through association. The current state is extrapolated from the state estimation value at the last moment, and the cost matrix is calculated by combining the observation value at the current moment. Set the associated door to limit the scope of association. For the observations in the correlation gate, the nearest neighbor, global nearest neighbor and other methods can be used for allocation. Taking vehicle detection as an example, nearest neighbor allocation can deal with most scenarios.

Next, Kalman filter is applied to solve the motion state of each radar target. Assuming that the target's motion state is (r _x ,r _y ,v _x ,v _y ) and the observation value is (ρ,θ,v), the observation equation is as follows:

If it is nonlinear, use extended Kalman filter to solve. In order to speed up the convergence speed of the target state, according to the priori that the car is mostly facing straight ahead, the horizontal velocity is initialized to 0, and the longitudinal velocity is equal to the radial velocity. Among them, ρ, θ, v are radial distance/angle/radial relative velocity respectively.

Clustering the to-be-processed targets after target association processing by using the point cloud top view;

The detection target is obtained according to the clustering result.

Among them, the cluster generated by clustering is a collection of a set of data objects, which are similar to objects in the same cluster and different from objects in other clusters.

Here, after the lasers are fused, the clustering condition is that the label values of the connected areas of the laser point cloud associated with multiple millimeter wave radar targets in the point cloud top view are the same, and the target point closest to the car is retained.

Optionally, after acquiring the target to be processed detected by the millimeter wave radar in the preset area, the method further includes:

Clustering the target to be processed, and obtaining clustering targets from each region after clustering;

The static target and dynamic target separation processing is performed on the cluster target.

Here, we mainly deal with the situation where there are multiple reflection points on a single target, which are divided into two situations. One is the situation where there are multiple reflection points for long-shaped and short-distance targets (such as buses, long trucks, etc.), and the closest target to the car is output through clustering. The other is the case of false targets caused by multipath effects. Clustering can be performed by judging the proximity of the target state (position, speed). The speed of the false target caused by the multipath effect is generally very close to the real target, but the position is far away. Therefore, compared with the first case, the threshold of strict speed and loose position is adopted.

Optionally, the separating processing of static targets and dynamic targets on the clustering targets includes:

Determining a speed interval according to the vehicle speed on which the millimeter wave radar is installed and a preset threshold value, where the preset threshold value is determined according to the angle of the cluster target measured by the millimeter wave radar;

According to the vehicle speed and the speed interval, separation processing of static targets and dynamic targets is performed on the cluster targets.

Exemplarily, first, the vehicle speed v_car can be obtained from the wheel speed; the radial speed v of the radar target is measured. In theory, if the target relative speed is approximately equal to the vehicle speed, the target is considered to be absolutely stationary.

Because the reflection points of some targets appear close to the car and have a large angle, the radial speed of the target cannot be approximated to the speed of the car at this time, so it is necessary to set the angle-related threshold threshold. The calculation method is as follows:

Threshold=max(k*v_car*cos(π/2-θ),min_threshold)

Among them, k is the adjustable ratio, min_threshold is the minimum allowable threshold to prevent the threshold from being too small at low speeds, and θ is the angle from the reflection point to the origin. Finally, according to whether the target speed is within the v_car±threshold range, the static target and the dynamic target are separated and stored separately, and the target speed can be converted into an absolute speed.

Use the point cloud top view to optimize the separated dynamic target to obtain the detection target.

The target detection method provided in this embodiment effectively eliminates false detections in the returned data of millimeter wave radar data through methods such as denoising, clustering, and separation of static and dynamic targets, and uses Kalman filtering methods to achieve target position, speed, etc. In addition, the method of fusion laser point cloud enhancement removes false targets in the returned data of millimeter wave radar data, further reduces false detections, and overcomes the problems of low resolution of millimeter wave radar and insensitivity to weak reflectors, thereby improving The accuracy of target detection results is improved, and the fusion of laser point clouds also solves the problem of poor clustering of nearby targets by millimeter wave radar; in addition, the fusion of millimeter wave and laser point clouds has low processing delay and can be used as Track the front-end input of the fusion module.

FIG. 4 is a schematic flowchart of another target detection method provided by an embodiment of the application. This embodiment, on the basis of the embodiment in FIG. 2, describes in detail the specific implementation process of this embodiment. As shown in Figure 4, the method includes:

S401: Obtain a target to be processed detected by the millimeter wave radar in a preset area.

S402: Perform denoising processing on the target to be processed.

Exemplarily, the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar are acquired, where the radial distance/angle/radial relative velocity can be expressed as (ρ, θ, v). Then eliminate targets whose angles are outside the preset angle range; eliminate targets with a shorter duration (targets with a duration lower than the preset time threshold), for example, only retain targets with a duration longer than 5 frames (250ms); eliminate target states ( Targets with sudden changes in distance/speed (targets whose radial distance difference between adjacent frames is greater than a preset distance threshold, and targets whose radial relative velocity difference between adjacent frames is greater than a preset speed threshold) are considered Wrong observations.

S403: Perform clustering on the target to be processed in the previous step, and obtain clustering targets from each area after clustering.

S404: Perform static target and dynamic target separation processing on the clustering target.

S405: Perform filtering processing on the target in the previous step, and perform target association processing on the filtered target.

S406: Obtain a target point cloud detected by the lidar in the preset area.

In the embodiments of the present application, the auxiliary sensor is a lidar and the target auxiliary information is a target point cloud as an example for description.

In addition, the data processing of millimeter-wave radar can also fuse the information of vision sensors to eliminate misdetected targets. Moreover, with the development of deep learning, the use of convolutional neural networks to achieve two-dimensional image target detection is relatively mature. Therefore, it is also possible to generate an image area of interest around the radar target, and then combine the target frame output by the convolutional network to calculate If the overlap ratio is lower than a certain ratio, it is considered that the radar target has misdetected the target and will be eliminated.

S407: Perform denoising and/or removing obstacles in the air on the target point cloud.

S408: Perform ground segmentation processing on the target point cloud in the previous step; obtain a top view of the point cloud according to the ground segmentation processing result.

S409: Perform multipath target determination on the target detected by the millimeter wave radar using the cloud top view; perform multipath target filtering on the target according to the determination result.

S410: Use the point cloud top view to determine the noise points in the target detected by the millimeter wave radar; filter out the noise points on the target according to the noise points.

S411: Use the point cloud top view to cluster the to-be-processed targets after the target association processing; obtain corresponding detection targets according to the clustering results.

Here, after the corresponding detection target is acquired, the acquired detection target can also be projected to each camera system/world system to meet the needs of multiple application scenarios.

The target detection method provided in this embodiment integrates laser point clouds to optimize millimeter-wave radar data, assists in removing false targets caused by millimeter-wave multipath effects, auxiliary target clustering, etc., and overcomes the low resolution of millimeter-wave radars. This improves the accuracy of target detection results, and also builds a low-latency, independent-working millimeter wave target detection system that does not rely heavily on visual detection.

FIG. 5 is a schematic structural diagram of a target detection device provided by an embodiment of the application. For ease of description, only parts related to the embodiments of the present application are shown. As shown in FIG. 5, the target detection device 50 includes: a millimeter wave radar information acquisition module 501, an auxiliary sensor information acquisition module 502, and a target optimization module 503.

Among them, the millimeter-wave radar information acquisition module 501 is used to acquire the target to be processed detected by the millimeter-wave radar in a preset area.

The auxiliary sensor information acquisition module 502 is configured to acquire the target auxiliary information detected by the auxiliary sensor in the preset area.

The target optimization module 503 is configured to optimize the target to be processed by using the target auxiliary information to obtain a detection target.

The device provided in this embodiment can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here in this embodiment.

Fig. 6 is a schematic structural diagram of another target detection device provided by an embodiment of the present invention. As shown in FIG. 6, on the basis of the embodiment in FIG. 5, this embodiment further includes: a ground segmentation module 504, a filter correlation module 505, a first preprocessing module 506, a clustering module 507 and a second preprocessing module 508.

In a possible design, the auxiliary sensor includes at least one of the following:

Lidar, vision sensor.

In a possible design, the auxiliary sensor is a lidar, and the target auxiliary information is a target point cloud.

In a possible design, the ground segmentation module 504 is configured to: after the auxiliary sensor information acquisition module 502 acquires the target point cloud detected by the lidar in the preset area, The cloud performs ground segmentation processing; the top view of the point cloud is obtained according to the ground segmentation processing result.

In a possible design, the target optimization module 503 is further configured to optimize the target to be processed by using the top view of the point cloud to obtain the detection target.

In a possible design, the target optimization module 503 uses the point cloud top view to optimize the target to be processed to obtain the detection target, including:

In a possible design, the filtering correlation module 505 is configured to optimize the target to be processed by the target optimization module 503 using the point cloud top view, and before obtaining the detection target, perform The processing target is filtered, and the filtered target to be processed is subjected to target association processing.

The detection target is obtained according to the clustering result.

In a possible design, the first preprocessing module 506 is configured to denoise and/or denoise the target point cloud before the ground segmentation module 504 performs ground segmentation processing on the target point cloud To deal with obstacles in the air.

In a possible design, the clustering module 507 is configured to perform processing on the target to be processed after the millimeter wave radar information acquisition module 501 acquires the target to be processed detected by the millimeter wave radar in a preset area Clustering, obtaining clustering targets from each area after clustering; performing static target and dynamic target separation processing on the clustering targets.

In a possible design, the second preprocessing module 508 is configured to cluster the target to be processed by the clustering module 507 and obtain the clustering target from each clustered area, Denoising processing is performed on the target to be processed.

In a possible design, the ground segmentation module 504 performs ground segmentation processing on the target point cloud, including:

In a possible design, the first preprocessing module 506 performs denoising and/or removal of obstacles in the air on the target point cloud, including:

In a possible design, the clustering module 507 performs static target and dynamic target separation processing on the clustering target, including:

In a possible design, the second preprocessing module 508 performs denoising processing on the target to be processed, including:

Acquiring the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar;

For the target whose angle is outside the preset angle range among the targets to be processed, the target whose duration is lower than the preset time threshold, the target whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or adjacent Targets whose radial relative speed difference of the frame is greater than the preset speed threshold are filtered out.

The device provided in the embodiment of the present application can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and the details of the embodiments of the present application are not repeated here.

FIG. 7 is a schematic diagram of the hardware structure of the target detection system provided by an embodiment of the application. As shown in FIG. 7, the target detection system 70 of this embodiment includes: a millimeter wave radar 701, an auxiliary sensor 702, a memory 703, and a processor 704;

The memory 703 is used to store program instructions;

The processor 704 is configured to execute program instructions stored in the memory, and when the program instructions are executed, the processor executes the following steps:

Lidar, vision sensor.

In a possible design, after acquiring the target point cloud detected by the lidar in the preset area, the processor is further configured to:

Performing ground segmentation processing on the target point cloud;

In a possible design, the processor is also used to:

In a possible design, the processor uses the point cloud top view to optimize the target to be processed, and when obtaining the detection target, it is specifically used to:

In a possible design, before the processor optimizes the target to be processed by using the top view of the point cloud to obtain the detection target, it is further configured to:

The detection target is obtained according to the clustering result.

In a possible design, before performing ground segmentation processing on the target point cloud, the processor is further configured to:

In a possible design, after acquiring the target to be processed detected by the millimeter wave radar in the preset area, the processor is further used to:

In a possible design, before the processor clusters the to-be-processed target and obtains the clustering target from each area after clustering, it is also used to:

Denoising processing is performed on the target to be processed.

In a possible design, the processor performs ground segmentation processing on the target point cloud, specifically for:

In a possible design, when the processor performs denoising and/or removing obstacles in the air on the target point cloud, it is specifically used for:

In a possible design, when the processor separates the static target from the dynamic target on the clustering target, it is specifically used for:

In a possible design, when the processor performs denoising processing on the target to be processed, it is specifically used to:

For the target whose angle is outside the preset angle range among the targets to be processed, the target whose duration is lower than the preset time threshold, the target whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or adjacent Targets whose radial relative speed difference of the frame is greater than the preset speed threshold are filtered out

In a possible design, the memory 703 may be independent or integrated with the processor 704.

When the memory 703 is set independently, the target detection system further includes a bus 705 for connecting the memory 703 and the processor 704.

In a possible design, the target detection system 70 may be a single device, and the device includes a complete set of the millimeter wave radar 701, the auxiliary sensor 702, the memory 703, and the processor 704 described above. In addition, taking vehicle detection as an example, the components of the target detection system 70 can be distributed and integrated on the vehicle, that is, the millimeter wave radar 701, the auxiliary sensor 702, the memory 703, and the processor 704 can be respectively set in different positions of the vehicle.

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the application. As shown in FIG. 8, the movable platform 80 of this embodiment includes: a movable platform body 801, a millimeter wave radar 802, an auxiliary sensor 803, and a target detection system 804; the millimeter wave radar 802 and the auxiliary sensor 803 are arranged in On the movable platform body 801, the movable platform body 801 and the target detection system 804 are connected wirelessly or wiredly.

The target detection system 804 is used to obtain the target to be processed detected by the millimeter-wave radar 802 in the preset area; obtain the target auxiliary information detected by the auxiliary sensor 803 in the preset area; The target to be processed is optimized to obtain the detection target.

Lidar, vision sensor.

In a possible design, after acquiring the target point cloud detected by the lidar in the preset area, the target detection system is further used to:

Performing ground segmentation processing on the target point cloud;

In a possible design, the target detection system is also used for:

In a possible design, the target detection system uses the point cloud top view to optimize the target to be processed to obtain the detection target, including:

In a possible design, before the target detection system uses the point cloud top view to optimize the target to be processed and obtains the detection target, it is also used to:

The detection target is obtained according to the clustering result.

In a possible design, before the target detection system performs ground segmentation processing on the target point cloud, it is also used to:

In a possible design, after the target detection system acquires the target to be processed detected by the millimeter wave radar in the preset area, it is further used to:

In a possible design, before the target detection system clusters the to-be-processed target and obtains the clustering target from each clustered area, it is also used to:

Denoising processing is performed on the target to be processed.

In a possible design, the target detection system performs ground segmentation processing on the target point cloud, including:

In a possible design, the target detection system performs denoising and/or removal of obstacles in the air on the target point cloud, including:

In a possible design, the target detection system performs static target and dynamic target separation processing on the clustered target, including:

In a possible design, the target detection system performs denoising processing on the target to be processed, including:

The movable platform provided by this embodiment includes: a movable platform body, a millimeter wave radar, an auxiliary sensor, and a target detection system. The target detection system obtains the auxiliary target by acquiring the target to be processed detected by the millimeter wave radar in a preset area. The target auxiliary information detected by the sensor in the preset area is optimized by using the target auxiliary information to optimize the target to be processed to obtain the detection target. That is, the auxiliary sensor is used to optimize the target detected by the millimeter wave radar, which overcomes the resolution of the millimeter wave radar. It is not high, and it is not sensitive to weak reflectors, which improves the accuracy of target detection results.

An embodiment of the present application provides a computer-readable storage medium having program instructions stored in the computer-readable storage medium, and when a processor executes the program instructions, the target detection method described above is implemented.

In the several embodiments provided by the present invention, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other divisions in actual implementation, for example, multiple modules can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or modules, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules may be integrated into one unit. The units formed by the above-mentioned modules can be realized in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated modules implemented in the form of software function modules may be stored in a computer readable storage medium. The above-mentioned software function module is stored in a storage medium and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute the various embodiments of the present application Part of the method.

It should be understood that the above-mentioned processor may be a central processing unit (Central Processing Unit, CPU for short), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), and application specific integrated circuits (Application Specific Integrated Circuits). Referred to as ASIC) and so on. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in combination with the invention can be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage, and may also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

The bus may be an Industry Standard Architecture (ISA) bus, Peripheral Component (PCI) bus, or Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

The above-mentioned storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Except for programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disks or optical disks. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in Application Specific Integrated Circuits (ASIC for short). Of course, the processor and the storage medium may also exist as discrete components in the electronic device or the main control device.

A person of ordinary skill in the art can understand that all or part of the steps in the foregoing method embodiments can be implemented by a program instructing relevant hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps including the foregoing method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention range.

Claims

A target detection method, characterized in that it comprises:

Obtain the target to be processed detected by the millimeter wave radar in the preset area;

Acquiring target auxiliary information detected by the auxiliary sensor in the preset area;

The target auxiliary information is used to optimize the target to be processed to obtain a detection target.
The method according to claim 1, wherein the auxiliary sensor comprises at least one of the following:

Lidar, vision sensor.
The method according to claim 1, wherein the auxiliary sensor is a lidar, and the target auxiliary information is a target point cloud.
The method according to claim 3, wherein after obtaining the target point cloud detected by the lidar in the preset area, the method further comprises:

Performing ground segmentation processing on the target point cloud;

The top view of the point cloud is obtained according to the result of the ground segmentation processing.
The method according to claim 4, wherein the method further comprises:

Use the point cloud top view to optimize the target to be processed to obtain the detection target.
The method according to claim 5, wherein the optimizing the target to be processed by using the top view of the point cloud to obtain the detection target comprises:

Using the point cloud top view to perform multipath target determination on the target to be processed;

Multi-path target filtering is performed on the target to be processed according to the determination result to obtain the detection target.
The method according to claim 5, wherein the optimizing the target to be processed by using the top view of the point cloud to obtain the detection target comprises:

Using the point cloud top view to determine noise points in the target to be processed;

Perform noise point filtering on the target to be processed according to the noise point to obtain the detection target.
The method according to any one of claims 5 to 7, characterized in that, before said using the point cloud top view to optimize the target to be processed and obtaining the detection target, the method further comprises:

Perform filtering processing on the target to be processed, and perform target association processing on the filtered target to be processed.
The method according to claim 8, wherein said using said point cloud top view to optimize said target to be processed to obtain said detection target comprises:

Clustering the to-be-processed targets after target association processing by using the point cloud top view;

The detection target is obtained according to the clustering result.
The method according to any one of claims 4 to 9, characterized in that, before the ground segmentation processing is performed on the target point cloud, the method further comprises:

Denoising and/or removing obstacles in the air is performed on the target point cloud.
The method according to any one of claims 5 to 9, characterized in that, after the acquiring the target to be processed detected by the millimeter wave radar in the preset area, the method further comprises:

Clustering the target to be processed, and obtaining clustering targets from each region after clustering;

The static target and dynamic target separation processing is performed on the cluster target.
The method according to claim 11, wherein said using said point cloud top view to optimize said target to be processed to obtain said detection target comprises:

Use the point cloud top view to optimize the separated dynamic target to obtain the detection target.
The method according to claim 11 or 12, characterized in that, before the clustering the target to be processed and obtaining the clustering target from each clustered area, the method further comprises:

Denoising processing is performed on the target to be processed.
The method according to any one of claims 4 to 9, wherein the performing ground segmentation processing on the target point cloud comprises:

Perform ground segmentation processing on the target point cloud according to a preset three-dimensional grid.
The method according to claim 10, wherein the denoising and/or air obstacle removal processing on the target point cloud comprises:

Obtaining the three-dimensional mesh density of the target point cloud according to the density histogram of the target point cloud;

Determine noise points and/or air obstacles in the target point cloud according to the three-dimensional grid density;

According to the determined noise points and/or aerial obstacles, the target point cloud is denoised and/or aerial obstacles removed.
The method according to claim 11, wherein said separating the static target and the dynamic target on the clustering target comprises:

Determining a speed interval according to the vehicle speed on which the millimeter wave radar is installed and a preset threshold value, where the preset threshold value is determined according to the angle of the cluster target measured by the millimeter wave radar;

According to the vehicle speed and the speed interval, separation processing of static targets and dynamic targets is performed on the cluster targets.
The method according to claim 13, wherein the denoising processing on the target to be processed comprises:

Acquiring the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar;

For the target whose angle is outside the preset angle range among the targets to be processed, the target whose duration is lower than the preset time threshold, the target whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or adjacent Targets whose radial relative speed difference of the frame is greater than the preset speed threshold are filtered out.
A target detection system is characterized by comprising millimeter wave radar, auxiliary sensors, memory and processor, wherein,

The memory is used to store program instructions;

The processor is configured to execute the program instructions, and when the program instructions are executed, the processor executes the following steps:

Obtain the target to be processed detected by the millimeter wave radar in the preset area;

Acquiring target auxiliary information detected by the auxiliary sensor in the preset area;

The target auxiliary information is used to optimize the target to be processed to obtain a detection target.
The system according to claim 18, wherein the auxiliary sensor comprises at least one of the following:

Lidar, vision sensor.
The system according to claim 18, wherein the auxiliary sensor is a lidar, and the target auxiliary information is a target point cloud.
The system according to claim 20, wherein after obtaining the target point cloud detected by the lidar in the preset area, the processor is further configured to:

Performing ground segmentation processing on the target point cloud;

The top view of the point cloud is obtained according to the result of the ground segmentation processing.
The system according to claim 21, wherein the processor is further configured to:

Use the point cloud top view to optimize the target to be processed to obtain the detection target.
The system according to claim 22, wherein the processor uses the point cloud top view to optimize the target to be processed, and when obtaining the detection target, it is specifically used for:

Using the point cloud top view to perform multipath target determination on the target to be processed;

Multi-path target filtering is performed on the target to be processed according to the determination result to obtain the detection target.
The system according to claim 22, wherein the processor uses the point cloud top view to optimize the target to be processed, and when obtaining the detection target, it is specifically used for:

Using the point cloud top view to determine noise points in the target to be processed;

Perform noise point filtering on the target to be processed according to the noise point to obtain the detection target.
The system according to any one of claims 22 to 24, wherein the processor is further configured to: before optimizing the target to be processed by using the top view of the point cloud to obtain the detection target:

Perform filtering processing on the target to be processed, and perform target association processing on the filtered target to be processed.
The system according to claim 25, wherein the processor uses the point cloud top view to optimize the target to be processed, and when obtaining the detection target, it is specifically used for:

Clustering the to-be-processed targets after target association processing by using the point cloud top view;

The detection target is obtained according to the clustering result.
The system according to any one of claims 21 to 26, wherein the processor is further configured to: before performing ground segmentation processing on the target point cloud:

Denoising and/or removing obstacles in the air is performed on the target point cloud.
The system according to any one of claims 22 to 26, wherein after acquiring the target to be processed detected by the millimeter wave radar in a preset area, the processor is further configured to:

Clustering the target to be processed, and obtaining clustering targets from each region after clustering;

The static target and dynamic target separation processing is performed on the cluster target.
The system according to claim 28, wherein the processor uses the point cloud top view to optimize the target to be processed, and when obtaining the detection target, it is specifically used for:

Use the point cloud top view to optimize the separated dynamic target to obtain the detection target.
The system according to claim 28 or 29, wherein the processor is further configured to: before clustering the target to be processed and obtaining clustering targets from each clustered area:

Denoising processing is performed on the target to be processed.
The system according to any one of claims 21 to 26, wherein the processor performs ground segmentation processing on the target point cloud, specifically for:

Perform ground segmentation processing on the target point cloud according to a preset three-dimensional grid.
The system according to claim 27, wherein when the processor performs denoising and/or removing obstacles in the air on the target point cloud, it is specifically configured to:

Obtaining the three-dimensional mesh density of the target point cloud according to the density histogram of the target point cloud;

Determine noise points and/or air obstacles in the target point cloud according to the three-dimensional grid density;

According to the determined noise points and/or aerial obstacles, the target point cloud is denoised and/or aerial obstacles removed.
The system according to claim 28, wherein the processor is specifically configured to: when performing static target and dynamic target separation processing on the clustering target:

Determining a speed interval according to the vehicle speed on which the millimeter wave radar is installed and a preset threshold value, where the preset threshold value is determined according to the angle of the cluster target measured by the millimeter wave radar;

According to the vehicle speed and the speed interval, separation processing of static targets and dynamic targets is performed on the cluster targets.
The system according to claim 30, wherein when the processor performs denoising processing on the target to be processed, it is specifically configured to:

Acquiring the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar;

For the target whose angle is outside the preset angle range among the targets to be processed, the target whose duration is lower than the preset time threshold, the target whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or adjacent Targets whose radial relative speed difference of the frame is greater than the preset speed threshold are filtered out.
A movable platform, which is characterized by comprising: a movable platform body, a millimeter wave radar, an auxiliary sensor, and a target detection system; the millimeter wave radar and the auxiliary sensor are arranged on the movable platform body, and the The movable platform body and the target detection system are connected wirelessly or wiredly;

The target detection system is used to obtain the target to be processed detected by the millimeter-wave radar in the preset area; to obtain the target auxiliary information detected by the auxiliary sensor in the preset area; Optimize to obtain the detection target.
The movable platform according to claim 35, wherein the auxiliary sensor comprises at least one of the following:

Lidar, vision sensor.
The mobile platform according to claim 35, wherein the auxiliary sensor is a lidar, and the target auxiliary information is a target point cloud.
The mobile platform according to claim 37, wherein the target detection system is further configured to: after acquiring the target point cloud detected by the lidar in the preset area:

Performing ground segmentation processing on the target point cloud;

The top view of the point cloud is obtained according to the result of the ground segmentation processing.
The movable platform according to claim 38, wherein the target detection system is further used for:

Use the point cloud top view to optimize the target to be processed to obtain the detection target.
The movable platform of claim 39, wherein the target detection system uses the point cloud top view to optimize the target to be processed to obtain the detection target, comprising:

Using the point cloud top view to perform multipath target determination on the target to be processed;

Multi-path target filtering is performed on the target to be processed according to the determination result to obtain the detection target.
The movable platform of claim 39, wherein the target detection system uses the point cloud top view to optimize the target to be processed to obtain the detection target, comprising:

Using the point cloud top view to determine noise points in the target to be processed;

Perform noise point filtering on the target to be processed according to the noise point to obtain the detection target.
The movable platform according to any one of claims 39 to 41, characterized in that, before the target detection system uses the point cloud top view to optimize the target to be processed and obtain the detection target, Used for:

Perform filtering processing on the target to be processed, and perform target association processing on the filtered target to be processed.
The mobile platform according to claim 42, wherein the target detection system uses the point cloud top view to optimize the target to be processed to obtain the detection target, comprising:

Clustering the to-be-processed targets after target association processing by using the point cloud top view;

The detection target is obtained according to the clustering result.
The mobile platform according to any one of claims 38 to 43, wherein before the target detection system performs ground segmentation processing on the target point cloud, it is further used for:

Denoising and/or removing obstacles in the air is performed on the target point cloud.
The movable platform according to any one of claims 39 to 43, wherein after the target detection system obtains the target to be processed detected by the millimeter wave radar in the preset area, it is further used for:

Clustering the target to be processed, and obtaining clustering targets from each region after clustering;

The static target and dynamic target separation processing is performed on the cluster target.
The movable platform of claim 45, wherein the target detection system uses the point cloud top view to optimize the target to be processed to obtain the detection target, comprising:

Use the point cloud top view to optimize the separated dynamic target to obtain the detection target.
The movable platform according to claim 45 or 46, characterized in that, before the target detection system clusters the target to be processed and obtains the cluster target from each clustered area, it is also used for :

Denoising processing is performed on the target to be processed.
The mobile platform according to any one of claims 38 to 43, wherein the target detection system performs ground segmentation processing on the target point cloud, comprising:

Perform ground segmentation processing on the target point cloud according to a preset three-dimensional grid.
The mobile platform according to claim 44, wherein the target detection system performs denoising and/or removal of obstacles in the air on the target point cloud, comprising:

Obtaining the three-dimensional mesh density of the target point cloud according to the density histogram of the target point cloud;

Determine noise points and/or air obstacles in the target point cloud according to the three-dimensional grid density;

According to the determined noise points and/or aerial obstacles, the target point cloud is denoised and/or aerial obstacles removed.
The mobile platform according to claim 45, wherein the target detection system performs static target and dynamic target separation processing on the cluster target, comprising:

Determining a speed interval according to the vehicle speed on which the millimeter wave radar is installed and a preset threshold value, where the preset threshold value is determined according to the angle of the cluster target measured by the millimeter wave radar;

According to the vehicle speed and the speed interval, separation processing of static targets and dynamic targets is performed on the cluster targets.
The mobile platform according to claim 47, wherein the target detection system performs denoising processing on the target to be processed, comprising:

Acquiring the radial distance, angle, and radial relative velocity of the target to be processed measured by the millimeter wave radar;

For the target whose angle is outside the preset angle range among the targets to be processed, the target whose duration is lower than the preset time threshold, the target whose radial distance difference between adjacent frames is greater than the preset distance threshold, and/or adjacent Targets whose radial relative speed difference of the frame is greater than the preset speed threshold are filtered out.
A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the computer-readable storage medium implements any one of claims 1 to 17 Target detection method.