CN115439484B - Detection method and device based on 4D point cloud, storage medium and processor - Google Patents
Detection method and device based on 4D point cloud, storage medium and processor Download PDFInfo
- Publication number
- CN115439484B CN115439484B CN202211403233.7A CN202211403233A CN115439484B CN 115439484 B CN115439484 B CN 115439484B CN 202211403233 A CN202211403233 A CN 202211403233A CN 115439484 B CN115439484 B CN 115439484B
- Authority
- CN
- China
- Prior art keywords
- target
- point
- cluster
- search
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The application provides a detection method, a detection device, a storage medium and a processor based on 4D point cloud, wherein a point cloud clustering algorithm, a neighborhood radius self-adaptive algorithm and a neighborhood point number self-adaptive algorithm are adopted for target identification, the distance between a target point and a coordinate origin in a 4D millimeter wave radar coordinate system is calculated, the neighborhood radius of a contour corresponding to a target clustering cluster corresponding to target 4D point cloud data is determined, and the actual number of search points is determined; under the condition that the actual number is larger than the minimum number, filtering out part of search points, and determining the rest search points to form a target clustering cluster; calculating the contour information of the target clustering cluster according to the target clustering cluster, and determining a corresponding target object of the target clustering cluster; and the track tracking is carried out according to the target clustering cluster, so that the accurate output of the target type is realized, and the aims of no need of acquiring a large amount of sample data in advance for supervised learning, rapid deployment and low requirement on a vehicle-mounted hardware platform are achieved.
Description
Technical Field
The application relates to the field of automatic driving AI (artificial intelligence) technical perception of vehicles, in particular to a detection method, a detection device, a storage medium, a processor and a detection system based on 4D point cloud.
Background
The method is widely applied to detection and acquisition of road traffic vehicle information, but although a common traffic radar or traffic monitoring sensor can accurately acquire vehicle speed and distance information, the method is short of a means for acquiring vehicle speed information, and the 4D millimeter wave radar is a novel millimeter wave radar sensor capable of measuring target three-dimensional space coordinates (Range (radial distance), azimuth (horizontal angle), elevation (pitch angle)) and Velocity. On the basis of the traditional 3D millimeter wave radar, the detection capability of the height is increased. Therefore, the environment map can be clearly established, and the measured traffic data is more accurate.
In the prior art, target detection algorithms based on 4D millimeter wave radar point cloud comprise two types, one type is based on unsupervised machine learning algorithm, and the other type is based on supervised deep learning algorithm. The deep learning algorithm needs to acquire a large amount of training data and samples in advance, and the sample data needs to contain abundant features to facilitate feature recognition of a network, so that the calculated amount is overlarge, and the requirement on a vehicle-mounted hardware platform is high.
Based on the problems, the application provides a point cloud target detection and tracking method based on a 4D millimeter wave radar without acquiring a large amount of training data and samples in advance.
Disclosure of Invention
The application mainly aims to provide a detection method and a detection device based on a 4D point cloud, so as to solve the problems that in the prior art, the target detection algorithm of the 4D millimeter wave radar point cloud is too large in calculation amount and the requirement on a vehicle-mounted hardware platform is high.
According to an aspect of the embodiments of the present invention, there is provided a detection method based on a 4D point cloud, applied to a 4D millimeter wave radar, the method including:
acquiring target 4D point cloud data, determining any point corresponding to the target 4D point cloud data as a target point, and calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, wherein the 4D millimeter wave radar coordinate system is a coordinate system established by taking the 4D millimeter wave radar center point as the coordinate origin, and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, determining a neighborhood radius of a contour corresponding to a target clustering cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target clustering cluster;
determining points, of which the distance from the target point is smaller than the neighborhood radius, in the points corresponding to the target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in the target cluster according to the distance from the search points to the target point, a second threshold and the set number of the search points at least under the condition that the distance from at least one search point to the target point is larger than the second threshold;
under the condition that the actual number is larger than the minimum number, filtering partial search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the residual search points to form a target clustering cluster;
and calculating the contour information of the target cluster according to the target 4D point cloud data, and determining the type of the target object corresponding to the contour of the target cluster.
Optionally, the method further includes: and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is less than or equal to a first threshold value, adopting a neighborhood radius threshold value of an initial preset target cluster as the neighborhood radius of the target cluster.
Optionally, determining a neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to a distance between the target point and a coordinate origin in the 4D millimeter wave radar coordinate system and the initial radius, including: calculating a compensation value of an initial radius according to the Euclidean distance between a search point and a coordinate origin in a 4D millimeter wave radar coordinate system, the set maximum distance of a cluster radius which is kept unchanged, the increasing step length of the cluster neighborhood radius and a scaling factor of the cluster neighborhood radius; according to the compensation value of the initial radius and the initial radius, adopting R cluster = R (1 +. DELTA.R) determining neighborhood radius of the target cluster, wherein R cluster And (4) representing the neighborhood radius of the target clustering cluster, wherein R is the initial radius, and delta R is the compensation value of the initial radius.
Optionally, calculating a compensation value of the initial radius according to the euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, the set maximum distance at which the clustering radius remains unchanged, the increasing step size of the clustering neighborhood radius, and the scaling factor of the clustering neighborhood radius, including: by using
Calculating a compensation value of the initial radius, wherein D is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, and R is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system thre For a set maximum distance, R, at which the cluster radius remains constant step For increasing step size of cluster neighborhood radius, R factor Is the scaling factor of the cluster neighborhood radius.
Optionally, the method further includes: and if the distance between the search point and the target point is less than or equal to a set second threshold value, adopting the set search point as the number of the minimum search points in the target clustering.
Optionally, determining the minimum number of search points in the target cluster according to at least the distance between the search point and the target point, a second threshold, and the set number of search points includes: according to n = min _ pt ((d) 0 -d)/d step +1)d factor Determining the minimum number of the search points in the target clustering cluster, wherein n is the minimum number of the search points, min _ pt is the initial preset minimum number of the search points, and d 0 Is a threshold value of the distance between the search point and the target point, d is the distance between the search point and the target point, d step Varying step size for the number of search points, d factor Is a search point number variation factor.
Optionally, filtering out part of the search points according to the distance between the search point and the target point and the speed difference between the search point and the target point, and determining the remaining search points to form a target cluster, including: obtaining the distance between the search point and the target point, judging whether the distance between the search point and the target point is greater than a first distance threshold, and filtering the search point if the distance is greater than the first distance threshold, wherein the first distance threshold comprises the following steps: the distance between the search point and the target point is larger than the neighborhood radius of the target cluster; acquiring the speed direction of a target point, calculating an included angle between the speed direction of the target point and an X axis of a 4D millimeter wave radar coordinate system, determining the transverse distance between a search point and the target point according to the included angle between the speed direction of the target point and the X axis of the 4D millimeter wave radar coordinate system and the sine value of the distance between the search point and the target point, judging whether the transverse distance between the search point and the target point is greater than a second distance threshold, and filtering the search point if the transverse distance is greater than the second distance threshold, wherein the second distance threshold comprises: a lateral distance threshold between the search point and the target point; acquiring the speed of a search point and the speed of a target point, calculating the speed difference between the search point and the target point, judging whether the speed difference between the search point and the target point is greater than a first speed threshold, and filtering the search point if the speed difference is greater than the first speed threshold, wherein the first speed threshold comprises the following steps: a velocity difference threshold between the velocity of the search point and the velocity of the target point; acquiring the speed direction of a search point and the speed direction of a target point, calculating an included angle between the search point and the speed direction of the target point, judging whether the included angle between the search point and the speed direction of the target point is greater than a second speed threshold, and filtering the search point if the included angle is greater than the second speed threshold, wherein the second speed threshold comprises: searching an included angle threshold value between the speed direction of the point and the speed direction of the target point; and taking the rest search points in the target clustering cluster as the points in the target clustering cluster.
Optionally, calculating profile information of the target cluster according to the target 4D point cloud data, and determining a type of the target object corresponding to the profile of the target cluster, including: acquiring the speeds of all points in the target clustering cluster, and calculating the mean value of the speeds of all points in the target clustering cluster to obtain the speed of the target clustering cluster; under the condition that the speed of a target cluster is larger than a preset speed threshold value, calculating the orientation of a target object corresponding to the target cluster according to the speed of the target cluster and a 4D millimeter wave radar coordinate, calculating the Euclidean distance between every two points in the target cluster according to the orientation of the target object corresponding to the target cluster, determining the first maximum value of the projection length of the Euclidean distance between the two points in the target cluster in the Z axis of a 4D millimeter wave radar coordinate system in the speed direction, taking the first maximum value as the height of the target cluster, calculating the projection length of the Euclidean distance between every two points in the speed direction, determining the second maximum value of the projection length of the Euclidean distance between the two points in the target cluster in the speed direction, taking the second maximum value as the length of the target cluster, calculating the projection length of the Euclidean distance between every two points in the vertical direction of the speed direction, determining the third maximum value of the projection length of the Euclidean distance between the two points in the target cluster in the speed direction, taking the third maximum value as the width of the target cluster, and obtaining the target cluster size; calculating an included angle between the direction of the target cluster and the X axis of the coordinate system of the 4D millimeter wave radar based on the coordinate system of the target cluster and the coordinate system of the 4D millimeter wave radar, rotating the coordinate system of the 4D millimeter wave radar to the direction same as that of the coordinate system of the target cluster based on the included angle, calculating three-dimensional coordinates of all points in the target cluster under the rotated coordinate system of the 4D millimeter wave radar, acquiring the maximum value and the minimum value of the three-dimensional coordinates of all the points, and calculating to obtain the position information of the target center point of the target cluster; according to the position information of the target center point of the target clustering cluster, the three-dimensional size and the orientation of the target object corresponding to the target clustering cluster; and calculating to obtain the contour information of the target cluster, and determining the type of the target object corresponding to the contour of the target cluster according to the contour information of the target cluster.
Optionally, the method further comprises: tracking the flight path according to the target clustering cluster, creating, updating or deleting flight path data, and determining the target flight path data, wherein the method comprises the following steps: and associating the target clustering cluster with the track data, finding the track data with the highest similarity with the target clustering cluster from the track data under the condition that the target clustering cluster is not associated with the track data, determining the track data with the highest similarity from the target clustering cluster and the track data to be associated, obtaining an association result, creating, updating or deleting the track data according to the association result, and determining the target track data.
Optionally, when the association result is that the target cluster does not have associated track data, creating track data by using the current target cluster; under the condition that the association result is that the target cluster is associated with the track data, updating the track data corresponding to the target cluster, wherein the updating comprises the following steps: comparing the position and the speed of the corresponding target object of the target clustering cluster with the corresponding track data to obtain a comparison result, and if the comparison result is greater than a set threshold value, taking the position and the speed of the corresponding target object of the current target clustering cluster as the current track data; and determining the association times of the track data associated with the target clustering cluster under the condition that the association result is that the track data is not associated with the target clustering cluster, and deleting the track data which is not associated with any target clustering cluster and has the association times larger than a set threshold if the association times are larger than the set threshold.
Optionally, the method further comprises: storing the profile information of the target clustering cluster associated with the flight path into a data queue, and setting a length threshold of the data queue; calculating the average value of the contour information of all target clustering clusters within the length threshold of the data queue to obtain a current measured value; and comparing the current measurement value with the last measurement value, if the current measurement value is larger than the last measurement value, updating the size of the tracking frame to be the current measurement value, and otherwise, keeping the size of the tracking frame unchanged.
According to another aspect of the embodiments of the present invention, there is also provided a detection apparatus based on a 4D point cloud, including: the system comprises a data acquisition module, a data preprocessing module and a point cloud clustering module; the data acquisition module is used for acquiring 4D point cloud data; the data preprocessing module is used for filtering noise points of the 4D point cloud data to determine target 4D point cloud data; the point cloud clustering module is used for determining any point corresponding to the target 4D point cloud data as a target point, calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, and determining the neighborhood radius of the contour corresponding to the target clustering cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and the initial radius under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target clustering cluster; determining points, of which the distance to a target point is smaller than the neighborhood radius, in points corresponding to the target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in the target clustering cluster according to the distance between the search points and the target point, a second threshold and the set number of the search points under the condition that the distance between at least one search point and the target point is larger than the second threshold; under the condition that the actual number is larger than the minimum number, filtering partial search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the residual search points to form a target clustering cluster; and calculating the contour information of the target cluster according to the target 4D point cloud data, and determining the type of the target object corresponding to the contour of the target cluster.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, which includes a stored program, wherein the program executes any one of the above-mentioned detection methods based on a 4D point cloud.
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes any one of the above detection methods based on a 4D point cloud when running.
According to another aspect of the embodiments of the present invention, there is also provided a detection system based on a 4D point cloud, including: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the above-described 4D point cloud based detection methods.
In the embodiment of the invention, a point cloud clustering algorithm, a neighborhood radius adaptive algorithm and a neighborhood point number adaptive algorithm are combined for target identification, 4D point cloud data of a target are obtained through a 4D millimeter wave radar, the distance between the target and a coordinate origin in a 4D millimeter wave radar coordinate system is calculated, the neighborhood radius of a contour corresponding to a target clustering cluster corresponding to the 4D point cloud data of the target is determined according to the distance between the target and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius under the condition that the distance between the target and the coordinate origin in the 4D millimeter wave radar coordinate system is larger than a first threshold, the actual number of search points is determined, and the minimum number of the search points in the target clustering cluster is determined according to the distance between the search points and the target, a second threshold and the set number of the search points under the condition that the distance between at least one search point and the target is larger than a second threshold; the accuracy of target clustering is greatly improved through a neighborhood radius adaptive algorithm and a neighborhood point number adaptive algorithm, the missing rate of distant targets is effectively reduced, and meanwhile, the target splitting phenomenon frequently occurring in sparse point cloud clustering is effectively reduced; under the condition that the actual quantity is larger than the minimum quantity, filtering partial search points, determining the rest search points to form a target clustering cluster, effectively filtering the influence of false measurement, and reducing the false alarm rate of a clustering result; the contour information of the target cluster is calculated according to the target cluster, the target type is obtained from the 4D millimeter wave radar, the contour information of the target cluster is accurately calculated, the practicability is high, and rich cluster target information can be provided; the method achieves the purposes of no need of acquiring a large amount of sample data in advance for supervised learning and low requirement on the vehicle-mounted hardware platform, thereby achieving the technical effects of strong generalization capability, good robustness, high reliability and small calculated amount, and further solving the technical problems of overlarge calculated amount and higher requirement on the vehicle-mounted hardware platform due to the fact that a large amount of sample data is acquired in advance for supervised learning.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 shows a flow chart of a 4D point cloud based detection method according to an embodiment of the present application;
FIG. 2 illustrates a flow chart of a multi-threshold clustering method;
FIG. 3 illustrates a contour flow diagram for determining a target cluster;
FIG. 4 shows a schematic diagram of clustering effects;
FIG. 5 is a schematic diagram showing the orientation and size of a target object corresponding to a target cluster;
FIG. 6 is a schematic view illustrating a track overlap determination;
FIG. 7 is a schematic diagram of a 4D point cloud-based detection device;
fig. 8 shows a schematic diagram of a data acquisition module structure.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solution better understood by those in the field of the present invention, the technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without any creative effort shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.
As mentioned in the background art, in the target detection algorithm of the 4D millimeter wave radar point cloud in the prior art, a large amount of training data and samples need to be acquired in advance, and the sample data needs to contain rich features to facilitate feature identification by a network, which results in an excessively large amount of calculation and a high requirement for a vehicle-mounted hardware platform.
According to an embodiment of the application, a detection method based on a 4D point cloud is provided.
Fig. 1 is a flowchart of a detection method based on a 4D point cloud according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, obtaining target 4D point cloud data, determining any point corresponding to the target 4D point cloud data as a target point, and calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, wherein the 4D millimeter wave radar coordinate system is a coordinate system established by taking a 4D millimeter wave radar center point as the coordinate origin, and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, determining a neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target cluster;
step S102, determining points, of which the distance to a target point is smaller than the neighborhood radius, in points corresponding to target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in a target cluster according to the distance between the search points and the target point, a second threshold and the set number of the search points under the condition that the distance between at least one search point and the target point is larger than the second threshold;
step S103, under the condition that the actual number is larger than the minimum number, filtering out part of search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the rest search points to form a target cluster;
and step S104, calculating the contour information of the target cluster according to the target 4D point cloud data, and determining the type of the target object corresponding to the contour of the target cluster.
Specifically, according to the distance between a target point and a coordinate origin in a 4D millimeter wave radar coordinate system and an initial radius, determining a neighborhood radius of a contour corresponding to a target cluster corresponding to target 4D point cloud data, greatly improving the accuracy of target clustering, determining the minimum number of search points in the target cluster according to the distance between the search points and the target point, a second threshold value and the set number of the search points, effectively reducing the omission factor of a distant target, and simultaneously effectively reducing the target splitting phenomenon frequently occurring in sparse point cloud clustering; under the condition that the actual quantity is larger than the minimum quantity, filtering partial search points, determining the rest search points to form a target clustering group, effectively filtering the influence of false measurement, and reducing the false alarm rate of a clustering result; the contour information of the target cluster is calculated according to the target cluster, the target type is obtained from the 4D millimeter wave radar, the contour information of the target cluster is accurately calculated, the practicability is high, and rich cluster target information can be provided.
In specific implementation, determining a neighborhood radius of a profile corresponding to a target cluster corresponding to target 4D point cloud data according to a distance between a target point and a coordinate origin in a 4D millimeter wave radar coordinate system and an initial radius, includes: presetting an initial radius R, and according to the Euclidean distance between a search point and a coordinate origin in a 4D millimeter wave radar coordinate system, the set maximum distance that the clustering radius is kept unchanged, the increasing step length of the clustering neighborhood radius, and the clustering neighborhoodA scaling factor for the domain radius, calculating a compensation value for the initial radius, comprising: by using
Calculating a compensation value of the initial radius; according to the compensation value of the initial radius and the initial radius, adopting R cluster = R (1 +. DELTA.R) determine neighborhood radius of target cluster, where R cluster The neighborhood radius of the target clustering cluster is defined, wherein R is an initial radius, and delta R is a compensation value of the initial radius; d is the Euclidean distance between the search point and the origin of coordinates in the 4D millimeter wave radar coordinate system, R thre For a set maximum distance, R, at which the cluster radius remains constant step For increasing step size of cluster neighborhood radius, R factor Is the scaling factor of the cluster neighborhood radius.
Specifically, the first threshold is set to 40m, and if the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than 40m, a neighborhood radius adaptive algorithm is adopted: r cluster And = R (1 +. DELTA.R), updating the initial radius R to obtain the neighborhood radius of the target cluster, determining the subsequent search points in the point cloud within the neighborhood radius of the target cluster, and adopting a neighborhood radius self-adaptive algorithm to greatly reduce the splitting condition of the target at the remote search points and the missing identification caused by 4D point cloud sparsity.
The method further comprises the following steps: and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is less than or equal to a first threshold value, adopting a neighborhood radius threshold value of an initial preset target cluster as the neighborhood radius of the target cluster.
Specifically, if the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is less than or equal to 40m, the preset initial radius R is adopted as the neighborhood radius of the target cluster, the neighborhood radius of the target cluster is 2.5m, if the distance between the search point and the target point is less than 2.5m, the search point is considered as the search point of the target cluster, and if the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is less than or equal to the first threshold, the initial radius is adopted as the neighborhood radius of the target cluster, so that the initial radius is prevented from being updated again, and the calculation amount is reduced.
In specific implementation, determining the minimum number of search points in the target cluster according to at least the distance between the search point and the target point, a second threshold and the set number of the search points includes: according to n = min _ pt ((d) 0 -d)/d step +1)d factor Determining the number of minimum search points in the target clustering cluster, wherein n is the number of minimum search points, min _ pt is the number of initial preset minimum search points, d 0 Is a threshold value of the distance between the search point and the target point, d is the distance between the search point and the target point, d step Varying step size for the number of search points, d factor Is a search point number variation factor.
The second threshold is a distance threshold between the search point and the target point, if the distance between at least one search point and the target point is greater than the set second threshold, the number of preset search points needs to be updated, and since the larger the neighborhood radius is, the more the search points are, if the existing search point exists, the minimum number of search points needs to be updated, and the preset search points cannot be applied, the number of the minimum search points is determined, the larger the neighborhood radius of the target cluster is, the larger the preset search number is, and the more accurate the search condition of the search points of the target cluster is ensured.
In specific implementation, the method further includes: and if the distance between the search point and the target point is smaller than or equal to a set second threshold value, the set search point is adopted as the minimum number of search points in the target clustering.
In the determination of the number of the minimum search points, when the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is smaller than or equal to the first threshold, it is determined again that the distance between the search point and the target point is smaller than or equal to the set second threshold, and it can be seen that the above conditions are all within the threshold range set by the initial conditions, and therefore the number of the minimum search points is also unchanged, and thus the calculation amount is reduced.
In specific implementation, as shown in the flowchart of the multi-threshold clustering method in fig. 2, according to the distance between the search point and the target point and the velocity difference between the search point and the target point, filtering out part of the search points, and determining the remaining search points to form a target clustering cluster, including the following steps:
step S201, obtaining a distance between the search point and the target point, determining whether the distance between the search point and the target point is greater than a first distance threshold, and if so, filtering the search point, wherein the first distance threshold includes: the distance between the search point and the target point is larger than the neighborhood radius of the target cluster;
the distance calculation formula between the search point and the target point is as follows:
Step S202, obtaining the speed direction of a target point, calculating the included angle between the speed direction of the target point and the X axis of a 4D millimeter wave radar coordinate system, determining the transverse distance between a search point and the target point according to the included angle between the speed direction of the target point and the X axis of the 4D millimeter wave radar coordinate system and the sine value of the distance between the search point and the target point, judging whether the transverse distance between the search point and the target point is greater than a second distance threshold, and filtering the search point if the transverse distance is greater than the second distance threshold, wherein the second distance threshold comprises the following steps: a lateral distance threshold between the search point and the target point;
before the second distance threshold judgment condition, the first distance threshold requirement needs to be met, and the calculating of the included angle of the X axis of the 4D millimeter wave radar coordinate system in the speed direction of the target point includes: by passing
Calculating the included angle of the X axis of the 4D millimeter wave radar coordinate system of the speed direction of the target point, wherein v y Velocity vector, v, along the y-axis for the target point x Is the velocity vector of the target point along the X-axis.
In an embodiment, the calculating of the sine value of the distance between the search point and the target point comprises: projecting the distance between the search point and the target point to the speed direction of the target point, and adopting the method according to the distance projection between the search point and the target point and the included angle between the speed direction of the target point and the X axis of the 4D millimeter wave radar coordinate system
Calculating to obtain the sine value of the distance between the search point and the target point, wherein d is the distance between the search point and the target point, d y Is the distance projection between the search point and the target point.
Step S203, obtaining the speed of the search point and the speed of the target point, calculating the speed difference between the search point and the target point, determining whether the speed difference between the search point and the target point is greater than a first speed threshold, and if the speed difference is greater than the first speed threshold, filtering the search point, wherein the first speed threshold includes: a velocity difference threshold between the velocity of the search point and the velocity of the target point;
the calculating the speed difference between the search point and the target point according to the speed of the search point and the speed of the target point specifically includes: by using
Calculating the velocity difference between the search point and the target point, wherein v 1 For the velocity of the search point, v 2 Is the velocity of the target point.
Step S204, obtaining the speed direction of the search point and the speed direction of the target point, calculating the included angle between the search point and the speed direction of the target point, and whether the included angle between the search point and the speed direction of the target point is greater than a second speed threshold, if so, filtering the search point, wherein the second speed threshold comprises: searching an included angle threshold value between the speed direction of the point and the speed direction of the target point; taking the rest search points in the target clustering cluster as target point clouds;
the speed direction of the search point and the speed direction of the target point are obtainedAnd calculating an included angle between the search point and the target point speed direction, wherein the included angle comprises the following steps: by using
Calculating the included angle between the search point and the target point in the speed direction, wherein v y1 For searching point delays
Velocity vector of shaft, v x1 Velocity vector, v, extending the X-axis for the search point y2 Extend to the target point
Velocity vector of shaft, v x2 Is the velocity vector of the target point along the axis.
The multi-threshold clustering condition in the steps S201 to S204 ensures that all search points in the same target cluster are clustered correctly.
In specific implementation, fig. 3 shows a flowchart of determining the contour of the target cluster (fig. 3 shows simplified contents of the following steps S301 to S305), and the above-mentioned calculating the contour information of the target cluster according to the target 4D point cloud data and determining the type of the target object corresponding to the contour of the target cluster includes the following steps:
step S301, acquiring the speed of all points in the target clustering cluster according to
Calculating the average value of the speed summation of all the points in the target clustering group to obtain the speed of the target clustering group;
step S302, under the condition that the speed of the target cluster is larger than a preset speed threshold value, calculating the speed directions of all points in the target cluster according to the speeds of all points in the target cluster, and adopting the 4D millimeter wave radar coordinates according to the speed of the target cluster and the 4D millimeter wave radar coordinates
Calculating target objects corresponding to target clustering clustersAccording to the orientation of the target objects corresponding to the target cluster, and then according to
when the target cluster comprises one point, setting the minimum three-dimensional size in the target cluster, wherein the minimum length of the target cluster is L min Minimum width of target cluster is W min Minimum height of target cluster is H min To a minimum length of L min Length as a target clusterLSo that the minimum width is W min As the width W of the target cluster, the minimum height is made H min Height as a target Cluster
；
Step S303, coordinate system based on target cluster
And 4D millimeter wave radar coordinate system
Calculating the included angle between the target clustering direction and the X axis of the 4D millimeter wave radar coordinate system
Based on the angle
Coordinate system of 4D millimeter wave radar
Rotating to coordinate system of clustering with target
In the same direction, adopt
Calculating three-dimensional coordinates of all points in the target cluster under the rotated 4D millimeter wave radar coordinate system, and obtaining the maximum value of the three-dimensional coordinates of all the points
And minimum value
By using
Step S304, calculating to obtain contour information of the target cluster by adopting the following formula 1, formula 2, formula 3 and formula 4 according to the target central point, the three-dimensional size and the orientation of the target object corresponding to the target cluster;
determining the type of the target object corresponding to the contour of the target cluster according to the obtained contour information of the target cluster, wherein x is center 、y center And z center The coordinate of the center point of the clustering target, L the length of the target clustering cluster, and W the width of the target clustering cluster.
Specifically, the clustering can be completed according to the above steps S301-S304, the clustering effect is shown in fig. 4, and the orientation and size of the target object corresponding to the target cluster are shown in fig. 5.
Step S305, obtaining the contour information of the target cluster according to the above calculation, and determining the type of the target object corresponding to the contour of the target cluster, including: setting different target speed threshold ranges and different target size threshold ranges according to different target object types, if the speed of the target cluster is within the set target speed threshold range and the contour information of the target cluster is within the target size threshold range, determining that the target object type with the speed of the target cluster within the set target speed threshold range and the contour information of the target cluster within the target size threshold range is an initial target object type, and meanwhile, determining that the number of times of confirmation of the target object type is +1, and if the number of times of confirmation of the target object type is greater than the set number of times of confirmation threshold, determining that the initial target object type meeting the conditions is the target object corresponding to the contour of the target cluster. For example, determining the type of the target object corresponding to the contour of the target cluster includes: the target type comprises a common vehicle and a truck, a common vehicle speed threshold range and a truck speed threshold range are set for the common vehicle, a common vehicle size threshold range and a truck size threshold range are set for the common vehicle, the speed of a target cluster is compared with the common vehicle speed threshold range or the truck speed threshold range, if the speed of the target cluster is within the common vehicle speed threshold range and the contour information of the target cluster is within the common vehicle size range, the target object type is considered to be the common vehicle, and if the speed of the target cluster is within the truck speed threshold range and the contour information of the target cluster is within the truck size range, the target object type is considered to be the truck; further comprising: and if the speed of the target cluster is within the range of the speed threshold of the common vehicle and within the range of the speed threshold of the truck, the target object type is considered as the truck, and the truck has priority.
The steps S301 to S305 are to calculate the contour information of the target cluster according to the target 4D point cloud data, determine the type of the target object corresponding to the contour of the target cluster, and accurately calculate the contour information of the target cluster according to the three-dimensional information of the 4D point cloud data, since the three-dimensional information can be directly obtained by the 4D millimeter wave radar, the amount of calculation is reduced, the hardware cost of the autonomous driving vehicle can be effectively reduced, and the same sensing effect of the target cluster can be obtained.
In specific implementation, the method further includes: and after determining that the point, the distance between which and the target point is less than the neighborhood radius, in the points corresponding to the target 4D point cloud data is the search point, adding an access mark is _ visited to the search point.
The access marks are added to the search points, so that the access times are reduced, and the clustering efficiency is ensured.
The above procedure for accessing the token is as follows: after the target point is determined, determining the neighborhood radius of the contour corresponding to the target cluster corresponding to the target 4D point cloud data according to the target point, traversing all the points in the neighborhood radius of the contour corresponding to the target cluster, if the point is not visited in the searching process, adding the point into the target cluster, marking the point added into the target cluster as visited until all the points in the neighborhood radius of the contour corresponding to the target cluster are visited, adding the point cloud of the neighborhood radius of the contour corresponding to the target cluster into the target cluster, adding no new point into the target cluster to obtain a complete target cluster, and continuously repeating the steps to traverse all the points to obtain a plurality of target clusters.
In specific implementation, the method further includes: 105, tracking the flight path according to the target clustering cluster, creating, updating or deleting flight path data, and determining target flight path data, wherein the steps comprise: and associating the target clustering cluster with the track data according to the Hungarian matching algorithm, under the condition that the target clustering cluster is not associated with the track data, finding out the track data with the highest similarity to the target clustering cluster from the track data according to the nearest neighbor association algorithm, determining the track data with the highest similarity from the target clustering cluster and the track data to be associated, obtaining an association result, creating, updating or deleting the track data according to the association result, and determining the target track data.
The Hungarian matching algorithm is the prior art, specific means are not described in detail here, and the accuracy of track association can be effectively improved, the miss association rate is reduced, and stable track of tracks is guaranteed by combining the Hungarian algorithm (HM) with the nearest neighbor tracking algorithm (GNN).
In specific implementation, the finding out the track data with the highest similarity to the target cluster from the track data according to the nearest neighbor association algorithm and determining the association between the target cluster and the track data with the highest similarity from the track data includes: and if the speed difference is less than a set speed threshold, calculating the position difference between the position of the target clustering cluster and the position of the track.
The calculating the position difference between the position of the target cluster and the position of the track includes: the position difference includes a lateral position difference Δ x lat And difference in longitudinal position Δ x lon If the transverse position difference is smaller than the set transverse position difference threshold value and the longitudinal position difference is smaller than the set longitudinal position difference threshold value, adopting the value of delta x =deltax lon +4Δx lat And calculating the position difference of the position of the target clustering cluster and the position of the flight path. The track data with the highest similarity in the target clustering cluster can be accurately obtained by calculating the position difference between the position of the target clustering cluster and the position of the track, so that later association is facilitated, and the position difference and the target index are combined into a group of data pairs to be stored in a data queue.
In a specific implementation, the updating the track data according to the association result includes: updating the position of the flight path, updating the speed of the flight path and updating the size of the flight path, wherein the updating the size of the flight path comprises the following steps: storing the size of the target clustering group associated with the flight path into a data queue, setting a length threshold of the data queue, calculating the average value of all sizes in the length threshold of the data queue, determining the current size, updating if the current size is larger than the size stored into the data queue last time, and otherwise, keeping the current size unchanged.
The position of the updated track and the speed of the updated track are both the prior art, the acquired position information and speed confidence of the target clustering cluster are directly compared with track data, the track data is updated according to the acquired position information and speed confidence of the target clustering cluster, the position of the updated track and the speed of the track are updated, the size of the updated track is convenient for track tracking, the moving position of the target can be accurately acquired, and the track of the target can be more accurately output; when the size of the flight path is updated, the smoothness of the data is ensured by calculating the average value of all sizes in the length threshold of the data queue.
In a specific implementation, the creating, updating or deleting the track data corresponding to the target cluster according to the association result to determine the target track data includes: under the condition that the correlation result is that the target clustering cluster does not have correlated track data, establishing track data by using the current target clustering cluster; updating the track data corresponding to the target cluster under the condition that the association result is that the target cluster is associated with the track data; and deleting the current track data if the correlation result is that the track data does not have the target clustering cluster and the frequency of the track data correlating with the target clustering cluster is greater than a set threshold value.
The track data corresponding to the target clustering cluster is created, updated or deleted according to the correlation result, and the target track data is determined, so that the accuracy and the stability of the three-dimensional size of the target are improved, and the follow-up utilization of the contour information of the target is facilitated to realize richer perception functions.
Fig. 6 is a schematic diagram illustrating a track overlap judgment, and as shown in fig. 6, the updating of the track data corresponding to the target cluster includes: merging the tracks, judging whether the tracks of the current target cluster are overlapped with the boundary frames of the tracks at the previous moment, if the boundary frames of the two tracks are overlapped, reserving the track with the earliest time of occurrence, judging the overlapping condition of the boundary frames of the two tracks, namely judging whether the boundary frames of the two tracks are crossed, and the method comprises the following steps: the first is whether the line segments of the two bounding boxes cross each other, the second is whether the multiple vertexes of the first bounding box are in the second bounding box, and the third is whether the multiple vertexes of the second bounding box are in the first bounding box.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
The embodiment of the present application further provides a detection apparatus based on a 4D point cloud, and it should be noted that the detection apparatus based on a 4D point cloud according to the embodiment of the present application may be used to execute the detection method based on a 4D point cloud provided in the embodiment of the present application. The following describes a detection apparatus based on a 4D point cloud according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of a detection apparatus based on a 4D point cloud according to an embodiment of the present application. As shown in fig. 7, the apparatus includes: the system comprises a data acquisition module, a data preprocessing module and a point cloud clustering module;
the data acquisition module 101 is used for acquiring 4D point cloud data;
the data preprocessing module 102 is configured to perform noise filtering on the 4D point cloud data to determine target 4D point cloud data;
the point cloud clustering module 103 is used for acquiring spatial three-dimensional information of target 4D point cloud data and determining a target point, and if the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system is greater than a first threshold value, determining a neighborhood radius of a target clustering cluster by adopting a neighborhood radius adaptive algorithm; determining all search points in the target cluster according to the neighborhood radius of the target cluster, including: determining the search point as the search point in the target clustering cluster according to the distance between the search point and the target point, and carrying out access marking on the search point; under the condition that the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system is larger than a first threshold, the distance between the search point and the target point is larger than a second threshold, and the number of the minimum search points in the target clustering cluster is determined according to a neighborhood point number adaptive algorithm; under the condition that the number of the search points of the target clustering cluster is larger than the number of the minimum search points, filtering all the search points in the target clustering cluster according to a multi-scale clustering threshold confirmation method, determining target point clouds in the target clustering cluster, and taking the target point clouds as members of the target clustering cluster; and calculating the contour information of the target clustering cluster according to the target point cloud, and determining the target type.
In a specific implementation, as shown in fig. 8, the data acquisition module 101 includes: the system comprises a data driving module 1001, a data generating module 1002 and a data recording module 1003, wherein the data driving module 1001 is used for analyzing data frames of the 4D millimeter wave radar and analyzing digital signals into corresponding data information according to a data transmission protocol, each frame of data comprises currently obtained traffic environment point cloud information, and each point in the point cloud comprises information such as a radial distance, a horizontal angle, a pitch angle, a radial speed and a radar scattering area; the data generation module 1002 is configured to convert the point cloud data into a required specific format after the data analysis is completed, where the required specific format includes, but is not limited to, txt, json, xml, and other data storage formats; a data recording module 1003, configured to compress and store data in a specific format; and the subsequent analysis and secondary development of the data packet are facilitated.
In specific implementation, the 4D millimeter wave radar includes a lot of noise points in the point cloud data due to the propagation characteristics of the electromagnetic waves and the existence of a lot of interference objects in the environment. In order to eliminate the influence of noise points on the sensing performance, the noise points need to be filtered. Therefore, the data preprocessing module is used for carrying out noise point filtering on the 4D point cloud data to determine target 4D point cloud data, and the noise point filtering method comprises the following steps: firstly, judging the motion state of a target (a target formed by a target cluster) according to the motion speed, setting different RCS (radar cross section) thresholds at different motion speeds, wherein the larger the motion speed is, the larger the RCS threshold is correspondingly; and when the RCS value of a certain point is smaller than a set threshold, filtering the point as a noise point. Secondly, when the 4D millimeter wave radar moves, filtering all points at which the relative speed of the movement speed of the 4D millimeter wave radar is 0 and the RCS value is smaller than a second threshold value; because in the real world, it is difficult to have a vehicle that is completely stationary relative to the movement of the 4D millimeter wave radar while the 4D millimeter wave radar is moving.
In specific implementation, the apparatus further comprises: and the track management module is used for tracking the track of the target cluster, creating, updating or deleting track data and determining the target track data.
The detection system based on the 4D point cloud comprises a processor and a memory, wherein each execution module and the like of the device are stored in the memory as a program unit, and the processor executes the program unit stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, the 4D millimeter wave radar obtains the target type by adjusting kernel parameters, and the contour information of the target clustering cluster is accurately calculated, so that the method has extremely strong feasibility and can provide rich clustering target information; the method has the advantages that the track tracking is carried out according to the target clustering cluster, the target size is more accurate and stable, the accurate output of the target type is realized, the purposes of no need of acquiring a large amount of sample data in advance for supervised learning, rapid deployment and low requirement on a vehicle-mounted hardware platform are achieved.
The memory may include volatile memory in a computer readable medium, random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
An embodiment of the present invention provides a storage medium on which a program is stored, the program implementing a detection method based on a 4D point cloud when executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the detection method based on 4D point cloud is executed when the program runs.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized: step S101, obtaining target 4D point cloud data, determining any point corresponding to the target 4D point cloud data as a target point, calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, and determining the neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target cluster; step S102, determining points, of which the distance to a target point is smaller than the neighborhood radius, in points corresponding to target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in a target cluster according to the distance between the search points and the target point, a second threshold and the set number of the search points under the condition that the distance between at least one search point and the target point is larger than the second threshold; step S103, under the condition that the actual number is larger than the minimum number, filtering out part of search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the rest search points to form a target cluster; step S104, calculating contour information of the target cluster according to the target 4D point cloud data, and determining the type of a target object corresponding to the contour of the target cluster; 105, tracking the flight path according to the target clustering cluster, creating, updating or deleting flight path data, and determining target flight path data, wherein the steps comprise: and associating the target cluster with the track data according to the Hungarian matching algorithm, finding the track data with the highest similarity with the target cluster in the track data according to the nearest neighbor association algorithm under the condition that the target cluster is not associated with the track data, determining that the target cluster is associated with the track data with the highest similarity in the track data to obtain an association result, and creating, updating or deleting the track data according to the association result to determine the target track data. The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device: step S101, obtaining target 4D point cloud data, determining any point corresponding to the target 4D point cloud data as a target point, calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, and determining the neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target cluster; step S102, determining points, of which the distance to a target point is smaller than the neighborhood radius, in points corresponding to target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in a target cluster according to the distance between the search points and the target point, a second threshold and the set number of the search points under the condition that the distance between at least one search point and the target point is larger than the second threshold; step S103, under the condition that the actual number is larger than the minimum number, filtering out part of search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the rest search points to form a target cluster; step S104, calculating contour information of the target cluster according to the target 4D point cloud data, and determining the type of a target object corresponding to the contour of the target cluster; 105, tracking the flight path according to the target clustering cluster, creating, updating or deleting flight path data, and determining target flight path data, wherein the steps comprise: and associating the target clustering cluster with the track data according to the Hungarian matching algorithm, under the condition that the target clustering cluster is not associated with the track data, finding out the track data with the highest similarity to the target clustering cluster from the track data according to the nearest neighbor association algorithm, determining the track data with the highest similarity from the target clustering cluster and the track data to be associated, obtaining an association result, creating, updating or deleting the track data according to the association result, and determining the target track data.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of a unit may be a division of a logic function, and an actual implementation may have another division, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or may not be executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:
the method comprises the steps of performing target identification by combining a point cloud clustering algorithm, a neighborhood radius self-adaptive algorithm and a neighborhood point number self-adaptive algorithm, obtaining target 4D point cloud data through a 4D millimeter wave radar, calculating the distance between a target point and a coordinate origin in a 4D millimeter wave radar coordinate system, determining the neighborhood radius of a contour corresponding to a target clustering cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is larger than a first threshold, determining the actual number of search points, and determining the minimum number of the search points in the target clustering cluster according to at least the distance between the search points and the target point, the second threshold and the set number of the search points under the condition that the distance between at least one search point and the target point is larger than a second threshold; the accuracy of target clustering is greatly improved through a neighborhood radius adaptive algorithm and a neighborhood point number adaptive algorithm, the missing rate of distant targets is effectively reduced, and meanwhile, the target splitting phenomenon frequently occurring in sparse point cloud clustering is effectively reduced; under the condition that the actual quantity is larger than the minimum quantity, filtering partial search points, determining the rest search points to form a target clustering cluster, effectively filtering the influence of false measurement, and reducing the false alarm rate of a clustering result; calculating the contour information of the target cluster according to the target cluster, acquiring the target type from the 4D millimeter wave radar, and accurately calculating the contour information of the target cluster, so that the method has extremely strong implementability and can provide rich clustered target information; the method and the device have the advantages that the target type is accurately output, the purposes that a large amount of sample data does not need to be collected in advance for supervision learning and the requirement on the vehicle-mounted hardware platform is low are achieved, and the technical effects of high generalization capability, high robustness, high reliability and small calculated amount are achieved, so that the technical problems that the calculated amount is too large and the requirement on the vehicle-mounted hardware platform is high due to the fact that a large amount of sample data is collected in advance for supervision learning are solved.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (13)
1. A detection method based on 4D point cloud is characterized by being applied to a 4D millimeter wave radar, and the method comprises the following steps:
acquiring target 4D point cloud data, determining any point corresponding to the target 4D point cloud data as a target point, and calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, wherein the 4D millimeter wave radar coordinate system is a coordinate system established by taking the 4D millimeter wave radar center point as the coordinate origin, and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, determining a neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and an initial radius, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target cluster;
determining points, of the points corresponding to the target 4D point cloud data, of which the distance from the target point is smaller than the neighborhood radius as search points to obtain the actual number of the search points, and determining the minimum number of the search points in the target clustering cluster according to the distance from the search points to the target point, a second threshold and the set number of the search points under the condition that the distance from at least one search point to the target point is larger than the second threshold;
under the condition that the actual number is larger than the minimum number, filtering out part of the search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the residual search points to form the target clustering cluster;
calculating the contour information of the target cluster according to the target 4D point cloud data, determining the type of a target object corresponding to the contour of the target cluster,
determining a neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and a coordinate origin in the 4D millimeter wave radar coordinate system and the initial radius, wherein the neighborhood radius comprises:
calculating a compensation value of an initial radius according to the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, the set maximum distance for keeping the cluster radius unchanged, the increasing step length of the cluster neighborhood radius and the scaling factor of the cluster neighborhood radius;
according to whatThe compensation value of the initial radius and the initial radius adopt R cluster = R (1 +. DELTA.R) determining neighborhood radius of the target cluster, wherein R cluster Is the neighborhood radius of the target cluster, R is the initial radius, and Δ R is the compensation value of the initial radius,
calculating a compensation value of an initial radius according to the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, the set maximum distance that the clustering radius is kept unchanged, the increasing step length of the clustering neighborhood radius and the scaling factor of the clustering neighborhood radius, wherein the method comprises the following steps:
by using
Calculating a compensation value of the initial radius, wherein D is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, and R is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system thre For a set maximum distance, R, at which the cluster radius remains constant step For increasing step size of cluster neighborhood radius, R factor Is the scaling factor of the cluster neighborhood radius.
2. The method of claim 1, further comprising:
and under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is smaller than or equal to a first threshold value, adopting a neighborhood radius threshold value of the target cluster which is initially preset as the neighborhood radius of the target cluster.
3. The method of claim 1, further comprising:
and if the distance between the search point and the target point is smaller than or equal to the set second threshold, the set search point is used as the minimum number of the search points in the target clustering group.
4. The method of claim 1, wherein determining the minimum number of search points within the target cluster according to at least the distance between the search point and the target point, the second threshold, and a set number of search points comprises:
according to n = min _ pt ((d) 0 -d)/d step +1)d factor Determining the minimum number of the search points in the target clustering cluster, wherein n is the minimum number of the search points, min _ pt is the initial preset minimum number of the search points, d 0 Is a threshold value of the distance between the search point and the target point, d is the distance between the search point and the target point, d step Varying step size for the number of search points, d factor Is a search point number variation factor.
5. The method of claim 1, wherein filtering out a portion of the search points according to the distance between the search point and the target point and the difference in velocity between the search point and the target point, and determining the remaining search points to form the target cluster comprises:
obtaining the distance between the search point and the target point, judging whether the distance between the search point and the target point is greater than a first distance threshold, and filtering the search point if the distance is greater than the first distance threshold, wherein the first distance threshold comprises: the distance between the search point and the target point is greater than the neighborhood radius of the target cluster;
acquiring the speed direction of the target point, calculating an included angle between the speed direction of the target point and an X axis of the 4D millimeter wave radar coordinate system, determining a transverse distance between the search point and the target point according to the included angle between the speed direction of the target point and the X axis of the 4D millimeter wave radar coordinate system and a sine value of the distance between the search point and the target point, judging whether the transverse distance between the search point and the target point is greater than a second distance threshold, and filtering the search point if the transverse distance between the search point and the target point is greater than the second distance threshold, wherein the second distance threshold comprises: a lateral distance threshold between the search point and the target point;
acquiring the speed of the search point and the speed of the target point, calculating the speed difference between the search point and the target point, judging whether the speed difference between the search point and the target point is greater than a first speed threshold, and filtering the search point if the speed difference is greater than the first speed threshold, wherein the first speed threshold comprises: a velocity difference threshold between the velocity of the search point and the velocity of the target point;
acquiring the speed direction of the search point and the speed direction of the target point, calculating an included angle between the search point and the speed direction of the target point, and whether the included angle between the search point and the speed direction of the target point is greater than a second speed threshold, and if so, filtering the search point, wherein the second speed threshold comprises: the included angle threshold value between the speed direction of the search point and the speed direction of the target point; and taking the rest of the search points in the target clustering cluster as the points in the target clustering cluster.
6. The method of claim 5, wherein calculating contour information of the target cluster according to the target 4D point cloud data, and determining a type of a target object corresponding to a contour of the target cluster comprises:
acquiring the speeds of all the points in the target clustering cluster, and calculating the mean value of the speeds of all the points in the target clustering cluster to obtain the speed of the target clustering cluster;
under the condition that the speed of the target clustering cluster is greater than a preset speed threshold value, calculating the orientation of a target object corresponding to the target clustering cluster according to the speed of the target clustering cluster and the 4D millimeter wave radar coordinate;
calculating the Euclidean distance between every two points in the target cluster according to the orientation of a target object corresponding to the target cluster, determining a first maximum value of the projection length of the Euclidean distance between the two points in the target cluster on the Z axis of a 4D millimeter wave radar coordinate system, taking the first maximum value as the height of the target cluster, calculating the projection length of the Euclidean distance between every two points in the speed direction, determining a second maximum value of the projection length of the Euclidean distance between the two points in the target cluster on the speed direction, taking the second maximum value as the length of the target cluster, calculating the projection length of the Euclidean distance between every two points in the direction perpendicular to the speed direction, determining a third maximum value of the projection length of the Euclidean distance between the two points in the target cluster on the speed direction, and taking the third maximum value as the width of the target cluster to obtain the three-dimensional size of the target cluster;
calculating the orientation of the target object corresponding to the target cluster and the coordinate system of the 4D millimeter wave radar based on the coordinate system of the target cluster and the coordinate system of the 4D millimeter wave radar
Rotating the coordinate system of the 4D millimeter wave radar to the direction same as that of the coordinate system of the target cluster based on the included angle, calculating three-dimensional coordinates of all points in the target cluster under the rotated coordinate system of the 4D millimeter wave radar, acquiring the maximum value and the minimum value of the three-dimensional coordinates in all the points, and calculating to obtain position information of a target center point of the target cluster;
calculating to obtain contour information of the target cluster according to the position information of the target central point of the target cluster, the three-dimensional size and the orientation of a target object corresponding to the target cluster, and determining the type of the target object corresponding to the contour of the target cluster according to the contour information of the target cluster.
7. The method of claim 1, further comprising:
associating the target cluster with track data;
when the target clustering cluster is not associated with the track data, finding out the track data with the highest similarity with the target clustering cluster from the track data, and determining that the target clustering cluster is associated with the track data with the highest similarity in the track data to obtain an association result;
and creating, updating or deleting the track data according to the association result, and determining target track data.
8. The method according to claim 7, wherein creating, updating or deleting track data corresponding to the target cluster according to the association result, and determining target track data comprises:
under the condition that the association result is that the target clustering cluster does not have associated track data, establishing track data by using the current target clustering cluster;
comparing the position and the speed of a corresponding target object of the target cluster with corresponding track data to obtain a comparison result under the condition that the association result is that the target cluster is associated with the track data, and taking the position and the speed of the corresponding target object of the current target cluster as the current track data if the comparison result is greater than a set threshold value;
and determining the association times of the track data associated with the target clustering cluster under the condition that the association result is that the track data is not associated with the target clustering cluster, and deleting the track data which is not associated with any target clustering cluster and has the association times larger than a set threshold if the association times are larger than the set threshold.
9. The method of claim 8, wherein the method comprises:
storing the profile information of the target clustering cluster associated with the flight path into a data queue, and setting a length threshold of the data queue;
calculating the average value of the contour information of all the target clustering clusters within the length threshold of the data queue to obtain a current measured value;
and comparing the current measurement value with the last measurement value, if the current measurement value is larger than the last measurement value, updating the size of the tracking frame to the current measurement value, and otherwise, keeping the size of the tracking frame unchanged.
10. A detection device based on 4D point cloud, characterized by comprising: the system comprises a data acquisition module, a data preprocessing module and a point cloud clustering module;
the data acquisition module is used for acquiring 4D point cloud data;
the data preprocessing module is used for filtering noise points of the 4D point cloud data to determine target 4D point cloud data;
the point cloud clustering module is used for determining any point corresponding to the target 4D point cloud data as a target point, calculating the distance between the target point and a coordinate origin in a 4D millimeter wave radar coordinate system, and determining the neighborhood radius of the contour corresponding to the target clustering cluster corresponding to the target 4D point cloud data according to the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system and the initial radius under the condition that the distance between the target point and the coordinate origin in the 4D millimeter wave radar coordinate system is greater than a first threshold value, wherein the initial radius is smaller than the maximum size of the contour corresponding to the target clustering cluster; determining points, of which the distance from the target point is smaller than the neighborhood radius, in the points corresponding to the target 4D point cloud data as search points to obtain the actual number of the search points, and determining the minimum number of the search points in the target cluster according to the distance from the search points to the target point, a second threshold and the set number of the search points at least under the condition that the distance from at least one search point to the target point is larger than the second threshold; under the condition that the actual number is larger than the minimum number, filtering partial search points according to the distance between the search points and the target point and the speed difference value between the search points and the target point, and determining the residual search points to form a target clustering cluster; calculating the contour information of the target cluster according to the target 4D point cloud data, determining the type of the target object corresponding to the contour of the target cluster,
determining a neighborhood radius of a contour corresponding to a target cluster corresponding to the target 4D point cloud data according to the distance between the target point and a coordinate origin in the 4D millimeter wave radar coordinate system and the initial radius, wherein the neighborhood radius comprises:
calculating a compensation value of an initial radius according to the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, the set maximum distance for keeping the cluster radius unchanged, the increasing step length of the cluster neighborhood radius and the scaling factor of the cluster neighborhood radius;
according to the compensation value of the initial radius and the initial radius, adopting R cluster = R (1 +. DELTA.R) determine neighborhood radius of the target cluster, where R cluster Is the neighborhood radius of the target cluster, R is the initial radius, delta R is the compensation value of the initial radius,
calculating a compensation value of an initial radius according to the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, the set maximum distance that the clustering radius is kept unchanged, the increasing step length of the clustering neighborhood radius and the scaling factor of the clustering neighborhood radius, wherein the method comprises the following steps:
by using
Calculating a compensation value of the initial radius, wherein D is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system, and R is the Euclidean distance between the search point and the coordinate origin in the 4D millimeter wave radar coordinate system thre For a set maximum distance, R, at which the cluster radius remains constant step For increasing step size of cluster neighborhood radius, R factor Is the scaling factor of the cluster neighborhood radius.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program performs the method of any one of claims 1 to 9.
12. A processor, characterized in that the processor is configured to run a program, wherein the program when running performs the method of any of claims 1 to 9.
13. A detection system based on 4D point cloud, comprising: one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403233.7A CN115439484B (en) | 2022-11-10 | 2022-11-10 | Detection method and device based on 4D point cloud, storage medium and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211403233.7A CN115439484B (en) | 2022-11-10 | 2022-11-10 | Detection method and device based on 4D point cloud, storage medium and processor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115439484A CN115439484A (en) | 2022-12-06 |
| CN115439484B true CN115439484B (en) | 2023-03-21 |
Family
ID=84252688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211403233.7A Active CN115439484B (en) | 2022-11-10 | 2022-11-10 | Detection method and device based on 4D point cloud, storage medium and processor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115439484B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110148144B (en) * | 2018-08-27 | 2024-02-13 | 腾讯大地通途（北京）科技有限公司 | Point cloud data segmentation method and device, storage medium and electronic device |
| CN111199555B (en) * | 2019-12-13 | 2023-10-13 | 意诺科技有限公司 | Millimeter wave radar target identification method |
| CN112689775B (en) * | 2020-04-29 | 2022-06-14 | 华为技术有限公司 | Radar point cloud clustering method and device |
| CN113671481B (en) * | 2021-07-21 | 2024-04-09 | 西安电子科技大学 | 3D multi-target tracking processing method based on millimeter wave radar |
| CN115062683A (en) * | 2022-04-24 | 2022-09-16 | 重庆长安汽车股份有限公司 | 4D millimeter wave radar clustering method based on DBSCAN and storage medium |
| CN114942434B (en) * | 2022-04-25 | 2024-02-02 | 四川八维九章科技有限公司 | Fall gesture recognition method and system based on millimeter wave Lei Dadian cloud |
-
2022
- 2022-11-10 CN CN202211403233.7A patent/CN115439484B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115439484A (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11037305B2 (en) | Method and apparatus for processing point cloud data | |
| CN111583369B (en) | Laser SLAM method based on facial line angular point feature extraction | |
| CN111210429B (en) | Point cloud data partitioning method and device and obstacle detection method and device | |
| Lari et al. | An adaptive approach for the segmentation and extraction of planar and linear/cylindrical features from laser scanning data | |
| CN110753892A (en) | Method and system for instant object tagging via cross-modality verification in autonomous vehicles | |
| CN111932943B (en) | Dynamic target detection method and device, storage medium and roadbed monitoring equipment | |
| CN114137509B (en) | Millimeter wave Lei Dadian cloud clustering method and device | |
| CN110799982A (en) | Method and system for object-centric stereo vision in an autonomous vehicle | |
| CN110869559A (en) | Method and system for integrated global and distributed learning in autonomous vehicles | |
| Erbs et al. | Moving vehicle detection by optimal segmentation of the dynamic stixel world | |
| EP3008488A1 (en) | Lidar-based classification of object movement | |
| CN111553946B (en) | Method and device for removing ground point cloud and method and device for detecting obstacle | |
| CN110674705A (en) | Small-sized obstacle detection method and device based on multi-line laser radar | |
| CN111308500B (en) | Obstacle sensing method and device based on single-line laser radar and computer terminal | |
| WO2023124133A1 (en) | Traffic behavior detection method and apparatus, electronic device, storage medium, and computer program product | |
| CN109583393B (en) | Lane line end point identification method and device, equipment and medium | |
| WO2022141116A1 (en) | Three-dimensional point cloud segmentation method and apparatus, and movable platform | |
| CN113537316A (en) | Vehicle detection method based on 4D millimeter wave radar point cloud | |
| CN111160132B (en) | Method and device for determining lane where obstacle is located, electronic equipment and storage medium | |
| JP7418476B2 (en) | Method and apparatus for determining operable area information | |
| WO2022099620A1 (en) | Three-dimensional point cloud segmentation method and apparatus, and mobile platform | |
| CN115439484B (en) | Detection method and device based on 4D point cloud, storage medium and processor | |
| Liu et al. | Target detection from 3D point-cloud using Gaussian function and CNN | |
| Tamayo et al. | Improving object distance estimation in automated driving systems using camera images, LiDAR point clouds and hierarchical clustering | |
| CN110782475A (en) | Multipath component processing method, terminal and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |