CN114419152A

CN114419152A - Target detection and tracking method and system based on multi-dimensional point cloud characteristics

Info

Publication number: CN114419152A
Application number: CN202210041471.1A
Authority: CN
Inventors: 张漫; 季宇寒; 李寒; 李世超; 曹如月; 张振乾; 苗艳龙
Original assignee: China Agricultural University
Current assignee: China Agricultural University
Priority date: 2022-01-14
Filing date: 2022-01-14
Publication date: 2022-04-29
Anticipated expiration: 2042-01-14
Also published as: CN114419152B

Abstract

The invention relates to a target detection and tracking method based on multi-dimensional point cloud characteristics, belonging to the technical field of mobile platform target detection and tracking, and comprising the following steps: s1, collecting fusion data; s2, detecting a target; s3, tracking the target; and S4, generating an environment map. According to the invention, through analysis and combination based on multi-dimensional point cloud characteristics, accurate and stable detection and tracking of moving platforms such as unmanned vehicles and robots on dynamic and static targets in a complex ground environment can be realized, and reliable references are provided for obstacle avoidance, behavior prediction and the like.

Description

Target detection and tracking method and system based on multi-dimensional point cloud characteristics

Technical Field

The invention belongs to the technical field of mobile platform target detection and tracking, and relates to a target detection and tracking method and system based on multi-dimensional point cloud characteristics.

Background

Target detection and tracking are key technologies of navigation platforms such as unmanned vehicles and autonomous mobile robots. Target detection is a measure of the current determined state of the target; target tracking is a prediction of the likely future state of a target. Detecting and tracking interdependence promotion relationship, wherein the detection can provide a measurement value for tracking so as to correct tracking model parameters; tracking may provide predictive values for detection to update metrology regions of interest. The target detection and tracking method can realize measurement and estimation of target information, and has important significance in navigation obstacle avoidance (such as unmanned driving), target following (such as multi-machine cooperation) and other applications.

The laser radar (Light detection and ranging) acquires the spatial information and the reflection intensity information of the surrounding environment through laser ranging, and has the advantages of high precision, long distance, high speed, no influence of illumination and the like. Three-dimensional LiDAR (3D LiDAR) has multiple sets of laser transmitters and receivers, can provide richer environmental information, and has been widely used in the fields of unmanned vehicles, robots, and the like.

Currently, target detection and tracking mainly includes two types: detection-based tracking and initialization-based tracking.

Detection Based Tracking (DBT) contains two key components, the detector and the tracker. Starting from initialization, identifying a target in original data through a detector, extracting an abstract target model through features, predicting the state of the target by using a tracker, associating actual measured data with predicted data, and updating parameters of the tracker. The DBT carries out detection-tracking iteration on each frame of data to realize closed loop, and continuously outputs a prediction result of a tracking target.

The Detection Free Tracking (DFT) based on initialization needs to manually specify an initial tracking target, determine a tracking frame, and complete a Detection tracking task together with a tracker. And the DFT continuously outputs the prediction result of the tracking target through the iteration of the initial tracking frame and the tracker.

Compared with the two tracking modes, the detection-based tracking method does not need manual initialization, can process the situations of addition, disappearance, combination, splitting and the like of a tracking target, has good tracking robustness and functional integrity, and is more suitable for being applied to mobile platforms such as unmanned vehicles or robots.

In particular, for detection-based tracking methods, the performance depends on the goodness of the detection method, i.e., on the discrimination of different target features by the detector. The traditional target detection method based on point cloud data mainly focuses on the induction of simple attributes such as target position, size, shape and the like, and the anti-interference capability is poor; the machine learning target detection method based on the point cloud data needs to use a large number of high-quality samples for training, and target types which do not appear in a training set cannot be well detected.

Disclosure of Invention

In view of the above technical problems, an object of the present invention is to provide a target detection and tracking method based on multidimensional point cloud features, which can realize accurate and stable detection and tracking of moving platforms such as unmanned vehicles and robots on dynamic and static targets in a complex ground environment through analysis and combination based on multidimensional point cloud features, and provide reliable references for obstacle avoidance, behavior prediction, and the like.

In order to achieve the purpose, the invention provides the following technical scheme:

the invention discloses a target detection tracking method based on multi-dimensional point cloud characteristics, which comprises the following steps:

s1, collecting fusion data;

installing configuration data collection equipment on a mobile platform, the data collection equipment comprising a three-dimensional LiDAR (3D LiDAR), a Global Navigation Satellite System (GNSS), and an Inertial Navigation System (INS); acquiring environmental point cloud information through a 3D LiDAR, acquiring the position and course information of a mobile platform through a GNSS, and acquiring the attitude information of the mobile platform through an INS; the data acquired by each data acquisition device are in the same time slot through multi-sensor time synchronization, so that the data correlation is ensured; and carrying out coordinate system conversion on the point cloud data in the same time slot through multi-coordinate system space registration to obtain fused point cloud data under a geodetic coordinate system.

S1.1, installing and configuring multiple sensors;

the method comprises the steps that a three-dimensional laser radar (3D LiDAR), a Global Navigation Satellite System (GNSS) and an Inertial Navigation System (INS) are arranged and configured on a mobile platform, environmental point cloud information is obtained through the 3D LiDAR, position and course information of the mobile platform is obtained through the GNSS, and attitude information of the mobile platform is obtained through the INS. The mobile platform comprises an unmanned vehicle, a robot and the like.

In the invention, the mobile platform is an unmanned tractor, and the application scene is a farmland automatic operation environment. Target detection and tracking in complex environments requires accurate, reliable, and rich raw data. The position and attitude estimation under the condition of uneven ground needs to be carried out on the high-precision positioning and attitude sensor. Assembling a 3D LiDAR at the top of the unmanned tractor, and acquiring three-dimensional point cloud information in the environment in real time; assembling a double-antenna RTK-GNSS mobile station at the top of the unmanned tractor, and receiving a base station deviation signal to obtain the positioning and course information of the high-precision mobile platform; and an AHRS is assembled at the position, close to the steering center, of the top of the unmanned tractor, and attitude information of the mobile platform is obtained.

S1.2, synchronizing the time of multiple sensors;

the multi-sensor time synchronization is characterized in that hardware synchronization or software synchronization is selected to standardize the data acquisition processes of the 3D LiDAR, the GNSS and the INS into a unified acquisition time slot, so that the 3D LiDAR, the GNSS and the INS in each acquisition time slot can acquire data according to a designed time sequence, and the acquired multi-sensor data with high time correlation degree is acquired. Different reading modes are selected for different types of devices, such as: the data effectiveness is verified in a multi-level buffering mode, the mean value filtering and the down-sampling of the buffer area are used, the data stability is improved, and the data effectiveness of received signals is controlled more accurately by using a trigger instruction. Wherein the content of the first and second substances,

the hardware synchronization mode is that the synchronization signal generation equipment sends acquisition trigger signals to the 3D LiDAR, the GNSS and the INS through physical connection, and the 3D LiDAR, the GNSS and the INS acquire the signals simultaneously after receiving the trigger signals. The trigger signal is a PPS (pulse per second) timing signal of the GNSS receiver or a synchronous signal generated by a special synchronous signal generator.

The software synchronization mode is that the acquisition software controls the acquisition flow of the sensor in the time slot by reading the hardware clock or instruction delay of the acquisition terminal and combining the data buffer area, thereby realizing time synchronization.

In the invention, the acquisition terminal of the unmanned tractor is a vehicle-mounted mobile workstation. Through a software synchronization mode, the vehicle-mounted mobile workstation reads a hardware clock of the vehicle-mounted mobile workstation, and rapidly reads data in a 3D LiDAR buffer area, an RTK-GNSS buffer area and an AHRS buffer area in sequence, so that the consistency of data acquisition time of each sensor is ensured.

After time synchronization, the data acquired by the multiple sensors in the same time slot have a corresponding relation in time, and the data in each time slot can be further subjected to spatial registration.

S1.3, spatial registration of a multi-coordinate system;

the multi-coordinate system space registration is to unify a sensor coordinate system corresponding to the 3D LiDAR data, a vehicle body coordinate system corresponding to the INS data and a geodetic coordinate system corresponding to the GNSS data through rigid body transformation, and transformation matrixes among the coordinate systems can be obtained through the installation position of equipment and the position of the mobile platform relative to a reference origin.

In the invention, the point cloud data collected by the unmanned tractor is spherical coordinate data with 3D LiDAR as an origin, and is converted into a space rectangular coordinate system with the 3D LiDAR as the origin through coordinate projection, namely a sensor coordinate system; the 3D LiDAR is stably installed at the top of the unmanned tractor, and point cloud data under a sensor coordinate system can be converted to a space rectangular coordinate system with the steering center of the unmanned tractor as an origin through rigid body coordinates, namely a vehicle body coordinate system; through the real-time pose information of the unmanned tractor, the point cloud under the vehicle body coordinate system can be converted into a plane rectangular coordinate system with the platform initial position as the origin, namely a geodetic coordinate system.

The point cloud data after time synchronization and space registration accurately fuses the surrounding environment information and the platform information, namely the fused point cloud data.

S2, detecting a target;

dividing the fused point cloud data obtained in the step S1 into a plurality of mutually independent point cloud clusters through target segmentation; carrying out bounding box fitting on the cloud clusters of each point to obtain the minimum bounding of the attached cloud clusters; extracting the characteristics of the point cloud cluster in the minimum enclosure to obtain the multi-dimensional point cloud characteristics of the point cloud cluster; and screening out a tracking target and a non-tracking target based on the classification features in the multi-dimensional point cloud features, and further dividing the tracking target into specific categories.

And S2.1, the target segmentation sequentially carries out ground point filtering, outlier filtering and point cloud clustering on the fused point cloud data to obtain point cloud clusters. The method specifically comprises the following steps:

s2.1.1, ground point filtering;

the fused point cloud data comprises a large amount of ground point clouds, the point clouds belonging to different targets can be separated through ground point filtering, meanwhile, interference points irrelevant to target detection and identification are reduced, and the accuracy and the real-time performance of point cloud detection and tracking are improved.

Performing ground point filtering processing on the fused point cloud data by adopting a sensor-based ground point filtering method, a neighborhood distribution-based ground point filtering method or a morphology-based ground point filtering method to obtain non-ground point cloud;

the sensor-based ground point filtering method calculates the theoretical value (called 'foot point') of the intersection point of each layer of 3D LiDAR scanning lines and the ground plane point by using a conical curve through the installation position of the 3D LiDAR, the angle of each layer of 3D LiDAR scanning lines and the posture of the mobile platform relative to the ground. Further, calculating the Euclidean distance between the fused point cloud data and the theoretical 'foot point' in the radial direction, and filtering out the ground point cloud with the distance smaller than a threshold value.

The neighborhood distribution-based ground point filtering method comprises the steps of carrying out grid space division on fused point cloud data to obtain regularly arranged independent grid spaces, carrying out statistics on point cloud distribution in each grid, and obtaining characteristic data such as the maximum value, the minimum value, the average value, the range and the like of the height of the point cloud distribution; and screening the ground grids according to an empirical threshold value in an actual scene, and filtering ground points contained in the ground grids from the fused point cloud data.

The morphological-based ground point filtering method morphologically approximates the ground point cloud to a standard plane; constructing a proper plane model, performing model parameter iteration through a plane fitting method, obtaining a point cloud index which accords with the plane model in the fused point cloud data, distinguishing an inner point and an outer point of the model, and further realizing segmentation of the ground point cloud.

In the invention, a random sampling consistency (RANSAC) plane fitting method of limiting normal vector direction is adopted to extract a ground point cloud index so as to obtain non-ground point cloud. And the normal vector direction is limited by calculating an included angle between the normal vector direction of the fitting plane obtained by the RANSAC method and the vertical direction, if the included angle is smaller than a threshold value, the fitting plane is regarded as the ground, otherwise, the fitting plane is regarded as interference, and other fitting planes meeting the conditions are continuously searched until the conditions are met.

S2.1.2, filtering outliers;

according to the principle of 3D LiDAR, the resolution of the point cloud data decreases with the increase of the acquisition distance, the point cloud data far away from the 3D LiDAR cannot completely represent the characteristics of the target due to being too sparse, and the existing noise points are easy to cause false detection. In addition, the point cloud data exceeding the maximum height of the mobile platform does not contribute to the analysis of the passable area of the mobile platform.

Performing outlier filtering processing on the non-ground point cloud obtained by S2.1.1 by adopting a statistical filtering method, a radius filtering method or a conditional filtering method to obtain filtered point cloud data;

the statistical filtering performs statistical analysis on the neighborhood of each point, and calculates the average distance from the point to all the neighboring points. Assuming a gaussian distribution with a shape determined by the mean and standard deviation, points with mean distances within a threshold range are retained and points outside the range are filtered out.

The radius filtering counts the number of points contained in the neighborhood of each point with the set radius, the points with the number exceeding the threshold are reserved, and the points with the number not exceeding the threshold are filtered.

And the conditional filtering sets a specified condition, judges whether each point meets the condition, reserves the point meeting the condition, and filters the point not meeting the condition. The condition may be a restriction of the point in certain dimensions or a combination thereof.

In the invention, two layers of clipping windows are established through a conditional filter, an outer layer of clipping window filters out too high and too far point clouds, and an inner layer of clipping window filters out the self point clouds of the unmanned tractor, thereby obtaining filtered point cloud data.

S2.1.3, point cloud clustering;

partitioning the filtered point cloud data by adopting a clustering method based on division, a clustering method based on hierarchy or a clustering method based on density to obtain mutually independent point cloud clusters;

the clustering method based on division iteratively obtains inner points of different clusters by setting clustering conditions (distance, point number and the like) to realize clustering, and typical methods include Euclidean clustering, conditional Euclidean clustering, K-means and the like.

The hierarchical clustering method is characterized in that a top-down octree segmentation principle or a bottom-up seed point growth principle is utilized, a plurality of hierarchical segmentations are carried out on point cloud data, and finally a proper hierarchy is selected as a clustering result.

The density-based clustering method assumes that the independent clusters have higher point cloud density, and takes the high-density clusters separated by the low-density point cloud as a clustering target, and a typical method is DBSCAN and the like.

In the invention, a classical Euclidean clustering method is used for realizing point cloud clustering, and further point cloud clusters are obtained. In particular, the setting of the euclidean distance threshold is based on the actual size of the unmanned tractor, reducing the false alarm rate generated by false passable regions.

S2.2, fitting a bounding box;

the geometric features of the point cloud clusters are one of important screening bases, and on the premise that the specific types of the point cloud clusters are not clear, a space model of the point cloud clusters is constructed preliminarily in a bounding box fitting mode and is used as the geometric features related to space constraint calculation.

Performing bounding box (frame) fitting on each point cloud cluster sequentially through an Oriented Bounding Box (OBB), a Projection bounding box (PBR) and an Upright Bounding Box (UBB) to obtain the minimum bounding of the attached point cloud cluster;

firstly, considering the characteristic vector direction of a point cloud cluster, selecting an orientation bounding box OBB, wherein the boundary direction of the orientation bounding box OBB is determined by the characteristic vector direction of the point cloud cluster, and the difference of the point cloud in distribution can be described; secondly, considering that the LiDAR may not be able to acquire complete point cloud data of the target under the condition of obvious occlusion in the environment, sequentially converting the orientation bounding box OBB into a projection bounding box PBR and an erecting bounding box UBB so as to better fit the point cloud cluster; finally, the two-dimensional geometric features of the point cloud cluster are obtained by projecting the bounding box PBR, and the three-dimensional geometric features of the point cloud cluster are obtained by erecting the bounding box UBB.

The basic flow of the directional bounding box OBB being converted into a projection bounding box PBR and further into an erected bounding box UBB is as follows:

(1) putting 8 vertexes Pt of the OBB_1-8The central coordinate P is projected to the vertical XOY plane to respectively obtain ground projection points Gt of 8 vertexes_1-8And projected point G of the center point_p；

(2) Projection point Gt to the ground_1-8The distances between previously edge-connected points are compared (e.g., Gt)₁And Gt₂，Gt₁And Gt₄，Gt₁And Gt₅) Where the maximum is the principal direction of the horizontal direction vector, i.e. the constituent direction of the PBR, and the direction vector is denoted v₁；

(3) With G_pA point on a straight line, vector v₁For the linear direction, a reference straight line L is constructed according to a straight line point directional equation₁(ii) a In the same way, with G_pIs a point on a straight line, perpendicular to the vector v₁Vector v of₂For the linear direction, a reference straight line L is constructed₂；

The point-wise equation of the straight line is as follows:

wherein x is₀、y₀Are each G_pThe horizontal and vertical coordinates of (1); u and v are each v₁Or v₂A component of (a);

(4) projection point Gt to the ground_1-8Respectively calculate to a straight line L₁And L₂The arithmetic distance of (i.e., distance with positive and negative);

the arithmetic distance of a point to a straight line is calculated as follows:

wherein x is₀、y₀Are each Gt_1-8The horizontal and vertical coordinates of (1); A. b, C is a parameter of a general equation Ax + By + C of a straight line being 0; d is the arithmetic distance from the point to the straight line;

(5) respectively taking the maximum value and the minimum value of the arithmetic distance of the two reference straight lines to obtain 4 maximum values X_max、X_min、Y_max、Y_min. Based on the 4 maxima, 4 initial vertices (X) are constructed_max，Y_max)、(X_max，Y_min)、(X_min，Y_max)、(X_min，Y_min)；

(6) From a known origin of coordinates and G_pConstructing translation vectors according to coordinates, constructing a rotation matrix according to the main direction calculated in the step (2), and performing rotation translation transformation on the 4 initial vertexes in the step (5) to obtain 4 vertexes Gt _ boundary of the projection bounding box PBR_1-4；

(7) The maximum values of the height in the point cloud cluster corresponding to the OBB, namely max _ z _ value and Gt _ boundary are obtained_1-4Combining to obtain the Gt _ boundary_1-4Is vertically aboveThe other 4 vertices of the square, Gt _ boundary_5-8. Thus, 8 vertexes Gt _ boundary of the upright bounding box UBB are obtained_1-8；

According to the method, the bounding box fitting process of the point cloud cluster is combined with OBB, PBR and UBB bounding boxes (frames), the OBB is used as the basis of bounding fitting, fitting distortion possibly existing in the point cloud cluster is corrected through PBR and UBB, and a basis is provided for multi-dimensional geometric feature extraction.

S2.3, extracting multi-dimensional point cloud features;

extracting the multidimensional point cloud characteristics from three different dimensions of point cloud distribution characteristics, point cloud neighborhood characteristics and point cloud material characteristics, and extracting the multidimensional point cloud characteristics of the point cloud cluster in the minimum enclosure to obtain the multidimensional point cloud characteristics of the point cloud cluster; wherein the content of the first and second substances,

the point cloud distribution characteristics are point characteristics based on point cloud clusters, and represent the number, the gravity center, the height maximum value, the height minimum value, the height average value and the height range of the point cloud serving as a high-density point distribution area; wherein, the number is the total number of points in the point cloud cluster; the gravity center is the mean value of x, y and z coordinates of all points in the point cloud cluster; the maximum height value is the maximum z coordinate value of the midpoint of the point cloud cluster; the minimum height value is the minimum z coordinate value of the midpoint of the point cloud cluster; the height average value is the z coordinate average value of the midpoint of the point cloud cluster; the height range is the difference between the maximum height and the minimum height.

The point cloud neighborhood features represent morphological features of the point cloud cluster as a whole based on geometrical features of the point cloud cluster, and comprise geometrical features under three different models, namely an orientation bounding box OBB, a projection bounding box PBR and an erecting bounding box UBB; wherein the content of the first and second substances,

geometric features of the oriented bounding box OBB include vertex, direction, center, length, width, height, volume, and three-dimensional density; the vertex is 8 vertex xyz coordinates of the OBB; direction is 3 eigenvectors of OBB; the center is the geometric center coordinate of the OBB; the length is the length of the longest side of the OBB; the width is the length of the shortest side of the OBB; the height is the length of the OBB secondary long side; the volume is the product of the length, the width and the height of the OBB; the three-dimensional density is the ratio of the total point number of the point cloud cluster to the volume of the OBB.

The geometric features of the projection bounding box PBR include vertex, direction, center, length, width, area, aspect ratio, two-dimensional density, and center distance; the vertex is 4 vertex xy coordinates of the PBR; direction is 2 eigenvectors of PBR; the center is PBR geometric center coordinates; the length is the length of the long side of the PBR; the width is the length of the short side of the PBR; the area is the length-width product of the PBR; the aspect ratio is the ratio of the length to the width of the PBR; the two-dimensional density is the ratio of the total point number of the point cloud cluster to the PBR area; the center distance is the length of the line segment between the PBR geometric center and the acquisition platform center.

The geometric features of the erected bounding box UBB include vertex, orientation, center, length, width, height, volume, three-dimensional density, and area-to-height ratio; 8 vertex xyz coordinates with a vertex of UBB; 3 eigenvectors with direction UBB; the center is UBB geometric center coordinates; length UBB for the longest side; a length of the shortest side of width UBB; a height of UBB the length of the minor side; length, width, height product of volume UBB; the three-dimensional density is the ratio of the total point number of the point cloud cluster to the volume of UBB; the area height ratio is the ratio of the area of the PBR to the height of UBB.

The point cloud material characteristics are based on information of point cloud clusters in reflectivity dimensions, and the statistical analysis is carried out on the reflectivity of the point cloud set, wherein the statistical analysis comprises average reflectivity, maximum reflectivity, minimum reflectivity, extremely poor reflectivity and reflectivity variance. The average reflectivity is the average reflectivity of the midpoint of the point cloud cluster; the maximum reflectivity is the maximum reflectivity of the midpoint of the point cloud cluster; the minimum reflectivity is the minimum reflectivity of the midpoint of the point cloud cluster; the extremely poor reflectivity is the difference between the maximum value and the minimum value of the reflectivity; the reflectivity variance is the reflectivity variance of the midpoint of the point cloud cluster.

The multi-dimensional point cloud features referred to in the present invention include vectors/scalars with dimensions and vectors/scalars without dimensions. The vectors/scalars with dimensions are all corresponding to international unit systems, and the influence caused by different dimensions is eliminated by a normalization method.

In the present invention, the advantages of the multi-dimensional point cloud feature are illustrated by a specific example. The combined features (linearity, planarity, divergence, total variance, feature entropy, trace, curvature change) formed by the feature factors of the point cloud feature vector are specifically defined as follows:

linearity:

planarity:

divergence:

the total variance is as follows:

characteristic entropy:

tracing:

curvature change:

in the formula, λ₁，λ₂，λ₃Is the eigenvalue of the point cloud eigenvector, and₁≥λ₂≥λ₃；e₁，e₂，e₃for the ratio of characteristic values, by formula

And (4) calculating.

S2.4, classifying the targets;

and screening out a tracking target and a non-tracking target based on a classification feature group in the multi-dimensional point cloud feature, and further dividing the tracking target into specific categories based on the tracking feature group.

The multi-dimensional point cloud features can be used as feature factors through decorrelation and normalization to form a more complex classification feature group and a tracking feature group. The classification feature group can be used as a criterion for point cloud classification; the tracking feature groups can form a difference function, and provide key criteria for data association.

Specifically, the multi-dimensional point cloud features described in S2.3 have correlation with each other, and are subjected to decorrelation operations such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), Wavelet packet transform (Wavelet packet transform), and the like, so as to become mutually Independent normalized feature factors with consistent dimensions. And carrying out weighted combination on the normalized feature factors to obtain two types of feature groups: a classification feature set and a tracking feature set.

Classification feature set: the classification features have obvious class correlation, the same class difference is small, the different classes have large difference, and different target types can be distinguished through analysis.

Tracking feature set: the category correlation of the tracking features is weak, and the different target individuals have significant difference, so that a difference degree function can be constructed to distinguish different targets in tracking.

The target classification comprises tracking target screening based on a typical threshold value of a classification feature set and target morphological classification based on training learning of the classification feature set.

Establishing a classification characteristic group under a specific scene according to experience, and dividing the point cloud cluster to be classified into a tracking obstacle needing further classification and a non-tracking obstacle needing no further classification through a threshold; and (3) distinguishing and conjecturing the specific category of the tracking target by using a machine learning (including deep learning) method and through high-quality point cloud supervision training of various target points.

S2.4.1, screening the tracked target based on the typical threshold of the classification feature group;

the point cloud cluster reserved by target segmentation comprises a tracking target and a non-tracking target. The tracking target is a target with potential movement capability, and the movement state of the tracking target is required to be tracked; the non-tracking target is a target without motion capability, and does not need to be tracked.

In a limited application scene, the types of the tracking targets which can appear can be expected, an empirical threshold value can be obtained through data statistics, and most of the non-tracking targets can be screened out by using a typical threshold value. Specifically, the following categories of common non-tracking targets are provided:

suspension: targets exceeding the buffer height of the acquisition platform (traffic is not affected and ignored in the environment map);

and (3) residual ground: residual ground targets (which do not influence the traffic and are ignored in the environment map) after the point cloud filtering processing;

building facade: objects such as wall surfaces of buildings, fence fences and the like (without tracking, in an environment map, a vertical projection layer representation);

multi-barrier cluster: a plurality of targets formed by independent clusters with similar distances, and the PBR of the targets comprises a larger passable area (without tracking, represented by a vertical projection layer in an environment map);

s2.4.2, training the learned target form classification based on the classification feature set;

and after the tracked targets are screened, the tracked targets need to be further classified into specific target types, and a training set for machine learning is established according to the classification feature group of the point cloud and the manually marked class labels. Depending on the specific application scenario, there may be some differences in the alternative categories of tracking targets. For example, in an urban road environment, the common types of tracked objects include pedestrians, vehicles, and other objects. The machine learning method for carrying out the target morphology classification can select Adaboost or Pointnet/Pointnet + +, and the like.

Adaboost is one of the most successful boost methods at present, and a strong classifier capable of accurately classifying training samples is finally obtained by constructing a series of weak classifiers (combining features in a classification feature set) and continuously adaptively adjusting the weight of each weak classifier in the iterative training learning process.

The Pointernet/Pointernet + + is a typical deep learning method, can directly perform type training by using point cloud data and training marks as input, has great advantages compared with a point cloud target detection and identification method based on image deep learning and needing depth map conversion, and can mine deep association of three-dimensional point cloud on target classification.

In the invention, a top-down two-stage classification method is adopted to classify the targets of the point cloud cluster.

A first stage: and (4) screening the tracking targets, namely dividing the point cloud clusters into the tracking targets and the non-tracking targets by using a typical threshold. Wherein, the non-tracking targets are not further classified (such as suspended objects, residual ground, building facades and multi-obstacle clusters), and the tracking targets are classified into specific categories.

And a second stage: and (3) carrying out object form classification, namely forming a weak classifier by utilizing the independent flat weight characteristics of the classification characteristic group aiming at the tracked object, and training and integrating the weak classifier into a strong classifier through an Adaboost algorithm to realize specific classification (such as pedestrians, vehicles and the like) of the tracked object.

S3, tracking the target;

constructing a morphological model of the tracking target according to the category of the tracking target, and determining a kinematic model of the tracking target through an interactive multi-model (IMM); selecting a corresponding tracker to predict a target state according to the kinematic model; designing a multi-stage region of interest, and matching a target detection value in a tracking region with a filter prediction value by using data association; managing a target tracking mark through a tracking state machine; and based on the geodetic coordinate system, judging the dynamic and static properties of the target and estimating a dynamic target motion vector.

S3.1, tracking the model;

in order to stably and reliably track the target, a tracking model needs to be established to describe the position and the contour of the target. Therefore, according to the specific category of the tracking target, different morphological models and kinematic models are selected, so that the tracking accuracy can be improved.

The morphological model represents the center, size, shape, etc. of the tracked target. For specific classified tracking targets in a specific application scene, selecting morphological models such as a cylinder, a cuboid, a parallelepiped and the like is favorable for accurately mastering the position and contour information of the tracking targets;

the kinematics model represents the motion mode of the tracked target, self-adaptive judgment can be carried out on different tracked targets through an interactive multi-model (IMM), and the motion possibility of the target is described by combining with classical models such as a uniform velocity model (CV), a uniform acceleration model (CA), a constant turning rate and a velocity amplitude (CTRV).

The appropriate tracking model has positive significance on the performance of target tracking. The attached morphological model can improve the stability of a target observation value; accurate kinematic models can improve the accuracy of state estimation.

S3.1.1, morphological model;

pedestrians and vehicles are representative two tracking targets, and morphological models of the pedestrians and the vehicles cannot be summarized.

In the present invention, the category-based morphological model building process is described by taking pedestrians, vehicles, and other types as examples.

The pedestrian targets are morphologically described by adopting the cylindrical models, the diameters and the heights of the cylindrical models are subjected to standard constraint, and the size of each cylindrical model is limited to enable the cylindrical model to only contain the independent pedestrian targets. The pedestrian motion is highly random and may change orientation and direction of motion in a short time, so the morphological orientation of the cylindrical model is not set.

The vehicle target adopts the cuboid model to carry out morphological description, and carries out fitting completion to the boundary of the cuboid model, calibrates the central point position, avoids because mutual sheltering or from sheltering from between the target and lead to the incomplete problem of morphological fitting. The vehicle usually has a definite orientation, and steering is also a continuous process, so a vector parallel to the long side of the cuboid is set as the morphological orientation.

Other objects are morphologically described using a straight parallelepiped model, other types of objects may be atypical classes of objects or aggregates of multiple pedestrians, multiple vehicles, pedestrians and vehicles, vehicles/pedestrians and other classes, whose morphological features are difficult to describe by a single specific model, thus preserving the locations of their boundaries and center points, and not setting their morphological orientation.

S3.1.2, kinematic model;

the kinematic models of pedestrians and vehicles need to consider not only their categories, but also specific application scenarios.

In the invention, adaptive kinematic models such as Interactive Multiple Models (IMM) are used to meet the actual use requirements.

The pedestrian target adopts a Constant velocity model (CV) to perform kinematics description so as to cope with the randomness of the motion of the pedestrian, and the pedestrian target (relatively slow speed) can be timely tracked by using the Constant velocity model under higher acquisition, detection and tracking frequency.

The vehicle target is described by a Constant acceleration model (CA) and a Constant turn rate and velocity magnitude model (CTRV), possible behaviors (straight line driving or steering) of the vehicle target are judged by IMM self-adaptation, and a proper kinematic model is selected.

The other targets are subjected to kinematic description by adopting a constant-speed model so as to deal with complicated and irregular atypical targets or aggregation targets, and the uncertainty of the model can be reduced as much as possible by selecting the constant-speed model.

S3.2, state estimation;

the state estimation is one of key technologies of target tracking, and the predicted value of the state estimation directly influences the tracking performance as the result of the target tracking.

At present, state estimation is mainly based on a classical correlation filtering mode, and a tracking target is estimated through trackers such as Kalman filtering, particle filtering and the like. The kalman filter is divided into a linear Kalman Filter (KF), an Extended Kalman Filter (EKF), and an Unscented Kalman Filter (UKF) according to whether the model is linear or not. Particle filtering can deal with the problem of non-Gaussian distribution, but has larger operation overhead, and is not suitable for real-time tracking application in a complex environment.

In a specific situation, the state estimation problem needs to consider the adaptation of the method to the model, including the measurement model of each sensor and the state model of the target.

In the invention, the output of the 3D LiDAR is a data point under a space rectangular coordinate system, and the data point has a linear measurement model; and tracking the state model of the target, and selecting a proper state model and a tracker corresponding to the state model in a self-adaptive mode according to different categories.

For the pedestrian target and other targets, a CV model is used for description, a state transition matrix of the CV model is linear, and linear Kalman filtering is used for state estimation.

And describing a vehicle target in a non-steering state by using a CA model, wherein a state transition matrix is linear, and performing state estimation by using linear Kalman filtering.

And describing a vehicle target in a steering state by using a CTRV model, wherein a state conversion matrix is nonlinear by introducing a steering angle, and estimating by using unscented Kalman filtering.

The main reason why the extended kalman filtering is not used is that the time overhead of the jacobian matrix calculation is unstable, and the effect is not as good as that of unscented kalman filtering in real-time application.

S3.3, associating data;

in the flow of the target tracking method, data association is a key step for linking the detector measurement value and the tracker prediction value. The data association matches the current state (a priori probability) predicted based on the state at the previous moment with the current measurements actually obtained by the sensor and detector, and then gives the state (a posteriori probability) at the current moment. Meanwhile, parameters of the tracker are updated and iterated according to the state of the current moment, and the reliability of the tracker is improved or maintained.

According to the principle of data association method, it can be divided into deterministic method (such as nearest neighbor, global nearest neighbor) and probabilistic method (such as joint probability data association, multi-hypothesis tracking). The former generally constructs a difference function, and obtains perfect matching by solving the minimum value of the difference function; the latter analyzes the complete matching by the angle of solving the optimal posterior probability through probability distribution.

In the invention, a tracking target range is determined by constructing a multi-level region of interest and a target tracking window, a data association bipartite graph is constructed by adopting a global nearest neighbor method, and optimal complete matching solution is carried out by adopting a KM algorithm. Particularly, a type conversion cost matrix is introduced into the construction of the difference function, so that the influence of the target class on data association is improved.

S3.3.1, tracking window;

under complex environmental conditions, if the assumption of cross-matching is made between all detected targets determined by the detector and all tracking tracks managed by the tracker, the problem of "combinatorial explosion" is caused, a large amount of computing resources are consumed, and obvious errors may be generated. For this problem, the constraint of the tracking window needs to be performed on both the moving platform and the tracking trajectory.

For the constraint of the mobile platform, dividing a multi-level region of interest (ROI) according to the distance from the PBR center point of the tracking target to the center of the mobile platform: vertical projection ROI, target tracking ROI, emergency braking ROI. Wherein the content of the first and second substances,

vertical projection ROI: and only vertically projecting the target in the ROI to construct a vertical projection area of the environment grid map.

Target tracking ROI: and carrying out target tracking and dynamic and static judgment on the target in the ROI to construct a static target area and a dynamic target area of the environment grid map.

Emergency braking ROI: and if the target appears in the ROI, immediately sending a braking instruction to the mobile platform, carrying out emergency obstacle avoidance, and constructing an emergency braking area of the environment grid map.

For the tracking trajectory constraint, since the tracking target position usually has a continuously changing characteristic, only the target detected around the tracker is likely to be the detection of the matching tracker. Therefore, a cylindrical target tracking window is constructed centering on the tracker, and only the detection value inside the target tracking window has an opportunity to become a detection target matching the tracker. In particular, the parameters of the target tracking window may be dynamically adjusted according to different tracking states to adapt to different tracking prediction confidence levels.

In the invention, a multi-stage region of interest consisting of a vertical projection ROI, a target tracking ROI and an emergency braking ROI and a target tracking window based on dynamic adjustment parameters of a tracking state are used for jointly determining a tracking window for tracking a target, thereby improving the tracking efficiency and avoiding 'combined explosion'.

S3.3.2, global nearest neighbor;

global Nearest Neighbor (GNN) is a classical and efficient data association method, and can also react quickly in a complex environment. The core of the global nearest neighbor is to construct a difference function by integrating the tracking characteristics of different target objects so as to represent the correlation accuracy. And optimizing the minimum value of the difference function to obtain the optimal data correlation result.

In the invention, the feature used by the diversity factor function is a weighted combination of tracking feature groups, and a key factor (category factor) is introduced, and the specific expression form of the factor is a type conversion cost matrix. According to different application scenes and practical situations, a targeted difference function can be constructed to seek the optimal effect. The following is a simple example of a function for the degree of difference:

wherein, CHANGE is a value of the variance function, and the smaller the value is, the higher the correlation degree between the measurement and the prediction is; distance represents the normalized Distance between the measured value and the predicted value; num_t+1And Num_tRespectively representing the number of target point clouds at t +1 and t; AI_t+1、AI_t、AI_RangeRespectively representing the average reflectivity of the point cloud at the t +1 moment, the average reflectivity of the point cloud at the t moment and the reflectivity range; classsdifference represents the value of the type conversion cost matrix; A. b, C, D represent the weight values of the respective additions.

S3.3.3, type conversion cost matrix;

the type conversion cost matrix is a real symmetric square matrix (with the same row and column order), the rows of which represent the detection target classes, the columns of which represent the tracker classes, and the corresponding values represent the conversion cost for converting from one class to another, which is inversely proportional to the probability. By setting conversion cost values among different types, an important parameter which is a class and is dimensionless and difficult to normalize can be introduced into the difference degree function and further used as an important factor.

The following is a simple example of the present invention with respect to the type conversion cost matrix:

wherein, ClassDifference represents the values of the type conversion cost matrix; pedestrian, vehicle, other represent pedestrians, vehicles, other tracked objects, respectively, "×" connections represent cost values for transitions between two different classes of objects, and the values of the matrix are referred to by way of example only.

S3.3.4, solving the optimal complete matching;

for solving the minimum value of the difference function, the minimum value is usually abstracted into weighted bipartite graph description, and the minimum weight is completely matched. Specifically, the left vertex set of the bipartite graph is a detection target, the right vertex set is a tracker, wherein the detection target located in a target tracking window of the tracker has an edge connection with the detection target, and a weight of the edge is a value of a difference function; the detected target outside the target tracking window of the tracker is not connected with the target, and the weight value of the edge of the detected target is equivalent to a maximum value.

In general, if each edge of the bipartite graph has a weight (which may be negative), a perfect matching scheme is solved such that the sum of the weights of all matching edges is maximized, denoted as the best perfect match. For the application needing to solve the minimum weight complete matching, only the weight of each side needs to be inverted, and the optimal complete matching is solved.

If the weight of each edge is the same, solving the matching problem of the bipartite graph by using a classical Hungarian method; for the unequal-weight bipartite graph matching problem, a KM (Kuhn and Munkres) method can be adopted, the KM method is a greedy extension of the Hungarian method, and the solution of the optimal perfect matching of the maximum (minimum) weight of the weighted bipartite graph is realized by introducing the concepts of a feasible benchmarking and an equal subgraph. By the KM method, the bipartite graph formed by the difference function values can be solved, and the optimal complete matching between the current-time detector and the tracker is obtained.

In the invention, the matching problem between the detection target and the tracker is abstracted into the weighted bipartite graph minimum weight complete matching problem, and the optimal complete matching is obtained by utilizing the connectivity judgment of the difference function value and the target tracking window.

S3.4, tracking management;

trace management designs a trace state machine with 6 states, including trace initial, trace hold, trace lost, trace retrieve, trace terminated, trace invalid. Meanwhile, a tracking state conversion rule among all tracking states is specified, and reliable and stable tracking management of a tracking target is realized.

S3.4.1, tracking state machine design;

each target in the tracking ROI may exhibit various states, with respective life cycles, that need to be managed using specific tracking state markers.

Tracking initialization: when the target is detected and meets the tracking condition, initializing a tracker;

track and hold: the target is continuously detected and meets the tracking condition, and the tracker updates the parameters by using the matched detector information;

loss of tracking: the tracked target cannot be detected or cannot meet the tracking condition, and the tracker continues to predict but does not update;

tracking and recovering: the target which is lost to be tracked is detected in the recovery time limit and meets the tracking condition, and the tracker is matched and updated again;

and (4) terminating tracking: the target losing the tracking can not be detected or can not meet the tracking condition within the recovery time limit, and the tracker is deleted;

and (3) invalid tracking: the target has not yet performed trace initialization or has terminated the generated invalid state.

The invention designs a tracking state machine, specifies 6 tracking states to represent the current tracking state of a target, and establishes a set of tracking state marks for management in order to quantitatively describe each tracking state and the corresponding conversion relation thereof.

The tracking status flag is defined as an integer from 0 to 10, where 0 represents tracking invalid, 1-4 represents tracking initial, 5 represents tracking hold, 6-9 represents tracking loss/recovery, and 10 represents tracking termination.

S3.4.2, tracking state transition rules;

in a complex and changeable environment, the method accords with logic, meets the quantitative tracking state conversion rule of a closed loop, can well process the conditions of shielding, merging, splitting and the like between targets which appear or disappear suddenly and the targets, and improves the robustness and efficiency of tracking.

In the invention, a set of complete closed-loop tracking state conversion rules is constructed on the basis of 6 typical states and based on tracking state marks.

An example of a typical tracking state transition period is as follows:

the tracking state mark gradually establishes tracking from 0 to 1-4; in the initial stage of tracking, the effective tracking makes the mark increase progressively, and the ineffective tracking makes the mark decrease progressively until reaching 5 stable tracking states or reaching 0 ineffective; if the target is tracked continuously and effectively in the tracking and keeping state, keeping the mark as 5; if invalid tracking occurs, the mark is increased progressively to enter a tracking loss state, and if valid tracking occurs before the mark reaches 10, the mark bit is decreased progressively; if the number of the tracking recovery zone bits is decreased to 5, returning to the tracking holding state; and if the tracking flag bit reaches 10, the target is failed to be retrieved, the tracking termination state 0 is entered, the tracker is deleted, and the tracking invalid state is entered.

Particularly, for the tracker in tracking loss and retrieval states, a target tracking window with a larger radius is adopted to make up for the increment of uncertainty of the tracker caused by loss of updating, and the success rate of retrieving the tracked target is improved.

S3.5, judging a moving target and a static target;

compared with relative observation taking a mobile platform as an observation center, the target information detected and tracked based on the fused point cloud data can directly judge the displacement of the target information relative to a geodetic coordinate system, namely real dynamic and static information, through the absolute position of the target information. The dynamic and static target judgment is very critical to the generation of a static target area and a dynamic target area in an environment map, and further influences the reliability and accuracy of subsequent processing such as local path planning, behavior prediction and the like.

Judging a dynamic target and a static target, and judging and ensuring the validity of a result through stable tracking based on tracking state keeping statistics; removing the maximum value and the next maximum value through a sliding window, and eliminating the influence caused by a single singular point; and calculating the average effective displacement, and combining a typical threshold value to give the dynamic and static attributes of the target.

S3.5.1, stable tracking judgment;

only in the case of stable tracking, the position information of the target has a reference value. Therefore, when the tracking state of the target is the tracking hold, a stable count is performed in conjunction with the tracking management state machine. And counting the value of the target stable count in the fixed time window, if the value exceeds a preset threshold value, determining the target stable count as stable tracking, and performing the next dynamic and static judgment, otherwise, marking the motion state of the target as unknown.

S3.5.2, singular point filtering;

interference factors may occur in a complex environment, which causes acquisition noise of a sensor or detection misjudgment of a detector, and mainly shows sudden target position change caused by sudden change of pose or point cloud. And the dynamic and static target judgment adopts sliding window processing on the interference factors which possibly occur, removes the maximum value, the secondary large value, the minimum value and the secondary small value, and eliminates the displacement influence caused by a single singular point.

S3.5.3, mean effective displacement;

in fact, the displacement of a stationary target between two frames is not completely zero, subject to factors such as sensor precision, time synchronization precision, detection algorithm stability, feature extraction algorithm accuracy, and the like. The average effective displacement is the average value of the displacement which is judged to be in a stable state through stable tracking and data are filtered by singular points. By setting a threshold, a target with an average effective displacement smaller than the threshold is regarded as a static target, whereas a target with an average effective displacement larger than the threshold is regarded as a dynamic target, and a vector (velocity direction and magnitude) of motion thereof is calculated.

In the invention, the displacement data of the tracking target is processed by stable tracking judgment and singular point filtering, the average effective displacement is calculated and compared with a threshold value, and finally the dynamic and static attribute judgment of the tracking target with a certain reference value based on a geodetic coordinate system and a motion vector relative to the ground are obtained.

S4, generating an environment map;

the environment map is used as an intuitive expression form of a target tracking result and is also input in a path planning process, and the selection of map elements influences the efficiency and accuracy of local path planning. Generally, the environment map may be classified into a grid map, a geometric map, and a topological map according to the expression form. The grid map can accurately describe the details of the environment, and has the advantages of simple description, controllable complexity, flexible composition and the like in the aspect of navigation obstacle avoidance compared with a geometric map and a topological map.

According to the specific conditions of the mobile platform and the environment, determining the elements of the environment map, respectively representing the targets in different obstacle avoidance levels, finally outputting the grid environment map in a matrix form, and carrying out visual visualization in a bitmap form.

S4.1, selecting map elements;

in the map element selection, firstly, a grid map is selected as a carrier according to a specific scene of an application, and key elements such as a coordinate origin, a coordinate axis, resolution and the like are defined. Particularly, the resolution, i.e., granularity, of the grid map is a key parameter, and an excessively large granularity may cause a decrease in the precision of map information and may not accurately describe the details of a target, while an excessively small granularity may cause a burden of storage and operation, which affects the real-time performance of an application.

The grid map is based on a geodetic coordinate system, and the initial position of the mobile platform is selected by the origin of coordinates to reduce the area overhead of the map; adopting a standard northeast sky axial direction, and keeping the same as a geodetic coordinate system; 0.5m was chosen as the side length of each grid to balance accuracy with real-time.

S4.2, representing a map;

s4.2.1 matrix representation;

the grid environment map is used as input of applications such as navigation obstacle avoidance or behavior prediction and is represented by a matrix with high time and space efficiency. Specifically, the region types represented by each lattice in the lattice are classified, and the tracking target is ID-encoded.

Specifically, the present invention classifies the region types as follows:

passable area: the encoding is done with the number 0, representing the secure area that can be accessed.

A vertical projection area: the number 1 is used for encoding, and represents an area where an untracked object exists and can not pass.

Static target area: and (4) coding by adding ID after the number 2 to represent a static obstacle in the tracked target, and setting a smaller obstacle avoidance buffer area.

Dynamic target area: and (3) coding by adding ID after the number 3 to represent a dynamic obstacle in a tracking target, and setting an asymmetric buffer zone according to the movement direction and speed.

Emergency braking area: the number 4 is used for coding, representing the target of entering the emergency braking range, as one of the conditions for the departure of the emergency braking command.

The contents of each grid in the matrix representation of the grid environment map are constructed as follows:

ZoneType+TargetID

wherein, ZoneType represents the region type, and the ZoneType belongs to {0,1,2,3,4 }; TargetID represents the number of targets, whose number of bits can be specified to suit a particular application.

S4.2.2, bitmap representation;

in an autonomous driving application, the driver (or operator) must have a higher priority for decision making than an autonomous driving system, and therefore an environmental map needs to be able to be visualized in an intuitive form for the driver to understand and make decisions.

Different areas are marked with different colors or filled by bitmap representation so as to describe the distribution of the targets in the current environment.

Specifically, the bitmap is represented as: passable areas (white or no fill), orthographic areas (black or opaque full fill), static target areas (blue or diagonal fill), dynamic target areas (yellow or translucent full fill), emergency stop areas (red or cross-hatched fill).

The invention provides a system for applying the target detection tracking method based on multi-dimensional point cloud characteristics, which comprises a fusion data acquisition module, a target detection module, a target tracking module and an environment map generation module.

Compared with the prior art, the invention has the beneficial effects that:

according to the invention, through analysis and combination based on multi-dimensional point cloud characteristics, accurate and stable detection and tracking of moving platforms such as unmanned vehicles and robots on dynamic and static targets in a complex ground environment can be realized, and reliable references are provided for obstacle avoidance, behavior prediction and the like.

Drawings

FIG. 1 is a flow chart of a target detection tracking method based on multi-dimensional point cloud features according to the present invention;

FIG. 2 is a diagram of the method of the present invention;

FIG. 3 is a multi-dimensional point cloud feature hierarchy diagram of the present invention;

FIG. 4 is a diagram of the flag transition rules of the trace state machine of the present invention;

FIG. 5 is a point cloud data collected by LiDAR in an embodiment of the present invention;

FIG. 6 is a diagram illustrating the effect of spatio-temporal corrected fused point cloud data in an embodiment of the present invention;

FIG. 7 is a diagram illustrating the effect of filtered point cloud data according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating an exemplary point cloud cluster effect;

FIG. 9 is a diagram illustrating the effect of target detection according to an embodiment of the present invention;

FIG. 10 is a diagram illustrating the tracking effect of the target in the embodiment of the present invention;

FIG. 11 is an environmental map effect diagram according to an embodiment of the present invention;

fig. 12 is a system block diagram of the present invention.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

In this embodiment, an unmanned tractor in a farmland environment is used as a mobile platform, and a specific implementation method of data acquisition and processing of target detection and identification of the present invention is illustrated.

As shown in fig. 1 and fig. 2, the target detection and tracking method based on multi-dimensional point cloud features of the present invention includes the following steps:

s1, collecting fusion data;

s1.1, installing and configuring multiple sensors;

assembling a 3D LiDAR at the top of the unmanned tractor, and acquiring three-dimensional point cloud information in the environment in real time; the LiDAR point cloud data obtained is shown in FIG. 5.

A double-antenna RTK-GNSS mobile station is assembled at the top of the unmanned tractor, and positioning and course information of the high-precision mobile platform is obtained after the base station deviation signal is received.

And an AHRS is assembled at the position, close to the steering center, of the top of the unmanned tractor, and attitude information of the mobile platform is obtained.

S1.2, synchronizing the time of multiple sensors;

the integrated data acquisition terminal of the unmanned tractor is a vehicle-mounted mobile workstation. Through a software synchronization mode, the vehicle-mounted mobile workstation reads a hardware clock of the vehicle-mounted mobile workstation, and rapidly reads data in a 3D LiDAR buffer area, an RTK-GNSS buffer area and an AHRS buffer area in sequence, so that the consistency of data acquisition time of each sensor is ensured. Specifically, the synchronous frequency of data acquisition is 10Hz, and the real-time performance of the actual application of the unmanned tractor under the farmland environment can be met.

S1.3, spatial registration of a multi-coordinate system;

the spatial registration of the point cloud data from a sensor coordinate system to a geodetic coordinate system is realized by coordinate projection conversion, rigid body coordinate conversion and real-time pose conversion of the point cloud data acquired by the unmanned tractor. The spatio-temporally corrected fused point cloud data obtained is shown in fig. 6.

S2, detecting a target;

s2.1, target segmentation;

s2.1.1, ground point filtering;

and extracting ground point cloud indexes from the fused point cloud data by a plane fitting method for limiting random sampling consistency in the normal vector direction, and further acquiring non-ground point clouds. Specifically, the normal vector direction of the fitting plane is restricted: only the fitting plane when the normal vector direction and the vertical direction of the fitting plane are less than 30 degrees is regarded as an effective ground model, and non-ground point clouds are filtered accordingly; otherwise, the fitting plane is reselected for verification.

S2.1.2, filtering outliers;

and establishing two layers of clipping windows through a conditional filter, filtering out the over-high and over-far point cloud through the outer layer of clipping window, filtering out the self point cloud of the unmanned tractor through the inner layer of clipping window, and further obtaining filtered point cloud data. Specifically, the shape of the outer layer cutting window is a cuboid, point cloud data outside the cuboid is filtered, and only internal point cloud is reserved; the shape of the inner layer cutting window is a cuboid, point clouds in the cuboid are filtered, and only external point clouds are reserved.

Different targets in the filtered point cloud data after ground point filtering and outlier filtering have relatively discrete distribution, and the next step of point cloud clustering can be performed. The filtered point cloud data obtained is shown in fig. 7.

S2.1.3, point cloud clustering;

and (4) realizing point cloud clustering by using a classical Euclidean clustering method so as to obtain point cloud clusters. In particular, the setting of the euclidean distance threshold of 2.5m is based on the actual size of the unmanned tractor, i.e., the width of the unmanned tractor of 2.05m (the setting of the threshold is expanded through the safety buffer), reducing the false alarm rate generated by false passable regions. The obtained point cloud cluster is shown in fig. 8.

S2.2, fitting a bounding box;

in the bounding box fitting process of the point cloud cluster, OBB, PBR and UBB bounding boxes (boxes) are combined, the OBB is used as the basis of bounding fitting, fitting distortion possibly existing in the point cloud cluster is corrected through PBR and UBB, and a basis is provided for multi-dimensional geometric feature extraction. UBB of the obtained point cloud cluster is a rectangle enclosing box as in fig. 9 (numbers 1 and 3).

S2.3, extracting multi-dimensional point cloud features;

and carrying out systematic induction and arrangement according to the point cloud distribution characteristics, the point cloud neighborhood characteristics and the point cloud material characteristics of the point cloud clusters through the multi-dimensional point cloud characteristics, combining a classification characteristic group and a tracking characteristic group through the multi-dimensional point cloud characteristics, and providing bases for point cloud classification and target tracking respectively. The multi-dimensional point cloud features are shown in fig. 3.

S2.4, classifying the targets;

and (4) carrying out target classification on the point cloud cluster by adopting a top-down two-stage classification method.

a first stage: and (4) screening the tracking targets, namely dividing the point cloud clusters into the tracking targets and the non-tracking targets by using a typical threshold. Wherein, the non-tracking targets are not further classified (such as suspended objects, residual ground, building facades and multi-obstacle clusters), and the tracking targets are classified into specific categories. The tracked objects are point cloud clusters having rectangular bounding boxes (numbers 1 and 3) as in fig. 9, and the untracked objects are point cloud clusters having no bounding boxes as in fig. 9.

and a second stage: and (3) carrying out object form classification, namely forming a weak classifier by utilizing the independent flat weight characteristics of the classification characteristic group aiming at the tracked object, and training and integrating the weak classifier into a strong classifier through an Adaboost algorithm to realize specific classification (such as pedestrians, vehicles and the like) of the tracked object. The specific type of the tracking target is shown as a text label (vector, Pedestrian) in fig. 9.

S3, tracking the target;

s3.1, tracking the model;

s3.1.1, morphological model;

the class-based morphological model building process is described in terms of pedestrians, vehicles, and other types. The pedestrian target is morphologically described by adopting a cylindrical model, and the size is limited but the morphological orientation is not set; the vehicle target adopts a cuboid model for morphological description, boundary completion and center position calibration are carried out, and the long side direction is set as the morphological orientation; other objects are morphologically described using a straight parallelepiped model, without morphological orientation.

S3.1.2, kinematic model;

the interactive multi-model self-adaptive kinematics model is more suitable for actual use requirements.

The pedestrian target and other targets are subjected to kinematic description by adopting a CV model; the vehicle targets are described using a CA model (straight-driving) or CTRV model (steering), with appropriate kinematic models being selected by the IMM.

S3.2, state estimation;

the output of the 3D LiDAR is a data point under a space rectangular coordinate system and has a linear measurement model; and tracking the state model of the target, and selecting a proper state model and a tracker corresponding to the state model in a self-adaptive mode according to different categories.

For pedestrian targets and other targets, a CV model is used for description, a state transition matrix of the CV model is linear, and KF is used for state estimation.

For a vehicle target in a non-steering state, a CA model is used for description, a state transition matrix is linear, and KF is used for state estimation.

For a vehicle target in a steering state, a CTRV model is used for description, a state conversion matrix is nonlinear due to the introduction of a steering angle, and UKF is used for estimation.

S3.3, associating data;

s3.3.1, tracking window;

and determining a tracking window for tracking the target by using a multi-stage region of interest consisting of a vertical projection ROI, a target tracking ROI and an emergency braking ROI and a target tracking window for dynamically adjusting parameters based on the tracking state. The vertical projection ROI coincides with the boundary of the skin clipping window, and the target tracking ROI and the emergency braking ROI are shown as the cylinder boundary (number 4, only partially shown) and the cylinder boundary (number 2) in fig. 10. The center point and the boundary of the target tracking window are shown as solid spheres (numbers 5 and 6) and cylindrical boundaries (numbers 1 and 3) in fig. 10.

S3.3.2, global nearest neighbor;

and carrying out data association between the detection target and the tracker by adopting a global nearest neighbor method based on the difference function.

S3.3.3, type conversion cost matrix;

a key type conversion cost matrix is introduced and used as a class factor to quantitatively describe the conversion cost value between targets of different classes.

S3.3.4, solving the optimal complete matching;

the matching problem between a detection target and a tracker is abstracted into a weighted bipartite graph minimum weight complete matching problem, and a KM algorithm is selected for solving by utilizing the connectivity judgment of the difference function value and a target tracking window to obtain the optimal complete matching. The best perfect match results are indicated by numbers after "Tracker" and "Detection" in fig. 10.

S3.4, tracking management;

s3.4.1, tracking state machine design;

a tracking state machine is designed, and 6 tracking states are specified to represent the current tracking condition of the target. Specifically, tracking states include: initial tracking, keeping tracking, loss tracking, recovery tracking, termination tracking and invalid tracking. In order to quantitatively describe each tracking state and the corresponding conversion relation thereof, a set of tracking state marks is established for management. And the tracking state mark corresponds to the tracking state.

S3.4.2, tracking state transition rules;

based on the 6 typical states, a set of complete closed-loop tracking state transition rules is constructed based on the tracking state flags, as shown in fig. 4.

S3.5, judging a dynamic target and a static target;

s3.5.1, stable tracking judgment;

the displacement data of the tracked target is subjected to stable tracking judgment, so that the interference of invalid data caused by unstable detectors and the like is eliminated, and only targets with tracking and holding states exceeding a threshold number in a specified time interval meet the stable tracking condition.

S3.5.2, singular point filtering;

and further eliminating the interference of single singular point data to the whole data by removing the maximum value, the second largest value, the minimum value and the second smallest value in the effective displacement data.

S3.5.3, mean effective displacement;

and averaging the data subjected to stable tracking judgment and singular point filtering to obtain average effective displacement, and referring the average effective displacement and a set threshold value to judge the dynamic and static attributes and the motion vector of the tracking target. The dynamic and Static attributes are indicated as "Move" or "Static" in fig. 10, and the velocity values of the motion vectors are indicated as numbers in fig. 10 in the unit of "m/s".

S4, generating an environment map;

s4.1, selecting map elements;

S4.2, representing a map;

s4.2.1, matrix representation;

wherein the value ZoneType represents the region type and TargetID represents the target ID

ZoneType+TargetID

wherein, ZoneType represents the region type, and the ZoneType belongs to {0,1,2,3,4 }; TargetID represents the number of the target. In particular, the amount of the solvent to be used,

S4.2.2, bitmap representation;

An example of the bitmap representation is shown in FIG. 11, where passable areas are represented by no fill, orthographic areas are represented by opaque full fill, and dynamic target areas are represented by semi-transparent full fill.

As shown in fig. 12, the present invention provides a system for applying the target detection and tracking method based on multi-dimensional point cloud features, which includes a fusion data acquisition module, a target detection module, a target tracking module, and an environment map generation module.

Claims

1. A target detection tracking method based on multi-dimensional point cloud features is characterized by comprising the following steps:

s1, collecting fusion data;

installing and configuring data acquisition equipment on a mobile platform, wherein the data acquisition equipment comprises a three-dimensional laser radar, a global navigation satellite system and an inertial navigation system; acquiring environmental point cloud information through a three-dimensional laser radar, acquiring position and course information of a mobile platform through a global navigation satellite system, and acquiring attitude information of the mobile platform through an inertial navigation system; the data acquired by each data acquisition device are in the same time slot through multi-sensor time synchronization, so that the data correlation is ensured; performing coordinate system conversion on point cloud data in the same time slot through multi-coordinate system space registration to obtain fused point cloud data under a geodetic coordinate system;

s2, detecting a target;

dividing the fused point cloud data obtained in the step S1 into a plurality of mutually independent point cloud clusters through target segmentation; carrying out bounding box fitting on the cloud clusters of each point to obtain the minimum bounding of the attached cloud clusters; extracting the characteristics of the point cloud cluster in the minimum enclosure to obtain the multi-dimensional point cloud characteristics of the point cloud cluster;

screening out a tracking target and a non-tracking target based on classification features in the multi-dimensional point cloud features, and further dividing the tracking target into specific categories;

s3, tracking the target;

constructing a morphological model of the tracked target according to the category of the tracked target, determining a kinematic model of the tracked target through an interactive multi-model, and selecting a corresponding tracker according to the kinematic model to predict the state of the tracked target; designing a multi-stage region of interest, and matching a target detection value in a tracking region with a filter prediction value by using data association; managing a target tracking mark through a tracking state machine; based on a geodetic coordinate system, judging the dynamic and static attributes of the target, and estimating a dynamic target motion vector;

s4, generating an environment map;

2. The method for detecting and tracking targets based on multi-dimensional point cloud features of claim 1, wherein in step S1, the multi-sensor time synchronization is implemented by selecting a hardware synchronization or software synchronization mode to standardize the data acquisition processes of the three-dimensional lidar, the global navigation satellite system and the inertial navigation system to a uniform acquisition time slot, so as to ensure that the three-dimensional lidar, the global navigation satellite system and the inertial navigation system perform data acquisition according to a designed time sequence in each acquisition time slot and obtain multi-sensor data with high time correlation;

the hardware synchronization mode is that the synchronization signal generation equipment sends acquisition trigger signals to the three-dimensional laser radar, the global navigation satellite system and the inertial navigation system through physical connection, and the three-dimensional laser radar, the global navigation satellite system and the inertial navigation system simultaneously acquire the acquisition trigger signals after receiving the trigger signals;

the software synchronization mode is that the acquisition software controls the acquisition processes of the three-dimensional laser radar, the global navigation satellite system and the inertial navigation system in a time slot by reading the hardware clock or instruction delay of the acquisition terminal and combining a data buffer area to realize time synchronization;

the multi-coordinate system space registration is to unify a sensor coordinate system corresponding to the three-dimensional laser radar data, a vehicle body coordinate system corresponding to the inertial navigation system data and a geodetic coordinate system corresponding to the global navigation satellite system data through rigid body transformation, and transformation matrixes among the coordinate systems are obtained through the installation position of equipment and the position of the mobile platform relative to a reference origin.

3. The method for detecting and tracking the target based on the multi-dimensional point cloud features of claim 1, wherein in the step S2, the target segmentation processes the fused point cloud data sequentially through ground point filtering, outlier filtering and point cloud clustering to obtain point cloud clusters; the method specifically comprises the following steps:

s2.1.1, ground point filtering;

the sensor-based ground point filtering method is characterized in that theoretical values of intersection points of scanning lines of each layer of the three-dimensional laser radar and ground plane points, namely theoretical foot points, are calculated by using a conic curve through the installation position of the three-dimensional laser radar, the angles of the scanning lines of each layer of the three-dimensional laser radar and the posture of the mobile platform relative to the ground; calculating the Euclidean distance between the fused point cloud data and the theoretical foot point in the radial direction, and filtering out the ground point cloud with the distance smaller than a threshold value;

the neighborhood distribution-based ground point filtering method comprises the steps of carrying out grid space division on fused point cloud data to obtain regularly arranged independent grid spaces, carrying out statistics on point cloud distribution in each grid, and obtaining characteristic data such as the maximum value, the minimum value, the average value, the range and the like of the height of the point cloud distribution; screening a ground grid according to an experience threshold value in an actual scene, and filtering ground points contained in the ground grid from the fused point cloud data;

the morphological-based ground point filtering method morphologically approximates the ground point cloud to a standard plane; constructing a proper plane model, performing model parameter iteration through a plane fitting method, obtaining a point cloud index which accords with the plane model in fused point cloud data, distinguishing an inner point and an outer point of the model, and further realizing segmentation of the ground point cloud;

s2.1.2, filtering outliers;

the statistical filtering carries out statistical analysis on the neighborhood of each point and calculates the average distance from the point to all the adjacent points; points with average distance within the threshold range are retained, and points outside the range are filtered;

the radius filtering counts the number of points contained in a neighborhood of each point with a set radius, the points with the number exceeding a threshold value are reserved, and the points with the number not exceeding the threshold value are filtered;

the condition filtering sets a specified condition, whether each point meets the condition is judged, the point meeting the condition is reserved, and the point not meeting the condition is filtered; establishing two layers of clipping windows through a conditional filter, filtering out overhigh and overlong point clouds through the outer layer of clipping window, filtering out the self point clouds of the unmanned tractor through the inner layer of clipping window, and further obtaining filtered point cloud data;

s2.1.3, point cloud clustering;

the clustering method based on division iteratively obtains the inner points of different clusters by setting clustering conditions so as to realize clustering;

the hierarchical clustering method is characterized in that a top-down octree segmentation principle or a bottom-up seed point growth principle is utilized to perform multi-level segmentation on point cloud data, and finally a proper level is selected as a clustering result;

the density-based clustering method assumes that the individual clusters have a higher point cloud density, and takes high-density clusters separated by low-density point clouds as clustering targets.

4. The method for detecting and tracking the target based on the multi-dimensional point cloud features of claim 1, wherein in the step S2, bounding box fitting is performed on each point cloud cluster sequentially through an orientation bounding box, a projection bounding box and an erecting bounding box to obtain the minimum bounding of the attached point cloud cluster;

the directional bounding box is converted into a projection bounding box and further into an erecting bounding box, comprising the steps of:

(1) 8 vertices Pt of the bounding box to be oriented_1-8Projecting the vertical XOY plane by the central coordinate P to respectively obtain ground projection points Gt of 8 vertexes_1-8And projected point G of the center point_p；

(2) Projection point Gt to the ground_1-8The distances between previously edge-connected points are compared (e.g., Gt)₁And Gt₂，Gt₁And Gt₄，Gt₁And Gt₅) Where the maximum is the principal direction of the horizontal direction vector, i.e. the direction in which the projection encloses the frame, the direction vector being denoted v₁；

(3) With G_pA point on a straight line, vector v₁For the linear direction, a reference straight line L is constructed according to a straight line point directional equation₁(ii) a In the same way, with G_pIs a point on a straight line, hangs downPerpendicular to the vector v₁Vector v of₂For the linear direction, a reference straight line L is constructed₂；

The point-wise equation of the straight line is as follows:

the arithmetic distance of a point to a straight line is calculated as follows:

(5) respectively taking the maximum value and the minimum value of the arithmetic distance of the two reference straight lines to obtain 4 maximum values X_max、X_min、Y_max、Y_min(ii) a Based on the 4 maxima, 4 initial vertices (X) are constructed_max，Y_max)、(X_max，Y_min)、(X_min，Y_max)、(X_min，Y_min)；

(7) The highest values of max _ z _ value and Gt _ boundary in the point cloud cluster corresponding to the orientation bounding box_1-4Combining to obtain the Gt _ boundary_1-4Another 4 vertices Gt boundary vertically above_5-8(ii) a Thus, 8 vertexes Gt _ boundary of the upright bounding box are obtained_1-8。

5. The method for detecting and tracking targets based on multi-dimensional point cloud features of claim 1, wherein in step S2, the multi-dimensional point cloud feature extraction includes extracting multi-dimensional point cloud features of point cloud clusters within the minimum bounding volume from three different dimensions of point cloud distribution features, point cloud neighborhood features and point cloud texture features; wherein the content of the first and second substances,

the point cloud distribution characteristics are point characteristics based on point cloud clusters, and represent the number, the gravity center, the height maximum value, the height minimum value, the height average value and the height range of the point cloud serving as a high-density point distribution area; wherein, the number is the total number of points in the point cloud cluster; the gravity center is the mean value of x, y and z coordinates of all points in the point cloud cluster; the maximum height value is the maximum z coordinate value of the midpoint of the point cloud cluster; the minimum height value is the minimum z coordinate value of the midpoint of the point cloud cluster; the height average value is the z coordinate average value of the midpoint of the point cloud cluster; the height range is the difference between the maximum height value and the minimum height value;

the point cloud neighborhood features represent morphological features of the point cloud cluster as a whole based on the geometrical features of the point cloud cluster, and comprise geometrical features under three different models, namely an orientation bounding box, a projection bounding box and an erecting bounding box; wherein the content of the first and second substances,

geometric features of the oriented bounding box include vertex, direction, center, length, width, height, volume, and three-dimensional density; the vertex is 8 vertex xyz coordinates of the directional bounding box; the direction is 3 eigenvectors of the directional bounding box; the center is the geometric center coordinate of the directional bounding box; the length is the length of the longest side of the directional bounding box; the width is the length of the shortest side of the directional bounding box; the height is the length of the secondary side of the oriented bounding box; the volume is the product of the length, the width and the height of the directional bounding box; the three-dimensional density is the ratio of the total point number of the point cloud clusters to the volume of the directional bounding box;

the geometric features of the projection bounding box include vertex, direction, center, length, width, area, aspect ratio, two-dimensional density, and center distance; the vertex is 4 vertex xy coordinates of the projection bounding box; the direction is 2 eigenvectors of the projection bounding box; the center is the geometric center coordinate of the projection bounding box; the length is the length of the long side of the projection bounding box; the width is the length of the short side of the projection bounding box; the area is the product of the length and the width of the projection surrounding frame; the aspect ratio is the ratio of the length to the width of the projection bounding box; the two-dimensional density is the ratio of the total point number of the point cloud cluster to the area of a projection bounding box; the central distance is the length of the line segment between the geometric center of the projection enclosure frame and the center of the acquisition platform;

the geometric characteristics of the erected bounding box comprise vertex, direction, center, length, width, height, volume, three-dimensional density and area-height ratio; the vertex is 8 vertex xyz coordinates of the upright bounding box; the direction is 3 eigenvectors of the erecting bounding box; the center is the geometric center coordinate of the upright bounding box; the length is the length of the longest side of the upright bounding box; the width is the length of the shortest side of the upright bounding box; the height is the length of the secondary long side of the upright bounding box; the volume is the product of the length, the width and the height of the upright bounding box; the three-dimensional density is the ratio of the total point number of the point cloud clusters to the volume of the upright bounding box; the area height ratio is the ratio of the area of the projection bounding box to the height of the upright bounding box;

the point cloud material characteristics are based on the information of the point cloud clusters in the reflectivity dimension, and the statistical analysis is carried out on the reflectivity of the point cloud set, wherein the reflectivity comprises average reflectivity, maximum reflectivity, minimum reflectivity, extremely poor reflectivity and reflectivity variance; the average reflectivity is the average reflectivity of the midpoint of the point cloud cluster; the maximum reflectivity is the maximum reflectivity of the midpoint of the point cloud cluster; the minimum reflectivity is the minimum reflectivity of the midpoint of the point cloud cluster; the extremely poor reflectivity is the difference between the maximum value and the minimum value of the reflectivity; the reflectivity variance is the reflectivity variance of the midpoint of the point cloud cluster.

6. The method for detecting and tracking the target based on the multi-dimensional point cloud features of claim 1, wherein in the step S2, the multi-dimensional point cloud features are normalized feature factors which are independent of each other and have consistent dimensions through principal component analysis, independent component analysis and wavelet packet transformation; and carrying out weighted combination on the normalized feature factors to obtain two types of feature groups: classifying the feature set and tracking the feature set;

classification feature set: the classification features have obvious class correlation, the same class difference is small, the different class differences are large, and different target types are distinguished through analysis;

tracking feature set: the tracking characteristics are weak in category correlation, and different target individuals have obvious difference, so that a difference function is constructed to distinguish different targets in tracking;

and based on the classification feature group, the point cloud cluster is divided into a tracking target and a non-tracking target by using a typical threshold, aiming at the tracking target, a weak classifier is formed by using the independent flat weight features of the classification feature group, and the weak classifier is trained and integrated into a strong classifier by a machine learning method, so that the specific classification of the tracking target is realized.

7. The method for detecting and tracking the target based on the multi-dimensional point cloud features of claim 1, wherein in the step S3, different morphological models and kinematic models are selected according to specific categories of the tracked target;

the morphological model represents the center, size and shape of a tracking target;

the kinematics model represents the motion mode of the tracked target, self-adaptive judgment is carried out on different tracked targets through an interactive multi-model, and the motion possibility of the target is described by combining a uniform velocity model, a uniform acceleration model, a constant turning rate model and a velocity amplitude model;

estimating a tracking target through a Kalman filtering tracker and a particle filtering tracker;

in the step S3, a tracking target range is determined by constructing a multi-level region of interest and a target tracking window, a global nearest neighbor method is used to construct a data association bipartite graph, and a KM algorithm is used to solve the optimal perfect matching, which includes the following specific processes:

dividing a multi-stage region of interest according to the distance from the center point of a projection bounding box of the calculated tracking target to the center of the mobile platform: vertically projecting an interested area, a target tracking interested area and an emergency braking interested area; wherein the content of the first and second substances,

vertical projection region of interest: only vertically projecting the target in the region of interest, and constructing a vertical projection region of the environment grid map;

target tracking region of interest: carrying out target tracking and dynamic and static judgment on a target in the region of interest, and constructing a static target region and a dynamic target region of an environment grid map;

emergency braking region of interest: if the target appears in the region of interest, immediately sending a braking instruction to the mobile platform, carrying out emergency obstacle avoidance, and constructing an emergency braking region of the environment grid map;

constructing a cylindrical target tracking window by taking a tracker as a center, wherein only a detection value in the target tracking window has an opportunity to become a detection target matched with the tracker;

constructing a targeted difference function according to different application scenes and actual situations:

wherein, CHANGE is a value of the variance function, and the smaller the value is, the higher the correlation degree between the measurement and the prediction is; distance represents the normalized Distance between the measured value and the predicted value; num_t+1And Num_tRespectively representing the number of target point clouds at t +1 and t; AI_t+1、AI_t、AI_RangeRespectively representing the average reflectivity of the point cloud at the t +1 moment, the average reflectivity of the point cloud at the t moment and the reflectivity range; classsdifference represents the value of the type conversion cost matrix; A. b, C, D represent the weight value of each addition item;

the matching problem between the detection target and the tracker is abstracted into the weighted bipartite graph minimum weight complete matching problem, and the optimal complete matching is obtained by utilizing the connectivity judgment of the difference function value and the target tracking window.

8. The method for detecting and tracking targets based on multi-dimensional point cloud features of claim 1, wherein in the step S3, the tracking state machine includes 6 states: tracking initialization, tracking maintenance, tracking loss, tracking recovery, tracking termination and tracking invalidation;

and (3) invalid tracking: the target does not carry out tracking initialization or track an invalid state generated after termination;

defining the tracking status flag as an integer from 0 to 10, wherein 0 represents tracking invalid, 1-4 represents tracking initial, 5 represents tracking hold, 6-9 represents tracking loss/recovery, and 10 represents tracking termination;

the tracking state mark gradually establishes tracking from 0 to 1-4; in the initial stage of tracking, the effective tracking makes the mark increase progressively, and the ineffective tracking makes the mark decrease progressively until reaching 5 stable tracking states or reaching 0 ineffective; if the target is tracked continuously and effectively in the tracking and keeping state, keeping the mark as 5; if invalid tracking occurs, the mark is increased progressively to enter a tracking loss state, and if valid tracking occurs before the mark reaches 10, the mark bit is decreased progressively; if the number of the tracking recovery zone bits is decreased to 5, returning to the tracking holding state; if the tracking flag bit reaches 10, the target is failed to be retrieved, the tracking termination state 0 is entered, the tracker is deleted, and the tracking invalid state is entered;

in step S3, the validity of the result is ensured by the stable tracking judgment based on the tracking state holding statistics; removing the maximum value and the next maximum value through a sliding window, and eliminating the influence caused by a single singular point; calculating average effective displacement, and combining a typical threshold value to give the dynamic and static attributes of the target;

combining a tracking management state machine, and if the tracking state of the target is tracking maintenance, performing one-time stable counting; counting the value of the target stable count in a fixed time window, if the value exceeds a preset threshold value, determining the target stable count as stable tracking, and performing the next step of dynamic and static judgment, otherwise, marking the motion state of the target as unknown;

the average effective displacement is the average value of the displacement which is judged to be in a stable state through stable tracking and data are filtered by singular points; and setting a threshold, regarding the target with the average effective displacement smaller than the threshold as a static target, and conversely, regarding the target with the average effective displacement larger than the threshold as a dynamic target, and calculating a motion vector of the target.

9. The method for detecting and tracking the target based on the multi-dimensional point cloud features of claim 1, wherein in step S4, firstly, according to the specific application scenario, a grid map is selected as a carrier, and a coordinate origin, a coordinate axis and a resolution are defined;

classifying the region type represented by each grid in the grid, and carrying out ID coding on a tracking target;

ZoneType+TargetID

wherein, ZoneType represents the region type, and the ZoneType belongs to {0,1,2,3,4 }; TargetID represents the number of the target;

10. A system for applying the multi-dimensional point cloud feature-based object detection and tracking method according to any one of claims 1-9, wherein the system comprises a fusion data acquisition module, an object detection module, an object tracking module and an environment map generation module.