CN113034419B - Method and device for objective quality evaluation of radar point cloud for machine vision tasks - Google Patents

Abstract

The invention provides a method and device for evaluating the objective quality of a radar point cloud oriented to a machine vision task, including: a spherical domain division step: dividing the first point cloud and the second point cloud into a plurality of adjacent spherical domains, respectively. The radius of the domain is related to the longest side a of the visual task target in the point cloud data; the calculation step of the direction vector of the point set: calculate the eigenvalues and eigenvectors of the corresponding point set based on each spherical domain, form the vector of the direction of the point set, and calculate the first The vector difference between one point cloud and the second point cloud in the same spherical domain; the model score acquisition steps: take the number of points in each spherical domain as the weight, and weight and sum the vector differences calculated by each spherical neighborhood, Get the model score. The invention can better estimate the service quality of specific tasks (such as point cloud classification, segmentation, identification, etc.) based on radar point clouds, and has better robustness.

Description

Method and device for objective quality evaluation of radar point cloud for machine vision tasks

technical field

The invention relates to the field of radar point cloud machine vision tasks and point cloud objective quality evaluation, and in particular, to a method and device for radar point cloud objective quality evaluation oriented to machine vision tasks.

Background technique

In recent decades, lidar scanning technology and systems have become more mature and widely used. Three-dimensional data can be obtained from lidar, which is also called radar point cloud and can reflect the basic structural information of the target. Target segmentation, target recognition, target classification, target registration and other specific vision tasks based on 3D laser point cloud data have achieved results. For example, the patent document CN 108981616A discloses a method for inverting the effective leaf area index of a plantation based on the UAV lidar empirical model, which is applied to the research fields of forest resource investigation, forest site quality evaluation and forest productivity estimation.

Unmanned driving technology based on laser point cloud is a hot research topic and has been applied to the actual driving process. However, the way in which vehicle-mounted LiDAR (Light Detection And Ranging, that is, laser detection and measurement, that is, lidar) acquires data leads to shadows, occlusions, etc. in point cloud data, which makes certain visual tasks unable to achieve ideal results, such as point clouds. The recognition rate of the segmentation algorithm becomes lower and the robustness becomes worse, resulting in a waste of computing resources. A data evaluation algorithm is needed to avoid this waste.

Due to the sparsity and inhomogeneity of LiDAR data (dense near, sparse far), and knowing that the target volume in a specific task often has the characteristics of a small proportion. The point-to-point and point-to-point distortion quality evaluation used in the existing pass-through conditions obviously cannot express the specific vision tasks in LiDAR, such as in the radar point cloud segmentation task, for noise data, distant points we do not pay attention to Due to sparseness, the error will be larger, and some of the points we are concerned about nearby will have a smaller error. And because the target volume is small and the noise is not uniform, the calculated RMSE is not directly related to the specific task results, so these several classical distortion evaluation models are less robust in radar point cloud data. In addition, due to the sparseness of LiDAR data, local properties such as normal vector and curvature are difficult to obtain and utilize accurately.

SUMMARY OF THE INVENTION

In view of the defects in the prior art, the purpose of the present invention is to provide a method and device for evaluating the objective quality of radar point cloud oriented to machine vision tasks.

In view of the defects in the prior art (two classical algorithms), the purpose of the present invention is to provide a radar point cloud evaluation method oriented to the quality of service of specific visual tasks, which can better estimate the specific tasks based on radar point clouds by using this model. (such as point cloud classification, segmentation, recognition, etc.) quality of service, and has better robustness.

Specifically, through the above model, the linear relationship between the index of a specific visual task and the model score can be obtained, and then the service quality of the LiDAR point cloud data can be evaluated, and the obtained data model score can estimate the service quality of the visual task.

To achieve the above object, the present invention has adopted the following technical solutions:

This paper provides an evaluation model based on service quality, that is, using the model to score data to estimate the correct rate of the data pair under a specific task, this kind of thinking has not been proposed before.

The specific model ideas are as follows.

1. Divide the reference point cloud a and the point cloud b into several adjacent spherical domains, the radius of the spherical domain is a multiplier of the size of the visual task target, and the radius of the divided spherical domain is the longest visual task target. A multiplier of no more than 2 times half the length of the side.

2. Calculate the eigenvalues and eigenvectors of its point set based on each sphere, obtain a vector representation representing the direction of the point set in the sphere according to a certain combination, and use the vector distance method to obtain the point cloud a and the point The vector difference of cloud b in the same spherical domain.

3. According to a certain attribute of each spherical neighborhood as a weight, the difference of the vectors calculated by each spherical neighborhood is weighted and summed to obtain the model score.

4. Verify the validity of the output score of the evaluation model, bring the reference point cloud a and point cloud b into a specific vision task, and calculate the corresponding accuracy index size. At this time, the relative score of the model should have a certain linearity with the corresponding index size. If the linearity is good, this model can be used to estimate the service quality corresponding to the data.

Preferably, the specific vision task includes any of the following:

Vision tasks such as point cloud classification, segmentation, recognition, and tracking.

Preferably, the multiplier related to the spherical domain includes any of the following:

1.2, 1.5, 1.7 and other numbers not greater than 2

Preferably, the noise types of the model verification method include any of the following:

Gaussian noise, random noise, regional noise and other noise types

Preferably, the combination of the model feature vectors includes any of the following:

Multiply and accumulate, select one or two eigenvalues and the corresponding eigenvector to multiply and accumulate, etc.

Preferably, the method for measuring the vector difference of the model includes any of the following:

L1 norm, L2 norm, Gaussian kernel function, etc.

Preferably, the visual task accuracy index of the model verification method includes any of the following:

IoU (intersection ratio), AP (average precision), recall and other indicators

The present invention also provides a radar point cloud objective quality evaluation system oriented to machine vision tasks, including: a noise adding module, a spherical domain segmentation module, a vector representation module for calculating spherical domain point sets, a vector difference calculation module, and a calculation quality evaluation module. Fractions module.

Input point cloud data module: input original point cloud data;

Noise-adding module: Add noise to the original point cloud data to obtain the noise-added point cloud data;

Spherical domain segmentation module: separate the original point cloud and the noised point cloud into the spherical domain;

Calculate the vector representation module of the spherical domain point set direction: calculate the eigenvalues and eigenvectors of the spherical domain point set, and calculate the vector representation according to the eigenvalues and eigenvectors;

Calculate the vector difference module: calculate the vector difference between the original point cloud and the noised point cloud in the same spherical domain;

Calculate the quality evaluation score module: use the attributes of each ball neighborhood as the weight to calculate the quality evaluation score.

Preferably, it also includes:

Model score verification module: Bring the first point cloud and the second point cloud into the point cloud segmentation task based on deep learning, and calculate the size of the corresponding accuracy index IoU.

Compared with the prior art, the present invention has the following beneficial effects:

1. Robustness enhancement, the present invention adopts a method based on the sum of weights, which preliminarily solves the problem that radar data is dense in the vicinity and sparse in the distance. Specifically, in practical applications, vehicle-mounted LiDAR pays more attention to near objects, and the target of a specific visual task is often near. Therefore, the present invention is based on the sum of the weights established by the task target, so the robustness will be enhanced. .

2. The complexity is reduced. Due to the large amount of point cloud data, the point-to-point, point-to-point approach will be very complicated. The present invention adopts the method of calculating the neighborhood of adjacent spheres, which greatly reduces the number of operations.

3. Based on structural similarity. The approach of point-to-point and point-to-point evaluation ignores the similarity of structure, and the evaluation of point cloud objects for visual tasks (segmentation, recognition, registration, etc.) will not be very accurate. The present invention takes into account the sparseness of radar point clouds, and therefore adopts the method based on the direction of the point set in the spherical neighborhood, and fully considers the structural similarity of the sparse point clouds, so the results will have a good correlation with the indicators of the visual task. .

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

1 is a flowchart of a method for evaluating the quality of radar point cloud data based on visual tasks in an embodiment of the present invention;

FIG. 2 is a model usage scenario in an embodiment of the present invention.

FIG. 3 is a functional block diagram of a visual task-oriented radar point cloud objective quality evaluation system according to an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the inventive concept. These all belong to the protection scope of the present invention.

First, the parameter setting and method selection of its specific invention implementation are introduced. The specific implementation process is as follows.

As shown in FIG. 1 , a method for evaluating the quality of radar point cloud data based on visual tasks provided by the present invention includes the following steps:

1. The reference point cloud a and the noised point cloud b are divided into several adjacent spherical domains respectively. The longest side of the visual task target in the observation point cloud data is x. In order to select the target that can be calculated as a whole as much as possible without affecting the robustness of the model, The radius of the sphere is set to 1.5*(x/2), and the number of sphere neighborhoods is given accordingly.

2. Calculate the eigenvalues and eigenvectors of the point set based on each spherical domain, set the eigenvalues a1, a2, a3, and the eigenvectors α1, α2, α3, and combine them into the direction L of the point set according to the following formula,

L=a1*α1+a2*α2+a3*α3

According to this combination method, the vector representing the direction of the point set in the spherical domain is obtained, and the L1 norm is used to obtain the vector difference between point cloud a and point cloud b in the same spherical domain, so that the vector size and direction difference can be calculated. are all manifested.

3. Taking the number of points in each sphere neighborhood as the weight, the difference of the vectors calculated by each sphere neighborhood is weighted and summed (that is, density weighting) to obtain the model score.

4. Verify the validity of its model, verify the validity of the model score, bring the reference point cloud a and point cloud b into the point cloud segmentation task based on deep learning, add gradient noise to the verification set, and calculate its corresponding accuracy index The size of the IoU (intersection and union ratio) is calculated at the same time as the evaluation model provided by the present invention. At this time, the relative score of the model has a certain linear relationship with the size of the corresponding index. This linearity can be measured by linear correlation indicators PLCC, KROCC, SROCC. If the linearity is good, this model can be used to estimate the service quality corresponding to the data.

Preferably, the specific vision task includes any of the following:

Vision tasks such as point cloud classification, segmentation, recognition, and tracking.

Preferably, the noise types of point cloud b in the model include any of the following:

Gaussian noise, random noise, regional noise and other noise types

Preferably, the multiplier related to the spherical domain includes any of the following:

1.2, 1.5, 1.7 and other numbers not greater than 2

Preferably, the combination of the model feature vectors includes any of the following:

Multiply and accumulate, select one or two eigenvalues and the corresponding eigenvector to multiply and accumulate, etc.

Preferably, the method for measuring the vector difference of the model includes any of the following:

L1 norm, L2 norm, radial basis function, etc.

Preferably, the visual task accuracy index of the model verification method includes any of the following:

IoU (intersection ratio), AP (average precision), recall and other indicators

The present invention also provides a radar point cloud objective quality evaluation system oriented to machine vision tasks, including: a noise adding module, a spherical domain segmentation module, a vector representation module for calculating spherical domain point sets, a vector difference calculation module, and a calculation quality evaluation module. Fractions module.

Input point cloud data module: input original point cloud data;

Noise-adding module: Add noise to the original point cloud data to obtain the noise-added point cloud data;

Spherical domain segmentation module: separate the original point cloud and the noised point cloud into the spherical domain;

Calculate the vector representation module of the spherical domain point set direction: calculate the eigenvalue and the eigenvector of the spherical domain point set, and calculate the vector representation according to the described eigenvalue and the eigenvector;

Calculate the vector difference module: calculate the vector difference between the original point cloud and the noised point cloud in the same spherical domain;

Calculate the quality evaluation score module: use the attributes of each ball neighborhood as the weight to calculate the quality evaluation score.

Preferably, it also includes:

Model score verification module: Bring the first point cloud and the second point cloud into the point cloud segmentation task based on deep learning, and calculate the size of the corresponding accuracy index IoU.

Based on the above statement, a specific application example is given below:

Taking the LiDAR target detection task in autonomous driving as an example, the point cloud data collected by LiDAR is very large. Massive point cloud data is not conducive to the next transmission and storage of the computer, and also sets up obstacles for the subsequent work. If the result of the target detection task can be estimated by using the model calculation before detection, and then the data with higher quality of model evaluation can be preferentially selected to increase the accuracy of the task.

As shown in Figure 2, an application scenario of the model evaluation algorithm is introduced, which is briefly described as follows: In unmanned driving, if the vehicle-mounted LiDAR obtains a set of radar data for a specific task, we can use our model algorithm (as shown in Figure 1). ) can obtain a score estimate for this data, from which it can be determined whether the data is feasible for its vision task. Therefore, the vehicle vision task can decide whether to use this data, which greatly improves frame utilization and saves computing resources.

specific,

1. Calculate the spherical neighborhood using the 3D reference point cloud a,

2. Based on the same spherical neighborhood, calculate the directional feature of the point set at the same time as the noised point cloud b, and obtain the directional feature.

3. Use a certain vector difference calculation method to obtain the point set direction feature difference. The commonly used methods here are L1 norm, L2 norm, etc.

4. Use the sphere neighborhood weighting to get the score F. The bigger the error. The higher the score.

5. According to the size of F, the influence of the point cloud b on the visual task can be estimated. If F>Q (threshold), it is considered that the data cannot complete the normal visual task, and the data is discarded; if F<Q, use This data performs specific vision tasks.

As shown in FIG. 3 , this embodiment also provides a device for evaluating the objective quality of radar point clouds for machine vision tasks, including: a noise adding module, a spherical segmentation module, a vector representation module for calculating spherical point sets, a calculation The vector difference module, the module for calculating the quality evaluation score.

Input point cloud data module: input original point cloud data;

Noise-adding module: Add noise to the original point cloud data to obtain the noise-added point cloud data;

Spherical domain segmentation module: separate the original point cloud and the noised point cloud into the spherical domain;

Calculate the vector representation module of the spherical domain point set direction: calculate the eigenvalue and the eigenvector of the spherical domain point set, and calculate the vector representation according to the described eigenvalue and the eigenvector;

Calculate the vector difference module: calculate the vector difference between the original point cloud and the noised point cloud in the same spherical domain;

Calculate the quality evaluation score module: use the attributes of each ball neighborhood as the weight to calculate the quality evaluation score.

To sum up, the objective quality evaluation method and system of radar point cloud for machine vision tasks of the present invention can be aimed at specific vision tasks (such as point cloud classification, segmentation, recognition, etc.) The validity of this input data for this task in turn improves resource utilization and the accuracy of the vision task.

Each functional module in the visual task-based radar point cloud data objective quality evaluation device provided in this implementation corresponds to the visual task-based radar point cloud data objective quality evaluation method in the above-mentioned embodiment. The structure and technical elements can be formed by corresponding conversion of the generation method, and the description is omitted here and will not be repeated here.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can use the methods and technical contents disclosed above to improve the present invention without departing from the spirit and scope of the present invention. The technical solutions are subject to possible changes and modifications. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solutions of the present invention belong to the technical solutions of the present invention. protected range.

Those skilled in the art know that, in addition to implementing the system provided by the present invention and its various devices, modules and units in the form of purely computer-readable program codes, the system provided by the present invention and its various devices can be implemented by logically programming the method steps. , modules, and units realize the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system provided by the present invention and its various devices, modules and units can be regarded as a kind of hardware components, and the devices, modules and units included in it for realizing various functions can also be regarded as hardware components. The device, module and unit for realizing various functions can also be regarded as both a software module for realizing a method and a structure within a hardware component.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essential content of the present invention. The embodiments of the present application and features in the embodiments may be arbitrarily combined with each other without conflict.

Claims (14)

1. A machine vision task-oriented sparse point cloud objective quality evaluation method is characterized by comprising the following steps:

inputting original point cloud data and noise point cloud data obtained by noise processing of the original point cloud;

respectively dividing the original point cloud and the noise point cloud into spherical areas, wherein the radius of each divided spherical area is a multiplier of the size of the target dimension of the visual task;

calculating a characteristic value and a characteristic vector of the ball domain point set;

calculating a direction vector of the point set in the spherical domain according to the characteristic value and the characteristic vector;

calculating a vector difference value of the original point cloud and the noise-added point cloud in the same spherical domain;

taking the attribute of each ball neighborhood as a weight, carrying out weighted summation on the vector difference values calculated by each ball neighborhood to obtain a quality evaluation score,

the method for calculating the direction vector of the point set in the spherical domain is a combined calculation method of multiplying the maximum characteristic value by the corresponding characteristic vector or accumulating the multiplied characteristic values by the characteristic vectors;

and the neighborhood attribute of each ball is the number of neighborhood points of each ball.

2. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1,

the sparse point cloud is a radar point cloud.

3. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1 or claim 2, further comprising:

and bringing the original point cloud and the noise point cloud into a visual task, calculating the corresponding accuracy index, and if the quality evaluation score and the accuracy index have a linear relationship, estimating the service quality corresponding to the data by using the sparse point cloud objective quality evaluation method, otherwise, not estimating the service quality corresponding to the data.

4. The machine vision task-oriented objective quality evaluation method for sparse point clouds according to claim 1,

the multiplier that the radius of the ball separating area is the size of the visual task target is that the radius of the ball separating area is not more than 2 times of the length of the longest edge of the visual task target.

5. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1 or claim 2,

the method for calculating the vector difference value of the original point cloud and the noisy point cloud in the same spherical domain is L1 norm or L2 norm or radial basis function.

6. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1 or claim 2, wherein the vision task comprises any one of the following:

and (4) carrying out point cloud classification, segmentation, identification and tracking on visual tasks.

7. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1 or claim 2, wherein the noise processing noise category comprises any one of:

gaussian noise, random noise, regional noise.

8. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 3, wherein the accuracy index comprises any one of the following:

cross-over ratio IoU, average accuracy AP, recall index.

9. The machine vision task-oriented sparse point cloud objective quality evaluation method as claimed in claim 1 or claim 2, comprising:

inputting sparse point cloud data;

calculating a quality evaluation score by using a method for calculating the quality evaluation score;

comparing the quality evaluation score with a preset threshold value: if the quality evaluation score is smaller than the threshold value, the data is valid and can be used continuously, otherwise, the data cannot be used and is discarded.

10. The utility model provides a machine vision task oriented sparse point cloud objective quality evaluation device which characterized in that includes:

an input point cloud data module: inputting original point cloud data;

a noise adding module: denoising the original point cloud data to obtain noisy point cloud data;

a spherical region segmentation module: respectively dividing the original point cloud and the noise point cloud into spherical areas, wherein the radius of each divided spherical area is a multiplier of the size of a visual task target;

a module for calculating the direction vector of the ball domain point set: calculating a characteristic value and a characteristic vector of the spherical domain point set, and calculating a direction vector of the spherical domain point set according to the characteristic value and the characteristic vector;

a calculate vector difference module: calculating the vector difference value of the original point cloud and the noise point cloud in the same spherical domain;

a quality evaluation score calculating module: calculating a quality evaluation score by taking the neighborhood attribute of each ball as a weight;

the method for calculating the direction vector of the sphere domain point set is a combined calculation method of multiplying the maximum characteristic value by the corresponding characteristic vector or accumulating the multiplied characteristic values and the characteristic vectors;

and the neighborhood attribute of each ball is the number of neighborhood points of each ball.

11. The machine vision task-oriented sparse point cloud objective quality assessment device as claimed in claim 10,

the sparse point cloud is a radar point cloud.

12. The device for objective quality evaluation of sparse point cloud for machine vision task according to claim 10 or claim 11, further comprising:

a data validity judging module: and bringing the original point cloud and the noise point cloud into a visual task, calculating the corresponding accuracy index size, and if the quality evaluation score and the accuracy index size have a linear relation, the sparse point cloud objective quality evaluation device can be used for estimating the service quality corresponding to the data, otherwise, the sparse point cloud objective quality evaluation device cannot be used for estimating the service quality corresponding to the data.

13. The machine vision task-oriented sparse point cloud objective quality evaluation device as claimed in claim 10 or 11,

the multiplier that the radius of the ball separating area is the size of the visual task target is that the radius of the ball separating area is not more than 2 times of the length of the longest edge of the visual task target.

14. The machine vision task-oriented sparse point cloud objective quality assessment device as claimed in claim 10 or claim 11,

the method for calculating the vector difference value of the original point cloud and the noise point cloud in the same spherical domain is L1 norm or L2 norm.

Images