WO2023155793A1

WO2023155793A1 - Point cloud processing method and apparatus, and laser radar

Info

Publication number: WO2023155793A1
Application number: PCT/CN2023/076111
Authority: WO
Inventors: 陈森柯; 王栋; 崔鹏飞; 夏冰冰; 石拓
Original assignee: 苏州一径科技有限公司
Priority date: 2022-02-16
Filing date: 2023-02-15
Publication date: 2023-08-24
Also published as: CN114612598A

Abstract

The present disclosure discloses a point cloud processing method. The method comprises: obtaining a first point cloud of a laser radar; traversing the first point cloud, and by means of looking up a point cloud neighbor table, calculating a first Euclidean distance from each point in the first point cloud to a nearest neighbor point; and determining outliers from within the first point cloud on the basis of at least the first Euclidean distances.

Description

A point cloud processing method, device and laser radar

Cross References to Related Applications

This disclosure claims priority to Chinese Patent Application No. 202210140658.7 filed in China on Feb. 16, 2022, the entire contents of which are hereby incorporated by reference.

technical field

The present disclosure relates to the technical field of laser radar, and in particular to a point cloud processing method, device and laser radar.

Background technique

The data form of LiDAR (light detection and ranging, LiDAR) is point cloud. Point cloud data contains a variety of feature information, such as: three-dimensional coordinate information, color information, reflection intensity information, etc.

At present, with the wide application of lidar, higher and higher requirements are put forward for the measurement accuracy of lidar, and higher requirements are also put forward for the reliability of point cloud data. In order to ensure the accuracy and reliability of the point cloud data, it is necessary to remove the noise points in the point cloud, such as outliers. However, the algorithms associated with identifying outliers are computationally intensive and inefficient.

Contents of the invention

The present disclosure provides a point cloud processing method, device and laser radar, so as to quickly identify outliers from the point cloud.

According to the first aspect of an embodiment of the present disclosure, a point cloud processing method is provided, including: obtaining the first point cloud of the lidar, the point cloud generated by the lidar is an unordered point cloud; traversing the first point cloud, by querying The point cloud neighbor table calculates the first Euclidean distance from each point in the first point cloud to the nearest neighbor point, wherein the point cloud neighbor table is generated based on the preset angle calibration data of the lidar; at least based on the first Euclidean distance, Identify outliers from the first point cloud.

In some exemplary embodiments according to the first aspect, at least based on the first Euclidean distance, determining outliers from the first point cloud includes: for the i-th point in the first point cloud, from the i-th Determine the number of the first point whose first Euclidean distance is less than the first threshold in the nearest neighbor of the point, and i is a positive integer; if the number of the first point is less than the second threshold, the i-th point is determined as an outlier.

In some exemplary embodiments according to the first aspect, after obtaining the input first point cloud, the method further includes: traversing the first point cloud, and calculating each point in the first point cloud by querying the point cloud neighbor table a second Euclidean distance to the second nearest neighbor; determining outliers from the first point cloud based on at least the first Euclidean distance, including: determining outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance point.

In some exemplary embodiments according to the first aspect, determining outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance includes: determining outliers from the first point cloud based on the first Euclidean distance Determine the suspected outliers; determine the outliers from the suspected outliers based on the second Euclidean distance.

In some exemplary embodiments according to the first aspect, determining suspected outliers from the first point cloud based on the first Euclidean distance includes: for the i-th point in the first point cloud, from the i-th Determine the number of the second point whose first Euclidean distance is less than the third threshold in the nearest neighbor of the point, i is a positive integer; if the number of the second point is less than the fourth threshold, then determine the i-th point as a suspected outlier .

In some exemplary embodiments according to the first aspect, determining the outliers from the suspected outliers based on the second Euclidean distance includes: for the jth suspected outlier in the first point cloud, from the Determine the number of third points whose second Euclidean distance is less than the fifth threshold among the second nearest neighbor points of j suspected outliers, and j is a positive integer; if the number of the third points is less than or equal to the sixth threshold, then the jth Suspected outliers are identified as outliers.

In some exemplary embodiments according to the first aspect, the method further includes: generating a second point cloud at a preset detection distance based on the angle calibration data of the lidar; the second point cloud is the same as the first point cloud Corresponding; traversing the second point cloud, searching for the nearest neighbor of each point; at least based on the nearest neighbor, generating a point cloud neighbor table.

In some exemplary embodiments according to the first aspect, after generating the second point cloud, the method It also includes: traversing the second point cloud, searching for the second nearest neighbor point of each point; generating a point cloud neighbor table based on the nearest neighbor point and the second nearest neighbor point.

According to the second aspect of the embodiments of the present disclosure, there is provided a point cloud processing device, the processing device includes: a data acquisition module, used to obtain the first point cloud of the laser radar, the point cloud generated by the laser radar is disordered Point cloud; the distance calculation module is used to traverse the first point cloud, and calculate the first Euclidean distance from each point in the first point cloud to the nearest neighbor by querying the point cloud neighbor table, wherein the point cloud neighbor table is based on the laser The radar preset angle calibration data is generated; the data analysis module is configured to determine outliers from the first point cloud based at least on the first Euclidean distance.

In some exemplary embodiments according to the second aspect, the data analysis module is further configured to, for the i-th point in the first point cloud, determine from the nearest neighbors of the i-th point that the first Euclidean distance is less than the first The quantity of the first point of a threshold, i is a positive integer; if the quantity of the first point is less than the second threshold, the i-th point is determined as an outlier.

In some exemplary embodiments according to the second aspect, the distance calculation module is also used to traverse the first point cloud, and calculate the second distance between each point in the first point cloud and the next nearest neighbor point by querying the point cloud neighbor table. Euclidean distance; the data analysis module is also used to determine outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance.

In some exemplary embodiments according to the second aspect, the data analysis module is further configured to determine suspected outliers from the first point cloud based on the first Euclidean distance; and, based on the second Euclidean distance, from Identify outliers among suspected outliers.

In some exemplary embodiments according to the second aspect, the data analysis module is further configured to, for the i-th point in the first point cloud, determine from the nearest neighbors of the i-th point that the first Euclidean distance is less than the first The number of the second points of the three thresholds, i is a positive integer; if the number of the second points is less than the fourth threshold, the i-th point is determined as a suspected outlier.

In some exemplary embodiments according to the second aspect, the data analysis module is further configured to, for the jth suspected outlier point in the first point cloud, determine from the second nearest neighbors of the jth suspected outlier point The number of third points whose second Euclidean distance is less than the fifth threshold, j is a positive integer; if the number of third points is less than or is equal to the sixth threshold, then the jth suspected outlier point is determined as an outlier point.

According to a third aspect of the embodiments of the present disclosure, there is provided a laser radar, including: a memory storing computer-executable instructions; a processor connected to the memory for executing the computer-executable instructions to implement the first aspect and the method described in any exemplary embodiment thereof.

According to a fourth aspect of the embodiments of the present disclosure, a computer storage medium is provided, the computer storage medium stores computer-executable instructions, and the computer-executable instructions can implement the first aspect and any exemplary method thereof after being executed by a processor. The method described in the embodiment.

Compared with the related art, the beneficial effect of the technical solution provided by the present disclosure is:

In the present disclosure, the nearest neighbor point of each point in the point cloud is determined by querying the point cloud neighbor table, and the outlier point in the point cloud is determined according to the Euclidean distance from each point to the nearest neighbor point. When analyzing the unknown point cloud, it is only necessary to look up the table to obtain the neighbor points, instead of searching for the neighbor points every time, the data calculation amount is small, convenient and efficient, thus effectively improving the processing efficiency of the point cloud.

Further, the present disclosure uses laser radar angle calibration data to generate a simulation point cloud, and generates a point cloud neighbor table according to the positional relationship of points in the simulation point cloud. Because the simulated point cloud and the unknown point cloud have the same launch angle, the arrangement of points in the simulated point cloud and the arrangement of points in the unknown point cloud are in one-to-one correspondence. The unknown point cloud uses the neighbor table generated by the simulated point cloud to determine the nearest neighbor point of any point in the unknown point cloud, and the nearest neighbor point determined by this method is more accurate.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit the scope of the present disclosure.

Description of drawings

FIG. 1 is a schematic structural diagram of a laser radar in the related art;

FIG. 2 is a flowchart of a point cloud processing method in an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a point cloud arrangement in an embodiment of the present disclosure;

FIG. 4 is a flow chart of point cloud processing in an embodiment of the present disclosure;

FIG. 5 is another flow chart of point cloud processing in an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a point cloud processing device in an embodiment of the present disclosure;

FIG. 7 is a configuration block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and techniques are presented for a thorough understanding of the embodiments of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure with unnecessary detail.

In order to illustrate the technical solutions described in the present disclosure, specific examples are used below to illustrate.

LiDAR is an object detection technology. The laser radar emits a laser beam through a laser. The laser beam encounters a target object and undergoes diffuse reflection. The detector receives the reflected beam and determines the distance, orientation, height, speed, Features such as posture and shape.

LiDAR has a wide range of applications. In addition to being used in the military field, it is also widely used in the field of life, including but not limited to: intelligent driving vehicles, intelligent driving aircraft, three-dimensional (3-Dimension, 3D) printing, virtual reality, augmented reality, service robots and other fields . Taking the intelligent driving technology as an example, the lidar is set in the intelligent driving vehicle, and the lidar can scan the surrounding environment by emitting laser beams quickly and repeatedly, so as to obtain the shape and position of one or more target objects in the surrounding environment , moving point clouds, etc.

It should be noted that the above intelligent driving technology may refer to unmanned driving, automatic driving, assisted driving and other technologies.

FIG. 1 is a schematic structural diagram of a laser radar in the related art. Referring to FIG. 1 , the laser radar 10 may include: a light emitting device 101 , a light receiving device 102 and a processor 103 . Wherein, both the light emitting device 101 and the light receiving device 102 are connected to the processor 103 .

Wherein, the connection relationship among the above-mentioned components may be an electrical connection, and may also be an optical fiber connection. More specifically, the light emitting device 101 and the light receiving device 102 may also respectively include a plurality of optical devices, and the connection relationship between these optical devices may also be a spatial optical transmission connection.

The processor 103 is used to control the light emitting device 101 and the light receiving device 102, so that the light emitting device 101 and the light receiving device 102 can work normally. Exemplarily, the processor 103 may provide driving voltages for the light emitting device 101 and the light receiving device 102 respectively, and the processor 103 may also provide control signals for the light emitting device 101 and the light receiving device 102 .

Exemplary, the processor 103 can be a general-purpose processor, such as a central processing unit (central processing unit, CPU), a network processor (network processor, NP), etc.; the processor 103 can also be a digital signal processor (digital signal processing , DSP), application specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

The light emitting device 101 also includes a light source (not shown in FIG. 1 ). It can be understood that the above-mentioned light source may refer to a laser, and the number of the laser may be one or more. Optionally, the laser may specifically be a pulsed laser diode (pulsed laser diode, PLD), a semiconductor laser, a fiber laser, or the like. The above-mentioned light sources are used to emit laser beams. Specifically, the processor 103 may send an emission control signal to the light source, thereby triggering the light source to emit the laser beam.

It can be understood that the above-mentioned laser beams may also be referred to as laser pulses, lasers, emitted light beams, and the like.

In the following, the detection process of the target object 104 by the laser radar will be briefly described in combination with the structure of the laser radar shown in FIG. 1 .

As shown in FIG. 1 , the laser beam propagates along the emission direction. When the laser beam encounters the target object 104 , it is reflected on the surface of the target object 104 , and the reflected beam is received by the light receiving device 102 of the laser radar. Here, the beam reflected by the target object 104 may be referred to as an echo beam (the laser beam and the echo beam are marked by solid lines in FIG. 1 ).

After receiving the echo beam, the light receiving device 102 performs photoelectric conversion on the echo beam, that is, converts the echo beam into an electrical signal, and the light receiving device 102 outputs the electrical signal corresponding to the echo beam to the processor 103, and the processor 103 can obtain the shape, position, and moving point cloud of the target object 104 according to the electrical signal of the echo beam.

In practical applications, the laser of the micro-electro-mechanical system (MEMS) scanning laser radar in the laser radar is fixedly connected, and the light can only propagate along the corresponding angle, so that the laser in the MEMS scanning laser radar Individual lasers often only have a limited field of view. In order to achieve the application requirements of large field of view or even full coverage of the laser radar, multiple lasers with different angles can be configured in the MEMS scanning laser radar. The field of view becomes a large field of view. At the same time, in order to prevent blind spots between different small fields of view and affect the detection accuracy, there is often a certain overlapping area between small fields of view. The existence of the overlapping area makes the point cloud with orderly distribution of point numbers in the small field of view (that is, ordered point cloud) become a point cloud with disordered point number distribution in the entire lidar field of view (that is, unordered point cloud).

In some special cases, random noise occurs in lidar, for example, avalanche photodiode (avalanche photon diode, APD) array part of the receiver is damaged, APD is exposed to direct sunlight, etc. The appearance of random noise will cause defects in the point cloud acquired by lidar scanning (usually manifested as outliers in the point cloud), which will affect the detection accuracy of lidar.

Because of the existence of unordered point cloud, the points in the point cloud have the characteristics that the sequence numbers of adjacent points are not adjacent. It is not possible to remove outliers from the point cloud based on the serial number of the point. Usually, the statistical outlier removal (SOR) in the point cloud library (PCL) algorithm is used to remove the outlier points in the point cloud. The SOR algorithm has the advantage of strong versatility, but the SOR algorithm needs to search for the nearest neighbor of each point in three-dimensional space after each input point cloud, which takes a very long time and is unstable.

In order to solve the above problems, an embodiment of the present disclosure provides a point cloud processing method. This method can be applied to lidars that generate disordered point clouds, such as MEMS scanning lidars.

It should be understood that the lidar referred to in the embodiments of the present disclosure may be all lidars that can obtain factory angle calibration files or lidars that can know the emission angle of the transmitter in the lidar. That is, all lidars that can obtain point cloud neighbor tables based on preset files and data. For the convenience of description, the following are collectively referred to as lidar.

FIG. 2 is a flow chart of a point cloud processing method in an embodiment of the present disclosure. Referring to the solid line in Fig. 2, the method includes steps S201 to S203.

S201. Acquire the first point cloud of the lidar.

Here, the point cloud generated by lidar is an unordered point cloud. Correspondingly, the first point cloud is an unordered point cloud.

It should be understood that the lidar can scan the target object to obtain a point cloud for describing the relevant characteristics of the target object, and use the point cloud as the first point cloud. Alternatively, lidar can be obtained by obtaining a historical point cloud that has been scanned and stored in the past, and use this point cloud as the first point cloud. Of course, the lidar can also obtain the first point cloud in other ways, which is not specifically limited in this embodiment of the present disclosure.

Exemplarily, the first point cloud may include information such as serial numbers of points, three-dimensional coordinates of points, and the like.

S202. Traverse the first point cloud, and calculate the first Euclidean distance from each point in the first point cloud to the nearest neighbor point by querying the point cloud neighbor table.

Wherein, the point cloud neighbor table may be generated based on the preset angle calibration data of the lidar. The point cloud neighbor table may also include the nearest neighbor points of each point in the lidar point cloud. Here, the nearest neighbor point can be understood as the point with the closest distance to each point in the point cloud in the three-dimensional space. In practical application, the nearest neighbor of a point can be one point or multiple points, then, the number of the nearest neighbor of a point can be set as K1, and K1 is a positive integer.

It should be noted that the number of nearest neighbors can be set according to the resources of the algorithm computing platform and the real-time requirements of the application scenario. Exemplarily, if computing resources are sufficient or computing real-time requirements are low, the number K1 of nearest neighbors can be set to 8; if computing resources are limited, or computing real-time requirements are high, the number K1 of nearest neighbors can be Set to 4. Of course, other situations may also exist, which are not specifically limited in this embodiment of the present disclosure.

It should be understood that the aforementioned point cloud neighbor table is pre-stored in the lidar. Then, after the lidar obtains the first point cloud through S101, it can traverse each point in the first point cloud (hereinafter referred to as point A As an example), by querying the point cloud neighbor table, K1 nearest neighbor points of point A are obtained, and the first Euclidean distance from point A to K1 nearest neighbor points is calculated. That is to say, the lidar determines the K1 nearest neighbor points of each point in the first point cloud by querying the point cloud neighbor table, and calculates the Euclidean distance between each point and its K1 nearest neighbor points. At this time, a total of K1 points are calculated First Euclidean distance.

In the embodiment of the present disclosure, the point cloud for generating the point cloud proximity table and the first point cloud are generated by a laser beam emitted by a laser on the same lidar. Since the emission angle of the laser is fixed and the emission beam propagates along a straight line, the arrangement of points in the lidar point cloud is relatively fixed. Then, the nearest neighbor point of any point in the first point cloud can be obtained by querying the point cloud neighbor table.

S203. Determine outliers from the first point cloud based on the first Euclidean distance.

It should be understood that after the lidar determines the K1 nearest neighbors of point A and the corresponding first Euclidean distances, it can determine whether point A is a distance from group points.

Specifically, S203 may include: for the i-th (i=1, 2, 3...) point in the first point cloud, determine from the nearest neighbors of the i-th point that the first Euclidean distance is less than the first threshold Number of first points. If the number of the first points is less than the second threshold, the ith point is determined as an outlier point.

It should be understood that the lidar obtains the first Euclidean distance from the i-th point to the nearest neighbor point through S202, and there are K1 in total. Then, the lidar executes S203 to find out the nearest neighbor point (that is, the first point) whose first Euclidean distance is less than the first threshold from the K1 nearest neighbor points, and determines whether the i-th point is a distance from the first point according to the number of first points. group points. If the number of the first point is less than the second threshold, the lidar determines that the i-th point is an outlier point, otherwise, the lidar determines that the i-th point is a non-outlier point, and repeats the above steps for the i+1th point .

In practical applications, the above-mentioned first point is a set of nearest neighbor points whose first Euclidean distance is smaller than the first threshold, and this set is a subset of K1 nearest neighbor points. The number of first points may be less than or equal to K1.

It should be noted that the lidar can traverse the first point cloud, perform the above judgment process for each point, and determine whether each point is an outlier point one by one. For example, taking point A as an example, the lidar determines from the K1 nearest neighbor points of point A that satisfy the condition that the first Euclidean distance is less than the first threshold The first point, a total of S1. If the number of the first points (S1) is less than the second threshold, the laser radar may determine point A as an outlier point. Conversely, the lidar determines point A as a non-outlier and repeats the above steps for the next point.

It should be understood that the first threshold and the second threshold are set according to actual conditions such as the application scenario and accuracy requirements of the lidar. The first threshold and the second threshold may be empirical values.

Exemplarily, it is assumed that the first threshold is 5 cm, and the second threshold is 2. If the lidar determines that the first Euclidean distance is less than 5 cm (that is, the first threshold) among the 4 nearest neighbor points (K1=4) of point A through S203, there are 3 first points (can be recorded as point B, point C and point D), and because 3 is greater than 2 (ie the second threshold), it can be determined that point A is not an outlier. If the lidar determines that the first point whose first Euclidean distance is less than 5 cm (i.e. the first threshold) is 1 (can be recorded as point E) among the 4 nearest neighbor points (K1=4) of point A through S203, and Since 1 is smaller than 2 (ie, the second threshold), it can be determined that point A is an outlier.

In practical applications, S202 to S203 are executed for each point in the first point cloud to determine whether each point in the first point cloud is an outlier point.

So far, the outliers are determined from the first point cloud according to the nearest neighbor points, which is not only simple and convenient, but also has a small amount of calculation, and can effectively improve the screening efficiency of outliers.

In some exemplary embodiments, the above-mentioned point cloud neighbor table may further include the second nearest neighbor points of each point in the point cloud. Here, the second nearest neighbor point can be understood as the second closest point to each point in the point cloud in three-dimensional space. In practical applications, the second nearest neighbor of a point can be one point or multiple points, then, the number of nearest neighbors of a point can be set as K2, and K2 is a positive integer.

It should be noted that the number of next-nearest neighbor points can be set according to the resources of the algorithm computing platform and the real-time requirements of the application scenario. Exemplarily, if the calculation resources are sufficient or the calculation real-time requirements are low, the number K2 of the second nearest neighbor points can be set to 16; if the calculation resources are limited, or the calculation real-time requirements are high, the number K2 of the second nearest neighbor points can be Set to 8. Of course, other situations may also exist, which are not specifically limited in this embodiment of the present disclosure.

Then, referring to the dotted line in FIG. 2 , the above method may further include S204 and S205. in, S204 is executed after S201, and may be executed simultaneously with S202, or may be executed successively with S202.

S204, traversing the first point cloud, and calculating the second Euclidean distance from each point in the first point cloud to the next nearest neighbor point by querying the point cloud neighbor table;

S205. Determine outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance.

It should be understood that after S201, the lidar can also traverse each point in the first point cloud, and obtain each point in the first point cloud by querying the point cloud neighbor table (still take point A as an example for description) K2 second-nearest neighbors, and calculate the second Euclidean distance from point A to its K2 second-nearest neighbors, a total of K2 second Euclidean distances. Then, the lidar executes S205 to determine whether point A is an outlier based on the first Euclidean distance from point A to the nearest neighbor point and the second Euclidean distance from point A to the second nearest neighbor point, and then determine from the first point cloud Outlier.

Specifically, S205 may include: determining suspected outliers from the first point cloud based on the first Euclidean distance; determining outliers from the suspected outliers based on the second Euclidean distance.

It should be understood that through S202 and S204, after the lidar obtains the first Euclidean distance and the second Euclidean distance of each point in the first point cloud, it can Quantity, to determine whether point A is a suspected outlier. Then, if the point A is a suspected outlier point, it is determined whether the point A is an outlier point according to the quantity of the second nearest neighbor points satisfying the second Euclidean distance of the second preset condition. Through two judgments, the non-outlier points are avoided from being mistakenly deleted, making the identification of outlier points more accurate.

Specifically, S205 may include: for the i-th (i=1, 2, 3...) point in the first point cloud, determining from the nearest neighbors of the i-th point that the first Euclidean distance is less than the third threshold The number of second points. If the number of the second points is less than the fourth threshold, the i-th point is determined as a suspected outlier point.

It should be understood that the lidar obtains the first Euclidean distance from the i-th point to the nearest neighbor point through S202, and there are K1 in total. Then, the lidar executes S205 to find out the nearest neighbor point (that is, the second point) whose first Euclidean distance is less than the third threshold from the K1 nearest neighbor points, and determine whether the i-th point is suspected according to the number of second points Outlier. If the number of the second point is less than the fourth threshold, the lidar determines that the i-th point is a suspected outlier point, otherwise, the lidar determines that the i-th point is a non-suspected outlier point, and for the i-th point Repeat the above steps for i+1 points.

In practical applications, the above-mentioned second point is a set of nearest neighbor points whose first Euclidean distance is smaller than the third threshold, and this set is a subset of K1 nearest neighbor points. The number of second points may be less than or equal to K1.

It should be noted that the lidar can traverse the first point cloud, perform the above judgment process for each point, and determine whether each point is a suspected outlier point one by one. For example, taking point A as an example, the lidar determines the second point satisfying the condition that the first Euclidean distance is less than the third threshold from the K1 nearest neighbor points of point A, a total of S2 points. If the number of the second points (S2) is less than the fourth threshold, the lidar may determine point A as a suspected outlier point. Conversely, LiDAR identifies point A as not a suspected outlier and repeats the above steps for the next point.

It should be understood that the third threshold and the fourth threshold are set according to actual conditions such as the application scenario and accuracy requirements of the lidar. The third threshold and the fourth threshold may be empirical values. The third threshold may be the same as or different from the first threshold, and the second threshold may be the same as or different from the fourth threshold, which is not specifically limited in this embodiment of the present disclosure. The second point and the first point may be the same or different, which is not specifically limited in this embodiment of the present disclosure.

Exemplarily, it is assumed that the third threshold is 5 cm, and the fourth threshold is 2. If the lidar determines that the first Euclidean distance is less than 5 cm (that is, the first threshold) among the 4 nearest neighbor points (K1=4) of point A through S205, there are 3 first points (can be recorded as point B, point C and point D), and because 3 is greater than 2 (ie the second threshold), it can be determined that point A is not a suspected outlier. If the lidar determines that the first point whose first Euclidean distance is less than 5 cm (i.e. the first threshold) is 1 (can be recorded as point E) among the 4 nearest neighbor points (K1=4) of point A through S203, and Because 1 is smaller than 2 (ie the second threshold), it can be determined that point A is a suspected outlier point.

Further, S205 may also include: for the jth (=1, 2, 3...) suspected outlier point in the first point cloud, determining the second Euclidean distance from the second nearest neighbor of the jth suspected outlier point The number of third points less than the fifth threshold. If the number of the third points is less than or equal to the sixth threshold, the jth suspected outlier point is determined as an outlier point.

It should be understood that the lidar obtains the second distance between the jth suspected outlier point and the second nearest neighbor point through S204. Euclidean distance, a total of K2. Then, the lidar executes S205 to find out the second nearest neighbor point (that is, the third point) whose second Euclidean distance is less than the fifth threshold from the K2 second nearest neighbor points, and determine the jth suspected outlier point according to the number of the third point Whether it is an outlier point. If the number of the third point is less than or equal to the sixth threshold, the lidar determines that the jth suspected outlier point is an outlier point, otherwise, the lidar determines that the jth suspected outlier point is a non-outlier point, and for the first Repeat the above steps for j+1 points.

In practical applications, the above-mentioned third point is a set of next-nearest neighbor points whose second Euclidean distance is smaller than the fifth threshold, and this set is a subset of K2 second-nearest neighbor points. The number of third points may be less than or equal to K2.

It should be noted that the lidar can traverse the first point cloud, perform the above judgment process for each suspected outlier point, and determine whether each suspected outlier point is an outlier point one by one. For example, taking point A as an example, the lidar determines the third point satisfying the condition that the second Euclidean distance is less than the fifth threshold from the K2 second nearest neighbor points of point A, totaling S3 points. If the number of the third points (S3) is less than or equal to the sixth threshold, the laser radar may determine point A as an outlier point. Conversely, the lidar determines point A as a non-outlier and repeats the above steps for the next point.

It should be understood that the fifth threshold and the sixth threshold are set according to actual conditions such as the application scenario and accuracy requirements of the lidar. The fifth threshold and the sixth threshold may be empirical values. The fifth threshold is greater than the third threshold, and the sixth threshold is greater than the fourth threshold.

Exemplarily, it is assumed that point A is a suspected outlier. The fifth threshold is 8 cm, and the sixth threshold is 2. If the lidar determines that the second Euclidean distance is less than 8 cm (that is, the third threshold) among the 8 second-nearest neighbor points (K2=8) of point A through S205, there are 4 third points (can be recorded as point F, point G , point H and point I), and because 4 is greater than 2 (ie the fourth threshold), it can be determined that point A is not an outlier. If the lidar determines that there are 2 third points whose second Euclidean distance is less than 8 cm among the 8 second-nearest neighbor points (K2=8) of point A through S203 (can be recorded as point H and point J), and because 1 is less than 2, then it can be determined that point A is an outlier.

In practical applications, S202 to S205 are executed for each point in the first point cloud to determine whether each point in the first point cloud is an outlier point.

So far, the outlier points are determined from the first point cloud according to the nearest neighbor point and the second nearest neighbor point, not only judging The accuracy of breaking outliers is higher, and it can effectively reduce the occurrence of mistakenly deleted points.

In the embodiment of the present disclosure, it can be seen from the above steps S201-S205 that by querying the point cloud neighbor table, the nearest neighbor point of each point in the point cloud is determined, and the point cloud is determined according to the Euclidean distance from each point to the nearest neighbor point Outliers in . When processing an unknown point cloud, it is only necessary to look up the table to obtain the neighbor points, instead of searching for the neighbor points every time, the data calculation amount is small, convenient and efficient, thus effectively improving the processing efficiency of the point cloud.

In some exemplary embodiments, the above method may further include the step of generating a point cloud neighbor list. Then, still referring to the solid line in Figure 2, before S201, the above method may also include:

S206, generating a second point cloud at a preset detection distance based on the angle calibration data of the lidar;

S207, traverse the second point cloud, and search for the nearest neighbor point of each point;

S208. Generate a point cloud neighbor table at least based on the nearest neighbor point.

In practical applications, the above steps of generating point cloud neighbor tables can be performed when the lidar leaves the factory, or it can be executed every time the lidar is powered on and initialized. In other words, lidar does not need to be performed every time before ranging.

In practical applications, the angle calibration data of the lidar is stored before the lidar leaves the factory. According to the angle calibration data, the laser radar can obtain the emission angle of each laser in the laser radar. The preset detection distance (which can be recorded as distance) can be understood as the preset scanning range of the lidar, which is set according to the actual situation. distance is a positive integer.

It should be understood that in S206, the lidar can generate a frame of simulated point cloud according to the emission angle of each laser and the preset detection distance (that is, the point cloud or second point cloud). Since the emission angle of the lidar laser is fixed and the laser travels along a straight line, the overlapping area of the simulated point cloud is fixed and has nothing to do with the emission distance of the laser. The arrangement of each point and its neighbors in the simulation point cloud is also relatively fixed. Then, the arrangement of points in any unknown point cloud generated by the laser of the lidar is consistent with the simulation point cloud, that is, the arrangement of points in the first point cloud is the same as the arrangement of points in the second point cloud is corresponding. Here, the simulation point cloud can be understood as At the same detection distance, the two-dimensional angle calibration data is restored to the three-dimensional angle space, so that the found neighbor points are more in line with the actual situation.

Exemplarily, FIG. 3 is a schematic diagram of a point cloud arrangement in an embodiment of the present disclosure. Assume that the preset detection distance (distance) is 2 meters, and the lidar generates a simulation point cloud as shown in FIG. 3 according to the angle calibration data and distance=2. In the simulation point cloud, lasers with different angles form different fields of view, and different fields of view form overlapping areas. The enlarged schematic part in Figure 3 is used to represent the distribution of point clouds in the overlapping area. Among them, the circles represent points, and the numbers in the circles represent the serial numbers of the points. It can be seen from Figure 3 that point 49178 is adjacent to point 3294, point 3302, point 3314, point 49394, point 49402, point 2014, point 48761 and point 48537, that is, the positions of the points are adjacent but the serial numbers of the point clouds are not the same adjacent. Among them, point 3302, point 49394, point 48761 and point 49402 are the nearest neighbors of point 49178, and point 3294, point 3314, point 2014 and point 8537 are the second nearest neighbors of point 49178. Further, the lidar records the nearest neighbor point and the second nearest neighbor point of each point in the simulation point cloud to generate a point cloud neighbor table.

It should be understood that, in S207 to S208, the lidar traverses the second point cloud, and searches for the nearest neighbor point of each point in the second point cloud. Specifically, the lidar can search the K1 nearest neighbors of each point through a search algorithm (such as the KD tree (K-dimensional Tree) algorithm), and record these nearest neighbors to generate a fixed point cloud neighbor table. Of course, the lidar may also use other search algorithms to search for the nearest neighbor of each point, which is not specifically limited in this embodiment of the present disclosure.

In some exemplary embodiments, still referring to the dotted line in FIG. 2 , after S206 , it may further include: S209 , traversing the second point cloud, and searching for the second nearest neighbor point of each point in the second point cloud. Specifically, lidar can search for K2 next-nearest neighbors of each point through the KD tree algorithm. S210. Generate a point cloud neighbor table based on the nearest neighbor point and the second nearest neighbor point, that is, record these nearest neighbor points and the second nearest neighbor point to generate a fixed point cloud neighbor table.

It should be noted that the same search algorithm may be used to search for the nearest neighbor point and the second nearest neighbor point, or different search algorithms may be used to search for the nearest neighbor point and the second nearest neighbor point, which is not specifically limited in this embodiment of the present disclosure.

The method for processing the above point cloud will be described below with specific examples.

FIG. 4 is a flow chart of point cloud processing in an embodiment of the present disclosure. FIG. 5 is another flow chart of point cloud processing in an embodiment of the present disclosure. The point cloud processing method will be described with reference to FIG. 4 and FIG. 5 . In the following embodiments, point A in the first point cloud is taken as an example.

In one embodiment, the processing method of point cloud can comprise the following steps:

S401, input the first point cloud, enter S402;

S402, judging whether the traversal is completed; if yes, go to S407; if not, go to S403;

S403, query the neighbor table, and calculate the first Euclidean distance from point A to K1 nearest neighbor points in the first point cloud; enter S404;

S404, judging whether the number of K1 first Euclidean distances smaller than the first set threshold is greater than 2; if yes, proceed to S405; if not, proceed to S406;

S405, do not process this point, go to S402;

S406, the point is an outlier point, enter into S402;

S407, outputting the point cloud after denoising, that is, the data from which outliers are removed.

It can be seen from steps S401-S407 that the outlier point can be determined based on the Euclidean distance from any point in the first point cloud to the nearest neighbor point by traversing the input first point cloud and querying the point cloud neighbor table.

In another embodiment, the point cloud processing method may include the following steps:

S501, read the angle calibration file, generate a point cloud neighbor table, and enter S502;

S502, input the first point cloud, enter S503;

S503, judging whether the traversal is completed; if yes, go to S510; if not, go to S504;

S504, query the neighbor table, calculate the first Euclidean distance from point A to K1 nearest neighbor points in the first point cloud, and enter S505;

S505, judging whether the number of K1 first Euclidean distances less than the first set threshold is greater than 2; if yes, proceed to S506; if not, proceed to S507;

S506, do not process this point, go to S503;

S507, the point is a suspected outlier point, query the neighbor table, and calculate the suspected outlier point to K2 times The second Euclidean distance of the nearest neighbor point; enter S508;

S508, judging whether the number of K2 second Euclidean distances smaller than the second set threshold is less than 3; if yes, proceed to S509; if not, proceed to S506;

S509, the point is an outlier point, enter into S503;

S510, outputting the point cloud after denoising, that is, the data of removing outliers.

It can be seen from steps S501-S510 that the first point cloud input is traversed, the point cloud neighbor table is generated and the point cloud neighbor table is queried, based on the Euclidean distance from any point in the first point cloud to the nearest neighbor point and the second nearest neighbor point, namely Outliers can be identified.

In the embodiment of the present disclosure, the nearest neighbor point of each point in the point cloud is determined by querying the point cloud neighbor table, and the outlier point in the point cloud is determined according to the Euclidean distance from each point to the nearest neighbor point. When analyzing the unknown point cloud, it is only necessary to look up the table to obtain the neighbor points, instead of searching for the neighbor points every time, the data calculation amount is small, convenient and efficient, thus effectively improving the processing efficiency of the point cloud.

Based on the same inventive concept, an embodiment of the present disclosure provides a point cloud processing device, which can be a chip or a system-on-chip in a laser radar device, and can also be used in a laser radar device to implement the above-mentioned various embodiments. The function module of the method. The device can realize the point cloud processing functions in the above-mentioned embodiments, and these functions can be realized by executing corresponding software through hardware. These hardware or software include one or more modules with corresponding functions mentioned above. FIG. 6 is a schematic structural diagram of a point cloud processing device in an embodiment of the present disclosure. Referring to FIG. 6, the processing device 600 may include: a data acquisition module 601 for acquiring the first point cloud of the lidar, The point cloud generated by the lidar is an unordered point cloud; the distance calculation module 602 is used to traverse the first point cloud, and calculate the first Euclidean distance from each point in the first point cloud to the nearest neighbor point by querying the point cloud neighbor table , where the point cloud neighbors The table is generated based on the preset angle calibration data of the lidar; the data analysis module 603 is configured to determine outliers from the first point cloud based at least on the first Euclidean distance. The value of the first value can be set according to the actual situation.

In some exemplary embodiments, the data analysis module 603 is further configured to, for the i-th point in the first point cloud, determine from the nearest neighbors of the i-th point that the first Euclidean distance is less than the first threshold The number of one point, i is a positive integer; if the number of the first point is less than the second threshold, the i-th point is determined as an outlier point.

In some exemplary embodiments, the distance calculation module 602 is also used to traverse the first point cloud, and calculate the second Euclidean distance from each point in the first point cloud to the second nearest neighbor point by querying the point cloud neighbor table; the data The analysis module 603 is further configured to determine outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance. The value of the third value can be set according to the actual situation.

In some exemplary embodiments, the data analysis module 603 is further configured to determine suspected outliers from the first point cloud based on the first Euclidean distance; Identify outliers.

In some exemplary embodiments, the data analysis module 603 is further configured to, for the i-th point in the first point cloud, determine from the nearest neighbors of the i-th point that the first Euclidean distance is less than the third threshold The number of two points, i is a positive integer; if the number of the second point is less than the fourth threshold, the i-th point is determined as a suspected outlier.

In some exemplary embodiments, the data analysis module 603 is further configured to, for the jth suspected outlier in the first point cloud, determine the second Euclidean distance from the second nearest neighbors of the jth suspected outlier The number of the third points less than the fifth threshold, j is a positive integer; if the number of the third points is less than or equal to the sixth threshold, the jth suspected outlier point is determined as an outlier point.

In some exemplary embodiments, the processing device 600 further includes:

The point cloud generation module is used to generate a second point cloud at a preset detection distance based on the angle calibration data of the lidar; the second point cloud is in one-to-one correspondence with the first point cloud;

Search module, for traversing the second point cloud, search for the nearest neighbor point and second nearest neighbor point of each point;

The point cloud neighbor table generation module is used to generate a point cloud neighbor table based on the nearest neighbor point and the second nearest neighbor point of each point.

It should be noted that, the specific implementation process of the above-mentioned data acquisition module 601, distance calculation module 602 and data analysis module 603 can refer to the detailed description of the embodiment in Fig. 2 to Fig. 5 , and for the sake of brevity, details are not repeated here.

The data acquisition module 601 , the distance calculation module 602 and the data analysis module 603 mentioned in the embodiments of the present disclosure may be one or more processors.

Based on the same inventive concept, an embodiment of the present disclosure provides a laser radar, including: a memory storing computer-executable instructions; a processor connected to the memory for executing the computer-executable instructions and being able to implement the above-mentioned one or The point cloud processing method described in multiple embodiments.

Based on the same inventive concept, the present disclosure provides a computer storage medium. The computer storage medium stores computer-executable instructions. After the computer-executable instructions are executed by a processor, the point cloud as described in one or more of the above-mentioned embodiments can be realized. processing method.

FIG. 7 shows a configuration block diagram of an electronic device 700 according to an embodiment of the present disclosure. Electronic device 700 may be any type of general or special purpose computing device, such as a desktop computer, laptop computer, server, mainframe computer, cloud-based computer, tablet computer, wearable device, vehicle electronics, or the like. As shown in FIG. 7 , an electronic device 700 includes an input/output (Input/Output, I/O) interface 701 , a network interface 702 , a processor 703 and a memory 704 .

I/O interface 701 is a collection of components that can receive input from a user and/or provide output to a user. I/O interface 701 may include, but is not limited to, buttons, keyboards, keypads, LCD displays, LED displays, or other similar display devices, including display devices with touch screen capabilities to enable interaction between the user and the electronic device.

Network interface 702 may include various adapters and circuitry implemented in software and/or hardware to enable communication with the lidar using wired or wireless protocols. The wired protocol is, for example, any one or more of serial port protocol, parallel port protocol, Ethernet protocol, USB protocol or other wired communication protocols. The wireless protocol is, for example, any IEEE 802.11 Wi-Fi protocol, cellular network communication protocol, or the like.

Memory 704 includes a single memory or one or more memories or storage locations including, but not limited to, Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Read Only Memory (ROM) ), EPROM, EEPROM, flash memory, logic blocks of FPGAs, hard disks, or any other layers of the memory hierarchy. Memory 704 may be used to store any type of instructions, software or algorithms, including instructions 705 for controlling the general functions and operations of electronic device 700 .

The processor 703 controls general operations of the electronic device 700 . Processor 703 may include but not limited to CPU, hardware microprocessor, hardware processor, multi-core processor, single-core processor, microcontroller, application specific integrated circuit (ASIC), DSP or other similar processing devices, capable of executing according to Any type of instructions, algorithms or software for controlling the operation and functionality of the electronic device 700 of the embodiments described in this disclosure. Processor 703 can be various implementations of digital circuitry, analog circuitry, or mixed-signal (combination of analog and digital) circuitry that performs functions in a computing system. The processor 703 may include, for example, portions or circuits such as an integrated circuit (IC), a separate processor core, an entire processor core, a separate processor, a programmable hardware device such as a field programmable gate array (FPGA), and/or A system that includes multiple processors.

Communication between components of electronic device 700 may be established using internal bus 706 .

Electronic device 700 is communicatively coupled to the lidar to be calibrated to control the operation of the lidar. For example, the point cloud processing method according to the present disclosure may be stored in the memory 704 of the electronic device 700 in the form of computer readable instructions. The processor 703 implements the point cloud processing method by reading the stored computer-readable instructions.

Although electronic device 700 is described using certain components, in alternative embodiments, different components may be present in electronic device 700 . For example, electronic device 700 may include one or more additional processors, memory, network interfaces, and/or I/O interfaces. Additionally, one or more of the components may not be present in electronic device 700 . Additionally, although separate components are shown in FIG. 7 , in some embodiments some or all of a given component may be integrated into one or more of the other components in electronic device 700 .

The present disclosure can be implemented as any combination of an apparatus, system, integrated circuit, and computer program or program product on a non-transitory computer readable medium.

It should be understood that the computer-readable storage medium or the computer-executable instructions in the program product according to the embodiments of the present disclosure may be configured to perform operations corresponding to the above-mentioned device and method embodiments. When referring to the above-mentioned apparatus and method embodiments, the embodiments of the computer-readable storage medium or the program product will be obvious to those skilled in the art, so the description will not be repeated. Computer-readable storage media and program products for carrying or including the computer-executable instructions described above also fall within the scope of the present disclosure. Such storage media may include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

In addition, it should be understood that the series of processes and devices described above may also be implemented by software and/or firmware. In the case of realization by software and/or firmware, corresponding programs constituting the corresponding software are stored in the storage medium of the related device, and various functions can be performed when the programs are executed.

For example, a plurality of functions included in one unit in the above embodiments may be realized by separate devices. Alternatively, a plurality of functions implemented by a plurality of units in the above embodiments may be respectively implemented by separate devices. In addition, one of the above functions may be realized by a plurality of units. Such a configuration is included in the technical scope of the present disclosure.

Those skilled in the art can understand that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic, rather than the implementation process of the embodiments of the present disclosure. constitute any limitation.

The above embodiments are only used to illustrate the technical solution of the present disclosure, not to limit it. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions described in the foregoing embodiments can still be modified, or some of the technical features can be replaced. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included within the protection scope of the present disclosure.

Claims

A method for processing a point cloud, comprising:

Obtain the first point cloud of the laser radar, the point cloud produced by the laser radar is an unordered point cloud;

Traversing the first point cloud, calculating the first Euclidean distance from each point in the first point cloud to the nearest neighbor point by querying the point cloud neighbor table, wherein the point cloud neighbor table is based on the lidar generated from the preset angle calibration data;

Outliers are determined from the first point cloud based at least on the first Euclidean distance.
The method according to claim 1, wherein said determining outliers from said first point cloud based at least on said first Euclidean distance comprises:

For the i-th point in the first point cloud, determine the number of first points whose first Euclidean distance is less than a first threshold from the nearest neighbors of the i-th point, where i is a positive integer;

If the number of the first points is less than a second threshold, the ith point is determined as the outlier point.
The method according to claim 1, wherein, after said obtaining the first point cloud of input, said method further comprises:

Traversing the first point cloud, calculating the second Euclidean distance from each point in the first point cloud to the next nearest neighbor point by querying the point cloud neighbor table;

The determining outliers from the first point cloud based at least on the first Euclidean distance includes:

Outliers are determined from the first point cloud based on the first Euclidean distance and the second Euclidean distance.
The method according to claim 3, wherein said determining outliers from said first point cloud based on said first Euclidean distance and said second Euclidean distance comprises:

determining suspected outliers from the first point cloud based on the first Euclidean distance;

Based on the second Euclidean distance, outliers are determined from the suspected outliers.
The method according to claim 4, wherein, based on the first Euclidean distance, from Determining suspected outliers in the first point cloud includes:

For the i-th point in the first point cloud, determine the number of second points whose first Euclidean distance is less than a third threshold from the nearest neighbors of the i-th point, where i is a positive integer;

If the number of the second points is less than the fourth threshold, the ith point is determined as the suspected outlier point.
The method according to claim 4, wherein said determining outliers from said suspected outliers based on said second Euclidean distance comprises:

For the jth suspected outlier point in the first point cloud, determine the number of third points whose second Euclidean distance is less than the fifth threshold from the second nearest neighbor points of the jth suspected outlier point, j is a positive integer;

If the number of the third points is less than or equal to the sixth threshold, the jth suspected outlier point is determined as the outlier point.
The method according to claim 1, said method further comprising:

Based on the angle calibration data of the lidar, at a preset detection distance, a second point cloud is generated; the second point cloud is in one-to-one correspondence with the first point cloud;

Traversing the second point cloud, searching for the nearest neighbor and second nearest neighbor of each point;

The point cloud neighbor table is generated based on the nearest neighbor point and the second nearest neighbor point of each point.
A point cloud processing device, comprising:

The data acquisition module is used to obtain the first point cloud of the laser radar, and the point cloud generated by the laser radar is a disordered point cloud;

The distance calculation module is used to traverse the first point cloud, and calculate the first Euclidean distance from each point in the first point cloud to the nearest neighbor by querying the point cloud neighbor table, wherein the point cloud neighbor table is generated based on the preset angle calibration data of the lidar;

A data analysis module, configured to determine outliers from the first point cloud based at least on the first Euclidean distance.
The device according to claim 8, wherein the data analysis module is further configured to determine the i-th point from the nearest neighbors of the i-th point for the i-th point in the first point cloud a European style The number of first points whose distance is less than the first threshold, i is a positive integer;

If the number of the first points is less than a second threshold, the ith point is determined as the outlier point.
The device according to claim 8, wherein the distance calculation module is also used for traversing the first point cloud, by querying the point cloud neighbor table, calculating the time-to-time distance of each point in the first point cloud The second Euclidean distance of the neighbor point;

The data analysis module is further configured to determine outliers from the first point cloud based on the first Euclidean distance and the second Euclidean distance.
The device according to claim 10, wherein the data analysis module is further configured to determine suspected outliers from the first point cloud based on the first Euclidean distance; and based on the second Euclidean distance , determining outliers from the suspected outliers.
The device according to claim 11, wherein the data analysis module is further configured to determine the i-th point from the nearest neighbors of the i-th point for the i-th point in the first point cloud A number of second points whose Euclidean distance is less than the third threshold, i is a positive integer; if the number of the second points is less than the fourth threshold, the ith point is determined as the suspected outlier point.
The device according to claim 11, wherein the data analysis module is further configured to, for the jth suspected outlier point in the first point cloud, obtain the second nearest neighbor of the jth suspected outlier point Determine the number of third points whose second Euclidean distance is less than the fifth threshold in the points, j is a positive integer; if the number of the third points is less than or equal to the sixth threshold, then the jth suspected outlier points are identified as the outliers.
The apparatus of claim 8, further comprising:

A point cloud generation module, configured to generate a second point cloud at a preset detection distance based on the angle calibration data of the lidar; the second point cloud is in one-to-one correspondence with the first point cloud;

A search module, configured to traverse the second point cloud, and search for the nearest neighbor and the second nearest neighbor of each point;

A point cloud neighbor table generating module, configured to generate the point cloud neighbor table based on the nearest neighbor point and the second nearest neighbor point of each point.
A lidar, comprising:

a memory storing computer-executable instructions;

A processor, connected to the memory, configured to implement the method according to any one of claims 1 to 7 by executing the computer-executable instructions.
A computer storage medium, the computer storage medium stores computer-executable instructions, and the computer-executable instructions can implement the method according to any one of claims 1 to 7 after being executed by a processor.