CN117830140B - Denoising method and device for foggy weather point cloud for unmanned control system - Google Patents
Denoising method and device for foggy weather point cloud for unmanned control system Download PDFInfo
- Publication number
- CN117830140B CN117830140B CN202410239016.1A CN202410239016A CN117830140B CN 117830140 B CN117830140 B CN 117830140B CN 202410239016 A CN202410239016 A CN 202410239016A CN 117830140 B CN117830140 B CN 117830140B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- cluster
- clustering
- point
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Landscapes
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The embodiment of the application provides a denoising method and device for a foggy weather point cloud for an unmanned control system. The method comprises the following steps: acquiring point cloud data to be processed; clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters; for each cluster, determining label information corresponding to the cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises a real point cloud and a noise point cloud; and removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data. According to the technical scheme, the noise point cloud can be accurately identified and removed in the foggy environment, so that the safety of the unmanned vehicle in foggy driving is improved, and the accuracy of a control instruction sent by a vehicle control system is improved.
Description
Technical Field
The application relates to the technical field of unmanned control systems, in particular to a method and a device for denoising foggy weather point clouds for an unmanned control system.
Background
With the rapid development of unmanned technology, the unmanned technology is widely applied in life. Unmanned vehicles often sense the surrounding environment through lidar, such as object detection, real-time localization, and mapping, among others. However, due to factors such as weather and environment, the unmanned vehicle often encounters heavy fog weather, so that a large amount of noise point clouds exist in the 3D point clouds obtained by laser radar scanning, misoperation is easily caused to control of an unmanned control system, and the running safety of the unmanned vehicle is seriously affected. Therefore, how to accurately identify and remove the noise point cloud in the foggy environment, so as to improve the accuracy of the vehicle control system in foggy driving and the safety of the unmanned vehicle in foggy driving becomes a technical problem to be solved.
Disclosure of Invention
The embodiment of the application provides a denoising method and device for a foggy point cloud for an unmanned control system, which can accurately identify and remove the foggy point cloud in a foggy environment at least to a certain extent, further improve the safety of an unmanned vehicle in foggy driving and improve the accuracy of a control instruction sent by a vehicle control system.
Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.
According to an aspect of the embodiment of the application, there is provided a denoising method of a foggy weather point cloud for an unmanned control system, comprising:
Acquiring point cloud data to be processed;
clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters;
For each cluster, determining label information corresponding to the cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises a real point cloud and a noise point cloud;
and removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data.
According to an aspect of the embodiment of the present application, there is provided a denoising apparatus for foggy weather point cloud for an unmanned control system, including:
the acquisition module is used for acquiring point cloud data to be processed;
the clustering module is used for clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters;
The label determining module is used for determining label information corresponding to each cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises real point cloud and noise point cloud;
and the processing module is used for removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data.
According to an aspect of the embodiments of the present application, there is provided a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements a denoising method for a foggy weather point cloud for a unmanned control system as described in the above embodiments.
According to an aspect of an embodiment of the present application, there is provided an electronic apparatus including: one or more processors; and a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method for denoising a foggy weather point cloud for an unmanned control system as described in the above embodiment.
According to an aspect of embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the denoising method of the foggy weather point cloud for the unmanned control system provided in the above embodiment.
In the technical scheme provided by some embodiments of the application, the point cloud data to be processed is obtained, and the points in different distance intervals in the point cloud data to be processed are clustered by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clusters, so that the rationality and the effectiveness of a clustering result are ensured, and the accuracy of the subsequent noise point cloud identification is further improved. Then, for each cluster, determining label information corresponding to each cluster according to the centroid point corresponding to each cluster and/or attribute information of the contained point, wherein the label information comprises real point cloud and noise point cloud, and removing the cluster with the label information being the noise point cloud from point cloud data to be processed to obtain target point cloud data. Therefore, the noise point cloud can be accurately identified and removed in the foggy environment, and the safety of the unmanned vehicle in foggy running is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:
FIG. 1 shows a flow diagram of a method of denoising a foggy weather point cloud for an unmanned control system according to one embodiment of the application;
FIG. 2 illustrates a block diagram of a denoising device for foggy weather point clouds for an unmanned control system according to one embodiment of the present application;
fig. 3 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
Fig. 1 shows a flow diagram of a method for denoising a foggy weather point cloud for an unmanned control system according to an embodiment of the present application. The method can be applied to terminal equipment or a server, wherein the terminal equipment can comprise one or more of a smart phone, a tablet computer, a portable computer, a desktop computer and a vehicle-mounted terminal; the server may be a physical server or a cloud server.
In a specific application scenario, at least one laser radar can be installed on an unmanned truck (hereinafter referred to as an unmanned truck), the laser radar harness is at least 16 lines, the height of the laser radar harness is at least 1.5m, and the laser radar can be connected with a vehicle-mounted terminal on the unmanned truck so as to transmit point cloud data to the vehicle-mounted terminal. The vehicle-mounted terminal can execute the denoising method of the foggy day point cloud for the unmanned control system according to the received point cloud data, so that the safety of automatic running of the unmanned collector card in foggy days is ensured, and the accuracy of a control instruction sent by the vehicle control system is improved.
Referring to fig. 1, the denoising method of the foggy weather point cloud for the unmanned control system at least includes steps S110 to S140, and is described in detail as follows (hereinafter, the method is applied to a vehicle-mounted terminal for example to be described, hereinafter, referred to as a terminal):
in step S110, point cloud data to be processed is acquired.
In this embodiment, the terminal is in communication connection with a laser radar on the vehicle, where the laser radar may scan the surrounding environment in real time during the running process of the vehicle, and send the point cloud data obtained by scanning to the terminal, and the terminal may use the received point cloud data as point cloud data to be processed for subsequent processing.
It should be understood that when the weather is other weather, the terminal may execute the general point cloud data processing logic, and when the weather is foggy, the terminal may execute the denoising method of foggy point cloud for the unmanned control system provided by the application, so as to ensure that corresponding point cloud data processing logic can be adopted in different weather environments, and ensure the effectiveness and rationality of point cloud data processing.
In step S120, for points in different distance intervals in the point cloud data to be processed, clustering is performed on the points by using a clustering rule corresponding to the distance intervals, so as to obtain a plurality of clusters.
In this embodiment, it should be appreciated that, for lidar, as the distance is greater, the point cloud perceived for the same object will be more sparse, the sparsity of which is primarily dependent on how many strands of the lidar are. While the influencing factors of the harness mainly come from the vertical resolution (vertical_resolution) and the horizontal resolution (horizon _resolution). Taking the point cloud p0 with the distance r as an example, the distance length and width of the closest point p1 of the plane perpendicular to the laser beam is about (vertical_resolution x, horizon _resolution x).
Thus, to ensure the rationality and effectiveness of the clustering result, the skilled person can preset the clustering rules corresponding to different distance intervals, and it should be understood that the distance is the distance from the pointing cloud to the laser radar. In one example, the point cloud may be divided into intervals of 10m, i.e., (0-10 m ], (10-20 m ], (20-30 m ]. … …. In other examples, the point cloud may be divided into intervals of other lengths, which is not particularly limited.
Then, according to different distance intervals, a person skilled in the art may set a corresponding clustering rule, taking a DBSCAN clustering algorithm as an example, different distance intervals may correspond to different neighborhood radii (Epsilon, hereinafter referred to as dis) and minimum points in the neighborhood (minPts), and the person skilled in the art may set according to previous experience, for example, dis=0.4m, minpts=5 at a 10m position, dis=k=0.8m, minpts=5 at a 20m position, dis=k=k×1.2, minpts=5 at a 30m position, where k is used to adjust the distance ratio, and the person skilled in the art may make an adaptive adjustment according to the actual laser radar effect, and default to 1. For example, the point p2, which is within the distance interval of (10-20 m), the core point is selected within the distance interval so that not less than 5 other points can be found within the range of 0.8 m.
In addition, the initial values of the minimum number of points in the neighborhood and the neighborhood radius may be associated with the vertical resolution and the horizontal resolution. It will be appreciated that with the same vertical and horizontal resolution, the point-to-point correspondence between points in the point cloud increases as the distance from the lidar increases. Assuming a vertical resolution of 2 ° and a horizontal resolution of 0.2 °, the horizontal resolution is about hr=10×0.2×pi/180=0.035 m at 10m, the vertical resolution is about vr=10×2×pi/180=0.35 m, and dis may be selected to be a value appropriately larger than max (hr, vr), where 0.4m is taken, minpts=dis/(2×min (hr, vr))=0.4/(2×0.035) =5.7, and rounded down to 5.
And then, according to the characteristics of the target objects in the actual scene, the threshold value can be appropriately adjusted, if the individual small target objects need to be further detected, the values of dis=0.4m and minPts=5 can be synchronously reduced, for example, dis=0.2 m and minPts=2 are taken, and the like. Therefore, the reasonability of the neighborhood radius and the minimum point number setting in the neighborhood can be ensured, and the accuracy of the subsequent clustering result is further improved.
In an example, the terminal may acquire clustering rules corresponding to different distance intervals from its own storage space, so as to cluster points in different distance intervals in the point cloud data to be processed respectively, so as to obtain a plurality of clusters. Through the mode, the perception condition of a real object can be obviously improved, and the phenomenon of cutting off the object of the vehicle is avoided.
In step S130, for each cluster, tag information corresponding to the cluster is determined according to the centroid point corresponding to the cluster and/or attribute information of the included point, where the tag information includes a real point cloud and a noise point cloud.
In this embodiment, the terminal may identify each cluster to determine tag information corresponding to the cluster, i.e. determine whether it is a real point cloud or a noisy point cloud. Specifically, the terminal may determine tag information corresponding to each cluster according to the centroid point corresponding to the cluster and/or attribute information of a point included in the cluster. The centroid point is determined by the terminal according to the coordinate information of the points contained in the cluster, and the average value of the coordinates of all the points on the XYZ axes is determined to obtain centroid points (xcenter, ycenter, zcenter). The attribute information of the points may include, but is not limited to, coordinate information, intensity information, and the like.
The person skilled in the art can preset the corresponding judgment rule according to the prior experience, judge based on the judgment rule and the attribute information of the centroid point corresponding to each cluster and/or the contained point, and identify the real point cloud and the noise point cloud from each cluster.
In one embodiment of the present application, for each cluster, determining label information corresponding to the cluster according to attribute information of a centroid point and/or a contained point corresponding to the cluster includes:
If the height of the centroid point corresponding to the cluster is lower than a first threshold value, determining that the label information corresponding to the cluster is a real point cloud;
And/or
If the variance of the distances from other points except the centroid point to the centroid point in the cluster is larger than a second threshold value, determining that the label information corresponding to the cluster is a real point cloud;
And/or
If the number of points with the point cloud intensity larger than the third threshold value in the cluster is larger than the fourth threshold value, determining the label information corresponding to the cluster as real point cloud;
And/or
If the number of the points with the heights smaller than a fifth threshold value in the cluster is larger than a sixth threshold value, determining that the label information corresponding to the cluster is a real point cloud, wherein the fifth threshold value is smaller than the first threshold value.
In this embodiment, a person skilled in the art may collect the original point cloud data of the heavy fog for a plurality of times in advance, so as to count centroid points of each cluster corresponding to the heavy fog in the environment, and obtain the lowest height of the centroid points as the first threshold. When the height of the centroid point corresponding to the cluster is lower than the first threshold, the label information corresponding to the cluster can be determined to be a real point cloud, namely, filtering processing is not performed on the real point cloud. For example, the first threshold is 1m, and if the height of the centroid point corresponding to the cluster is smaller than 1m, the cluster is determined to be a real point cloud.
The terminal can also calculate the square of the difference value between all the points in each cluster and the centroid point of the cluster, further obtain a corresponding variance, compare the variance with a preset second threshold, and if the variance is larger than the second threshold, determine that the label information corresponding to the cluster is a real point cloud. Wherein, the second threshold value can be obtained by a person skilled in the art according to the statistics of the pre-acquired original point cloud data of the big fog.
The terminal can also count according to the point cloud intensities of the points contained in the cluster, and when the number of points with the point cloud intensity larger than the third threshold value in a certain cluster is larger than the fourth threshold value, the label information corresponding to the cluster is determined to be a real point cloud, and the object corresponding to the cluster is determined to be a real object.
The terminal may further perform statistics according to the heights of the points included in the cluster, and determine that the tag information corresponding to the cluster is a real point cloud when the number of points with heights smaller than a fifth threshold in a certain cluster is greater than a sixth threshold, where the fifth threshold is smaller than the first threshold.
It should be noted that, the "and/or" of the present application, that is, one skilled in the art may select any one or more of the above determination rules to apply according to actual implementation needs, and the present application is not limited thereto.
Therefore, through the judging rule, whether each cluster corresponds to a real object or noise point cloud generated by a foggy day can be accurately identified, and further the subsequent filtering effect is ensured.
Referring to fig. 1, in step S140, the cluster with the tag information being noise point cloud is removed from the point cloud data to be processed, so as to obtain target point cloud data.
In this embodiment, the terminal may remove points in the cluster determined as the noise point cloud by the tag information from the point cloud data to be processed, and the remaining points are the target point cloud data, so as to achieve the purpose of removing the noise point cloud, so as to be used by subsequent recognition.
In other examples, the terminal may also directly integrate and store the points in the cluster in which the tag information is determined to be the real point cloud in addition, that is, the cluster in which the tag information is the noise point cloud is removed, so as to obtain the target point cloud data.
It should be noted that, according to the actual implementation needs, those skilled in the art may select a corresponding implementation manner, which is not limited in particular.
Based on the embodiment shown in fig. 1, in one embodiment of the present application, for points in different distance intervals in the point cloud data to be processed, clustering is performed on the points by using a clustering rule corresponding to the distance interval, so as to obtain a plurality of clusters, including:
dividing according to the point cloud intensities corresponding to points in the point cloud data to be processed to obtain a first class of point clouds and a second class of point clouds, wherein the first class of point clouds consists of points with the point cloud intensities smaller than an intensity threshold, and the second class of point clouds consists of points with the point cloud intensities larger than or equal to the intensity threshold;
Clustering points in different distance intervals in the first class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of first clustering clusters;
clustering points in different distance intervals in the second class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of second clustering clusters;
Removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data, wherein the method comprises the following steps:
And merging the first cluster and the second cluster with the tag information being the real point cloud, and re-clustering the first cluster and the second cluster with the intersection ratio larger than a seventh threshold in the merging process to obtain the target point cloud data.
In this embodiment, the terminal may divide points in the point cloud data to be processed according to a preset intensity threshold, divide points with the point cloud intensity smaller than the intensity threshold into first class point clouds, and divide points with the point cloud intensity greater than or equal to the intensity threshold into second class point clouds. Wherein the intensity threshold may be preset by a person skilled in the art based on prior experience.
Then, clustering is carried out on the first class point cloud and the second class point cloud respectively, namely, the points in different distance intervals in the first class point cloud and the second class point cloud are clustered by adopting a clustering rule corresponding to the distance intervals, so that a plurality of corresponding first clustering clusters and a plurality of corresponding second clustering clusters are obtained.
Then, based on the tag information determining rule as described above, a plurality of first cluster clusters and a plurality of second cluster clusters are identified to determine tag information corresponding to each of the first cluster and the second cluster.
And then merging the first cluster and the second cluster of which the label information is the real point cloud, and in the merging process, if the merging ratio between any pair of the first cluster and the second cluster is larger than a seventh threshold value, re-clustering the pair of the first cluster and the second cluster. The intersection ratio can be determined by the coincidence degree between the minimum circumscribed polygonal frames of each cluster. And finally, outputting the combined first cluster and second cluster as target point cloud data.
It can be understood that by dividing the point cloud data into two types of point clouds according to the point cloud intensity, the actual object and the noise point clouds generated by fog can be distinguished from each other in terms of intensity, so that the accuracy of the clustering result is ensured. For example, an actual vehicle may be clustered around a stack of fog, if not separated into two categories, with a high probability of clustering the vehicle and fog into one object, which is not in good agreement with the fact.
In other embodiments, different clustering thresholds may be set during clustering according to actual features of the low-intensity point cloud (i.e., the first category point cloud) and the high-intensity point cloud (i.e., the second category point cloud), so that the clustering result is more fit to the actual situation.
In one embodiment of the present application, obtaining point cloud data to be processed includes:
Acquiring point cloud data to be processed;
According to the point cloud data to be processed, determining the point cloud intensity of each point in the point cloud within a preset range of the laser radar and the point cloud density within the preset range;
and if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a preset foggy day judging rule, denoising the foggy day point cloud according to the point cloud data to be processed.
In this embodiment, after the terminal obtains the point cloud data to be processed, in order to determine whether to execute the denoising method of the foggy day point cloud for the unmanned control system, the terminal may determine, according to the point cloud data to be processed, the point cloud intensity of each point in the point cloud located in the predetermined range of the laser radar and the point cloud density in the predetermined range. Assuming a threshold of 0.5m and a lidar mounting position of (x, y, z), then the corresponding predetermined range is all points within [ x+0.5, x-0.5], [ y, y-0.5], [ z-0.5, z+0.5 ]. The threshold may be adapted by one skilled in the art based on the lidar and the operating environment mist characteristics, among other things.
Then, the terminal can count the point cloud intensity of the points in the preset range and the corresponding point cloud density, and judge the points by adopting a preset foggy day judging rule so as to determine whether the points are in a foggy day environment currently. In an example, when the ratio of the number of points in the predetermined range, where the point cloud intensity is less than a certain threshold, to the total number of points in the predetermined range reaches (i.e., is greater than or equal to) a predetermined ratio, and the point cloud density (i.e., the total number divided by the spatial volume) is greater than the density threshold, it indicates that the point cloud data to be processed is currently in a foggy environment, the denoising process of the foggy point cloud may be performed. Otherwise, general point cloud processing logic may be executed.
For example, the terminal may count points with a point cloud intensity < intensity in a predetermined range, intensity=12, and if the proportion of points with an intensity less than intensity to the total number of points in the predetermined range reaches 80%, and the point cloud density P1/V > dense turns on the big fog mode (i.e. performs denoising processing of the fog point cloud), dense=10, where V is a space [ x+0.5, x-0.5], [ y, y-0.5], [ z-0.5, z+0.5] volume is 1×1×0.5=0.5.
Therefore, the terminal can automatically judge whether the vehicle is in a foggy environment, and timely start and execute the denoising logic of foggy point clouds, so that the safety of the unmanned vehicle is ensured, and the accuracy of a control instruction sent by a vehicle control system is improved.
In one embodiment of the present application, if the point cloud intensity of each point in the predetermined range and the corresponding point cloud density conform to a preset foggy day determination rule, performing denoising processing of foggy day point clouds on the point cloud data to be processed, including:
If the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a first foggy day judging rule, first denoising processing of foggy day point clouds is executed aiming at the point cloud data to be processed;
And if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a second foggy day judging rule, executing second denoising processing of foggy day point clouds aiming at the point cloud data to be processed, wherein the first foggy day judging rule and the second foggy day judging rule respectively correspond to different severe weather intensities, and the point cloud intensity thresholds in the first foggy day judging rule and the second foggy day judging rule are different.
In this embodiment, a person skilled in the art may set a corresponding foggy day determination rule according to different severe weather intensities, and when the point cloud intensity of each point in the predetermined range and the corresponding point cloud density meet the first foggy day determination rule, perform first denoising processing of the foggy day point cloud with respect to the point cloud data to be processed, and when the second foggy day determination rule is met, perform second denoising processing of the foggy day point cloud with respect to the point cloud data to be processed.
For example, a first fog day determination rule of intensity=6 and density=5 is set to 1-level fog, a second fog day determination rule of intensity=12 and density=10 is set to 2-level fog, and so on. It should be noted that the foregoing numbers are merely exemplary, and those skilled in the art may determine the corresponding foggy day determination rule according to actual implementation needs, which is not limited in particular.
Therefore, the foggy environment can be classified through different regular point cloud intensity thresholds, different denoising treatments are adopted for coping, the denoising effect of the foggy point cloud can be ensured, and the driving safety of the unmanned vehicle in foggy days is improved.
In another embodiment of the present application, obtaining point cloud data to be processed includes:
and acquiring point cloud data to be processed according to the received denoising request for the foggy point cloud.
In this embodiment, the vehicle-mounted terminal may be in communication connection with a handheld terminal of a manager, and when the vehicle-mounted terminal is in a foggy environment, the manager may send a denoising request for foggy point cloud to the vehicle-mounted terminal through the handheld terminal, and after receiving the request, the vehicle-mounted terminal may acquire point cloud data to be processed to perform denoising processing of foggy point cloud. Therefore, a manager can manually start the denoising logic of the foggy weather point cloud, the response speed to the foggy weather environment is guaranteed, and the driving safety of the unmanned vehicle in foggy weather is further guaranteed.
The following describes an embodiment of the apparatus of the present application, which may be used to implement the denoising method of the foggy weather point cloud for the unmanned control system in the above embodiment of the present application. For details not disclosed in the embodiment of the apparatus of the present application, please refer to the embodiment of the denoising method for foggy weather point cloud for the unmanned control system.
Fig. 2 shows a block diagram of a denoising device for foggy weather point clouds for an unmanned control system according to an embodiment of the present application.
Referring to fig. 2, a denoising apparatus of a foggy weather point cloud for an unmanned control system according to an embodiment of the present application includes:
the acquisition module is used for acquiring point cloud data to be processed;
the clustering module is used for clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters;
The label determining module is used for determining label information corresponding to each cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises real point cloud and noise point cloud;
and the processing module is used for removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data.
In an embodiment, for each cluster, determining label information corresponding to the cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point includes:
If the height of the centroid point corresponding to the cluster is lower than a first threshold value, determining that the label information corresponding to the cluster is a real point cloud;
And/or
If the variance of the distances from other points except the centroid point to the centroid point in the cluster is larger than a second threshold value, determining that the label information corresponding to the cluster is a real point cloud;
And/or
If the number of points with the point cloud intensity larger than the third threshold value in the cluster is larger than the fourth threshold value, determining the label information corresponding to the cluster as real point cloud;
And/or
If the number of the points with the heights smaller than a fifth threshold value in the cluster is larger than a sixth threshold value, determining that the label information corresponding to the cluster is a real point cloud, wherein the fifth threshold value is smaller than the first threshold value.
In an embodiment, for points in different distance intervals in the point cloud data to be processed, clustering the points by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clusters, including:
dividing according to the point cloud intensities corresponding to points in the point cloud data to be processed to obtain a first class of point clouds and a second class of point clouds, wherein the first class of point clouds consists of points with the point cloud intensities smaller than an intensity threshold, and the second class of point clouds consists of points with the point cloud intensities larger than or equal to the intensity threshold;
Clustering points in different distance intervals in the first class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of first clustering clusters;
clustering points in different distance intervals in the second class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of second clustering clusters;
Removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data, wherein the method comprises the following steps:
And merging the first cluster and the second cluster with the tag information being the real point cloud, and re-clustering the first cluster and the second cluster with the intersection ratio larger than a seventh threshold in the merging process to obtain the target point cloud data.
In an embodiment, obtaining point cloud data to be processed includes:
Acquiring point cloud data to be processed;
According to the point cloud data to be processed, determining the point cloud intensity of each point in the point cloud within a preset range of the laser radar and the point cloud density within the preset range;
and if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a preset foggy day judging rule, denoising the foggy day point cloud according to the point cloud data to be processed.
In an embodiment, if the point cloud intensity of each point in the predetermined range and the corresponding point cloud density conform to a preset foggy day determination rule, performing denoising processing of foggy day point clouds on the point cloud data to be processed, including:
If the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a first foggy day judging rule, first denoising processing of foggy day point clouds is executed aiming at the point cloud data to be processed;
And if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a second foggy day judging rule, executing second denoising processing of foggy day point clouds aiming at the point cloud data to be processed, wherein the first foggy day judging rule and the second foggy day judging rule respectively correspond to different severe weather intensities, and the point cloud intensity thresholds in the first foggy day judging rule and the second foggy day judging rule are different.
In an embodiment, obtaining point cloud data to be processed includes:
and acquiring point cloud data to be processed according to the received denoising request for the foggy point cloud.
Fig. 3 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
It should be noted that, the computer system of the electronic device shown in fig. 3 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.
As shown in fig. 3, the computer system includes a central processing unit (Central Processing Unit, CPU) 301 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 302 or a program loaded from a storage section 308 into a random access Memory (Random Access Memory, RAM) 303. In the RAM 303, various programs and data required for the system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other through a bus 304. An Input/Output (I/O) interface 305 is also connected to bus 304.
The following components are connected to the I/O interface 305: an input section 306 including a keyboard, a mouse, and the like; an output portion 307 including a Cathode Ray Tube (CRT), a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), and a speaker, etc.; a storage section 308 including a hard disk or the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 309 performs communication processing via a network such as the internet. The drive 310 is also connected to the I/O interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 309, and/or installed from the removable medium 311. When executed by a Central Processing Unit (CPU) 301, performs the various functions defined in the system of the present application.
It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
As another aspect, the present application also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (7)
1. The denoising method of the foggy weather point cloud for the unmanned control system is characterized by comprising the following steps of:
Acquiring point cloud data to be processed;
Clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters; the distance is the distance between the point cloud and the laser radar; the clustering rule is a DBSCAN clustering algorithm, and the clustering is carried out by corresponding different neighborhood radiuses and the minimum points in the neighborhood through different distance intervals, or the clustering is carried out by associating the initial values of the minimum points in the neighborhood and the neighborhood radiuses with the vertical resolution and the horizontal resolution; dividing according to the point cloud intensities corresponding to points in the point cloud data to be processed to obtain a first class of point clouds and a second class of point clouds, wherein the first class of point clouds consists of points with the point cloud intensities smaller than an intensity threshold, and the second class of point clouds consists of points with the point cloud intensities larger than or equal to the intensity threshold; clustering points in different distance intervals in the first class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of first clustering clusters; clustering points in different distance intervals in the second class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of second clustering clusters; removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data, wherein the method comprises the following steps: combining the first cluster and the second cluster with the tag information being the real point cloud, and re-clustering the first cluster and the second cluster with the intersection ratio larger than a seventh threshold in the combining process to obtain target point cloud data;
for each cluster, determining label information corresponding to the cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises a real point cloud and a noise point cloud; if the height of the centroid point corresponding to the cluster is lower than a first threshold value, determining that the label information corresponding to the cluster is a real point cloud; and/or if the variance of the distances from other points except the centroid point to the centroid point in the cluster is larger than a second threshold value, determining that the label information corresponding to the cluster is a real point cloud; and/or if the number of points with the point cloud intensity greater than the third threshold value in the cluster is greater than the fourth threshold value, determining the label information corresponding to the cluster as the real point cloud; and/or if the number of the points with the height smaller than a fifth threshold value in the cluster is larger than a sixth threshold value, determining that the label information corresponding to the cluster is a real point cloud, wherein the fifth threshold value is smaller than the first threshold value;
and removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data.
2. The method of claim 1, wherein obtaining point cloud data to be processed comprises:
Acquiring point cloud data to be processed;
According to the point cloud data to be processed, determining the point cloud intensity of each point in the point cloud within a preset range of the laser radar and the point cloud density within the preset range;
and if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a preset foggy day judging rule, denoising the foggy day point cloud according to the point cloud data to be processed.
3. The method according to claim 2, wherein if the point cloud intensity and the corresponding point cloud density of each point in the predetermined range conform to a preset foggy day determination rule, performing a foggy day point cloud denoising process for the point cloud data to be processed, including:
If the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a first foggy day judging rule, first denoising processing of foggy day point clouds is executed aiming at the point cloud data to be processed;
And if the point cloud intensity of each point in the preset range and the corresponding point cloud density accord with a second foggy day judging rule, executing second denoising processing of foggy day point clouds aiming at the point cloud data to be processed, wherein the first foggy day judging rule and the second foggy day judging rule respectively correspond to different severe weather intensities, and the point cloud intensity thresholds in the first foggy day judging rule and the second foggy day judging rule are different.
4. The method of claim 1, wherein obtaining point cloud data to be processed comprises:
and acquiring point cloud data to be processed according to the received denoising request for the foggy point cloud.
5. Denoising device of foggy weather point cloud for unmanned control system, characterized by comprising:
the acquisition module is used for acquiring point cloud data to be processed;
The clustering module is used for clustering points in different distance intervals in the point cloud data to be processed by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of clustering clusters; the distance is the distance between the point cloud and the laser radar; the clustering rule is a DBSCAN clustering algorithm, and the clustering is carried out by corresponding different neighborhood radiuses and the minimum points in the neighborhood through different distance intervals, or the clustering is carried out by associating the initial values of the minimum points in the neighborhood and the neighborhood radiuses with the vertical resolution and the horizontal resolution; dividing according to the point cloud intensities corresponding to points in the point cloud data to be processed to obtain a first class of point clouds and a second class of point clouds, wherein the first class of point clouds consists of points with the point cloud intensities smaller than an intensity threshold, and the second class of point clouds consists of points with the point cloud intensities larger than or equal to the intensity threshold; clustering points in different distance intervals in the first class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of first clustering clusters; clustering points in different distance intervals in the second class point cloud by adopting a clustering rule corresponding to the distance intervals to obtain a plurality of second clustering clusters; removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data, wherein the method comprises the following steps: combining the first cluster and the second cluster with the tag information being the real point cloud, and re-clustering the first cluster and the second cluster with the intersection ratio larger than a seventh threshold in the combining process to obtain target point cloud data;
The label determining module is used for determining label information corresponding to each cluster according to the centroid point corresponding to the cluster and/or attribute information of the contained point, wherein the label information comprises real point cloud and noise point cloud; if the height of the centroid point corresponding to the cluster is lower than a first threshold value, determining that the label information corresponding to the cluster is a real point cloud; and/or if the variance of the distances from other points except the centroid point to the centroid point in the cluster is larger than a second threshold value, determining that the label information corresponding to the cluster is a real point cloud; and/or if the number of points with the point cloud intensity greater than the third threshold value in the cluster is greater than the fourth threshold value, determining the label information corresponding to the cluster as the real point cloud; and/or if the number of the points with the height smaller than a fifth threshold value in the cluster is larger than a sixth threshold value, determining that the label information corresponding to the cluster is a real point cloud, wherein the fifth threshold value is smaller than the first threshold value;
and the processing module is used for removing the clustering cluster with the tag information being noise point cloud from the point cloud data to be processed to obtain target point cloud data.
6. A computer-readable medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements a method for denoising a foggy weather point cloud for an unmanned control system according to any one of claims 1 to 4.
7. An electronic device, comprising:
One or more processors;
a storage means for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of denoising a foggy weather point cloud for an unmanned control system according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410239016.1A CN117830140B (en) | 2024-03-04 | 2024-03-04 | Denoising method and device for foggy weather point cloud for unmanned control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410239016.1A CN117830140B (en) | 2024-03-04 | 2024-03-04 | Denoising method and device for foggy weather point cloud for unmanned control system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117830140A CN117830140A (en) | 2024-04-05 |
| CN117830140B true CN117830140B (en) | 2024-05-10 |
Family
ID=90523151
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410239016.1A Active CN117830140B (en) | 2024-03-04 | 2024-03-04 | Denoising method and device for foggy weather point cloud for unmanned control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117830140B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659547A (en) * | 2018-06-29 | 2020-01-07 | 比亚迪股份有限公司 | Object recognition method, device, vehicle and computer-readable storage medium |
| CN111582054A (en) * | 2020-04-17 | 2020-08-25 | 中联重科股份有限公司 | Point cloud data processing method and device and obstacle detection method and device |
| WO2021238306A1 (en) * | 2020-05-25 | 2021-12-02 | 华为技术有限公司 | Method for processing laser point cloud and related device |
| CN114692734A (en) * | 2022-03-11 | 2022-07-01 | 三一智矿科技有限公司 | Mining area dust filtering method, device, equipment and medium based on point cloud |
| CN116485674A (en) * | 2023-04-24 | 2023-07-25 | 智道网联科技(北京)有限公司 | Real-time denoising method and device for laser point cloud data, electronic equipment and storage medium |
| CN117572451A (en) * | 2024-01-11 | 2024-02-20 | 广州市杜格科技有限公司 | Traffic information acquisition method, equipment and medium based on multi-line laser radar |
-
2024
- 2024-03-04 CN CN202410239016.1A patent/CN117830140B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659547A (en) * | 2018-06-29 | 2020-01-07 | 比亚迪股份有限公司 | Object recognition method, device, vehicle and computer-readable storage medium |
| CN111582054A (en) * | 2020-04-17 | 2020-08-25 | 中联重科股份有限公司 | Point cloud data processing method and device and obstacle detection method and device |
| WO2021238306A1 (en) * | 2020-05-25 | 2021-12-02 | 华为技术有限公司 | Method for processing laser point cloud and related device |
| CN114692734A (en) * | 2022-03-11 | 2022-07-01 | 三一智矿科技有限公司 | Mining area dust filtering method, device, equipment and medium based on point cloud |
| CN116485674A (en) * | 2023-04-24 | 2023-07-25 | 智道网联科技(北京)有限公司 | Real-time denoising method and device for laser point cloud data, electronic equipment and storage medium |
| CN117572451A (en) * | 2024-01-11 | 2024-02-20 | 广州市杜格科技有限公司 | Traffic information acquisition method, equipment and medium based on multi-line laser radar |
Non-Patent Citations (2)
| Title |
|---|
| 基于三维激光雷达和深度图像的自动驾驶汽车障碍物检测方法;王新竹;李骏;李红建;尚秉旭;;吉林大学学报(工学版);20160315(第02期);全文 * |
| 铁路场景三维点云分割与分类识别算法;郭保青;余祖俊;张楠;朱力强;高晨光;;仪器仪表学报;20170915(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117830140A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111582054B (en) | Point cloud data processing method and device and obstacle detection method and device | |
| CN111210429A (en) | Point cloud data partitioning method and device and obstacle detection method and device | |
| CN109948635B (en) | Target identification method and device based on laser scanning | |
| CN110530375B (en) | Robot adaptive positioning method, positioning device, robot and storage medium | |
| CN113537133B (en) | Information processing method and device, electronic equipment, storage medium and product | |
| CN117830140B (en) | Denoising method and device for foggy weather point cloud for unmanned control system | |
| CN108345007B (en) | Obstacle identification method and device | |
| CN112562419A (en) | Off-line multi-target tracking-based weather avoidance zone setting method | |
| CN110363847B (en) | Map model construction method and device based on point cloud data | |
| CN116486130A (en) | Obstacle recognition method, device, self-mobile device and storage medium | |
| EP4080479A2 (en) | Method for identifying traffic light, device, cloud control platform and vehicle-road coordination system | |
| CN113920273B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN112435475B (en) | Traffic state detection method, device, equipment and storage medium | |
| CN113313654A (en) | Laser point cloud filtering and denoising method, system, equipment and storage medium | |
| CN112132215B (en) | Method, device and computer readable storage medium for identifying object type | |
| CN113610059A (en) | Vehicle control method and device based on regional assessment and intelligent traffic management system | |
| CN117392000B (en) | Noise removing method and device, electronic equipment and storage medium | |
| CN118038415A (en) | Laser radar-based vehicle identification method, device, medium and electronic equipment | |
| CN113763260B (en) | Water drop noise-based noise filtering method, device, equipment and storage medium | |
| CN115345919B (en) | Depth determination method and device, electronic equipment and storage medium | |
| CN113688721B (en) | Method and device for fitting lane lines | |
| CN117970269A (en) | Noise target filtering method and device, electronic equipment and storage medium | |
| CN114863148A (en) | Target identification method based on millimeter wave radar and terminal equipment | |
| CN115439830A (en) | Information processing method, apparatus, device, medium, vehicle, and program product | |
| CN117095380A (en) | Method and device for removing obstacle point cloud from point cloud map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |