CN112085843B - Tunnel class target feature real-time extraction and measurement method and device - Google Patents
Tunnel class target feature real-time extraction and measurement method and device Download PDFInfo
- Publication number
- CN112085843B CN112085843B CN202010854777.XA CN202010854777A CN112085843B CN 112085843 B CN112085843 B CN 112085843B CN 202010854777 A CN202010854777 A CN 202010854777A CN 112085843 B CN112085843 B CN 112085843B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- boundary
- clouds
- tunnel
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims description 10
- 238000000691 measurement method Methods 0.000 title claims description 4
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 3
- 238000009434 installation Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 125000000205 L-threonino group Chemical group [H]OC(=O)[C@@]([H])(N([H])[*])[C@](C([H])([H])[H])([H])O[H] 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000295 fuel oil Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Theoretical Computer Science (AREA)
- Remote Sensing (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application provides a method and a device for extracting and measuring tunnel target characteristics in real time, which solve the technical problem that the existing tunnel target characteristics are not easy to extract and measure. Comprising the following steps: eliminating the interference point cloud to obtain an effective point cloud; fitting the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle; compensating pitch angles for the rest effective point clouds after removing the ground point clouds; extracting a characteristic point cloud in the effective point cloud, and projecting the characteristic point cloud on a projection plane to generate a section point cloud; identifying a demarcation point cloud in the section point cloud, and distinguishing the section point cloud by taking the demarcation point cloud as a demarcation point; constructing a mathematical description model of the tunnel type target section according to the distinguished section point cloud; and calculating the height and width of the tunnel type target according to the mathematical description model. The application can extract the tunnel target characteristics in real time in the running process of the vehicle, simultaneously realize the mathematical description of the tunnel target characteristics and the measurement of the height and the width, meet the requirement of advanced pre-judgment and assist the driving of the vehicle.
Description
Technical Field
The application relates to the technical field of intelligent driving, in particular to a method and a device for extracting and measuring tunnel target characteristics in real time.
Background
Along with the continuous development of science and technology, automobile manufacturing and information technology development, the travel tool for people to travel is gradually changed from a carriage to a bicycle, a fuel oil automobile and an electric automobile, and intelligent driving is improved to a new schedule nowadays.
In the road conditions of automobile driving, tunnel-type target height-limiting road infrastructures such as culverts, tunnels and bridges are important factors influencing the traffic of vehicles and are also hot spots for research in the intelligent driving field. The detection and extraction of the tunnel type target height limiting facility characteristics are realized, and the safety and reliability of vehicle running can be effectively improved. However, the tunnel target features are not easy to extract and measure, and the technology for extracting and measuring the tunnel target features is also lacking in the prior art.
Disclosure of Invention
In view of the above problems, the embodiments of the present application provide a method for extracting and measuring tunnel target features in real time, which solves the technical problem that the existing tunnel target features are not easy to extract and measure.
In a first aspect, the present application provides a method for extracting and measuring tunnel target features in real time, including:
eliminating interference point clouds in three-dimensional point clouds acquired by the multi-line laser radar to obtain effective point clouds;
selecting a point cloud with the effective point cloud height of Cheng Zui to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
compensating pitch angles for the rest effective point clouds after removing the ground point clouds;
extracting point clouds with upward emission angles in the effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
identifying boundary point clouds in the section point clouds, and dividing the section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
constructing a mathematical description model of the tunnel type target section according to the distinguished section point cloud;
and calculating the height and width of the tunnel type target according to the mathematical description model.
In an embodiment, the removing the interference point cloud from the three-dimensional point cloud acquired by the multi-line laser radar, and obtaining the effective point cloud include:
determining not to be a redundant point cloud in the vehicle front area;
determining a point cloud with a data value not being the NAF as an invalid point cloud;
determining the point cloud with surrounding non-adjacent point clouds as an isolated point cloud;
and eliminating the redundant point cloud, the invalid point cloud and the isolated point cloud to obtain the valid point cloud.
In an embodiment, the selecting a point cloud with a height of Cheng Zui hours in the effective point cloud to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle includes:
rasterizing the effective point cloud data space, wherein each grid is a cuboid with the same bottom surface and the height changing according to the point cloud space distribution;
the point clouds in each grid are ordered according to the elevation, the lowest point cloud in each grid is compared with the lowest point cloud of the adjacent grids, the point cloud with the lowest elevation is obtained, and the point cloud with the lowest point cloud distance less than the threshold value range is used as the ground point cloud;
and carrying out plane fitting on the ground point cloud, solving a normal vector of a fitting plane, and solving a pitch angle according to the normal vector of the fitting plane.
In an embodiment, extracting a point cloud with an upward emission angle in the effective point cloud after the pitch angle compensation to form a characteristic point cloud of the tunnel-type target, and projecting the characteristic point cloud on a projection plane parallel to the axle and perpendicular to the advancing direction to generate a section point cloud includes:
determining a multi-line laser radar coordinate system XOY plane as a radar plane;
calculating an included angle theta of a single-beam laser point cloud and a radar plane in the effective point cloud
Screening a single-beam laser point cloud with an included angle theta between 10 and 15 degrees as a characteristic point cloud according to the radar installation angle and radar vertical field angle parameters;
and projecting the characteristic point cloud onto a plane perpendicular to the axle, and generating a cross-section point cloud of the tunnel type target.
In one embodiment, identifying the boundary point cloud in the cross-section point cloud, and dividing the cross-section point cloud into a top boundary point cloud, a left boundary point cloud, and a right boundary point cloud with the boundary point cloud as the boundary point includes:
selecting three adjacent point clouds, namely an upper point cloud, a middle point cloud and a lower point cloud, from the cross-section point clouds to form a scanning triangle, and dividing the cross-section point clouds into a left part and a right part for scanning according to the clockwise direction and the anticlockwise direction respectively from the front of the vehicle; calculating the angle of the vertex of the scanning triangle by taking the middle point cloud as the vertex;
taking the point with abrupt change of the angle value of the vertex of the scanning triangle as the demarcation point cloud in the section point cloud;
taking the point cloud before the demarcation point as the top boundary point cloud of the tunnel type target, taking the point cloud after the left half boundary point cloud as the left boundary point cloud, and taking the point cloud after the right half boundary point cloud as the right boundary point cloud;
the top boundary point cloud, the left boundary point cloud, and the right boundary point cloud are stored in a top boundary container, a left boundary container, and a right boundary container, respectively.
In an embodiment, the constructing a mathematical description model of the tunnel-type target section according to the differentiated section point cloud includes:
performing quadratic curve fitting on the top boundary point cloud to obtain a top boundary curve parameter U (a) 1 ,b 1 ,c)，U(a 1 ,b 1 C) represents the top conic coefficient, constructs a top conic model y=a 1 x 2 +b 1 x+c；
Performing straight line fitting on the left boundary point cloud and the right boundary point cloud to obtain a left boundary curve parameter L (a) 2 ,b 2 ) And R (a) 3 ,b 3 ) Wherein L (a) 2 ,b 2 ) Representing the left straight line coefficient; r (a) 3 ,b 3 ) Right straight line coefficient, constructing left straight line model y=a 2 x+b 2 And right straight line model y=a 3 x+b 3 。
In one embodiment, the calculating the height and width of the tunnel-like object according to the mathematical description model includes:
calculating a target width according to a left-right demarcation line model and a formula width=fabs (L_b-R_b);
wherein: l_b represents the left boundary straight line constant term coefficient; r_b represents right boundary straight line constant term coefficients, respectively.
According to the top demarcation curve model, according to the formula:calculating the target height;
wherein: h is a 0 Is the height of the multi-line laser radar from the ground.
In a second aspect, the present application provides a device for extracting and measuring tunnel target features in real time, where the device includes:
and the point cloud preprocessing module is used for: eliminating interference point clouds in three-dimensional point clouds acquired by the multi-line laser radar to obtain effective point clouds;
the gradient calculation module: the method comprises the steps of selecting a point cloud with the effective point cloud height of Cheng Zui to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
and the compensation module is used for: the method comprises the steps of compensating a pitch angle for the rest effective point cloud after removing the ground point cloud;
and an extraction module: the method comprises the steps of extracting point clouds with upward emission angles in effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
and a classification module: the method comprises the steps of identifying boundary point clouds in the cross-section point clouds, and dividing the cross-section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
the construction module comprises: the mathematical description model is used for constructing a tunnel type target section according to the distinguished section point cloud;
the calculation module: for calculating the height and width of the tunnel-like object from the mathematical description model.
In a third aspect, the present application provides an electronic device comprising:
a processor, a memory, an interface in communication with the gateway;
the memory is used for storing programs and data, and the processor calls the programs stored in the memory to execute the tunnel type target feature real-time extraction and measurement method provided by any one of the first aspects.
In a fourth aspect, the present application provides a computer readable storage medium comprising a program which, when executed by a processor, is adapted to carry out the method for real-time extraction and measurement of tunnel-like object features provided in any of the first aspects.
From the above description, the embodiment of the application provides a method and a device for extracting and measuring the tunnel target characteristics in real time.
Drawings
Fig. 1 is a flow chart of a method for extracting and measuring tunnel target features in real time according to an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating identifying a demarcation point cloud among a cross-section point cloud according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a device for extracting and measuring characteristics of a tunnel class object in real time according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the application.
Detailed Description
The present application will be further described with reference to the drawings and the detailed description below, in order to make the objects, technical solutions and advantages of the present application more apparent. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The method for extracting and measuring the tunnel type target characteristics in real time in the embodiment of the application is shown in figure 1. In fig. 1, the method comprises:
s101, eliminating interference point clouds in three-dimensional point clouds acquired by a multi-line laser radar to obtain effective point clouds;
specifically, the multi-line laser radar is arranged on the roof, three-dimensional point cloud information in front of the vehicle is acquired through the multi-line laser radar, but the three-dimensional point cloud acquired by the multi-line laser radar is doped with redundant point cloud, invalid point cloud or isolated point cloud and other interference point clouds, the three-dimensional point cloud acquired by the multi-line laser radar is preprocessed in the step to remove the interference point cloud, effective point cloud is acquired, subsequent calculation amount is reduced, and accuracy of the acquired point cloud is improved.
S102, selecting a point cloud with the height of Cheng Zui small in the effective point cloud to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
specifically, the preprocessed point cloud comprises a ground point cloud and a point cloud of tunnel type target characteristics above the ground, the actual running road of the vehicle generally has a gradient, a certain included angle exists between a multi-line laser radar measuring plane and a road plane, and further errors occur in the tunnel type target measuring result, so that the gradient (namely a pitch angle) of the road needs to be calculated, the ground point cloud is subjected to plane fitting to obtain a fitting plane aiming at the ground plane, and the gradient of the road can be further obtained by calculating the normal vector of the fitting plane.
S103, compensating a pitch angle for the rest effective point clouds after removing the ground point clouds;
specifically, pitch angle compensation is carried out on the effective point cloud which is remained after the ground point cloud is removed according to the road gradient obtained in the previous step, pitch angle compensation is carried out on the effective point cloud obtained by the multi-line laser radar, the included angle between the multi-line laser radar measuring plane and the road plane is reduced, and then the measuring error is reduced.
S104, extracting point clouds with upward emission angles in the effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
specifically, as the multi-line laser radar is installed on the roof and has a certain distance from the ground, the radar plane of the multi-line laser radar is higher than the ground, and the point cloud below the radar plane detects ground information and is not tunnel target characteristic information; the point cloud information above the radar plane is effective characteristic point cloud for describing tunnel type target characteristics, the characteristic point cloud comprises three-dimensional information of tunnel type targets in front of the vehicle, the characteristic point cloud is projected on a plane parallel to an axle and perpendicular to the advancing direction, and therefore the cross section point cloud comprising the cross section information of the tunnel type targets is integrated on the plane.
S105, identifying boundary point clouds in the cross-section point clouds, and dividing the cross-section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
specifically, it can be understood that the cross section of the tunnel type object is composed of two boundaries and a top edge connecting the two boundaries, and how to extract or identify the characteristic information of the tunnel type object represented by each part in the cross section point cloud needs to distinguish the side boundary from the top boundary, and the side boundary and the top boundary have obvious boundaries, so that it can be determined through the boundary point cloud which of the cross section point clouds belongs to the top boundary point cloud and which of the cross section point clouds belongs to the left boundary point cloud and the right boundary point cloud.
S106, constructing a mathematical description model of the tunnel type target section according to the distinguished section point clouds;
specifically, a mathematical description model of the tunnel type target geometric description can be constructed by adopting ceres curve fitting according to the top boundary point cloud, the left boundary point cloud and the right boundary point cloud which are obtained in the last step.
And S107, calculating the height and the width of the tunnel type target according to the mathematical description model.
Specifically, according to a mathematical description model of the geometric description of the tunnel type target, the width of the tunnel type target can be measured by calculating the distance between the left boundary point cloud and the right boundary point cloud, and the height of the tunnel type target can be measured by calculating the height of the highest point of the top boundary point cloud.
In the embodiment, the tunnel type target characteristics can be extracted in real time in the vehicle driving process, and meanwhile, the mathematical description of the tunnel type target characteristics and the measurement of the height and the width of the tunnel type target are realized, so that the requirement of intelligent driving advance pre-judgment is met, the vehicle driving is assisted, and the safety and the reliability of the vehicle driving are improved.
Based on the above embodiment, as a preferred embodiment, step 101 specifically includes the following steps:
determining not to be a redundant point cloud in the vehicle front area;
specifically, in order to meet the advanced prognosis of the vehicle running, the point cloud that is not in the range of 30m×20m in front of the vehicle head is determined as the redundant point cloud.
Determining a point cloud with a data value not being the NAF as an invalid point cloud;
specifically, the NAF is a non-adjacent representation, and the point cloud with the point cloud data value type not belonging to the NAF is determined as an invalid point cloud.
Determining the point cloud with surrounding non-adjacent point clouds as an isolated point cloud;
specifically, some point clouds in the point clouds acquired by the multi-line laser radar have larger distance from adjacent point clouds, cannot establish connection with the adjacent point clouds, influence the clustering algorithm result, and can be regarded as isolated point clouds from the point clouds with the distance of more than 0.3 m.
And eliminating the redundant point cloud, the invalid point cloud and the isolated point cloud to obtain the valid point cloud.
In this embodiment, the point cloud information acquired by the multi-line laser radar is preprocessed, and the redundant point clouds, the invalid point clouds, the isolated point clouds and other interference point clouds in the acquired point clouds are removed, and the interference point clouds do not participate in subsequent tunnel class target feature description and width and height measurement, so that the precision and accuracy of the tunnel class target feature description and width and height can be improved.
Based on the above embodiment, as a preferred embodiment, step 102 specifically includes the following steps:
rasterizing the effective point cloud data space, wherein each grid is a cuboid with the same bottom surface and the height changing according to the point cloud space distribution;
the point clouds in each grid are ordered according to the elevation, the lowest point cloud in each grid is compared with the lowest point cloud of the adjacent grids, the point cloud with the lowest elevation is obtained, and the point cloud with the lowest point cloud distance less than the threshold value range is used as the ground point cloud;
and carrying out plane fitting on the ground point cloud, solving a normal vector of a fitting plane, and solving a pitch angle according to the normal vector of the fitting plane.
In this embodiment, due to the gradient of the actual road, an included angle exists between the radar plane of the multi-line laser radar and the road plane, which causes errors between the tunnel type target measurement height and the actual tunnel height. Therefore, the pitch angle of the point cloud needs to be compensated, and the generation of errors is reduced. When the point cloud is compensated, the road gradient needs to be measured, and the road gradient can be quickly obtained through the steps.
Based on the above embodiment, as a preferred embodiment, step 104 specifically includes the following steps:
determining a multi-line laser radar coordinate system XOY plane as a radar plane;
specifically, the multi-line radar coordinate system is: the radar body is used as an origin, the right front of the radar body is used as a Y axis, the right side of the radar body is used as an X axis, the Z axis faces upwards, and the XOY plane of the multi-line laser radar coordinate system is defined as a radar plane.
Calculating an included angle theta of a single-beam laser point cloud and a radar plane in the effective point cloud
Wherein: p is p ix 、p iy 、p iz Respectively represent coordinate values of the ith point cloud.
Screening a single-beam laser point cloud with an included angle theta between 10 and 15 degrees as a characteristic point cloud according to the radar installation angle and radar vertical field angle parameters;
and projecting the characteristic point cloud onto a plane perpendicular to the axle, and generating a cross-section point cloud of the tunnel type target.
In the embodiment, whether the emission angle of the single-beam laser is above the radar plane is judged by calculating the included angle between the single-beam laser and the radar plane, the single-beam laser point cloud with the emission angle above the radar plane is extracted from the compensated residual effective point cloud, the ground point cloud acquired by the multi-line laser radar is eliminated, the calculated amount is reduced, and meanwhile, the influence of the ground point cloud on a measurement result is eliminated. And obtaining characteristic point clouds of tunnel targets above the radar plane after eliminating the ground point clouds, projecting the characteristic point clouds onto a plane perpendicular to the axle, obtaining cross-section point clouds of the tunnel targets, and further obtaining the information of the cross sections of the tunnel targets.
Based on the above embodiment, as a preferred embodiment, the step 105 specifically includes the following steps:
selecting three adjacent point clouds, namely an upper point cloud, a middle point cloud and a lower point cloud, from the cross-section point clouds to form a scanning triangle, and dividing the cross-section point clouds into a left part and a right part for scanning according to the clockwise direction and the anticlockwise direction respectively from the front of the vehicle; calculating the angle of the vertex of the scanning triangle by taking the middle point cloud as the vertex;
specifically, the cross section point cloud is actually consistent with the cross section of the tunnel type target, and the identification speed can be improved by dividing the cross section point cloud into two parts for scanning. The shape of the cross-section point cloud is shown in fig. 2, three adjacent point clouds A, B, C in the cross-section point cloud are selected, the point cloud B is selected as a vertex, and the angle value of the vertex B can be calculated according to the triangle cosine theorem.
Taking the point with abrupt change of the angle value of the vertex of the scanning triangle as the demarcation point cloud in the section point cloud;
specifically, it will be appreciated that the cross section of the tunnel-like object is actually composed of left and right sides and a top edge, and the connection between the left and right sides and the top edge is actually a demarcation point. The angle value of the vertex B of the scanning triangle also changes along with the scanning triangle in the scanning process, when the boundary of the top edge and the side edge is scanned, the angle value of the vertex is suddenly changed, and the point cloud with the suddenly changed angle value can be identified as the boundary point cloud.
Taking the point cloud before the demarcation point as the top boundary point cloud of the tunnel type target, taking the point cloud after the left half boundary point cloud as the left boundary point cloud, and taking the point cloud after the right half boundary point cloud as the right boundary point cloud;
the top boundary point cloud, the Left boundary point cloud and the Right boundary point cloud are respectively stored in a top boundary container U_lanes, a Left boundary container left_lanes and a Right boundary container right_lanes.
Specifically, the top boundary container u_lanes, the Left boundary container left_lanes and the Right boundary container right_lanes are actually point cloud arrays, and are used for storing classified point clouds to form a point cloud set with corresponding attributes, so that the top boundary point cloud, the Left boundary point cloud and the Right boundary point cloud can be clustered and fitted conveniently.
In this embodiment, the acquired cross-section point clouds identify the demarcation point clouds through scanning triangles, and then classify the cross-section point clouds through the demarcation point clouds, and store the classified cross-section point clouds in the boundary containers corresponding to the respective categories, so that the cross-section data of the tunnel targets are quickly identified, the cross-section data of the tunnel targets are classified and stored, the characteristics of the tunnel targets are classified and extracted, and the analysis of the cross-section data of the tunnel targets is facilitated.
Based on the above embodiment, as a preferred embodiment, step 106 specifically includes the following steps:
performing quadratic curve fitting on the top boundary point cloud to obtain a top boundary curve parameter U (a) 1 ,b 1 ,c)，U(a 1 ,b 1 C) represents the top conic coefficient, constructs a top conic model y=a 1 x 2 +b 1 x+c；
It can be understood that the top boundary of the tunnel type target is mostly parabolic, so that the top boundary point cloud is subjected to quadratic curve fitting by adopting a thres curve fitting method, and a quadratic curve model is constructed for the quadratic curve fitting method.
Performing straight line fitting on the left boundary point cloud and the right boundary point cloud to obtain a left boundary curve parameter L (a) 2 ,b 2 ) And R (a) 3 ,b 3 ) Wherein L (a) 2 ,b 2 ) Representing the left straight line coefficient; r (a) 3 ,b 3 ) Right straight line coefficient, constructing left straight line model y=a 2 x+b 2 And right straight line model y=a 3 x+b 3 ；
It will be appreciated that the left and right side boundaries of tunnel-like objects are mostly in the form of straight lines, so that straight line fitting is performed on the side boundaries, and a straight line model is constructed for this purpose.
In the embodiment, the accurate mathematical description of the tunnel targets is realized by a mathematical fitting method, and an accurate external operation environment description data source is provided for intelligent driving of the vehicle.
Based on the above embodiment, as a preferred embodiment, step 107 specifically includes the following steps:
according to the left-right demarcation line model, the formula width=fabs (l_b 2 -R_b 3 ) Calculating a target width;
wherein: l_b 2 Representing the coefficients of the left boundary linear constant term; r_b 3 Respectively represent right boundary straight line constant term coefficients.
According to the top demarcation curve model, according to the formula:calculating the target height;
wherein: h is a 0 Is the height of the multi-line laser radar from the ground.
It will be appreciated that the height of the multiline lidar from the ground is known data, which is primarily determined by the vehicle height.
In the embodiment, the measurement of the width and the height of the tunnel type targets is realized through the calculation of parameters in the mathematical description formed by the tunnel type targets, the perceptibility of intelligent driving on the tunnel type targets is improved, and the safety of vehicle driving is ensured.
In conclusion, the method and the device realize real-time measurement of the height and the width of the tunnel category, and the data updating frequency reaches 10HZ; the device has advanced measurement capability, and the measurement accuracy can reach 0.5m; the use condition is not changed by the ambient light, and the device can work normally in the environments such as backlight, night and the like.
Based on the same inventive concept, the embodiment of the application also provides a device for extracting and measuring the tunnel type target characteristics in real time, which can be used for realizing the method for extracting and measuring the tunnel type target characteristics in real time described in the embodiment, as described in the following embodiment. Because the principle of solving the problem of the device for extracting and measuring the tunnel target characteristics in real time is similar to that of the method for extracting and measuring the tunnel target characteristics in real time, the implementation of the device for extracting and measuring the tunnel target characteristics in real time can be implemented by referring to the method, and the repetition is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While a tunnel-type target feature real-time extraction and measurement apparatus described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and contemplated.
As shown in fig. 2, an embodiment of the present application provides a device for extracting and measuring tunnel target features in real time, which includes:
point cloud preprocessing module 201: eliminating interference point clouds in three-dimensional point clouds acquired by the multi-line laser radar to obtain effective point clouds;
gradient calculation module 202: the method comprises the steps of selecting a point cloud with the effective point cloud height of Cheng Zui to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
the compensation module 203: the method comprises the steps of compensating a pitch angle for the rest effective point cloud after removing the ground point cloud;
extraction module 204: the method comprises the steps of extracting point clouds with upward emission angles in effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
classification module 205: the method comprises the steps of identifying boundary point clouds in the cross-section point clouds, and dividing the cross-section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
the construction module 206: the mathematical description model is used for constructing a tunnel type target section according to the distinguished section point cloud;
calculation module 207: for calculating the height and width of the tunnel-like object from the mathematical description model.
The embodiment of the present application further provides a specific implementation manner of an electronic device capable of implementing all the steps in the method for extracting and measuring tunnel target features in real time in the foregoing embodiment, and referring to fig. 3, the electronic device 300 specifically includes the following contents:
a processor 310, a memory 320, a communication unit 330, and a bus 340;
wherein the processor 310, the memory 320, and the communication unit 330 perform communication with each other through the bus 340; the communication unit 330 is configured to implement information transmission between the server-side device and the terminal device.
The processor 310 is configured to invoke a computer program in the memory 320, and when the processor executes the computer program, implement all the steps in the method for extracting and measuring the characteristics of the tunnel class target in real time in one of the above embodiments.
Those of ordinary skill in the art will appreciate that: the Memory may be, but is not limited to, random access Memory (Random Access Memory; RAM; ROM; programmable Read-Only Memory; PROM; erasable ROM; erasable Programmable Read-Only Memory; EPROM; electrically erasable ROM; electric Erasable Programmable Read-Only Memory; EEPROM; etc.). The memory is used for storing a program, and the processor executes the program after receiving the execution instruction.
Further, the software programs and modules within the memory may also include an operating system, which may include various software components and/or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.
The processor may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a Network Processor (NP), and the like. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The present application also provides a computer readable storage medium comprising a program which, when executed by a processor, is adapted to carry out a method for extracting and measuring tunnel class target features in real time, provided by any of the foregoing method embodiments.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic or optical disks may store the program code, and the application is not limited by the specific type of media.
The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims (9)
1. A method for extracting and measuring tunnel target features in real time is characterized by comprising the following steps:
eliminating interference point clouds in three-dimensional point clouds acquired by the multi-line laser radar to obtain effective point clouds;
selecting a point cloud with the effective point cloud height of Cheng Zui to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
compensating pitch angles for the rest effective point clouds after removing the ground point clouds;
extracting point clouds with upward emission angles in the effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
identifying boundary point clouds in the section point clouds, and dividing the section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
constructing a mathematical description model of the tunnel type target section according to the distinguished section point cloud;
the construction of the mathematical description model of the tunnel type target section according to the distinguished section point cloud comprises the following steps:
performing quadratic curve fitting on the top boundary point cloud to obtain a top boundary curve parameter U (a) 1 ,b 1 ,c)，U(a 1 ,b 1 C) represents the top conic coefficient, constructs a top conic model y=a 1 x 2 +b 1 x+c；
Performing straight line fitting on the left boundary point cloud and the right boundary point cloud to obtain a left boundary curve parameter L (a) 2 ,b 2 ) And R (a) 3 ,b 3 ) Wherein L (a) 2 ,b 2 ) Representing the left straight line coefficient; r (a) 3 ,b 3 ) Representing the right straight line coefficient, constructing a left straight line model y=a 2 x+b 2 And right straight line model y=a 3 x+b 3 ；
And calculating the height and width of the tunnel type target according to the mathematical description model.
2. The method for extracting and measuring tunnel target features in real time according to claim 1, wherein the step of eliminating the interference point cloud in the three-dimensional point cloud acquired by the multi-line laser radar to obtain an effective point cloud comprises the steps of:
determining not to be a redundant point cloud in the vehicle front area;
determining a point cloud with a data value not being the NAF as an invalid point cloud;
determining the point cloud with surrounding non-adjacent point clouds as an isolated point cloud;
and eliminating the redundant point cloud, the invalid point cloud and the isolated point cloud to obtain the valid point cloud.
3. The method for extracting and measuring tunnel target features in real time according to claim 1, wherein the selecting a point cloud with a height of Cheng Zui hours in the effective point cloud to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle comprises:
rasterizing the effective point cloud data space, wherein each grid is a cuboid with the same bottom surface and the height changing according to the point cloud space distribution;
the point clouds in each grid are ordered according to the elevation, the lowest point cloud in each grid is compared with the lowest point cloud of the adjacent grids, the point cloud with the lowest elevation is obtained, and the point cloud with the lowest point cloud distance less than the threshold value range is used as the ground point cloud;
and carrying out plane fitting on the ground point cloud, solving a normal vector of a fitting plane, and solving a pitch angle according to the normal vector of the fitting plane.
4. The method for extracting and measuring characteristics of tunnel targets in real time according to claim 1, wherein extracting the point clouds with upward emission angles in the effective point clouds after pitch angle compensation to form characteristic point clouds of the tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to a forward direction to generate the section point clouds comprises:
determining a multi-line laser radar coordinate system XOY plane as a radar plane;
calculating an included angle theta of a single-beam laser point cloud and a radar plane in the effective point cloud
Screening a single-beam laser point cloud with an included angle theta between 10 and 15 degrees as a characteristic point cloud according to the radar installation angle and radar vertical field angle parameters;
and projecting the characteristic point cloud onto a plane perpendicular to the axle, and generating a cross-section point cloud of the tunnel type target.
5. The method for real-time extracting and measuring tunnel class target features according to claim 1, wherein identifying the boundary point cloud in the cross-section point cloud, and dividing the cross-section point cloud into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by using the boundary point cloud as the boundary point comprises:
selecting three adjacent point clouds, namely an upper point cloud, a middle point cloud and a lower point cloud, from the cross-section point clouds to form a scanning triangle, and dividing the cross-section point clouds into a left part and a right part for scanning according to the clockwise direction and the anticlockwise direction respectively from the front of the vehicle; calculating the angle of the vertex of the scanning triangle by taking the middle point cloud as the vertex;
taking the point with abrupt change of the angle value of the vertex of the scanning triangle as the demarcation point cloud in the section point cloud;
taking the point cloud before the demarcation point as the top boundary point cloud of the tunnel type target, taking the point cloud after the left half boundary point cloud as the left boundary point cloud, and taking the point cloud after the right half boundary point cloud as the right boundary point cloud;
the top boundary point cloud, the left boundary point cloud, and the right boundary point cloud are stored in a top boundary container, a left boundary container, and a right boundary container, respectively.
6. The method for real-time extracting and measuring tunnel class object features according to claim 1, wherein said calculating the height and width of the tunnel class object according to the mathematical description model comprises:
calculating a target width according to a left-right demarcation line model and a formula width=fabs (L_b-R_b);
wherein: l_b represents the left boundary straight line constant term coefficient; r_b represents right boundary straight line constant term coefficients respectively;
according to the top demarcation curve model, according to the formula:calculating the target height;
wherein: h is a 0 Is the height of the multi-line laser radar from the ground.
7. The utility model provides a tunnel class target feature draws and measuring device in real time which characterized in that, this device includes:
and the point cloud preprocessing module is used for: eliminating interference point clouds in three-dimensional point clouds acquired by the multi-line laser radar to obtain effective point clouds;
the gradient calculation module: the method comprises the steps of selecting a point cloud with the effective point cloud height of Cheng Zui to form a ground point cloud, performing plane fitting on the ground point cloud, and calculating a normal vector of a fitting plane to solve a pitch angle;
and the compensation module is used for: the method comprises the steps of compensating a pitch angle for the rest effective point cloud after removing the ground point cloud;
and an extraction module: the method comprises the steps of extracting point clouds with upward emission angles in effective point clouds after pitch angle compensation to form characteristic point clouds of tunnel targets, and projecting the characteristic point clouds on a projection plane parallel to an axle and perpendicular to the advancing direction to generate cross-section point clouds;
and a classification module: the method comprises the steps of identifying boundary point clouds in the cross-section point clouds, and dividing the cross-section point clouds into a top boundary point cloud, a left boundary point cloud and a right boundary point cloud by taking the boundary point clouds as boundary points;
the construction module comprises: the mathematical description model is used for constructing a tunnel type target section according to the distinguished section point cloud;
the construction of the mathematical description model of the tunnel type target section according to the distinguished section point cloud comprises the following steps:
performing quadratic curve fitting on the top boundary point cloud to obtain a top boundary curve parameter U (a) 1 ,b 1 ,c)，U(a 1 ,b 1 C) represents the top conic coefficient, constructs a top conic model y=a 1 x 2 +b 1 x+c；
Performing straight line fitting on the left boundary point cloud and the right boundary point cloud to obtain a left boundary curve parameter L (a) 2 ,b 2 ) And R (a) 3 ,b 3 ) Wherein L (a) 2 ,b 2 ) Representing the left straight line coefficient; r (a) 3 ,b 3 ) Representing the right straight line coefficient, constructing a left straight line model y=a 2 x+b 2 And right straight line model y=a 3 x+b 3 ；
The calculation module: for calculating the height and width of the tunnel-like object from the mathematical description model.
8. An electronic device, comprising:
a processor, a memory, an interface in communication with the gateway;
the memory is used for storing programs and data, and the processor calls the programs stored in the memory to execute the method for extracting and measuring the tunnel class target characteristics in real time according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a program for performing the tunnel class target feature real-time extraction and measurement method of any one of claims 1 to 6 when being executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010854777.XA CN112085843B (en) | 2020-08-24 | 2020-08-24 | Tunnel class target feature real-time extraction and measurement method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010854777.XA CN112085843B (en) | 2020-08-24 | 2020-08-24 | Tunnel class target feature real-time extraction and measurement method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112085843A CN112085843A (en) | 2020-12-15 |
| CN112085843B true CN112085843B (en) | 2023-11-28 |
Family
ID=73728517
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010854777.XA Active CN112085843B (en) | 2020-08-24 | 2020-08-24 | Tunnel class target feature real-time extraction and measurement method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112085843B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112904315B (en) * | 2021-01-12 | 2024-04-26 | 广州广电研究院有限公司 | Laser radar point cloud data correction method, device and medium |
| CN113283494B (en) * | 2021-05-21 | 2023-07-14 | 福建盛海智能科技有限公司 | Ground pit detection method and terminal |
| CN115656962B (en) * | 2022-12-26 | 2023-03-31 | 南京楚航科技有限公司 | Method for identifying height-limited object based on millimeter wave radar |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101078627A (en) * | 2007-06-28 | 2007-11-28 | 北京航空航天大学 | On-line calibration method for shield machine automatic guiding system based on optical fiber gyro and PSD laser target |
| JP2011069747A (en) * | 2009-09-26 | 2011-04-07 | Sohatsu System Kenkyusho:Kk | Tunnel data processing system and tunnel data processing method |
| CN107393006A (en) * | 2017-07-26 | 2017-11-24 | 河海大学 | A kind of method for weighing tunnel overall deformation |
| CN108920750A (en) * | 2018-05-24 | 2018-11-30 | 武汉八维时空信息技术股份有限公司 | The fusion of engineering-built Dynamic and Multi dimensional information and cooperation interaction system |
| WO2019026114A1 (en) * | 2017-07-31 | 2019-02-07 | 三菱電機株式会社 | Structure measurement device, measurement point correction device, and measurement point correction method |
| CN109359409A (en) * | 2018-10-31 | 2019-02-19 | 张维玲 | A kind of vehicle passability detection system of view-based access control model and laser radar sensor |
| CN110737977A (en) * | 2019-10-10 | 2020-01-31 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | tunnel surrounding rock deformation prediction method and prediction device |
| CN110765576A (en) * | 2019-09-19 | 2020-02-07 | 西南交通大学 | Three-dimensional space-time fluctuation analysis method for tunnel seismic safety assessment |
| CN111145345A (en) * | 2019-12-31 | 2020-05-12 | 山东大学 | Tunnel construction area three-dimensional model construction method and system |
| CN111524103A (en) * | 2020-04-10 | 2020-08-11 | 山东科技大学 | Circular tunnel central axis extraction method based on three-dimensional laser point cloud |
| CN111561878A (en) * | 2020-05-06 | 2020-08-21 | 上海市建筑科学研究院有限公司 | Point cloud error correction method based on mobile 3D laser scanning |
-
2020
- 2020-08-24 CN CN202010854777.XA patent/CN112085843B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101078627A (en) * | 2007-06-28 | 2007-11-28 | 北京航空航天大学 | On-line calibration method for shield machine automatic guiding system based on optical fiber gyro and PSD laser target |
| JP2011069747A (en) * | 2009-09-26 | 2011-04-07 | Sohatsu System Kenkyusho:Kk | Tunnel data processing system and tunnel data processing method |
| CN107393006A (en) * | 2017-07-26 | 2017-11-24 | 河海大学 | A kind of method for weighing tunnel overall deformation |
| WO2019026114A1 (en) * | 2017-07-31 | 2019-02-07 | 三菱電機株式会社 | Structure measurement device, measurement point correction device, and measurement point correction method |
| CN108920750A (en) * | 2018-05-24 | 2018-11-30 | 武汉八维时空信息技术股份有限公司 | The fusion of engineering-built Dynamic and Multi dimensional information and cooperation interaction system |
| CN109359409A (en) * | 2018-10-31 | 2019-02-19 | 张维玲 | A kind of vehicle passability detection system of view-based access control model and laser radar sensor |
| CN110765576A (en) * | 2019-09-19 | 2020-02-07 | 西南交通大学 | Three-dimensional space-time fluctuation analysis method for tunnel seismic safety assessment |
| CN110737977A (en) * | 2019-10-10 | 2020-01-31 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | tunnel surrounding rock deformation prediction method and prediction device |
| CN111145345A (en) * | 2019-12-31 | 2020-05-12 | 山东大学 | Tunnel construction area three-dimensional model construction method and system |
| CN111524103A (en) * | 2020-04-10 | 2020-08-11 | 山东科技大学 | Circular tunnel central axis extraction method based on three-dimensional laser point cloud |
| CN111561878A (en) * | 2020-05-06 | 2020-08-21 | 上海市建筑科学研究院有限公司 | Point cloud error correction method based on mobile 3D laser scanning |
Non-Patent Citations (3)
| Title |
|---|
| A practical application of photogrammetry to performing rib characterization measurements in an underground coal mine using a DSLR camera;Brent A.Slaker;Khaled M.Mohamed;;International Journal of Mining Science and Technology(第01期);全文 * |
| 一种基于点云匹配的激光雷达/IMU联合标定方法;薛庆全等;《电子技术应用》;全文 * |
| 基于移动激光扫描技术的隧道限界检测研究;史增峰;《中国优秀硕士学位论文全文数据库》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112085843A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112085843B (en) | Tunnel class target feature real-time extraction and measurement method and device | |
| CN109143207B (en) | Laser radar internal reference precision verification method, device, equipment and medium | |
| CN109635672B (en) | Unmanned road characteristic parameter estimation method | |
| CN111179274B (en) | Map ground segmentation method, device, computer equipment and storage medium | |
| EP4296713A1 (en) | Target detection method, storage medium, electronic device, and vehicle | |
| CN113238251B (en) | Target level semantic positioning method based on vehicle-mounted laser radar | |
| CN112034482A (en) | Road boundary real-time extraction and measurement method and device | |
| CN114485698B (en) | Intersection guide line generation method and system | |
| CN115342821A (en) | Unmanned vehicle navigation cost map construction method under complex unknown environment | |
| CN112700479A (en) | Registration method based on CNN point cloud target detection | |
| CN114821526A (en) | Obstacle three-dimensional frame detection method based on 4D millimeter wave radar point cloud | |
| CN115205803A (en) | Automatic driving environment sensing method, medium and vehicle | |
| CN112147638A (en) | Ground information acquisition method, device and system based on laser point cloud reflection intensity | |
| CN113345093B (en) | Filtering method for laser radar point cloud trailing points | |
| CN112699575A (en) | Method and system for measuring and calculating relative position in virtual vehicle test platform | |
| US11555928B2 (en) | Three-dimensional object detection with ground removal intelligence | |
| CN116299313A (en) | Laser radar-based intelligent vehicle passable area detection method | |
| CN115390050A (en) | Calibration method, device and equipment of vehicle-mounted laser radar | |
| CN114241083A (en) | Lane line generation method and device, electronic equipment and storage medium | |
| CN113436336A (en) | Ground point cloud segmentation method and device and automatic driving vehicle | |
| CN114910881A (en) | Negative obstacle detection method and device and vehicle | |
| CN112906519A (en) | Vehicle type identification method and device | |
| Mattson et al. | Reducing ego vehicle energy-use by LiDAR-based lane-level positioning | |
| CN115574805B (en) | Lane line relationship identification method and device, vehicle and storage medium | |
| US20230314169A1 (en) | Method and apparatus for generating map data, and non-transitory computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |