CN113479191A - Lane-line-free lane boundary detection system and method for parking and vehicle - Google Patents
Lane-line-free lane boundary detection system and method for parking and vehicle Download PDFInfo
- Publication number
- CN113479191A CN113479191A CN202110742580.1A CN202110742580A CN113479191A CN 113479191 A CN113479191 A CN 113479191A CN 202110742580 A CN202110742580 A CN 202110742580A CN 113479191 A CN113479191 A CN 113479191A
- Authority
- CN
- China
- Prior art keywords
- lane
- vehicle
- parking
- gridmap
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
- B60W30/06—Automatic manoeuvring for parking
Abstract
The invention discloses a lane boundary detection system without lane lines for parking, a method and a vehicle, comprising the following steps: a) acquiring obstacle point information by using an ultrasonic sensor; b) acquiring 3D visual point cloud information by using a visual sensor; c) fusing the obstacle point information and the visual point cloud information, and filtering out dynamic obstacles; d) projecting the information to a two-dimensional plane to generate a Gridmap grid map, wherein the Gridmap grid map is used for representing a local static environment; e) converting the Gridmap grid map into a binary image by using a threshold value; f) performing linear extraction by a Hough linear detection method provided by opencv; g) f, screening and filtering the line segments extracted in the step f by using the heading information of the vehicle, and outputting 1 boundary at most on the left and right; h) and finally, filtering the boundary line. The invention can realize the detection of the lane boundary under the scene without lane lines, so that the parking system can realize the function of automatically searching the parking spaces.
Description
Technical Field
The invention belongs to the technical field of automatic parking, and particularly relates to a lane boundary detection system and method without lane lines for parking and a vehicle.
Background
In recent years, the problem of difficulty in parking has become more prominent as the amount of automobiles kept has exponentially increased. This problem has attracted much attention from the outside world, especially in large host plants. Nowadays, intelligent driving is emerging, and vehicles equipped with autonomous parking systems are more and more popular. Along with automatic system's function of parking constantly upgrades, more and more host computer factories release the automatic parking stall function of searching, when the driver finds the place ahead has the parking stall, the driver can push down after the key of parking can get off or wait in the car, the vehicle can search the parking stall forward by oneself, it just can stop by oneself to judge that the parking stall is suitable when the system is listened to, thereby has promoted the convenience of parkking. The control process of the vehicle to automatically advance and search the parking space mainly depends on the advance of the lane center line, so the key step is to know the lane center line. Most roads have no lane center line and even no lane connecting line. If the system knows the lane boundaries, then the lane centerline can be abstracted and the vehicle can proceed with reference to the lane centerline. However, most of the existing host plants which release the automatic parking space searching function require that a lane line is arranged on a lane, and do not have the automatic parking space searching function aiming at scenes without the lane line.
In the field of automatic parking space searching, for example, a parking lot management system and method for automatically searching for vacant parking spaces disclosed in patent document CN104282173A, a parking space occupancy detector is provided and respectively disposed on each parking space for detecting whether a vehicle is parked in the parking space, and information about whether the parking space is occupied is transmitted to a parking lot controller in real time; the parking lot controller is used for receiving the parking space occupation information sent by each parking space occupation detector, storing and updating the parking space idle information in real time and displaying the parking space idle information on the entrance screen. The method is mainly used for reminding a driver of the remaining free parking spaces and depends on an intelligent parking lot. For another example, patent document CN107776570A discloses a full-automatic parking method and a full-automatic parking system, but it does not suggest how to automatically advance to search for a parking space. In the field of lane boundary detection,
therefore, there is a need to develop a new lane boundary detection system, method, vehicle and vehicle for parking without lane lines.
Disclosure of Invention
The invention aims to provide a lane boundary detection system and method without a lane line for parking and a vehicle, which can realize the extraction of lane boundaries under a scene without the lane line.
In a first aspect, the present invention provides a lane boundary detection method without lane lines for parking, including the steps of:
a) acquiring obstacle point information by using an ultrasonic sensor;
b) acquiring 3D visual point cloud information by using a visual sensor;
c) fusing the obstacle point information and the visual point cloud information, and filtering out dynamic obstacles;
d) projecting the information to a two-dimensional plane to generate a Gridmap grid map;
e) converting the Gridmap grid map into a binary image by using a threshold value;
f) performing linear extraction by a Hough linear detection method provided by opencv;
g) f, screening and filtering the line segments extracted in the step f by using the heading information of the vehicle, and outputting 1 boundary at most on the left and right;
h) and finally, filtering the boundary line.
Alternatively, when the vehicle is moving, the currently active raster image region should always cover the ROI around the vehicle body.
Optionally, the Gridmap grid map is always only translated and not rotated relative to a global coordinate system, where the global coordinate system uses a rear axle center of the vehicle as an origin of coordinates, an X-axis is directly in front of the vehicle, and a Y-axis is left of the vehicle.
Optionally, each grid of the Gridmap grid map contains three probabilities of free, occupied, and unknown, and the sum of the three values is equal to 1.
Optionally, the Gridmap raster map update output frequency is not less than 5 fps.
Optionally, the Gridmap grid map is 200 × 200 in size, with a grid resolution of 10cm × 10 cm.
In a second aspect, the present invention provides a lane boundary detection system for parking without a lane line, including:
the ultrasonic sensor is used for acquiring obstacle point information;
the visual sensor is used for acquiring 3D visual point cloud information;
a memory having a computer readable program stored therein;
a controller capable of executing the steps of the lane boundary detection method for a driverless vehicle parking according to any one of claims 1 to 6 when the controller calls the computer readable program.
In a third aspect, the present invention provides a vehicle employing the lane boundary detection system for parking without lane lines according to the present invention.
The invention has the following advantages: the method can realize lane boundary detection in the scene without lane lines, so that the parking system can automatically search parking spaces according to the detected lane boundaries.
Drawings
FIG. 1 is a flow chart of the present embodiment;
FIG. 2 is a schematic drawing of gridmap with walls on one side;
FIG. 3 is a graph of the boundary line extraction effect for a wall on one side;
FIG. 4 is a schematic diagram of gridmap for curb parking a parallel vehicle;
FIG. 5 is a diagram showing the effect of boundary line extraction for a roadside parked parallel vehicle;
FIG. 6 is a schematic diagram of gridmap for curb parking a vertical vehicle;
fig. 7 is a boundary line extraction effect diagram of a roadside parked vertical vehicle.
Detailed Description
As shown in fig. 1, in the present embodiment, a lane boundary detection method without lane lines for parking includes the following steps:
a) and acquiring obstacle point information by using the ultrasonic sensor.
b) And acquiring the 3D visual point cloud information by using a visual sensor.
c) And fusing the ultrasonic information and the visual point cloud information to filter out dynamic obstacles.
d) And projecting the information to a two-dimensional plane to generate a Gridmap grid map. The grid map is used for representing a local static environment, the size of the grid map is 200 × 200 (namely, the grid is 200 grids in length and 200 grids in width), and the grid resolution is 10cm × 10 cm. When the vehicle is moving, the currently active grid map region should always cover the ROI (i.e. the region of interest) around the vehicle body, and in this embodiment, the ROI around the vehicle body refers to x ∈ 10m, +10m, y ∈ [ -10m, +10m ] under the central coordinate system of the rear axle of the vehicle. The global coordinate system takes the center of the rear axle of the vehicle as the origin of coordinates, the right front of the vehicle is an X-axis, and the left of the vehicle is a Y-axis. The Gridmap raster map update output frequency is not less than 5 fps. And the Gridmap grid graph only translates and does not rotate relative to the global coordinate system all the time. Each grid of the Gridmap grid map contains three probabilities of free, occupied, and unknown, with the sum of the three values being equal to 1.
e) And converting the Gridmap grid map into a binary image by using a threshold value.
f) The line extraction is carried out by a Hough line detection method provided by opencv.
g) And f, screening and filtering the line segments extracted in the step f by using the heading information of the vehicle, and outputting 1 boundary at most on the left and right.
h) And finally, filtering the boundary line.
Example 1 is a scene with a wall on one side (upper side), when a vehicle is driving forward, an ultrasonic sensor detects the wall surface and outputs a string of points, and then visual point cloud information is fused to generate a gridmap grid map, referring to fig. 2, the darker the color represents that the grid has a higher probability of being an obstacle, the pixel value of the corresponding image is 255 when the probability of being an obstacle is greater than 60%, and the pixel value of the corresponding image is 0 when the probability of being an obstacle is less than 60%; the distance accuracy is set to be 1 pixel (corresponding to the actual quantity to be 10 cm), the angle accuracy is set to be 1 degree, the threshold parameter of the accumulation plane is 30 degrees, the minimum length of the line segment is 30 pixels (corresponding to the actual minimum length to be 3 m), and the maximum allowable interval of two line segments in the same direction which are determined as one line segment is 8 pixels (corresponding to the actual maximum allowable interval to be 80 cm). The results obtained using the Hough Linear detection method (i.e., HoughLinesP) provided by opencv are shown in FIG. 3.
Example 2 is a scene of parking a parallel vehicle on the roadside, in the process of forward driving of the vehicle, an ultrasonic sensor detects a vehicle body parked on the roadside vehicle and outputs a string of points, then visual point cloud information is fused to generate a gridmap grid map, see fig. 4, wherein a rectangular frame is a parking space detected by a sensing module, the darker the color of the grid represents that the grid has a higher probability of being an obstacle, the pixel value of a corresponding image is 255 when the probability of being an obstacle is greater than 60%, and the pixel value of the corresponding image is 0 when the probability of being an obstacle is less than 60%; the distance accuracy is set to be 1 pixel (corresponding to the actual quantity to be 10 cm), the angle accuracy is set to be 1 degree, the threshold parameter of the accumulation plane is 30 degrees, the minimum length of the line segment is 30 pixels (corresponding to the actual minimum length to be 3 m), and the maximum allowable interval of two line segments in the same direction which are determined as one line segment is 8 pixels (corresponding to the actual maximum allowable interval to be 80 cm). The results obtained using the HoughLinesP method supplied by opencv are shown in FIG. 5.
Example 3 is a scene of parking a vertical vehicle on the roadside, in the process of forward driving of the vehicle, the ultrasonic sensor detects the head of the vertical vehicle parked on the roadside and outputs a string of points, then visual point cloud information is fused to generate a gridmap, see fig. 6, wherein a rectangular frame is a parking space detected by the sensing module, the darker the color of a grid represents that the grid has a higher probability of being an obstacle, it is set that when the probability of being an obstacle is greater than 60%, the pixel value of a corresponding image is 255, and when the probability of being an obstacle is less than 60%, the pixel value of the corresponding image is 0; the distance accuracy is set to be 1 pixel (corresponding to the actual quantity to be 10 cm), the angle accuracy is set to be 1 degree, the threshold parameter of the accumulation plane is 30 degrees, the minimum length of the line segment is 30 pixels (corresponding to the actual minimum length to be 3 m), and the maximum allowable interval of two line segments in the same direction which are determined as one line segment is 8 pixels (corresponding to the actual maximum allowable interval to be 80 cm). The results obtained using the HoughLinesP method supplied by opencv are shown in FIG. 7.
Claims (8)
1. A lane boundary detection method for parking without a lane line, characterized by comprising the steps of:
a) acquiring obstacle point information by using an ultrasonic sensor;
b) acquiring 3D visual point cloud information by using a visual sensor;
c) fusing the obstacle point information and the visual point cloud information, and filtering out dynamic obstacles;
d) projecting the information to a two-dimensional plane to generate a Gridmap grid map, wherein the Gridmap grid map is used for representing a local static environment;
e) converting the Gridmap grid map into a binary image by using a threshold value;
f) performing linear extraction by a Hough linear detection method provided by opencv;
g) f, screening and filtering the line segments extracted in the step f by using the heading information of the vehicle, and outputting 1 boundary at most on the left and right;
h) and finally, filtering the boundary line.
2. The lane boundary detection method without a lane line for parking according to claim 1, characterized in that: when the vehicle is moving, the currently active raster image region should always cover the ROI around the vehicle body.
3. The lane boundary detection method without a lane line for parking according to claim 2, characterized in that: the Gridmap grid graph only translates and does not rotate relative to a global coordinate system all the time, wherein the global coordinate system takes the center of a rear axle of a vehicle as a coordinate origin, the right front of the vehicle is an X axle, and the left of the vehicle is a Y axle.
4. The lane boundary detection method without a lane line for parking according to claim 3, characterized in that: each grid of the Gridmap grid map contains three probabilities of free, occupied, and unknown, and the sum of the three values is equal to 1.
5. The lane boundary detection method without a lane line for parking according to claim 4, characterized in that: and the output frequency of Gridmap raster map updating is not less than 5 fps.
6. The lane boundary detection method without a lane line for parking according to claim 5, characterized in that: the Gridmap grid map is 200 × 200 in size with a grid resolution of 10cm × 10 cm.
7. A lane boundary detection system for parking without a lane line, comprising:
the ultrasonic sensor is used for acquiring obstacle point information;
the visual sensor is used for acquiring 3D visual point cloud information;
a memory having a computer readable program stored therein;
a controller, characterized by: the controller, when invoking the computer readable program, is capable of performing the steps of the lane boundary detection method for a driverless parking vehicle of any of claims 1 to 6.
8. A vehicle, characterized in that: the lane boundary detection system for parking a vehicle using a lane-less according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110742580.1A CN113479191B (en) | 2021-06-30 | 2021-06-30 | Lane-line-free lane boundary detection system and method for parking and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110742580.1A CN113479191B (en) | 2021-06-30 | 2021-06-30 | Lane-line-free lane boundary detection system and method for parking and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113479191A true CN113479191A (en) | 2021-10-08 |
| CN113479191B CN113479191B (en) | 2023-04-07 |
Family
ID=77936998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110742580.1A Active CN113479191B (en) | 2021-06-30 | 2021-06-30 | Lane-line-free lane boundary detection system and method for parking and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113479191B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113942458A (en) * | 2021-10-29 | 2022-01-18 | 禾多科技（北京）有限公司 | Control method, device, equipment and medium for vehicle-mounted camera adjusting system |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010140170A (en) * | 2008-12-10 | 2010-06-24 | Toyota Motor Corp | Lane boundary deriving device |
| US20120062747A1 (en) * | 2010-07-20 | 2012-03-15 | Gm Global Technology Operations, Inc. | Lane fusion system using forward-view and rear-view cameras |
| DE102013200387A1 (en) * | 2013-01-14 | 2014-07-17 | Robert Bosch Gmbh | Creation of a obstacle map |
| CN104850834A (en) * | 2015-05-11 | 2015-08-19 | 中国科学院合肥物质科学研究院 | Road boundary detection method based on three-dimensional laser radar |
| KR20160123668A (en) * | 2015-04-16 | 2016-10-26 | 한국전자통신연구원 | Device and method for recognition of obstacles and parking slots for unmanned autonomous parking |
| CN106627582A (en) * | 2016-12-09 | 2017-05-10 | 重庆长安汽车股份有限公司 | Path planning system and method for overtaking vehicle on adjacent lane in single-lane automatic drive mode |
| CN108805065A (en) * | 2018-05-31 | 2018-11-13 | 华南理工大学 | One kind being based on the improved method for detecting lane lines of geometric properties |
| CN109532821A (en) * | 2018-11-09 | 2019-03-29 | 重庆长安汽车股份有限公司 | Merge parking system |
| US20190156129A1 (en) * | 2016-07-22 | 2019-05-23 | Hitachi Automotive Systems, Ltd. | Traveling Road Boundary Estimation Apparatus and Traveling Assistance System Using Same |
| CN110044371A (en) * | 2018-01-16 | 2019-07-23 | 华为技术有限公司 | A kind of method and vehicle locating device of vehicle location |
| CN110147109A (en) * | 2019-05-21 | 2019-08-20 | 重庆长安汽车股份有限公司 | A kind of archetype development system of automated driving system |
| CN110287779A (en) * | 2019-05-17 | 2019-09-27 | 百度在线网络技术（北京）有限公司 | Detection method, device and the equipment of lane line |
| WO2019245982A1 (en) * | 2018-06-18 | 2019-12-26 | Zoox, Inc. | Occlusion aware planning |
| CN110838233A (en) * | 2019-10-12 | 2020-02-25 | 中国平安人寿保险股份有限公司 | Vehicle behavior analysis method and device and computer readable storage medium |
| CN110861639A (en) * | 2019-11-28 | 2020-03-06 | 安徽江淮汽车集团股份有限公司 | Parking information fusion method and device, electronic equipment and storage medium |
| CN111273305A (en) * | 2020-02-18 | 2020-06-12 | 中国科学院合肥物质科学研究院 | Multi-sensor fusion road extraction and indexing method based on global and local grid maps |
| CN111319615A (en) * | 2020-03-16 | 2020-06-23 | 湖北亿咖通科技有限公司 | Intelligent passenger-replacing parking method, computer-readable storage medium and electronic device |
| CN112776797A (en) * | 2021-02-27 | 2021-05-11 | 重庆长安汽车股份有限公司 | Original parking space parking establishment method and system, vehicle and storage medium |
-
2021
- 2021-06-30 CN CN202110742580.1A patent/CN113479191B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010140170A (en) * | 2008-12-10 | 2010-06-24 | Toyota Motor Corp | Lane boundary deriving device |
| US20120062747A1 (en) * | 2010-07-20 | 2012-03-15 | Gm Global Technology Operations, Inc. | Lane fusion system using forward-view and rear-view cameras |
| DE102013200387A1 (en) * | 2013-01-14 | 2014-07-17 | Robert Bosch Gmbh | Creation of a obstacle map |
| KR20160123668A (en) * | 2015-04-16 | 2016-10-26 | 한국전자통신연구원 | Device and method for recognition of obstacles and parking slots for unmanned autonomous parking |
| CN104850834A (en) * | 2015-05-11 | 2015-08-19 | 中国科学院合肥物质科学研究院 | Road boundary detection method based on three-dimensional laser radar |
| US20190156129A1 (en) * | 2016-07-22 | 2019-05-23 | Hitachi Automotive Systems, Ltd. | Traveling Road Boundary Estimation Apparatus and Traveling Assistance System Using Same |
| CN106627582A (en) * | 2016-12-09 | 2017-05-10 | 重庆长安汽车股份有限公司 | Path planning system and method for overtaking vehicle on adjacent lane in single-lane automatic drive mode |
| CN110044371A (en) * | 2018-01-16 | 2019-07-23 | 华为技术有限公司 | A kind of method and vehicle locating device of vehicle location |
| CN108805065A (en) * | 2018-05-31 | 2018-11-13 | 华南理工大学 | One kind being based on the improved method for detecting lane lines of geometric properties |
| WO2019245982A1 (en) * | 2018-06-18 | 2019-12-26 | Zoox, Inc. | Occlusion aware planning |
| CN109532821A (en) * | 2018-11-09 | 2019-03-29 | 重庆长安汽车股份有限公司 | Merge parking system |
| CN110287779A (en) * | 2019-05-17 | 2019-09-27 | 百度在线网络技术（北京）有限公司 | Detection method, device and the equipment of lane line |
| CN110147109A (en) * | 2019-05-21 | 2019-08-20 | 重庆长安汽车股份有限公司 | A kind of archetype development system of automated driving system |
| CN110838233A (en) * | 2019-10-12 | 2020-02-25 | 中国平安人寿保险股份有限公司 | Vehicle behavior analysis method and device and computer readable storage medium |
| CN110861639A (en) * | 2019-11-28 | 2020-03-06 | 安徽江淮汽车集团股份有限公司 | Parking information fusion method and device, electronic equipment and storage medium |
| CN111273305A (en) * | 2020-02-18 | 2020-06-12 | 中国科学院合肥物质科学研究院 | Multi-sensor fusion road extraction and indexing method based on global and local grid maps |
| CN111319615A (en) * | 2020-03-16 | 2020-06-23 | 湖北亿咖通科技有限公司 | Intelligent passenger-replacing parking method, computer-readable storage medium and electronic device |
| CN112776797A (en) * | 2021-02-27 | 2021-05-11 | 重庆长安汽车股份有限公司 | Original parking space parking establishment method and system, vehicle and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 吴彦文等: "基于多传感融合的车道线检测与跟踪方法的研究", 《计算机应用研究》 * |
| 姜灏: "一种自动驾驶车的环境感知系统", 《电子制作》 * |
| 龚建伟等: "一种自适应动态窗口车道线高速检测方法", 《北京理工大学学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113942458A (en) * | 2021-10-29 | 2022-01-18 | 禾多科技（北京）有限公司 | Control method, device, equipment and medium for vehicle-mounted camera adjusting system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113479191B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10628690B2 (en) | Systems and methods for automated detection of trailer properties | |
| US20220107651A1 (en) | Predicting three-dimensional features for autonomous driving | |
| JP7467485B2 (en) | Generating ground truth for machine learning from time series elements | |
| CN108959321B (en) | Parking lot map construction method, system, mobile terminal and storage medium | |
| CN113379805B (en) | Multi-information resource fusion processing method for traffic nodes | |
| CN111874006B (en) | Route planning processing method and device | |
| US9360332B2 (en) | Method for determining a course of a traffic lane for a vehicle | |
| US11003920B2 (en) | Detection and planar representation of three dimensional lanes in a road scene | |
| US10325163B2 (en) | Vehicle vision | |
| CN110320531B (en) | Obstacle identification method based on laser radar, map creation method and device | |
| US20230184560A1 (en) | Visual interface display method and apparatus, electronic device, and storage medium | |
| US10286901B2 (en) | Map of the surroundings for driving areas with random altitude profile | |
| US11662221B2 (en) | Change point detection device and map information distribution system | |
| CN108960035A (en) | The method and system of usage scenario detection pattern object of classification in perception scene figure | |
| CN110298311B (en) | Method and device for detecting surface water accumulation | |
| US11891091B2 (en) | Semantic occupancy grid management in ADAS/autonomous driving | |
| CN108725318B (en) | Automobile safety early warning method and device and computer readable storage medium | |
| CN113479191B (en) | Lane-line-free lane boundary detection system and method for parking and vehicle | |
| CN114537447A (en) | Safe passing method and device, electronic equipment and storage medium | |
| CN114694108A (en) | Image processing method, device, equipment and storage medium | |
| CN113838060A (en) | Perception system for autonomous vehicle | |
| CN114779276A (en) | Obstacle detection method and device | |
| CN114677658A (en) | Billion-pixel dynamic large-scene image acquisition and multi-target detection method and device | |
| CN113840079A (en) | Division of images acquired from unmanned vehicle cameras | |
| US20220270484A1 (en) | Parking lot control server, parking lot control method and vehicle control apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |