CN109583604B - Substation equipment fault marking method based on SLAM technology - Google Patents
Substation equipment fault marking method based on SLAM technology Download PDFInfo
- Publication number
- CN109583604B CN109583604B CN201811505591.2A CN201811505591A CN109583604B CN 109583604 B CN109583604 B CN 109583604B CN 201811505591 A CN201811505591 A CN 201811505591A CN 109583604 B CN109583604 B CN 109583604B
- Authority
- CN
- China
- Prior art keywords
- slam
- equipment
- coordinates
- power transformation
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Abstract
The invention provides a substation equipment fault marking method based on an SLAM technology, which comprises the following steps: s1, selecting a visual feature point mark near the power transformation equipment; s2, corresponding the visual feature point marks obtained in the S1 to coordinates in a BIM system; s3, generating a local three-dimensional map and a coordinate system through an SLAM technology; s4, recording the coordinates of the visual feature points in the SLAM coordinate system by using a computer vision technology; s5, based on the positions of the visual coordinates in the two sets of coordinate systems, associating the BIM system and the SLAM through three-dimensional coordinate transformation to generate a coordinate system of three-dimensional coordinates; and S6, recording the three-dimensional coordinates of the fault of the power transformation equipment in the local coordinate system generated by the SLAM, and synchronizing the information into the BIM system through coordinate transformation. The invention effectively improves the efficiency of eliminating the fault of the power transformation equipment, reduces the labor intensity of maintaining the power transformation equipment, and simultaneously facilitates the application of big data and machine learning due to the formation of a new structured infrared data structure.
Description
Technical Field
The invention relates to the technical field of power data acquisition methods, in particular to a substation equipment fault marking method based on an SLAM technology.
Background
With the improvement of the precision and the miniaturization of the sensor, the explosive development of the machine learning algorithm and the improvement of the chip processing capacity, the perception of the computer to the environment starts to be more and more like a human, and the computer can acquire the exact position in the space of the computer through the sensor by matching the acceleration sensor data and the visual feature extraction technology, so that the technology is used as an important basis of an industrial 4.0 technical system and has revolutionary influence on a plurality of industries including the power industry.
The power transformation equipment of the power system is various in types, different manufacturers and different product models can be provided for the same type of equipment, different software versions can be encountered for the secondary power transformation equipment of the same type along with popularization of intelligent power transformation, fragmented equipment causes great burden to maintenance personnel, and the maintenance personnel can change the state of the equipment at every work.
Therefore, the characteristics of the equipment are recorded and displayed for a front-line worker in an intuitive mode, the working efficiency of the worker is greatly improved, and the working errors are reduced. Moreover, through the data collection and presentation of multimedia, the working experience of a front-line worker can be partially saved in the enterprise knowledge base in this way. Therefore, the learning curve of the new staff is improved, and the adaptation time of the new staff is reduced.
Synchronous positioning and mapping (SLAM or Simultaneous localization and mapping) is a technical field that positions the self position and posture through repeatedly observed features during movement, and then constructs a map incrementally according to the self position, thereby achieving the purpose of Simultaneous positioning and map construction.
Disclosure of Invention
The invention aims to provide a substation equipment fault marking method based on an SLAM technology.
In order to achieve the purpose, the invention adopts the following technical scheme: a substation equipment fault marking method based on SLAM technology is characterized by comprising the following steps:
s1, selecting a visual feature point mark near the power transformation equipment;
s2, corresponding the visual feature point marks obtained in the S1 to coordinates in a BIM system;
s3, generating a local three-dimensional map and a coordinate system through an SLAM technology;
s4, recording the coordinates of the visual feature points in the SLAM coordinate system by using a computer vision technology;
s5, based on the positions of the visual coordinates in the two sets of coordinate systems, associating the BIM system and the SLAM through three-dimensional coordinate transformation to generate a coordinate system of three-dimensional coordinates;
and S6, recording the three-dimensional coordinates of the fault of the power transformation equipment in the local coordinate system generated by the SLAM, and synchronizing the information into the BIM system through coordinate transformation.
Further, in S1, near transformer equipment selects the marking device that visual characteristic point mark used and includes the detection case, be equipped with the detection chamber in the detection case, the externally mounted of detection case has first driving motor, be equipped with the spout on the detection case inner wall, be equipped with the pulley in the spout, install the shielded cell on the pulley, be equipped with the installation cavity in the shielded cell, be equipped with the intercommunication chamber way on the shielded cell, install the camera lens mount pad on the shielded cell, install the camera lens on the camera lens mount pad, it is used for connecting data storage module and camera lens to be equipped with the data line in the intercommunication chamber way.
Further, a plurality of second driving motors are installed to the below that just is located the shielded cell on the detection case inner wall, the piston is installed to one side that the detection case was kept away from to second driving motor, the piston axle is installed to one side that the detection case was kept away from to the piston, the limiting plate is installed to one side that the piston was kept away from to the piston axle, the limiting plate mount pad, install the limiting plate on the limiting plate mount pad.
Furthermore, a spring is arranged between the piston and the limiting plate mounting seat and sleeved on the periphery of the piston shaft.
Furthermore, the shooting lens adopts an RGB-D camera.
Further, in S1, the visual feature points include equipment location, equipment parameters and characteristics, and equipment repair history and defects.
After the technical scheme is adopted, the invention has the following advantages:
1. according to the position of the three-dimensional space coordinate and the visual characteristic point position provided by the SLAM technology in the space, the position of the power transformation equipment fault information corresponding to the coordinate system generated by the SLAM is recorded, and the visual characteristic point position corresponds to the space coordinate in the BIM system, so that the power transformation equipment fault position and the fault information with the three-dimensional coordinate are generated, the efficiency of removing the power transformation equipment fault is effectively improved, and the labor intensity of maintaining the power transformation equipment is reduced.
2. The visual characteristic point mark is selected near the power transformation equipment and marked by the marking device, the detection box is installed on the periphery of the power transformation equipment to be detected, the pulley is driven by the first driving motor to slide in the sliding groove, so that the shielding box is driven to surround the power transformation equipment to be detected, the power transformation equipment to be detected is shot by the shooting lens installed on the shielding box, shot information is stored in the data storage module, the shot information corresponds to coordinates in a BIM system by the data storage module, a new structured infrared data structure is formed, and the application of big data and machine learning is facilitated.
3. Install the second driving motor through the below that just is located the shielded cell on the detection case inner wall, it is flexible in the piston to drive the piston axle by the second driving motor, gather data's transformer equipment's size as required, through limiting plate and transformer equipment outer wall contact, stretch out and draw back in the piston by second driving motor drive piston axle, carry out spacing fixed by piston axle drive limiting plate to transformer equipment, effectively improved the spacing fixed stability to transformer equipment, stability when effectively having improved the detection case and using, transformer equipment data acquisition's reliability has further been improved.
4. Through establishing the spring between piston and limiting plate mount pad, stability when can effectively improve the installation detection case by the spring further improves data acquisition's stability, improves data acquisition's reliability.
5. The RGB-D camera can be used for simultaneously collecting the color and depth information of an object, and the position perception of the SLAM carrier is matched to perform centimeter-level three-dimensional modeling on a scene. The rapid modeling technology can reduce time for three-dimensional modeling of the transformer substation, so that the cost of large-scale application deployment is reduced.
6. The visual feature point information and the modeling result are associated through the SLAM technology, so that a new structured infrared data structure is formed, and the application of big data and machine learning is facilitated.
Drawings
The invention will be further described with reference to the accompanying drawings in which:
fig. 1 is a flowchart of a substation equipment fault marking method based on the SLAM technology according to the present invention;
fig. 2 is a schematic structural diagram of a marking device used in S1 in a substation equipment fault marking method based on the SLAM technology;
fig. 3 is a partially enlarged schematic view of a portion a in fig. 2.
Detailed Description
The invention is further described with reference to the following figures and specific examples. It is to be understood that the following terms "upper," "lower," "left," "right," "longitudinal," "lateral," "inner," "outer," "vertical," "horizontal," "top," "bottom," and the like are used merely to indicate an orientation or positional relationship relative to one another as illustrated in the drawings, merely to facilitate describing and simplifying the invention, and are not intended to indicate or imply that the device/component so referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore are not to be considered limiting of the invention.
As shown in fig. 1 to 3, the present invention provides a substation equipment fault marking method based on SLAM technology, including the following steps:
s1, selecting a visual feature point mark near the power transformation equipment;
s2, corresponding the visual feature point marks obtained in the S1 to coordinates in a BIM system;
s3, generating a local three-dimensional map and a coordinate system through an SLAM technology;
s4, recording the coordinates of the visual feature points in the SLAM coordinate system by using a computer vision technology;
s5, based on the positions of the visual coordinates in the two sets of coordinate systems, associating the BIM system and the SLAM through three-dimensional coordinate transformation to generate a coordinate system of three-dimensional coordinates;
and S6, recording the three-dimensional coordinates of the fault of the power transformation equipment in the local coordinate system generated by the SLAM, and synchronizing the information into the BIM system through coordinate transformation.
According to the position of the three-dimensional space coordinate and the visual characteristic point position provided by the SLAM technology in the space, the position of the power transformation equipment fault information corresponding to the coordinate system generated by the SLAM is recorded, and the visual characteristic point position corresponds to the space coordinate in the BIM system, so that the power transformation equipment fault position and the fault information with the three-dimensional coordinate are generated, the efficiency of removing the power transformation equipment fault is effectively improved, and the labor intensity of maintaining the power transformation equipment is reduced.
In this embodiment, in S1, near transformer equipment selects the marking device that visual characteristic point mark used to include detection case 1, be equipped with in the detection case 1 and detect chamber 9, the externally mounted of detection case 1 has first driving motor 2, be equipped with spout 3 on the detection case 1 inner wall, be equipped with pulley 10 in the spout 3, install shielding box 11 on the pulley 10, be equipped with installation cavity 13 in the shielding box 11, be equipped with intercommunication chamber way 15 on the shielding box 11, install taking lens mount pad 16 on the shielding box 11, install taking lens 17 on the taking lens mount pad 16, be equipped with data line 14 in the intercommunication chamber way 15 and be used for connecting data storage module 12 and taking lens 17. The visual characteristic point mark is selected near the power transformation equipment and is marked by the marking device, the detection box 1 is installed on the periphery of the power transformation equipment to be detected, the pulley 10 is driven by the first driving motor 2 to slide in the sliding groove 3, so that the shielding box 11 is driven to surround the power transformation equipment to be detected, the power transformation equipment to be detected is shot by the shooting lens 17 installed on the shielding box 11, shot information is stored in the data storage module 12, the shot information corresponds to coordinates in a BIM system through the data storage module 12, a new structured infrared data structure is formed, and application of big data and machine learning is facilitated.
In this embodiment, a plurality of second driving motor 5 are installed to the below that just is located shielding case 11 on the 1 inner wall of detection case, second driving motor 5 is kept away from one side of detection case 1 and is installed piston 6, piston 6 is kept away from one side of detection case 1 and is installed piston shaft 4, piston shaft 4 is kept away from one side of piston 6 and is installed limiting plate 8, limiting plate mount pad 5, install limiting plate 8 on the limiting plate mount pad 5. Install second driving motor 5 through the below that just is located shielding box 11 on detection case 1 inner wall, it is flexible in piston 6 to drive piston shaft 4 by second driving motor 5, data's transformer equipment's size is gathered as required, through limiting plate 8 and transformer equipment outer wall contact, by second driving motor 5 drive piston shaft 4 flexible in piston 6, carry out spacing fixed by piston shaft 4 drive limiting plate 8 to transformer equipment, effectively improved the stability to transformer equipment spacing fixed, stability when effectively having improved detection case 1 and using, transformer equipment data acquisition's reliability has further been improved.
In this embodiment, a spring 7 is disposed between the piston 6 and the limit plate mounting seat 5, and the spring 7 is sleeved on the outer periphery of the piston shaft 4. Through establish spring 7 between piston 6 and limiting plate mount pad 5, stability when installing detection case 1 can effectively be improved by spring 7, further improve data acquisition's stability, improve data acquisition's reliability.
In the present embodiment, the photographing lens 17 employs an RGB-D camera. The RGB-D camera can be used for simultaneously collecting the color and depth information of an object, and the position perception of the SLAM carrier is matched to perform centimeter-level three-dimensional modeling on a scene. The rapid modeling technology can reduce time for three-dimensional modeling of the transformer substation, so that the cost of large-scale application deployment is reduced.
In this embodiment, in S1, the visual feature points include equipment location, equipment parameters and characteristics, and equipment repair history and defects. The visual feature point information and the modeling result are associated through the SLAM technology, so that a new structured infrared data structure is formed, and the application of big data and machine learning is facilitated.
Other embodiments of the present invention than the preferred embodiments described above will be apparent to those skilled in the art from the present invention, and various changes and modifications can be made therein without departing from the spirit of the present invention as defined in the appended claims.
Claims (4)
1. A substation equipment fault marking method based on SLAM technology is characterized by comprising the following steps:
s1, selecting visual characteristic point marks near the power transformation equipment, wherein a marking device used for selecting the visual characteristic point marks near the power transformation equipment comprises a detection box, a detection cavity is arranged in the detection box, a first driving motor is installed outside the detection box, a sliding groove is formed in the inner wall of the detection box, a pulley is arranged in the sliding groove, a shielding box is installed on the pulley, an installation cavity is formed in the shielding box, a communicating cavity channel is formed in the shielding box, a shooting lens installation seat is installed on the shielding box, a shooting lens is installed on the shooting lens installation seat, and a data line used for connecting a data storage module and the shooting lens is arranged in the communicating cavity channel; a plurality of second driving motors are arranged on the inner wall of the detection box and below the shielding box, a piston is arranged on one side, away from the detection box, of each second driving motor, a piston shaft is arranged on one side, away from the detection box, of each piston, a limiting plate is arranged on one side, away from the piston, of each piston shaft, and a limiting plate is arranged on each limiting plate mounting seat;
s2, corresponding the visual feature point marks obtained in the S1 to coordinates in a BIM system;
s3, generating a local three-dimensional map and a coordinate system through an SLAM technology;
s4, recording the coordinates of the visual feature points in the SLAM coordinate system by using a computer vision technology;
s5, based on the positions of the visual coordinates in the two sets of coordinate systems, associating the BIM system and the SLAM through three-dimensional coordinate transformation to generate a coordinate system of three-dimensional coordinates;
and S6, recording the three-dimensional coordinates of the fault of the power transformation equipment in the local coordinate system generated by the SLAM, and synchronizing the information into the BIM system through coordinate transformation.
2. The SLAM technology-based power transformation equipment fault marking method as claimed in claim 1, wherein a spring is arranged between the piston and the limiting plate mounting seat, and the spring is sleeved on the outer periphery of the piston shaft.
3. The substation equipment fault marking method based on SLAM technology as claimed in claim 1, wherein the taking lens adopts an RGB-D camera.
4. The SLAM technology-based substation equipment fault marking method of claim 1, wherein in S1, the visual characteristic points comprise equipment location, equipment parameters and characteristics and equipment repair history and defects.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811505591.2A CN109583604B (en) | 2018-12-10 | 2018-12-10 | Substation equipment fault marking method based on SLAM technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811505591.2A CN109583604B (en) | 2018-12-10 | 2018-12-10 | Substation equipment fault marking method based on SLAM technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109583604A CN109583604A (en) | 2019-04-05 |
| CN109583604B true CN109583604B (en) | 2021-08-24 |
Family
ID=65928798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811505591.2A Active CN109583604B (en) | 2018-12-10 | 2018-12-10 | Substation equipment fault marking method based on SLAM technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109583604B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279987A (en) * | 2013-06-18 | 2013-09-04 | 厦门理工学院 | Object fast three-dimensional modeling method based on Kinect |
| CN104374395A (en) * | 2014-03-31 | 2015-02-25 | 南京邮电大学 | Graph-based vision SLAM (simultaneous localization and mapping) method |
| CN104932001A (en) * | 2015-07-08 | 2015-09-23 | 四川德马克机器人科技有限公司 | Real-time 3D nuclear radiation environment reconstruction monitoring system |
| CN205450060U (en) * | 2016-03-04 | 2016-08-10 | 南通鑫源电器制造有限公司 | Convenient to use's intelligent monitoring power box |
| CN106127739A (en) * | 2016-06-16 | 2016-11-16 | 华东交通大学 | A kind of RGB D SLAM method of combination monocular vision |
| CN207366010U (en) * | 2017-09-13 | 2018-05-15 | 中交隧道局电气化工程有限公司 | It is a kind of to be used for unattended computer room safety monitoring device |
| CN108227929A (en) * | 2018-01-15 | 2018-06-29 | 廖卫东 | Augmented reality setting-out system and implementation method based on BIM technology |
| CN207779556U (en) * | 2017-12-04 | 2018-08-28 | 国网河南省电力公司新乡供电公司 | Video temperature patrol inspection device in a kind of provision for sealing cabinet |
| CN108537848A (en) * | 2018-04-19 | 2018-09-14 | 北京工业大学 | A kind of two-stage pose optimal estimating method rebuild towards indoor scene |
-
2018
- 2018-12-10 CN CN201811505591.2A patent/CN109583604B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279987A (en) * | 2013-06-18 | 2013-09-04 | 厦门理工学院 | Object fast three-dimensional modeling method based on Kinect |
| CN104374395A (en) * | 2014-03-31 | 2015-02-25 | 南京邮电大学 | Graph-based vision SLAM (simultaneous localization and mapping) method |
| CN104932001A (en) * | 2015-07-08 | 2015-09-23 | 四川德马克机器人科技有限公司 | Real-time 3D nuclear radiation environment reconstruction monitoring system |
| CN205450060U (en) * | 2016-03-04 | 2016-08-10 | 南通鑫源电器制造有限公司 | Convenient to use's intelligent monitoring power box |
| CN106127739A (en) * | 2016-06-16 | 2016-11-16 | 华东交通大学 | A kind of RGB D SLAM method of combination monocular vision |
| CN207366010U (en) * | 2017-09-13 | 2018-05-15 | 中交隧道局电气化工程有限公司 | It is a kind of to be used for unattended computer room safety monitoring device |
| CN207779556U (en) * | 2017-12-04 | 2018-08-28 | 国网河南省电力公司新乡供电公司 | Video temperature patrol inspection device in a kind of provision for sealing cabinet |
| CN108227929A (en) * | 2018-01-15 | 2018-06-29 | 廖卫东 | Augmented reality setting-out system and implementation method based on BIM technology |
| CN108537848A (en) * | 2018-04-19 | 2018-09-14 | 北京工业大学 | A kind of two-stage pose optimal estimating method rebuild towards indoor scene |
Non-Patent Citations (2)
| Title |
|---|
| 基于智能视觉物联网的变电站红外监测系统;漆灿等;《电力系统保护与控制》;20180801;第46卷(第15期);第135~141页 * |
| 室内定位技术在变电检修中的应用;李振华 等;《中国高新科技》;20180915(第18期);第135~141页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109583604A (en) | 2019-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110238633B (en) | Novel auto-screwdriving machine towards flexible manufacturing | |
| CN104842361A (en) | Robotic system with 3d box location functionality | |
| CN111151463B (en) | Mechanical arm sorting and grabbing system and method based on 3D vision | |
| CN105666505A (en) | Robot system having augmented reality-compatible display | |
| US20130131853A1 (en) | Automated Identification Of Shoe Parts | |
| CN105217324A (en) | A kind of novel de-stacking method and system | |
| CN109638959B (en) | Power equipment remote signaling function debugging method and system based on AR and deep learning | |
| CN110569849B (en) | AR (augmented reality) -glasses-based multi-instrument simultaneous identification and spatial positioning method and system | |
| CN105373130A (en) | Special device accident on-site information detection system based on stereo modeling | |
| US20220292702A1 (en) | Image processor, imaging device, robot and robot system | |
| CN110927172B (en) | Online detection device and method for missing coating of sealant of integral fuel tank of airplane | |
| CN107918954A (en) | The method that 3D for 2D points of interest maps | |
| CN203376759U (en) | BNC terminal automatic detection machine | |
| CN104834284A (en) | Automatic production line based on visual identification | |
| CN210476118U (en) | Novel auto-screwdriving machine towards flexible manufacturing | |
| CN109583604B (en) | Substation equipment fault marking method based on SLAM technology | |
| CN113196165A (en) | Information projection system, control device, and information projection method | |
| CN105068139B (en) | A kind of characterization processes of piston cooling nozzle installment state | |
| CN113269729B (en) | Assembly body multi-view detection method and system based on depth image contrast | |
| CN204036474U (en) | Industrial robot sorting system | |
| CN110207619A (en) | The measuring system and method for cooperation mechanical arm are carried based on omni-directional moving platform | |
| CN114187312A (en) | Target object grabbing method, device, system, storage medium and equipment | |
| CN210277081U (en) | Floor sweeping robot | |
| CN113524216A (en) | Fruit and vegetable picking robot based on multi-frame fusion and control method thereof | |
| CN203242165U (en) | All-in-one machine for video velocity measurement and license plate identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |