CN114998199A - Infrared, ultraviolet and visible light fusion power grid equipment state detection system - Google Patents

Infrared, ultraviolet and visible light fusion power grid equipment state detection system Download PDF

Info

Publication number
CN114998199A
CN114998199A CN202210436022.7A CN202210436022A CN114998199A CN 114998199 A CN114998199 A CN 114998199A CN 202210436022 A CN202210436022 A CN 202210436022A CN 114998199 A CN114998199 A CN 114998199A
Authority
CN
China
Prior art keywords
light sensor
visible light
ultraviolet
infrared
fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210436022.7A
Other languages
Chinese (zh)
Inventor
张逸峰
施天宇
孙颖
谢励耘
杨建平
颜楠楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Mingxu Intelligent Technology Co ltd
State Grid Shanghai Electric Power Co Ltd
Original Assignee
Shanghai Mingxu Intelligent Technology Co ltd
State Grid Shanghai Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Mingxu Intelligent Technology Co ltd, State Grid Shanghai Electric Power Co Ltd filed Critical Shanghai Mingxu Intelligent Technology Co ltd
Priority to CN202210436022.7A priority Critical patent/CN114998199A/en
Publication of CN114998199A publication Critical patent/CN114998199A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8806Specially adapted optical and illumination features
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N25/00Investigating or analyzing materials by the use of thermal means
    • G01N25/72Investigating presence of flaws
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01RMEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/12Testing dielectric strength or breakdown voltage ; Testing or monitoring effectiveness or level of insulation, e.g. of a cable or of an apparatus, for example using partial discharge measurements; Electrostatic testing
    • G01R31/1218Testing dielectric strength or breakdown voltage ; Testing or monitoring effectiveness or level of insulation, e.g. of a cable or of an apparatus, for example using partial discharge measurements; Electrostatic testing using optical methods; using charged particle, e.g. electron, beams or X-rays
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Analytical Chemistry (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention relates to a state detection system of infrared, ultraviolet and visible light fusion power grid equipment, which comprises a front end, an edge calculation end, an access network pipe side and a server cloud end which are sequentially connected; the front end comprises an infrared light sensor, an ultraviolet light sensor and a visible light sensor; the infrared light sensor, the ultraviolet light sensor and the visible light sensor are all used for acquiring image information of the power grid equipment; the edge calculation end is used for carrying out data fusion on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor; the access network management side is used for data communication between the edge computing side and the server cloud side; and the server cloud is used for carrying out fault analysis according to the input image data through a deep learning algorithm. Compared with the prior art, the method can automatically fuse the characteristic information of the equipment state under different spectral bands, establish joint analysis, reduce the operation risk of the power grid and improve the intelligent level of the sensing terminal.

Description

Infrared, ultraviolet and visible light fusion power grid equipment state detection system
Technical Field
The invention relates to the technical field of power grid equipment maintenance, in particular to a system for detecting the state of infrared, ultraviolet and visible light fused power grid equipment.
Background
In order to accurately measure and analyze the defect information of the power transmission and transformation equipment, a large number of imaging technologies and image detection devices based on photoelectric sensing are applied to the operation maintenance and the state overhaul of the power grid equipment. However, the existing image detection equipment mostly adopts one or two spectral bands to carry out defect detection of the power equipment, the sensing dimensionality coverage of an image sensor is insufficient, and the visual signal characteristics of an event cannot be represented. The space-time correlation of data acquired by different devices is weak, and fusion and sharing are difficult to realize. For example, if it is desired to detect the multispectral image information of one power device, at least two devices are required to be used for detection, and the multispectral image information at the same time and at the same observation position cannot be obtained, so that cross-complementation of multiple detection technologies is not realized, and the synergistic effect in the defect/fault occurrence process is ignored, which results in low field detection efficiency. In addition, the image sensing data has low dimensionality and weak relevance, so that the analysis technology capability of the existing application system is insufficient.
Disclosure of Invention
The invention aims to provide an infrared-ultraviolet-visible light fusion power grid equipment state detection system based on the characteristic distribution and time domain incidence relation of the power equipment state under different spectral bands, so as to overcome the defects that in the prior art, a power sensor is low in sensing data acquisition dimensionality and relevance, weak in multi-spectral-band time sequence data intelligent analysis capability and poor in effective feedback instantaneity of mass data input and calculation systems.
The purpose of the invention can be realized by the following technical scheme:
a state detection system for infrared, ultraviolet and visible light fusion power grid equipment comprises a front end, an edge calculation end, an access network pipe side and a server cloud end which are sequentially connected;
the front end comprises an infrared light sensor, an ultraviolet light sensor and a visible light sensor; the infrared light sensor, the ultraviolet light sensor and the visible light sensor are all used for acquiring image information of the power grid equipment;
the edge calculation end is used for carrying out data fusion on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor;
the access network management side is used for data communication between the edge computing side and the server cloud side;
the server cloud is used for performing fault analysis according to input image data through a deep learning algorithm, performing parameter quantification on the obtained data, setting a network structure of the deep learning algorithm and performing hardware acceleration.
Further, the data fusion comprises multi-mode fusion, multi-level fusion and perception and knowledge reasoning fusion of image data detected by an infrared light sensor, an ultraviolet light sensor and a visible light sensor, wherein the multi-level fusion comprises decision result fusion, feature fusion and pixel fusion.
Further, the edge calculation terminal is further used for performing time domain correlation on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor by adopting an SIFT matching method and an SURF matching method.
Further, the edge calculation end is further configured to perform time-domain evolution situation awareness according to a time-domain correlation result of the image data.
Further, the server cloud is further used for performing time sequence evolution trend analysis according to the time domain correlation result and the time domain evolution situation perception result of the image data.
Further, the server cloud end obtains a fault analysis result by referring to a fault feature map in a fault image analysis library obtained in advance through a deep learning algorithm.
Further, the fault analysis result comprises an infrared heating defect, an ultraviolet discharge defect and a visible light external damage defect.
Further, the grid device includes: the device comprises a transformer, a breaker, a current-voltage transformer, a lightning arrester, a reactor, a bus, a capacitor, a combined electrical apparatus and a line tension tower.
And further, carrying out data processing after preprocessing, data cleaning and automatic labeling on the data acquired by the front end.
Further, the infrared light sensor, the ultraviolet light sensor and the visible light sensor are connected with each other for physical coupling.
Compared with the prior art, the invention has the following advantages:
(1) according to the invention, through front-end three-optical photoelectric sensing, high-precision multi-state fusion and lightweight structure design and combined with the design of acceleration of the terminal combined platform bilateral AI intelligent algorithm, the key problems of high spatial-temporal dimension alignment, wide-area time domain evolution, whole scene perception, algorithm self-evolution and the like of multi-mode data are fundamentally solved, and further intelligent upgrading of the equipment state perception system is realized.
(2) Compared with the traditional equipment state detection system, the method can automatically fuse the characteristic information of the equipment state in different spectral bands, establish joint analysis, reduce the operation risk of the power grid and improve the intelligent level of the sensing terminal.
Drawings
Fig. 1 is a schematic structural diagram of a state detection system for an infrared, ultraviolet, and visible light fusion power grid device provided in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.
Example 1
As shown in fig. 1, the present embodiment provides a system for detecting a state of an infrared, ultraviolet, and visible light fusion power grid device, including a front end, an edge computing end, an access network side, and a server cloud end, which are connected in sequence;
the front end comprises an infrared light sensor, an ultraviolet light sensor and a visible light sensor; the infrared light sensor, the ultraviolet light sensor and the visible light sensor are all used for acquiring image information of the power grid equipment;
the edge calculation end is used for carrying out data fusion on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor;
the access network management side is used for data communication between the edge computing side and the server cloud side;
the server cloud is used for performing fault analysis according to input image data through a deep learning algorithm, performing parameter quantification on the obtained data, setting a network structure of the deep learning algorithm and performing hardware acceleration.
Each part is described in detail below.
1. Front end
In this embodiment front end, the electric wire netting equipment through infrared light sensor, ultraviolet sensor and visible light sensor monitoring includes: the device comprises a transformer, a breaker, a current-voltage transformer, a lightning arrester, a reactor, a bus, a capacitor, a combined electrical apparatus and a line tension tower.
In the front end, the infrared light sensor, the ultraviolet light sensor and the visible light sensor are connected to each other for physical coupling.
2. Edge calculation terminal
The edge computing end carries out preprocessing, data cleaning and automatic labeling on data collected by the front end and then carries out data fusion, the data fusion comprises multi-mode fusion, multi-level fusion and sensing and knowledge reasoning fusion on image data detected by an infrared light sensor, an ultraviolet light sensor and a visible light sensor, and the multi-level fusion comprises decision result fusion, feature fusion and pixel fusion.
The edge calculation end is also used for performing time domain correlation on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor by adopting an SIFT matching method and an SURF matching method, and performing time domain evolution situation perception.
3. Server cloud
And the cloud end of the server acquires a fault analysis result by referring to a fault characteristic map in a fault image analysis library acquired in advance through a deep learning algorithm. The fault analysis result comprises infrared heating defects, ultraviolet discharge defects and visible light external damage defects. The server cloud has an equipment query function, a map query function and a time sequence data analysis function;
the equipment inquiry function realizes the inquiry of the production date, the affiliated interval and the running state of the equipment;
the map query function is used for querying the three-light shooting time, the equipment voltage level and the defect level;
and the time sequence data analysis function realizes time sequence evolution trend analysis and correlation analysis.
Specifically, in this embodiment, a multispectral fusion intelligent sensing system with high space-time dimension is constructed from the full links of the four layers of "cloud-pipe-edge-end";
at a sensing terminal, the physical coupling of photoelectric sensing is realized from the front end, the input end of a high space-time dimension panoramic sensing database is constructed, and the method has the characteristics of intelligence, high precision, easiness in deployment and the like.
At an edge computing end, multi-mode fusion, multi-level (decision, feature and pixel) fusion, perception and knowledge reasoning fusion are carried out on ultraviolet, infrared and visible light three-light images suitable for power grid equipment, and further, cooperation, mutual inspection, correlation analysis and state/trend perception and prediction of power grid equipment states across-domain events and multi-time scales based on three-light image pixel level accurate recognition are achieved.
On the side of an access network management, a three-light image deep learning lightweight method of a power grid device with an edge computing terminal and a server cloud is adopted, and a three-layer framework of a network structure, parameter quantification and hardware acceleration is adopted to realize that a deep network model can automatically obtain a power device defect diagnosis result on site in time, so that the problem that the instantaneity of mass data input and effective feedback of a computing system is poor in the past is solved, the ubiquitous and real-time performance of connection is improved, and the concurrent, real-time and systematic cooperative computing of mass data is supported, so that the safe operation of the device is ensured, and the power supply reliability is improved.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations can be devised by those skilled in the art in light of the above teachings. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims (10)

1. The infrared, ultraviolet and visible light fusion power grid equipment state detection system is characterized by comprising a front end, an edge calculation end, an access network pipe side and a server cloud end which are sequentially connected;
the front end comprises an infrared light sensor, an ultraviolet light sensor and a visible light sensor; the infrared light sensor, the ultraviolet light sensor and the visible light sensor are all used for acquiring image information of the power grid equipment;
the edge calculation end is used for carrying out data fusion on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor;
the access network management side is used for data communication between the edge computing side and the server cloud side;
the server cloud is used for performing fault analysis according to input image data through a deep learning algorithm, performing parameter quantification on the obtained data, setting a network structure of the deep learning algorithm and performing hardware acceleration.
2. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 1, wherein the data fusion comprises multi-modal fusion, multi-level fusion and perception and knowledge inference fusion of image data detected by an infrared light sensor, an ultraviolet light sensor and a visible light sensor, and the multi-level fusion comprises decision result fusion, feature fusion and pixel fusion.
3. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 1, wherein the edge computing terminal is further configured to perform time domain correlation on image data detected by the infrared light sensor, the ultraviolet light sensor and the visible light sensor by using a SIFT matching method and a SURF matching method.
4. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 3, wherein the edge calculation end is further configured to perform time-domain evolution situation awareness according to a time-domain correlation result of image data.
5. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 4, wherein the server cloud is further configured to perform time-series evolution trend analysis according to a time-domain correlation result and a time-domain evolution situation perception result of the image data.
6. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 1, wherein the server cloud obtains a fault analysis result by referring to a fault feature map in a fault image analysis library obtained in advance through a deep learning algorithm.
7. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 6, wherein the fault analysis result comprises an infrared heating defect, an ultraviolet discharge defect and a visible light external damage defect.
8. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 1, wherein the power grid equipment comprises: the device comprises a transformer, a circuit breaker, a current-voltage transformer, a lightning arrester, a reactor, a bus, a capacitor, a combined electrical appliance and a line strain tower.
9. The system for detecting the state of the infrared, ultraviolet and visible light fusion power grid equipment according to claim 1, wherein data collected at the front end is preprocessed, cleaned and automatically labeled and then processed.
10. The system for detecting the state of the infrared-ultraviolet-visible light fusion power grid device according to claim 1, wherein the infrared light sensor, the ultraviolet light sensor and the visible light sensor are connected to each other for physical coupling.
CN202210436022.7A 2022-04-24 2022-04-24 Infrared, ultraviolet and visible light fusion power grid equipment state detection system Pending CN114998199A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210436022.7A CN114998199A (en) 2022-04-24 2022-04-24 Infrared, ultraviolet and visible light fusion power grid equipment state detection system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210436022.7A CN114998199A (en) 2022-04-24 2022-04-24 Infrared, ultraviolet and visible light fusion power grid equipment state detection system

Publications (1)

Publication Number Publication Date
CN114998199A true CN114998199A (en) 2022-09-02

Family

ID=83025241

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210436022.7A Pending CN114998199A (en) 2022-04-24 2022-04-24 Infrared, ultraviolet and visible light fusion power grid equipment state detection system

Country Status (1)

Country Link
CN (1) CN114998199A (en)

Similar Documents

Publication Publication Date Title
CN109785289B (en) Transmission line defect detection method and system and electronic equipment
CN109242439B (en) Feature extraction and identification method based on associated data of substation equipment
CN101957325B (en) Substation equipment appearance abnormality recognition method based on substation inspection robot
CN112734692A (en) Transformer equipment defect identification method and device
CN108344931B (en) Power equipment safety analysis system based on ultraviolet spectrum technology
WO2006037874A1 (en) Method and device for detecting electric arc phenomenon on at least one electric cable
CN116723211B (en) Remote inspection device for large-space environment power equipment
CN113205039A (en) Power equipment fault image identification and disaster investigation system and method based on multiple DCNNs
CN108280449A (en) Power equipment image collecting method based on multispectral sensor group
CN112906620A (en) Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment
CN114743089B (en) Image recognition GIS fault diagnosis device and method based on SSA-SVM
CN115236451A (en) Power distribution network overhead line fault monitoring device and method
CN106355187A (en) Application of visual information to electrical equipment monitoring
WO2024119983A1 (en) Anomality detection method and apparatus for cable terminal of rail vehicle
CN112734637B (en) Thermal infrared image processing method and system for monitoring temperature of lead
CN117856442A (en) Power equipment remote monitoring system and method based on big data
CN117372954A (en) Charging station safety monitoring method, device, system and computer equipment
CN117367589A (en) All-weather auxiliary temperature measurement system for substation main equipment based on AI technology
CN114998199A (en) Infrared, ultraviolet and visible light fusion power grid equipment state detection system
CN106248215A (en) A kind of real-time calculating system of power transformer critical component running temperature
CN114243925B (en) Intelligent fusion terminal-based distribution substation allergy sensing method and system
Wang et al. Exploration on Automatic Management of GIS Using TL-CNN and IoT
CN109541347B (en) Sensor for electric energy metering
CN112101375A (en) Intelligent substation equipment state detection method and system based on deep learning
CN112200030A (en) Power system field operation action risk identification method based on graph convolution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination