CN111931577A - Intelligent inspection method for specific foreign matters of power grid line - Google Patents
Intelligent inspection method for specific foreign matters of power grid line Download PDFInfo
- Publication number
- CN111931577A CN111931577A CN202010646192.9A CN202010646192A CN111931577A CN 111931577 A CN111931577 A CN 111931577A CN 202010646192 A CN202010646192 A CN 202010646192A CN 111931577 A CN111931577 A CN 111931577A
- Authority
- CN
- China
- Prior art keywords
- power grid
- image
- grid line
- detection
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
- G07C1/20—Checking timed patrols, e.g. of watchman
Abstract
An intelligent inspection method for specific foreign matters in a power grid line belongs to the field of line detection. Adopting manual/unmanned vehicles/unmanned aerial vehicles to automatically shoot, collect and transmit pictures upwards below a power grid line by taking a power grid and the sky as backgrounds; automatically preprocessing the image by a picture scale scaling algorithm and storing the image into a target detection image folder; based on a training set data enhancement method, the fast-RCNN detection model performance when the number of the original training set pictures is insufficient is greatly improved, and a detection result is obtained after the images to be detected are subjected to model detection. The method combines the existing manual/unmanned vehicle/unmanned aerial vehicle routing inspection, and provides a shooting target at a specific angle to simplify the image background and improve the detection effect; the workload of manual image processing is reduced by compiling an automatic image size processing algorithm, automatic fault identification is realized based on a computer vision technology, the intelligent level of the whole inspection process is improved, and the labor cost is saved. The intelligent routing inspection system can be widely applied to the field of intelligent routing inspection of power grid lines.
Description
Technical Field
The invention belongs to the field of power grid line detection, and particularly relates to a method for detecting specific foreign matters such as kites and balloons in a power grid line.
Background
Various foreign matters, particularly bird nests, kites, balloons and the like can be hung on the power grid line, the existence of the foreign matters threatens the insulativity of the power grid line, and the power failure accident of a large area caused by the short circuit fault of the line can be caused, so that the normal production and life of people are influenced, and even huge economic loss can be caused. Foreign matter hanging wire is an important potential hidden danger threatening the safe and stable operation of the power grid.
The high-speed development of the power grid in China has the advantages that the total length of the power transmission line exceeds 118 kilometers, the scale has leaped the first place in the world, the power transmission line is scattered, the area is wide, the terrain is complex, and the natural environment is severe. The manpower line patrol is high in cost, long in time, difficult and high in risk.
The unmanned aerial vehicle and the image recognition can be used for completing the electric power inspection task in a 'quick, good and economical mode'. With the rapid development of unmanned aerial vehicle/vehicle inspection technology, computer technology and communication technology, image accumulation has become a conventional phenomenon in national network in recent years and the trend thereof is increased continuously, a bottleneck exists in the speed of processing images and the image learning of tens of thousands of images, descriptions and results are often different from person to person and are easy to form confusion, and inconvenience is brought to equipment operation and maintenance and management.
In recent years, the development of artificial intelligence, particularly the development of deep learning, brings a great breakthrough to the field of computer vision (cv), and also provides a chance for processing a large amount of inspection photos.
Disclosure of Invention
The invention aims to provide an intelligent inspection method for specific foreign matters in a power grid line. The method uses the latest technology developed by artificial intelligence for reference, automatically processes the inspection photo, and realizes the abnormal detection and identification of the photo. Further, a line fault database is initially established, and various foreign body suspensions and the like are involved; and the obtained information is better mined and utilized by combining a big data technology, and preliminary fault prediction and evaluation are realized.
The technical scheme of the invention is as follows: the intelligent inspection method for the specific foreign matters in the power grid line is characterized by comprising the following steps:
1) adopting an artificial/unmanned vehicle/unmanned aerial vehicle to automatically shoot pictures upwards below a power grid line by taking a power grid and the sky as backgrounds, and collecting and transmitting the pictures;
2) automatically preprocessing the image by a picture scale scaling algorithm and storing the image into a target detection image folder;
3) based on a training set data enhancement method, the fast-RCNN detection model performance when the number of the original training set pictures is insufficient is greatly improved, and a detection result is obtained after the images to be detected are subjected to model detection.
Specifically, when an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network is trained, a power grid line foreign matter hanging picture set under different shooting angles, different illumination intensities under different weather conditions, different picture definitions, different surrounding environments and different state conditions is used as a training sample.
Further, when an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network is trained, a training set is expanded by using a picture data enhancement method of rotating, changing brightness, chroma and contrast so as to further improve the performance of the training model.
Specifically, in an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network, a manual/unmanned vehicle/unmanned aerial vehicle is adopted to automatically shoot and collect images, the shooting angle takes a power grid and the sky as the background, namely the shooting angle is upward, so that the background complexity of the images to be detected is further simplified, and the detection precision is improved.
Further, in an automatic detection model of fast-RCNN based on the inclusion V2 feature extraction network, a picture scale scaling algorithm is written, and a picture to be detected is automatically processed to a proper scale to adapt to the size of a detected image result output window.
Further, in an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network, a GoogLeNet inclusion V2 convolutional neural network model is adopted to carry out a series of operations of convolution, pooling and activation function combination on an original image, so as to extract a feature map of a target image.
According to the intelligent inspection method, the workload of manual image processing is reduced by compiling the automatic image size processing algorithm, the automatic fault identification is realized based on the computer vision technology, the intelligent level of the whole inspection process is improved, and the labor cost is saved.
Compared with the prior art, the invention has the advantages that:
1) the invention combines the automatic inspection of unmanned vehicles/unmanned planes to realize automatic shooting, collection and transmission of pictures, and can better utilize the characteristics of convenience and intellectualization of automatic equipment;
2) according to the invention, a power grid line foreign matter hanging picture set under different shooting angles, different illumination intensities under different weather conditions, different picture definitions, different surrounding environments and different state conditions is adopted as a training sample, and higher detection precision can be obtained in scenes with complex environments and more background interference by improving the diversity of the training sample;
3) according to the invention, by compiling a picture scale scaling algorithm, the picture to be detected is automatically processed to a proper scale to adapt to the size of a result output window of the detected image, so that the labor cost for manually processing the picture to be detected is reduced;
4) according to the invention, an automatic detection model of the fast-RCNN based on the Incepton V2 feature extraction network is adopted to realize intelligent analysis of the foreign matter image of the power grid line, and the final detection effect and the intelligent level are improved.
Drawings
FIG. 1 is a block diagram of the intelligent inspection method of the present invention;
FIG. 2 is a schematic flow chart of the master RCNN algorithm of the present invention;
fig. 3a and 3b are schematic diagrams of the detection effect of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
As shown in fig. 1, the technical solution of the present invention provides an intelligent inspection method for foreign matters in a power grid line, which is characterized in that the intelligent inspection method comprises the following steps:
1) the manual/unmanned vehicle/unmanned aerial vehicle takes a power grid and the sky as backgrounds, and automatically shoots, collects and transmits pictures upwards below a power grid line;
2) automatically preprocessing the image by a picture scale scaling algorithm and storing the image into a target detection image folder;
3) based on a training set data enhancement method, the fast-RCNN detection model performance when the number of the original training set pictures is insufficient is greatly improved, and a detection result is obtained after the images to be detected are subjected to model detection.
The hardware configuration for realizing the method comprises a computer host (CPU is 32G) and an independent graphics card (RTX 2080Super) for accelerating the operation process of training the neural network model, the software configuration is Windows10, Anaconda3, CUDA Tool kit9.0+ CuDNN7.0 and Tensorflow-gpu1.9, and the detection result is obtained based on the automatic detection model of training fater-RCNN.
The convolutional neural network performs convolutional calculation on image data by using a convolutional core with a certain scale, can well extract the characteristics of image edges, textures and the like, has a very good effect in the fields of image analysis and target detection, and has a basic structure comprising an input layer, a convolutional layer, a pooling layer, a full-link layer and an output layer. In the convolutional layer, parameters in the convolutional core are used as weights and local inputs to carry out weighted summation and then add offset values, the obtained output result is activated by an activation function to obtain nonlinearity, and then the local maximum value or the local average value is selected by the pooling layer to reduce the data dimensionality so as to reduce the overfitting degree. The convolution layers and the pooling layers are alternately arranged, convolution and pooling operations are performed for multiple times, image features are fully extracted, and a more robust neural network is trained.
Increasing the depth (number of layers of a neural network) or the width (number of convolution kernels or neurons in each layer) of a model is the safest way to obtain a high-quality model, but the method has the disadvantages of too many parameters, increased computational complexity, easy generation of overfitting under the condition of limited training data set, and difficulty in optimizing the model due to the easy occurrence of the problem of gradient dispersion when the number of layers of the network is large.
In the technical scheme, a GoogLenet inclusion V2 is used as a feature extraction neural network model, the inclusion V2 model reduces the thickness of a feature map by adopting a 1 × 1 convolution kernel, and two 3 × 3 convolution kernels are used for replacing a 5 × 5 convolution kernel, so that the parameter amount is greatly reduced under the condition of ensuring that the expression effect is almost unchanged, and the overfitting degree is reduced; meanwhile, the BN regularization method is adopted to accelerate the training speed of the network and improve the classification accuracy after convergence.
And the feature map extracted by the GoogLenet enters an RPN (resilient packet network), and 9 anchor boxes are generated on each point of the feature map of the RPN for judging the category and the position of the target in the image. Firstly, performing dimension conversion on a feature map input into an RPN network by using 3-by-3 sliding window, further concentrating feature information, forming two fully-connected layers after 18-by-1 and 36-by-1 convolution kernels respectively, wherein one layer is a box-regression layer (reg), and 18 (2-by-9) convolution kernel outputs of 1-by-1 are used for judging whether images in 9 anchor boxes are foreground (fg) or background (bg); and the other is a box-classification layer (cls),36(4 × 9) convolution kernels with 1 × 1 are output as coordinate information (x, y, w, h) of 9 anchor boxes, the anchor boxes are subjected to out-of-range filtering, subjected to IOU (input output) calculation with a ground channel and then marked with positive and negative samples, subjected to coordinate offset calculation to generate more accurate region probes, and then overlapped anchor boxes are further filtered out through NMS non-maximum suppression. And the ROI pooling is to pool coordinates of four vertexes of the region, so that the candidate regions with different sizes have the same size of feature vectors after pooling, the feature vectors are used as the input of a full connection layer, a softmax function is adopted to classify the region samples into specific categories, the region samples are subjected to bounding box regression again, and a higher-precision rectangle box is obtained.
The flow chart of the fast RCNN algorithm adopted in the technical scheme of the invention is shown in FIG. 2, in this case, a fast _ RCNN _ initiation _ v2_ COCO model pre-trained on a COCO data set is adopted, and the pre-trained model can reduce the required sample amount of a training set, reduce the training period and improve the training efficiency.
Example (b):
power grid specific foreign matter-kite detection:
the kite has the characteristics of rich colors, various shapes and the like, various hanging states of the kite on a power grid line are damaged to a certain extent after being exposed to sunlight and rain in a natural environment for a long time, and states of the kite presented on an image are various due to different shooting angles, so that the difficulty of target detection is increased.
In the embodiment, 60 power grid line kite foreign matter graphs, 40 outdoor (in various scenes) kite-flying graphs and 40 kite pictures of kites at different angles are selected, in order to further increase the sample size, 140 pictures are subjected to rotation, brightness, color saturation and contrast change, picture data enhancement is performed on a training set to expand a data set, and the performance of a model is improved to a certain extent. The training set pictures are transformed as follows:
the original 140 pictures of the training set are expanded into 140 x (4 x 2+1) pictures, namely 1260 pictures, by the data enhancement method.
After the pictures are labeled, the pictures are used as a training set to perform fine-tune on a pre-trained false RCNN model, and the training parameters are set as follows:
1、step1-14000，learning-rate＝0.0002；
2、step14001-18000,learning rate＝0.00002；
3、step18001-20000,learning rate＝0.000002
the IOU thresholds for marking positive and negative samples are taken to be 0.7 and 0.3, respectively, as explained in detail below:
1. if the IoU values of the anchor box and the ground channel are the maximum, marking as a positive sample;
2. if IoU of the anchor box and ground channel is >0.7, marking as positive sample;
3. if IoU of the anchor box and ground channel is <0.3, it is marked as a negative sample.
And storing the trained model parameters, and waiting for the input of the detection picture set.
And calling an automatic image scale scaling algorithm to process the image to a uniform scale, and storing the image into a detection target folder.
The trained model is tested, pictures shot under the conditions of different light brightness, different angles and different distances are selected, the test results are shown in fig. 3(a) and fig. 3(b), and the trained model can achieve a good detection effect under various scenes.
According to the technical scheme, the existing manual/unmanned vehicle/unmanned aerial vehicle routing inspection is combined, a target is shot at a specific angle to simplify the image background, and the detection effect is improved; the workload of manual image processing is reduced by compiling an automatic image size processing algorithm, automatic fault identification is realized based on a computer vision technology, the intelligent level of the whole inspection process is improved, the labor cost is saved, and the final detection effect is improved.
The intelligent foreign matter inspection system can be widely applied to the field of intelligent inspection of specific foreign matters of power grid lines.
Claims (7)
1. An intelligent inspection method for specific foreign matters in a power grid line is characterized by comprising the following steps:
1) adopting an artificial/unmanned vehicle/unmanned aerial vehicle to automatically shoot pictures upwards below a power grid line by taking a power grid and the sky as backgrounds, and collecting and transmitting the pictures;
2) automatically preprocessing the image by a picture scale scaling algorithm and storing the image into a target detection image folder;
3) based on a training set data enhancement method, the fast-RCNN detection model performance when the number of the original training set pictures is insufficient is greatly improved, and a detection result is obtained after the images to be detected are subjected to model detection.
2. The intelligent power grid line foreign matter inspection method according to claim 1, wherein when an automatic detection model of fast-RCNN of an inclusion V2 feature extraction-based network is trained, a power grid line foreign matter hanging picture set at different shooting angles, different illumination intensities under different weather conditions, different picture definitions, different surrounding environments and different state conditions is used as a training sample.
3. The intelligent power grid line foreign matter inspection method according to claim 1, wherein when an initiation V2 feature extraction network-based fast-RCNN automatic detection model is trained, a training set is expanded by using a rotation, brightness, chromaticity and contrast image data enhancement method to further improve the performance of the training model.
4. The intelligent power grid line foreign matter inspection method according to claim 1, wherein in an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network, an image is automatically shot and collected by a manual/unmanned vehicle/unmanned aerial vehicle, and the shooting angle is based on a power grid and the sky, namely the shooting angle is upward, so that the background complexity of the image to be detected is further simplified, and the detection precision is improved.
5. The intelligent power grid line foreign matter inspection method according to claim 4, wherein in an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network, a picture scale scaling algorithm is written, and a picture to be detected is automatically processed to a proper scale to adapt to the size of a detection image result output window.
6. The intelligent power grid line foreign matter inspection method according to claim 1, wherein in an automatic detection model of fast-RCNN based on an inclusion V2 feature extraction network, a GoogLeNet inclusion V2 convolutional neural network model is adopted to perform a series of operations of convolution, pooling and activation function combination on an original image, so as to extract a feature map of a target image.
7. The intelligent inspection method for specific foreign matters in a power grid line according to claim 1, wherein the intelligent inspection method reduces the workload of manual image processing by compiling an automatic image size processing algorithm, realizes automatic fault identification based on a computer vision technology, improves the intelligent level of the whole inspection process and saves labor cost.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010646192.9A CN111931577A (en) | 2020-07-07 | 2020-07-07 | Intelligent inspection method for specific foreign matters of power grid line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010646192.9A CN111931577A (en) | 2020-07-07 | 2020-07-07 | Intelligent inspection method for specific foreign matters of power grid line |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111931577A true CN111931577A (en) | 2020-11-13 |
Family
ID=73312287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010646192.9A Pending CN111931577A (en) | 2020-07-07 | 2020-07-07 | Intelligent inspection method for specific foreign matters of power grid line |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111931577A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112651337A (en) * | 2020-12-25 | 2021-04-13 | 国网黑龙江省电力有限公司电力科学研究院 | Sample set construction method applied to training line foreign object target detection model |
| CN112909824A (en) * | 2021-03-24 | 2021-06-04 | 南方电网电力科技股份有限公司 | Method and device for identifying suspended foreign matters of power transmission line |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734143A (en) * | 2018-05-28 | 2018-11-02 | 江苏迪伦智能科技有限公司 | A kind of transmission line of electricity online test method based on binocular vision of crusing robot |
| CN109493337A (en) * | 2018-11-15 | 2019-03-19 | 中国石油大学(华东) | A kind of electric line foreign matter detection method based on improved Faster RCNN |
| CN110334612A (en) * | 2019-06-19 | 2019-10-15 | 上海交通大学 | Electric inspection process image object detection method with self-learning capability |
| CN110989657A (en) * | 2019-11-15 | 2020-04-10 | 国网河南省电力公司洛阳供电公司 | Mid-tower patrol inspection method based on unmanned aerial vehicle infrared detection |
| CN111079747A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bogie side frame fracture fault image identification method |
-
2020
- 2020-07-07 CN CN202010646192.9A patent/CN111931577A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734143A (en) * | 2018-05-28 | 2018-11-02 | 江苏迪伦智能科技有限公司 | A kind of transmission line of electricity online test method based on binocular vision of crusing robot |
| CN109493337A (en) * | 2018-11-15 | 2019-03-19 | 中国石油大学(华东) | A kind of electric line foreign matter detection method based on improved Faster RCNN |
| CN110334612A (en) * | 2019-06-19 | 2019-10-15 | 上海交通大学 | Electric inspection process image object detection method with self-learning capability |
| CN110989657A (en) * | 2019-11-15 | 2020-04-10 | 国网河南省电力公司洛阳供电公司 | Mid-tower patrol inspection method based on unmanned aerial vehicle infrared detection |
| CN111079747A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bogie side frame fracture fault image identification method |
Non-Patent Citations (2)
| Title |
|---|
| 张凡 等: "一种高压线异物检测算法", 《现代电子技术》, pages 36 - 40 * |
| 王纪武 等: "基于Faster R-CNN的多尺度高压塔鸟巢检测", 《北京交通大学学报》, pages 37 - 43 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112651337A (en) * | 2020-12-25 | 2021-04-13 | 国网黑龙江省电力有限公司电力科学研究院 | Sample set construction method applied to training line foreign object target detection model |
| CN112909824A (en) * | 2021-03-24 | 2021-06-04 | 南方电网电力科技股份有限公司 | Method and device for identifying suspended foreign matters of power transmission line |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784685B (en) | Power transmission line defect image identification method based on cloud edge cooperative detection | |
| CN111428748B (en) | HOG feature and SVM-based infrared image insulator identification detection method | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN112199993B (en) | Method for identifying transformer substation insulator infrared image detection model in any direction based on artificial intelligence | |
| WO2021022970A1 (en) | Multi-layer random forest-based part recognition method and system | |
| CN109902806A (en) | Method is determined based on the noise image object boundary frame of convolutional neural networks | |
| CN111695514B (en) | Vehicle detection method in foggy days based on deep learning | |
| CN109034184B (en) | Grading ring detection and identification method based on deep learning | |
| CN111814686A (en) | Vision-based power transmission line identification and foreign matter invasion online detection method | |
| CN114743119B (en) | High-speed rail contact net hanger nut defect detection method based on unmanned aerial vehicle | |
| CN112560634B (en) | Power line insulator rapid detection and positioning method and system based on field image | |
| Lan et al. | Defect detection from UAV images based on region-based CNNs | |
| CN111639530B (en) | Method and system for detecting and identifying power transmission tower and insulator of power transmission line | |
| CN105868734A (en) | Power transmission line large-scale construction vehicle recognition method based on BOW image representation model | |
| Yan et al. | Insulator detection and recognition of explosion based on convolutional neural networks | |
| CN111931577A (en) | Intelligent inspection method for specific foreign matters of power grid line | |
| CN115862073A (en) | Transformer substation harmful bird species target detection and identification method based on machine vision | |
| CN113297915A (en) | Insulator recognition target detection method based on unmanned aerial vehicle inspection | |
| CN109325426B (en) | Black smoke vehicle detection method based on three orthogonal planes time-space characteristics | |
| CN117197700B (en) | Intelligent unmanned inspection contact net defect identification system | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| CN110503049B (en) | Satellite video vehicle number estimation method based on generation countermeasure network | |
| CN116309407A (en) | Method for detecting abnormal state of railway contact net bolt | |
| CN116071299A (en) | Insulator RTV spraying defect detection method and system | |
| CN110334703B (en) | Ship detection and identification method in day and night image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |