CN114581722A - Two-stage multi-classification industrial image defect detection method based on twin residual error network - Google Patents
Two-stage multi-classification industrial image defect detection method based on twin residual error network Download PDFInfo
- Publication number
- CN114581722A CN114581722A CN202210285597.3A CN202210285597A CN114581722A CN 114581722 A CN114581722 A CN 114581722A CN 202210285597 A CN202210285597 A CN 202210285597A CN 114581722 A CN114581722 A CN 114581722A
- Authority
- CN
- China
- Prior art keywords
- network
- detection
- twin
- classification
- fpn
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a twin residual error network-based two-stage multi-classification defect detection method, which comprises the steps of inputting an image to be detected and template information thereof into a network together, extracting difference characteristics, combining the difference characteristics into a residual error network taking the template information as input according to a residual error network design idea, and extracting multi-classification characteristics; training a two-classification detection network by using the difference characteristics; and using a proposal detection frame of the detection network for training the multi-classification detection network to complete the task of defect multi-classification detection. In the invention, the twin network structure and the residual error network structure decouple the defect and template information, so that the network can learn the defect part and the classification part more clearly, and has strong migration capability, and simultaneously, the characteristics and the advantages of the two-classification detection network and the multi-classification detection network are combined, and the network structure which can accurately detect the defect and can accurately carry out multi-classification is designed.
Description
Technical Field
The invention relates to the technical field of image detection, in particular to a two-stage multi-classification industrial image defect detection method based on a twin residual error network.
Background
With the rapid development of artificial intelligence technology in the field of digital media processing and the rapid improvement of hardware performance in recent years, the performance of computer vision technology is continuously improved, and the deployment cost is continuously reduced. The landing conditions of vision-related artificial intelligence algorithms tend to mature and are in fact increasingly applied in various industrial scenarios. In the industrial field of computer vision, automatic defect detection is an important application scene of related applications. The defect detection requires a sensor to acquire industrial product information, send the industrial product information into a preset detection system, and output whether the product has defects and related information of the defects.
In the defect detection process, different defects on the product have different influences on the product, so manufacturers often have different detection standards for different types of defects. Then, in order to improve the detection accuracy, the detection system is required to have excellent defect multi-classification capability; meanwhile, when the background texture of the product is complex and the product is diverse in variety, the detection system needs to have certain migration capability, and defects can be correctly classified as much as possible under an application scene which is not learned. In the face of the above requirements, the current detection scheme has the following defects:
firstly, most of the existing defect detection schemes adopt images to be detected as independent input data, such schemes are likely to cause the network to learn some auxiliary information related to a specific background, and the model migration capability cannot be ensured under such a condition. The system has high coupling degree of defect information and background information during training, and the model is difficult to learn the information only aiming at the defect. Therefore, when the detection system meets a new product, the attention point is difficult to be placed on the defect, and a large amount of false detection or missing detection phenomena are easy to generate on the data;
secondly, the prior knowledge of the intermediate features is not sufficiently utilized by the current defect detection system, and if a feature layer with prior information can be constructed, the information can be utilized to supervise and train the middle of the model, so that the network is easier to train, and the performance is better;
thirdly, the multi-classification and the two-classification of the current defect detection system are completely separated, the advantages of the two are not combined, and the two are not decoupled on a network structure, so that the multi-classification capability and the migration capability of the network are bottleneck and difficult to promote.
Disclosure of Invention
The present invention provides a two-stage multi-classification industrial image defect detection method based on a twin residual error network, which has at least solved one of the problems of the prior art.
In view of this, the specific scheme of the invention is as follows:
a two-stage multi-classification industrial image defect detection method based on a twin residual error network comprises the following steps:
s1, acquiring an image to be detected and a corresponding template image and preprocessing image data;
s2, inputting the image to be detected and the template image into a twin backbone network to complete image data processing, and inputting the formed data representation characteristics into a twin FPN network to obtain twin FPN characteristics;
s3, inputting template data into a residual error network, extracting by combining twin network characteristics to obtain residual error network characteristics, and inputting the residual error network characteristics into a residual error FPN network to obtain residual error FPN characteristics;
s4, sending the twin FPN characteristics into a segmentation network for supervision training, and optimizing the characteristics;
s5, inputting the twin FPN characteristics into a two-classification detection network for two-classification supervision training, and extracting an offer detection frame of the detection network;
s6, training a multi-classification detection network by using the proposed detection frame and the residual FPN characteristics;
and S7, detecting the image sample by using a multi-classification detection network and outputting a detection result.
In the present invention, step S5 is:
s51, sending the twin FPN characteristics to an RPN network, and calculating to obtain an offer detection frame;
and S52, sending the proposed detection frame and the twin FPN characteristics into a two-classification detection network to obtain detection frame output and corresponding two-classification scores, and marking the detection frame as a label for training.
In the present invention, step S6 is:
s61, inputting residual FPN characteristics and an proposed detection box of a two-class detection network into the multi-class detection network;
and S62, marking the multi-classification detection frame of the defect as a label, and training a multi-classification detection network.
In the present invention, the number of the feature channels of the twin backbone network in the step S2 is the same as the number of the feature layers corresponding to the residual network in the step S3.
Further, the shallow feature extraction layer of the twin backbone network is split in the step S2 to become two input ends, the twin network shares common parameters, two sets of features are output respectively, the result of the feature subtraction is input to the subsequent part of the network and merged into a strand of data, and the normal feature extraction of the backbone network is subsequently recovered.
Further, the step S3 includes:
s31, sending the template image into a residual error network, and extracting the intermediate features of the input data;
s32, finding the last residual block before the intermediate feature extraction, and combining the corresponding size of the twin network feature extracted in the step S2 into a residual block structure to obtain a residual network feature;
and S33, sending the residual error network characteristics to an FPN characteristic extraction module to obtain residual error FPN characteristics with fixed channel number.
Preferably, the twin FPN feature and the residual FPN feature are both features of D4-D64.
Compared with the prior art, the invention has the beneficial effects that:
1. the method uses the twin network structure to extract and learn the characteristics of the defect area, mainly analyzes the difference characteristics, and decouples and analyzes the defect information and the background template information, so that the network structure can detect the defect part more sensitively, improve the detection precision of the defect and improve the migration capability of the network to the defect detection;
2. the invention constructs a difference characteristic structure with the background expectation of 0, adds a network structure before the part of characteristics of the segmentation network supervised learning in the network training process, and fully utilizes the prior information to ensure that the network is easier to train and has higher precision;
3. the invention designs a twin residual error network structure, decouples the defect part from the background information and fuses the characteristic level at the same time, so that the network has stronger migration detection capability;
4. the invention integrates the functions of the difference characteristic information and the background information in the detection network, uses the difference characteristic to search the proposed detection frame of the defect, and uses the fusion characteristic of the background information and the difference characteristic information to classify the detection frame, so that the detection system has the capability of multi-classification of the defect on the basis of keeping the classification performance.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a general schematic diagram of a two-stage multi-classification industrial image defect detection method according to the present invention.
FIG. 2 is a schematic flow chart of a two-stage multi-classification industrial image defect detection method according to the present invention.
FIG. 3 is a flow chart of the two-stage multi-classification industrial image defect detection method for extracting network features.
FIG. 4 is a schematic diagram of a network feature using method of the two-stage multi-classification industrial image defect detection method according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantageous effects of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and the detailed description. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The invention provides a twin residual network-based two-stage multi-classification defect detection method, which takes the information transmission mode of a residual network structure as a core design idea, uses the twin network structure to extract residual characteristic information, and combines the residual characteristic information with the residual network structure to obtain double performance improvement of detection capability and migration capability, wherein the overall schematic diagram of the detection method is shown in figure 1.
Specifically, a flowchart of the twin residual network-based two-stage multi-classification defect detection method is shown in fig. 2, and the method includes the following steps:
s1, acquiring an image to be detected and a corresponding template image, preprocessing image data, scaling the image to a uniform size, carrying out image contrast normalization, and carrying out affine transformation correction;
s2, extracting feature information of a twin network FPN, wherein the flow is shown in figure 3, and the method specifically comprises the following steps:
a) the twin network needs to be used together with a residual error network, so the twin backbone network needs to be designed elaborately, and the number of characteristic channels D4-D32 is the same as that of characteristic layers corresponding to the residual error network;
b) the shallow feature extraction layer (before the D2 feature) of the backbone network is split into two input ends, a twin network with common parameters is shared, and two groups of D2 features are output by the network. Inputting the output result of the subtraction of the two groups of features into a subsequent part of the network, merging the two groups of features into a strand of data, and subsequently recovering the normal feature extraction mode of the backbone network;
c) extracting difference information characteristics in the twin backbone network, and extracting characteristics of D4, D8, D16 and D32 from deep characteristics of the backbone network;
d) the twin network D4-D32 features are sent to an FPN feature extraction module to obtain the feature C with the fixed channel numbernumTwin FPN _ D4-FPN _ D64.
S3, extracting residual FPN characteristics of the template image, wherein the process is shown in FIG. 3, and the method specifically comprises the following steps:
a) sending the template image into a residual error network, and extracting D4-D32 intermediate features of input data;
b) finding the last residual block before the intermediate feature extraction, and combining the corresponding sizes of the twin network D4-D32 features extracted in the step two into a residual block structure to obtain the residual network D4-D32 features;
c) the residual error network D4-D32 features are sent to an FPN feature extraction module to obtain the feature C with the fixed channel numbernumResidual FPN _ D4-FPN _ D64 feature of (a).
S4, sending the twin FPN _ D4-FPN _ D64 characteristics into a segmentation network for supervised training, as shown in FIG. 4, specifically comprising the following steps:
a) compressing the channel number of the twin FPN _ D4-FPN _ D64 feature to be 1 by using two layers of 1 × 1 convolution to obtain a compressed D4-D64 single-channel feature;
b) using a sigmoid function to activate the characteristics, wherein the specific formula is as follows:
c) and respectively converting the segmentation labels of the images to be detected into five sizes of original images D4-D64, generating five groups of bool matrixes as labels, training a segmentation network, wherein the training targets are as follows: predicting the defect part as 1 and the background as 0;
d) the loss function uses two classes of cross entropy:
s5, using a two-classification detection network of a twin FPN characteristic training two stage, as shown in FIG. 4, specifically comprising the following steps:
a) sending the twin FPN characteristics into an RPN network, and calculating to obtain an offer detection frame;
b) and (4) sending the proposed detection frame and the twin FPN characteristics into a two-classification ROI detection network to obtain the output of the detection frame and corresponding two classification scores, and marking the detection frame as a label for training.
S6, training the multi-classification detection network by using the residual FPN characteristics and the proposed detection frame of the two-classification detection network, and specifically comprising the following steps:
a) inputting residual FPN characteristics and proposed detection boxes of the two-classification detection network into the multi-classification ROI detection network;
b) and (5) taking the multi-classification detection frame of the defect as a label to train the multi-classification detection network.
And S7, detecting the sample to be detected by using the multi-classification detection network structure to obtain a detection result.
In an embodiment of the invention, the twin residual error network-based two-stage multi-classification defect detection method specifically comprises the following steps:
1. acquiring an image to be detected of an industrial semiconductor device and a corresponding template image, and performing position calibration according to a difference gradient in the process of intercepting the template image so that the deviation between the template and the image to be detected is within 0.5 pixel;
2. extracting the twin FPN characteristics requires splitting the backbone network and transforming it into the twin network. Taking a Resnet50 backbone network as an example, splitting a network layer (including a first convolutional layer, a first BN layer, a first Relu layer and a first maxporoling layer) before layer1 into a twin structure, sharing parameters by the twin network, outputting two sets of characteristics, and subtracting the two sets of characteristics to be used as the input of a subsequent Resnet 50. Extracting D4, D8, D16 and D32 characteristics from the characteristic layer of Resnet50, and sending the characteristics into an FPN module (the number of channels C of the FPN module)numSet to 256) resulting in twin FPN _ D4, FPN _ D8, FPN _ D16, FPN _ D32, FPN _ D64 characteristics.
3. Extracting residual FPN characteristics of the template images, and extracting D4-D32 intermediate characteristics of input data in order to input the template images into a structure of a residual network (the Resnet50 is taken as an example in the residual network); determining the last residual block before the feature extraction, and combining the corresponding sizes of the twin network D4-D32 features extracted in the step 2 into a residual block structure to obtain residual network D4-D32 features; and (3) sending the residual error network D4-D32 characteristics to an FPN characteristic extraction module to obtain residual error FPN _ D4-FPN _ D64 characteristics with the fixed channel number of 256.
4. Supervised training is performed using a segmented network. The method comprises the steps of sending twin FPN _ D4-FPN _ D64 features into a segmentation network for supervision training, compressing the number of channels of the twin FPN _ D4-FPN _ D64 features to be 1 by using two layers of 1 × 1 convolution (output channels are 32 and 1), obtaining compressed D4-D64 single channel features, activating the features by using a sigmoid function, respectively converting segmentation labels of an image to be detected into five sizes of original images D4-D64, generating five groups of bool matrixes as labels, training the segmentation network, and enabling a loss function to be a binary cross entropy.
5. And (3) training a two-stage binary detection network by using the twin FPN characteristics. Taking the fast rcnn as an example, the two-stage detection network sends the twin FPN characteristics to the RPN network, and calculates to obtain a proposed detection frame; and (3) sending the proposed detection frame and the twin FPN characteristics into a two-classification ROI detection network to obtain the output of the detection frame and corresponding two classification scores, marking the detection frame as a label to train the detection network, and respectively using BCE loss and an L1 loss function.
6. The multi-class detection network is trained using the residual FPN features and proposed detection boxes of the two-class detection network. As shown in fig. 4, the residual FPN feature and the proposed detection box of the two-class detection network are input into the multi-class ROI detection network; and (5) taking the multi-classification detection frame of the defect as a label to train the multi-classification detection network.
7. And detecting the sample to be detected by using the multi-classification detection network structure to obtain a detection result. In this example, we used the existing data to separate into four data sets for migration data testing, and the results (mAP) of the baseline experiments and the invention are shown in Table 1.
Table 1:
network architecture | Migrating data set 1 | Migrating data set 2 | Migrating data set 3 | Migrating data set 4 |
farstercnn (Baseline) | 83.75% | 60.34% | 85.19% | 72.23% |
Fastercnn (invention) | 87.64% | 70.33% | 89.40% | 80.09% |
From the results in table 1, it can be observed that the data migration performance of the defect detection framework proposed by the present invention on each data set is better than that of the baseline algorithm, which indicates that the method proposed by the present invention has stronger data migration capability on the defect multi-classification task.
The invention is not limited solely to that described in the specification and embodiments, and additional advantages and modifications will readily occur to those skilled in the art, so that the invention is not limited to the specific details, representative apparatus, and illustrative examples shown and described herein, without departing from the spirit and scope of the general concept as defined by the appended claims and their equivalents.
Claims (7)
1. A two-stage multi-classification industrial image defect detection method based on a twin residual error network is characterized by comprising the following steps:
s1, acquiring an image to be detected and a corresponding template image and preprocessing image data;
s2, inputting the image to be detected and the template image into a twin backbone network to complete image data processing, and inputting the formed data representation characteristics into a twin FPN network to obtain twin FPN characteristics;
s3, inputting template data into a residual error network, extracting by combining twin network characteristics to obtain residual error network characteristics, and inputting the residual error network characteristics into a residual error FPN network to obtain residual error FPN characteristics;
s4, sending the twin FPN characteristics into a segmentation network for supervision training, and optimizing the characteristics;
s5, inputting the twin FPN characteristics into a two-classification detection network for two-classification supervision training, and extracting an offer detection frame of the detection network;
s6, training a multi-classification detection network by using the proposed detection frame and the residual FPN characteristics;
and S7, detecting the image sample by using a multi-classification detection network and outputting a detection result.
2. The industrial image defect detecting method according to claim 1, wherein the step S5 is:
s51, sending the twin FPN characteristics to an RPN network, and calculating to obtain an offer detection frame;
and S52, sending the proposed detection frame and the twin FPN characteristics into a two-classification detection network to obtain detection frame output and corresponding two-classification scores, and marking the detection frame as a label for training.
3. The industrial image defect detecting method according to claim 1, wherein the step S6 is:
s61, inputting residual FPN characteristics and an proposed detection box of a two-class detection network into the multi-class detection network;
and S62, marking the multi-classification detection frame of the defect as a label, and training a multi-classification detection network.
4. The industrial image defect detecting method of claim 1, wherein the number of the twin backbone network feature channels in the step S2 is the same as the number of the feature layers corresponding to the residual network in the step S3.
5. The industrial image defect detection method of claim 4, wherein in step S2, the shallow feature extraction layer of the twin backbone network is split into two input ends, the twin network shares common parameters, two sets of features are respectively output, the result of feature subtraction is input to the subsequent part of the network, and is merged into a strand of data, and the normal feature extraction of the backbone network is subsequently recovered.
6. The industrial image defect detection method according to claim 4, wherein said step S3 includes:
s31, sending the template image into a residual error network, and extracting the intermediate features of the input data;
s32, finding the last residual block before the intermediate feature extraction, and combining the corresponding size of the twin network feature extracted in the step S2 into a residual block structure to obtain a residual network feature;
and S33, sending the residual error network characteristics to an FPN characteristic extraction module to obtain residual error FPN characteristics with fixed channel number.
7. The industrial image defect detection method of claim 5 or 6, wherein the twin FPN feature and the residual FPN feature are both features of D4-D64.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210285597.3A CN114581722A (en) | 2022-03-22 | 2022-03-22 | Two-stage multi-classification industrial image defect detection method based on twin residual error network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210285597.3A CN114581722A (en) | 2022-03-22 | 2022-03-22 | Two-stage multi-classification industrial image defect detection method based on twin residual error network |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114581722A true CN114581722A (en) | 2022-06-03 |
Family
ID=81782768
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210285597.3A Pending CN114581722A (en) | 2022-03-22 | 2022-03-22 | Two-stage multi-classification industrial image defect detection method based on twin residual error network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114581722A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115546211A (en) * | 2022-11-29 | 2022-12-30 | 福建帝视智能科技有限公司 | Welding spot defect classification method, terminal and computer storage medium |
-
2022
- 2022-03-22 CN CN202210285597.3A patent/CN114581722A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115546211A (en) * | 2022-11-29 | 2022-12-30 | 福建帝视智能科技有限公司 | Welding spot defect classification method, terminal and computer storage medium |
CN115546211B (en) * | 2022-11-29 | 2023-04-11 | 福建帝视智能科技有限公司 | Welding spot defect classification method, terminal and computer storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110097568B (en) | Video object detection and segmentation method based on space-time dual-branch network | |
CN108830855B (en) | Full convolution network semantic segmentation method based on multi-scale low-level feature fusion | |
CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
KR20190063839A (en) | Method and System for Machine Vision based Quality Inspection using Deep Learning in Manufacturing Process | |
CN113160200B (en) | Industrial image defect detection method and system based on multi-task twin network | |
CN106156777B (en) | Text picture detection method and device | |
CN111126115B (en) | Violent sorting behavior identification method and device | |
EP4322106B1 (en) | Defect detection method and apparatus | |
JP2022027473A5 (en) | ||
CN111340796A (en) | Defect detection method and device, electronic equipment and storage medium | |
WO2024021461A1 (en) | Defect detection method and apparatus, device, and storage medium | |
CN113591831A (en) | Font identification method and system based on deep learning and storage medium | |
CN115861210B (en) | Transformer substation equipment abnormality detection method and system based on twin network | |
CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
CN111652846B (en) | Semiconductor defect identification method based on characteristic pyramid convolution neural network | |
CN116246059A (en) | Vehicle target recognition method based on improved YOLO multi-scale detection | |
CN117152459A (en) | Image detection method, device, computer readable medium and electronic equipment | |
CN111401368A (en) | News video title extraction method based on deep learning | |
CN114581722A (en) | Two-stage multi-classification industrial image defect detection method based on twin residual error network | |
Singh et al. | CNN based approach for traffic sign recognition system | |
CN113159071A (en) | Cross-modal image-text association anomaly detection method | |
CN117522735A (en) | Multi-scale-based dense-flow sensing rain-removing image enhancement method | |
CN114596242A (en) | Defect detection method, defect detection device, electronic equipment and computer readable storage medium | |
CN110826564A (en) | Small target semantic segmentation method and system in complex scene image | |
CN116363105A (en) | Method for identifying and positioning high-speed rail contact net parts based on Faster R-CNN |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |