CN114445693A - Knowledge distillation-based sustainable learning water obstacle detection system and method - Google Patents
Knowledge distillation-based sustainable learning water obstacle detection system and method Download PDFInfo
- Publication number
- CN114445693A CN114445693A CN202111553195.9A CN202111553195A CN114445693A CN 114445693 A CN114445693 A CN 114445693A CN 202111553195 A CN202111553195 A CN 202111553195A CN 114445693 A CN114445693 A CN 114445693A
- Authority
- CN
- China
- Prior art keywords
- network model
- unknown
- student
- module
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
Abstract
The invention relates to a sustainable learning aquatic obstacle detection system based on knowledge distillation, which comprises: a teacher network model module: the method is used for training the teacher network model to realize detection of the above-water obstacles; knowledge distillation module: the method is used for enabling the student network model to accurately imitate the target detection result of the teacher network model through a knowledge distillation method; student's obstacle detection network module: the student network model is used for acquiring the capability of detecting the above-water obstacles; unknown class detection module of student: the system comprises an unknown class data collection module, a student network model and a student data analysis module, wherein the unknown class data collection module is used for enabling the student network model to identify an unknown class target which is not learned and transmitting picture data of the class target to the unknown class data collection module; an unknown class data collection module: the method is used for collecting picture data of unknown targets and further training a teacher network model to identify new unknown targets.
Description
Technical Field
The invention relates to the field of target detection in machine vision, in particular to a sustainable learning overwater obstacle detection system and method based on knowledge distillation.
Background
In recent years, due to the problems of severe working environment, high cost of manned equipment and the like in water scenes, reliable design of water unmanned equipment becomes a research hotspot. Detection of obstacles on water has been extensively and intensively studied as one of core technologies required for realizing unmanned driving on water. Most of the existing water obstacle detection methods mainly focus on fully supervised learning, namely, a sufficient number of image data are used for driving a network to perform inefficient mechanical learning. However, the existing data set resources available to a land based environment are very rich and sophisticated, essentially comprising a variety of target images in various situations. Compared with the land environment, in the field of water, because the existing data sets on water are far from sufficient in quantity, and the large enough data sets are acquired on water, the offshore environment is complex, the acquisition difficulty and the equipment cost are high, and compared with the data acquisition on land, the manpower and financial cost are higher, technicians often cannot obtain ideal experimental results because the proper data sets cannot be found, which greatly limits the development of the field. Meanwhile, compared with the land environment, the obstacles possibly existing on the water are complex, appear less and are difficult to collect, and it is not practical to collect enough data for the system to learn if each object possibly appears. Under the condition, the existing deep learning neural network is difficult to meet the complex and constantly changing requirements in the actual scene.
In the conventional target detection method, the output of the neural network model can be selected only from a limited set of known classes, and in this case, if an image having an unknown class which is not present in the input data set is an image having an unknown class, the output of the model is forcibly given to the unknown class as one of the known classes. If a target detection model is trained with a variety of data sets with various vessels, after training is completed, a picture with plastic bottles is input to the model, and the model classifies the plastic bottles as one of the vessels. The accuracy of the neural network model is greatly reduced, and meanwhile, because the network has high probability to identify objects which are not seen in the neural network model in practical application, the application of the deep learning algorithm in the practical application is greatly hindered by the error. Therefore, a system and method capable of solving this problem is urgently needed.
When new visual information is received, the curiosity of human beings often drives people to pay more attention to unknown things, so that the human beings have more efficient learning capacity compared with the machines, and naturally, the characteristics of the human beings are simulated, so that the machines have the capacity of identifying unknown things, outputting the categories which cannot be identified by the human beings and collecting the data of the unknown categories for relearning, various problems in the practical application can be effectively solved, and the machines have efficient sustainable learning capacity.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a knowledge distillation-based system and a method for detecting obstacles on water for sustainable learning.
The purpose of the invention can be realized by the following technical scheme:
a knowledge distillation based sustainable learning above-water obstacle detection system, the system comprising:
a teacher network model module: the system is used for training a teacher network model through the existing picture data set, detecting the water obstacles of known class and unknown class and teaching the water obstacles to the student network model;
knowledge distillation module: the method is used for enabling the student network model to accurately imitate the target detection result of the teacher network model through a knowledge distillation method;
student's obstacle detection network module: the student network model is used for simulating the characteristics and the target detection result of the teacher network model, so that the function of detecting the obstacle on the water is realized;
unknown class detection module of student: the system comprises a student network model, an unknown class data collection module, a middle layer data processing module and a data processing module, wherein the student network model is used for clustering the characteristics of the middle layer of the student network model so as to enable the student network model to identify the unknown class target and transmit the picture data of the unknown class target to the unknown class data collection module;
an unknown class data collection module: the method is used for obtaining picture data of a plurality of unknown targets through a data enhancement method, and training a teacher network model to identify new unknown targets based on the picture data of the unknown targets.
The structure of the teacher network model is the network structure of fast-RCNN.
The structure of the student network model is the network structure of fast-RCNN.
The knowledge distillation module carries out knowledge distillation on a plurality of characteristic layers of the student network model, and the plurality of characteristic layers of the student network model for carrying out the knowledge distillation are respectively an intermediate layer for carrying out feature extraction, a classification layer of the RPN network and a regression layer.
A detection method of the sustainable learning aquatic obstacle detection system comprises the following steps:
step 1: collecting picture data of an overwater scene to obtain a picture data set, and calibrating obstacles in the picture data of the picture data set;
step 2: inputting the picture data of the data set into a teacher network model, and training the teacher network model;
and step 3: detecting the input picture data through the trained teacher network model to obtain a target detection result, and transmitting the target detection result to the knowledge distillation module;
and 4, step 4: the knowledge distillation module trains the student network model through the teacher network model based on a knowledge distillation method;
and 5: detecting obstacles in the picture data of the water scene by adopting the trained student network model to obtain a target detection result, detecting unknown targets in the picture data of the water scene by adopting a student unknown type detection module, and transmitting the picture data to an unknown data collection module;
step 6: and (3) after the number of the picture data of the unknown targets collected by the unknown data collection module reaches a set value, retraining the teacher network model, returning to the step 2, and realizing the sustainable learning of the above-water obstacle detection system.
In the step 4, the process of training the student network model by the knowledge distillation module through the teacher network model based on the knowledge distillation method specifically comprises the following steps:
and converting the characteristic graph in the teacher network model to make the characteristic graph have the same size as that of the student network model, and calculating distillation loss through the knowledge distillation module to force the characteristics extracted by the middle layer of the student network model to approach the characteristics extracted by the middle layer of the teacher network model.
In the step 5, the process of detecting the unknown targets in the picture data of the water scene by adopting the unknown class detection module of the student specifically comprises the following steps:
step 501: comparing and clustering the characteristics of the candidate block of the region of the fast-RCNN, separating the characteristics of the hidden layer through a minimized loss function, forcibly reducing the intra-class difference to increase the inter-class difference, and pulling the distance between the characteristics of different classes;
step 502: and giving labels corresponding to the features of the feature space, modeling according to probability density functions of the features of different classes, and distinguishing the labels as the different classes to judge whether the features belong to the features of the unknown class targets.
In step 501, the unknown class detection module adds a comparison clustering function to the conventional loss function to obtain a loss function, where the formula of the loss function is:
wherein p isiIs a feature of the i-th class object, fcAs the c-th feature of the input, lu(fc) As a function of the total loss for all classes, l (f)c,pi) Is the fractional loss function corresponding to the i-th class target, C is the number of target classes, D is the Euclidean distance function, D (f)c,pi) As feature vector fcAnd piAnd Δ is a set distance parameter for determining whether different features are similar, where c ≠ i denotes that the input c-th feature is the feature of the i-th class target, and c ≠ i denotes that the input c-th feature is not the feature of the i-th class target.
The unknown targets are specifically as follows:
one class of targets that do not exist in the existing picture data set that trains the teacher network model, i.e., the class of waterborne obstacles that are not input into the teacher network model and the student network model for learning.
The target detection result comprises the type of the obstacle and the position of the obstacle.
Compared with the prior art, the invention has the following advantages:
1. the detection system for the obstacle on water capable of continuously learning can distinguish the target of unknown type in the current picture and output the target, wherein the type is the 'curious' type of the target detection system, and after the picture data of the type is added, the system learns the picture data to realize the continuous learning;
2. according to the method, a knowledge distillation method is introduced, the detection capability of the above-water obstacles is obtained from the existing picture data set through a teacher network model, and the detection capability of the above-water obstacles is also obtained through a student network model through knowledge distillation;
3. in practical application, the student network is required to identify unknown targets which are not learned and output the unknown targets, the unknown targets are sent to the teacher network model for retraining after the unknown data collection module collects obstacles of the class, and the teacher network model imparts the learned targets of a new class to the student network model, so that sustainable learning is realized.
Drawings
FIG. 1 is a system block diagram of the present invention.
FIG. 2 is a flow chart of the method of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Examples
The invention provides a sustainable learning overwater obstacle detection system based on knowledge distillation.
A teacher network model module: training a teacher network model through the existing data set, detecting the above-water obstacles and learning new unknown targets at the same time to obtain target detection results, and teaching the target detection results to a student network model;
knowledge distillation module: the method is used for enabling the student network model to accurately simulate the target detection result of the teacher network model through the selected specific knowledge distillation method;
student's obstacle detection network module: simulating a target detection result of the teacher network model through the student network model to obtain the target detection capability of the obstacle on the water;
unknown class detection module of student: the intermediate layer characteristics of the student network model are clustered, so that unknown targets which are not learned by the student network model can be identified, and data of the unknown targets are transmitted to the unknown data collection module;
an unknown class data collection module: and acquiring data of the unknown class target enough for training by the selected data enhancement method so as to train the teacher network model to identify a new class.
In addition, the invention also provides a method for detecting the water obstacle based on attention to the unknown target, which comprises the following steps:
step 1: collecting picture data of an overwater scene to obtain a picture data set, and calibrating obstacles in the picture data of the picture data set;
step 2: determining a teacher network model structure, inputting picture data of a data set into the teacher network model, and training the teacher network model;
and step 3: detecting the input picture data through the trained teacher network model to obtain a target detection result, and transmitting the target detection result to the knowledge distillation module;
and 4, step 4: determining a student network model structure, determining a feature layer for knowledge distillation by comparing with a teacher network model, determining a structure of a knowledge distillation module, and training the student network model through the teacher network model based on a knowledge distillation method;
and 5: detecting obstacles in the picture data of the water scene by adopting the trained student network model to obtain a target detection result, detecting unknown targets in the picture data of the water scene by adopting a student unknown type detection module, and transmitting the picture data to an unknown data collection module;
step 6: and (3) after the quantity of the image data of the unknown targets collected by the unknown data collection module reaches a set value, returning to the step (2), and retraining the teacher network model to realize sustainable learning of the water obstacle detection system.
The teacher network model and the student network model adopt a fast-RCNN network structure, a middle layer for feature extraction, a classification layer of an RPN network and a regression layer are selected from a feature layer needing distillation, the size of the feature layer in the teacher network model is converted into the size of a feature map which is the same as that of the student network model, a knowledge distillation module is used for calculating distillation loss, and the purpose of knowledge distillation is to force the features extracted from the middle layer of the student network model to approach the features extracted from the middle layer of the teacher network model.
The unknown class detection module performs comparison clustering on the characteristics of the region candidate blocks of the Faster-RCNN network, so that the characteristics of different classes are forcibly separated, and the formula of a loss function adopted in the comparison clustering process is as follows:
wherein p isiIs a feature of the i-th class object, fcAs the c-th feature of the input, lu(fc) As a function of the total loss for all classes, l (f)c,pi) Is the fractional loss function corresponding to the i-th class target, C is the number of target classes, D is the Euclidean distance function, D (f)c,pi) As feature vector fcAnd piΔ is a set distance parameter for determining whether different features are similar, where c ≠ i denotes that the input c-th feature is the feature of the i-th class target, and c ≠ i denotes that the input c-th feature is not the feature of the i-th class target;
the unknown obstacle type is an obstacle type which is not learned by the teacher network model, namely an unknown target, and the unknown target is a type of target which is not included in picture data for training the teacher network model, namely a type of water obstacle which is not input into the teacher network model and the student network model.
A comparison clustering function is superposed on a conventional loss function by a loss function of an unknown class detection module, the loss function is minimized, the separation of hidden layer features can be ensured, the intra-class difference is forcibly reduced to increase the inter-class difference, after all the features are clustered, the distances of the features of different classes are pulled apart, labels corresponding to the features of a feature space are given, modeling is carried out according to the probability density functions of the features of the different classes to distinguish the different classes, and whether sample data belong to an unknown class or not is judged.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A system for sustainable learning aquatic obstacle detection based on knowledge distillation, the system comprising:
a teacher network model module: the system is used for training a teacher network model through the existing picture data set, detecting the water obstacles of known class and unknown class and teaching the water obstacles to the student network model;
knowledge distillation module: the student network model is used for accurately simulating a target detection result of the teacher network model through a knowledge distillation method;
student's obstacle detection network module: the student network model is used for simulating the characteristics and the target detection result of the teacher network model, so that the function of detecting the obstacle on the water is realized;
unknown class detection module of student: the system comprises a student network model, an unknown class data collection module, a middle layer data processing module and a data processing module, wherein the student network model is used for clustering the characteristics of the middle layer of the student network model so as to enable the student network model to identify the unknown class target and transmit the picture data of the unknown class target to the unknown class data collection module;
an unknown class data collection module: the method is used for obtaining picture data of a plurality of unknown targets through a data enhancement method, and training a teacher network model to identify new unknown targets based on the picture data of the unknown targets.
2. The knowledge-distillation-based sustainable learning aquatic obstacle detection system according to claim 1, wherein the teacher network model is a fast-RCNN network.
3. The knowledge-distillation-based sustainable learning aquatic obstacle detection system according to claim 1, wherein the student network model has a network structure of fast-RCNN.
4. The knowledge distillation-based sustainable learning aquatic obstacle detection system according to claim 1, wherein the knowledge distillation module performs knowledge distillation on a plurality of feature layers of the student network model, and the plurality of feature layers of the student network model performing knowledge distillation are an intermediate layer for feature extraction, a classification layer of the RPN network and a regression layer.
5. A detection method of the sustainable learning aquatic obstacle detection system according to any one of claims 1 to 4, comprising the following steps:
step 1: collecting picture data of an overwater scene to obtain a picture data set, and calibrating obstacles in the picture data of the picture data set;
step 2: inputting the picture data of the data set into a teacher network model, and training the teacher network model;
and step 3: detecting the input picture data through the trained teacher network model to obtain a target detection result, and transmitting the target detection result to the knowledge distillation module;
and 4, step 4: the knowledge distillation module trains the student network model through the teacher network model based on a knowledge distillation method;
and 5: detecting obstacles in the picture data of the water scene by adopting the trained student network model to obtain a target detection result, detecting unknown targets in the picture data of the water scene by adopting a student unknown type detection module, and transmitting the picture data to an unknown data collection module;
step 6: and (3) after the number of the picture data of the unknown targets collected by the unknown data collection module reaches a set value, retraining the teacher network model, returning to the step 2, and realizing the sustainable learning of the above-water obstacle detection system.
6. The detection method according to claim 5, wherein in the step 4, the process of training the student network model by the knowledge distillation module through the teacher network model based on the knowledge distillation method specifically comprises:
and converting the characteristic graph in the teacher network model to make the characteristic graph have the same size as that of the student network model, and calculating distillation loss through the knowledge distillation module to force the characteristics extracted by the middle layer of the student network model to approach the characteristics extracted by the middle layer of the teacher network model.
7. The detection method according to claim 5, wherein in the step 5, the process of detecting the unknown class object in the picture data of the water scene by using the student unknown class detection module specifically comprises the following steps:
step 501: comparing and clustering the characteristics of the candidate block of the region of the fast-RCNN, separating the characteristics of the hidden layer through a minimized loss function, forcibly reducing the intra-class difference to increase the inter-class difference, and pulling the distance between the characteristics of different classes;
step 502: and giving labels corresponding to the features of the feature space, modeling according to probability density functions of the features of different classes, and distinguishing the labels as the different classes to judge whether the features belong to the features of the unknown class targets.
8. The detecting method according to claim 7, wherein in step 501, the unknown class detecting module superimposes a contrast clustering function on a conventional loss function to obtain a loss function, and the formula of the loss function is:
wherein p isiIs a feature of the i-th class object, fcAs the c-th feature of the input, lu(fc) As a function of the total loss for all classes, l (f)c,pi) Is the fractional loss function corresponding to the i-th class target, C is the number of target classes, D is the Euclidean distance function, D (f)c,pi) As feature vector fcAnd piAnd Δ is a set distance parameter for determining whether different features are similar, where c ≠ i denotes that the input c-th feature is the feature of the i-th class target, and c ≠ i denotes that the input c-th feature is not the feature of the i-th class target.
9. The detection method according to claim 5, wherein the unknown class of objects is specifically:
one class of targets that do not exist in the existing picture data set that trains the teacher network model, i.e., the class of waterborne obstacles that are not input into the teacher network model and the student network model for learning.
10. The detection method according to claim 5, wherein the target detection result comprises the type of the obstacle and the position of the obstacle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111553195.9A CN114445693A (en) | 2021-12-17 | 2021-12-17 | Knowledge distillation-based sustainable learning water obstacle detection system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111553195.9A CN114445693A (en) | 2021-12-17 | 2021-12-17 | Knowledge distillation-based sustainable learning water obstacle detection system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114445693A true CN114445693A (en) | 2022-05-06 |
Family
ID=81363137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111553195.9A Pending CN114445693A (en) | 2021-12-17 | 2021-12-17 | Knowledge distillation-based sustainable learning water obstacle detection system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114445693A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168256A (en) * | 2023-04-19 | 2023-05-26 | 浙江华是科技股份有限公司 | Ship detection method, system and computer storage medium |
| CN117197590A (en) * | 2023-11-06 | 2023-12-08 | 山东智洋上水信息技术有限公司 | Image classification method and device based on neural architecture search and knowledge distillation |
-
2021
- 2021-12-17 CN CN202111553195.9A patent/CN114445693A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168256A (en) * | 2023-04-19 | 2023-05-26 | 浙江华是科技股份有限公司 | Ship detection method, system and computer storage medium |
| CN117197590A (en) * | 2023-11-06 | 2023-12-08 | 山东智洋上水信息技术有限公司 | Image classification method and device based on neural architecture search and knowledge distillation |
| CN117197590B (en) * | 2023-11-06 | 2024-02-27 | 山东智洋上水信息技术有限公司 | Image classification method and device based on neural architecture search and knowledge distillation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109949317B (en) | Semi-supervised image example segmentation method based on gradual confrontation learning | |
| CN111368815B (en) | Pedestrian re-identification method based on multi-component self-attention mechanism | |
| CN110414368B (en) | Unsupervised pedestrian re-identification method based on knowledge distillation | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN110414432A (en) | Training method, object identifying method and the corresponding device of Object identifying model | |
| CN111368690B (en) | Deep learning-based video image ship detection method and system under influence of sea waves | |
| CN109063649B (en) | Pedestrian re-identification method based on twin pedestrian alignment residual error network | |
| CN104200237A (en) | High speed automatic multi-target tracking method based on coring relevant filtering | |
| CN111222471B (en) | Zero sample training and related classification method based on self-supervision domain perception network | |
| CN113076994B (en) | Open-set domain self-adaptive image classification method and system | |
| CN110633632A (en) | Weak supervision combined target detection and semantic segmentation method based on loop guidance | |
| CN114445693A (en) | Knowledge distillation-based sustainable learning water obstacle detection system and method | |
| CN110765921B (en) | Video object positioning method based on weak supervised learning and video spatiotemporal features | |
| CN112036276B (en) | Artificial intelligent video question-answering method | |
| CN110598752B (en) | Image classification model training method and system for automatically generating training data set | |
| CN109902202B (en) | Video classification method and device | |
| CN112819065B (en) | Unsupervised pedestrian sample mining method and unsupervised pedestrian sample mining system based on multi-clustering information | |
| CN110598693A (en) | Ship plate identification method based on fast-RCNN | |
| CN102521340A (en) | Method for analyzing TV video based on role | |
| CN112149547A (en) | Remote sensing image water body identification based on image pyramid guidance and pixel pair matching | |
| CN112766218B (en) | Cross-domain pedestrian re-recognition method and device based on asymmetric combined teaching network | |
| CN112990282B (en) | Classification method and device for fine-granularity small sample images | |
| CN109446897B (en) | Scene recognition method and device based on image context information | |
| CN111860193A (en) | Text-based pedestrian retrieval self-supervision visual representation learning system and method | |
| CN112395953A (en) | Road surface foreign matter detection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |