CN111916204A - Brain disease data evaluation method based on self-adaptive sparse deep neural network - Google Patents
Brain disease data evaluation method based on self-adaptive sparse deep neural network Download PDFInfo
- Publication number
- CN111916204A CN111916204A CN202010653663.9A CN202010653663A CN111916204A CN 111916204 A CN111916204 A CN 111916204A CN 202010653663 A CN202010653663 A CN 202010653663A CN 111916204 A CN111916204 A CN 111916204A
- Authority
- CN
- China
- Prior art keywords
- data
- neural network
- sparse
- deep neural
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a brain disease data evaluation method based on a self-adaptive sparse deep neural network, which comprises the following steps: 1) collecting documented brain disease data; 2) summarizing each individual and the corresponding data characteristic and the change value thereof into a piece of unit data, and then constructing a data matrix by using the unit data corresponding to each individual; 3) dividing the data matrix obtained in the step 2) into a training set and a test set; 4) establishing a deep neural network model based on a sparse enhancement BP algorithm; 5) training the deep neural network model based on the sparse enhancement BP algorithm established in the step 4) by using the training set and the test set obtained in the step 3) to obtain a trained deep neural network model based on the sparse enhancement BP algorithm; 6) the method can effectively solve the over-fitting problem of the existing deep learning model for processing the high latitude small sample data.
Description
Technical Field
The invention belongs to the field of data processing, and relates to a brain disease data evaluation method based on a self-adaptive sparse deep neural network.
Background
High-dimensional small sample data is commonly found in the biomedical field, such as genomic data, medical image data, protein data and the like, and the data has the characteristics of small sample size but huge sample characteristics. This feature presents certain challenges to the process of data processing and analysis. When the sample size to sample feature ratio is small, classical machine learning algorithms tend to fail because irrelevant and redundant features may be contained in the high dimensional data. At present, deep learning is proved to be one of the strongest tools in big data analysis, but the traditional deep learning algorithm is still very limited in application of bioinformatics, mainly because when the sample size of data is far smaller than the feature number, the model is often trapped in overfitting, and then data processing is inaccurate. Therefore, it is necessary to find a deep network model with strong self-sparse learning capability for such data with high latitude small sample characteristics, so as to solve the existing data processing problem.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a brain disease data evaluation method based on an adaptive sparse deep neural network, which can effectively solve the over-fitting problem of processing high latitude small sample data by the existing deep learning model.
In order to achieve the above purpose, the brain disease data evaluation method based on the adaptive sparse deep neural network of the present invention comprises the following steps:
1) collecting documented brain disease data;
2) summarizing each individual and the corresponding data characteristics and the variation value thereof into a unit data in the acquired brain disease data, and then constructing a data matrix by using the unit data corresponding to each individual;
3) dividing the data matrix obtained in the step 2) into a training set and a test set;
4) establishing a deep neural network model based on a sparse enhancement BP algorithm;
5) training the deep neural network model based on the sparse enhancement BP algorithm established in the step 4) by using the training set and the test set obtained in the step 3) to obtain a trained deep neural network model based on the sparse enhancement BP algorithm;
6) and evaluating the brain disease data to be evaluated based on the trained deep neural network model based on the sparse enhancement BP algorithm.
The deep neural network model based on the sparse enhancement BP algorithm comprises an input layer, five hidden layers and an output layer, wherein the five hidden layers are formed by stacking five restricted Boltzmann machines.
Adding log-sum functional form to response and connection of hidden layer neurons after loss function of sparse enhancement BP algorithm
The sparse penalty term of (1).
The objective function of the sparse enhancement BP algorithm is as follows:
wherein, tau1And τ2Penalty coefficients respectively representing a log-sum response penalty term and a log-sum connection penalty term,1and2is a given constant.
The invention has the following beneficial effects:
the brain disease data evaluation method based on the self-adaptive sparse deep neural network comprises the steps that during specific operation, brain disease data is evaluated based on a deep neural network model of a sparse enhancement BP algorithm, wherein a hidden layer in the network model is formed by stacking five limited Boltzmann machines, meanwhile, a target function of the sparse enhancement BP algorithm is modified, and sparse penalty terms are added to response and connection of neurons of the hidden layer after a loss function of the sparse enhancement BP algorithm, so that the complexity of the model is reduced, and the problem of overfitting is solved.
Drawings
FIG. 1 is a diagram of an adaptive sparse deep neural network model architecture;
FIG. 2 is a diagram of an embedded restricted Boltzmann RBM model;
FIG. 3 is a flow chart of the application of the entire MRI data;
FIG. 4 is a graph of the results of comparing the two networks with the best sparsity to LSES-DNN;
FIG. 5 is a graph showing the response values of each of LSES-DNN, DP-DNN, DPC-DNN, Pr-DNN and SVD-DNN;
FIG. 6 is a graph of classification errors of 5 deep learning methods in training set and test.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings:
referring to fig. 1, the method for analyzing brain disease data based on the adaptive sparse deep neural network according to the present invention includes the following steps:
1) collecting the recorded brain disease data (high latitude small sample), and storing the recorded brain disease data in a computer;
2) in the acquired brain disease data, summarizing each individual and the corresponding data characteristics and the variation value thereof into a unit data, and then constructing a data matrix by using the unit data corresponding to each individual, wherein the data matrix comprises a sample volume N and a sample characteristic p, and the data usually has the characteristics of a high latitude small sample, namely N < < p;
3) dividing the data matrix obtained in the step 2) into a training set and a test set, and then inputting the training set and the test set obtained by division into a memory corresponding to a deep neural network model based on a sparse enhancement BP algorithm;
the traditional deep learning algorithm comprises a Pre-training process (Pre-training) and a Fine-tuning process (Fine-tuning), the sparse enhancement BP algorithm provided by the invention is mainly realized in the Fine-tuning process, the data are optimized and calculated through a deep neural network model based on the sparse enhancement BP algorithm, and the specific optimization process is as follows:
a) setting a deep learning basic framework, and establishing a deep neural network model comprising an input layer, five hidden layers and an output layer according to data characteristics for data in a training set, wherein the input layer comprises a plurality of nodes with data characteristics, namely the number of the nodes of the input layer is the sample characteristics p of the training set, the output layer comprises a plurality of nodes with medical diagnosis data characteristics, namely the number of label types corresponding to each sample, and is uniformly expressed by label hereinafter, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the input value of the previous layer.
b) And establishing a data model of each node of each layer of network by adopting a mathematical equation, and presetting related parameter values in the mathematical equation.
In the deep network, 5 Restricted Boltzmann Machines (RBMs) are selected for stacking, namely the number of hidden layers is 5, the number of neurons of five hidden layers is respectively 2000, 1000, 800, 500 and 200, the number of nodes of an input layer is a training set sample characteristic and is marked as p, the number of neurons in an output layer is a characteristic dimension of medical diagnosis data and is marked as MnAccording to the above setting, the deep neural network model includes 7 neural layers, which are an input layer, 5 hidden layers and an output layer.
c) In the deep neural network model, a Gaussian-Bernoulli RBM (GBRBM) is used as a first layer RBM, a Bernoulli-Bernoulli RBM (BBRBM) is used as other layer RBMs, the learning rates of the network are 0.001 and 0.1 respectively for the GBRBM and the BBRBM, the punishment parameters lambda 1 are 0.03 and 0.0002 respectively, and the punishment coefficients of the norm re-weighted-L1 are 0.001 and 0.001 respectively and 0.1 respectively;
d) initializing parameter values Ai of each layer of network, including weight Wi connected with the network and corresponding bias bi, learning the network structure of the training set data input layer, comparing the output value of each node in the output layer with the corresponding medical diagnosis data label, repeatedly correcting the parameter values of each layer of neural network, and sequentially circulating to obtain the parameter values of each layer of neural network node corresponding to the output value with the highest similarity of each node of the output layer and the brain disease data characteristics.
e) Inputting the test set data into a trained deep neural network model, judging whether the result of an output layer is equal to the original label of the test set data, and calculating the accuracy of the deep neural network model on the brain disease data prediction according to the number of misclassifications of the deep neural network model on the test set data.
It should be noted that the mathematical equation is a parametric mathematical equation, and may be a linear model or a neuron model, such as a sigmoid activation function or a convolution operation model, and the mathematical model is set in the following manner:
y＝g(X)＝fn(fn-1(fn-2(...f1(X))
wherein y is a medical diagnostic data feature in the output layer and the dimension is MnX is training set data and characteristic dimensions are p, f1To fnFor each layer of set operational equations, and each layer of equations fiFor example, from the first input layer to the first hidden layer, the training data with the dimension p is converted into the output with the dimension 200, and the analogy is performed according to the number of the neurons in each layer. Wherein each layer model fiThere is a matching set of parameters Ai, including the connection weights Wi and the offsets bi of the various layers.
To overcome the overfitting problem, for the fine tuning process, constraints are added to the response and connection of hidden neurons after the loss function of the original BP algorithm to reduce the complexity of the model, considering that the log-sum function has the best approximation to the common sparse norm L0, i.e., the log-sum function is approximated to the common sparse norm L0
Adding log-sum functional forms for responses and connections to the loss function
wherein J (W) is an objective function of the conventional BP algorithm, wherein tau1And τ2Penalty coefficients respectively representing a log-sum response penalty term and a log-sum connection penalty term,1and2is a given constant.
The architecture diagram of the adaptive sparse deep neural network model is shown in fig. 1, and the embedded restricted boltzmann RBM model is shown in fig. 2.
Comparative experiment
MRI has been widely used for early detection, diagnosis and disease treatment as an important medical imaging mode, and such biomedical data generally has the characteristics of high-dimensional small samples, i.e. has a large number of features and a low sample number, and may have an overfitting problem when a deep learning method is applied to perform corresponding brain structure and brain function abnormality on such data. In order to verify that the method can effectively avoid overfitting and improve the identification capability. We refer to philiadelphia Neuro-degenerative cowort (PNC) data from a large-scale experimental data cooperative study of brain behaviour by pennsylvania university and Philadelphia children hospital. This data included fMRI data for 878 adolescents aged 8-22 using standard pre-processing SPM12(http:// www.fil.ion.ucl.ac.uk/SPM /).
The present invention verifies that the brain development difference is a function of age using an adaptive sparse depth network and selects a data subset of the complete data set according to age (in months), wherein subjects above 216 months of age belong to a first class and subjects below 144 months of age belong to a second class, and the pearson correlation coefficient between them is first calculated from the regional average oxygenation level correlation (BOLD) signals of 264 regions of interest (ROIs) of the brain. Therefore, the dimension of the input data is 34716, the total sample size is 397, the whole MRI data application flow chart is shown in FIG. 3, and the performance of the network is verified respectively from the network sparsity and the classification accuracy by the invention:
sparsity of network connections
Comparing the sparsity of the common depth networks DNN, SDNN, DP-DNN, DPC-DNN, Pr-DNN, SVD-DNN and the invention (LSES-DNN), and further comparing and verifying the two networks with the best sparsity with the LSES-DNN, the specific comparison results are shown in FIG. 4 and Table 1, wherein FIG. 4 shows LSES-DNN, DP-DNN, DPC-DNN, the connection weight non-zero ratio of each layer of the three depth networks, the length of each petal represents the non-zero ratio of each depth network connected to the current layer, and it is obvious that LSES-DNN has sparseness compared with the other depth networks.
TABLE 1
Response sparsity of hidden neurons
FIG. 5 shows the difference of response values of each of LSES-DNN, DP-DNN, DPC-DNN, Pr-DNN and SVD-DNN, wherein the horizontal axis represents the activation probability of a certain layer, the vertical axis represents the ratio of the activation probabilities at different time intervals, and the hidden layer neuron activation ratio of LSES-DNN is significantly smaller than that of other deep networks.
Accuracy of classification
The invention compares 17 conventional and commonly used classifiers of LSES-DNN, uses 70% of original data as a training set and the remaining 30% as a test set for each classifier, repeats the whole process for 30 times, selects the same network parameters of LSES-DNN for all the classifiers of the deep network to train, and the average classification accuracy of the test set is shown in Table 2. In addition, fig. 6 shows the classification errors of 5 deep learning methods in the training set and the actual measurement. For the training data, the error of LSES-DNN decreased rapidly and converged around epochs-50, where the error was 1.8%; meanwhile, the error curves of other methods are converged, but the time is long. For the test data, the error of LSES-DNN decreased to 16.33% around epoch-30 and finally to 12.93% at epoch-75. The error of sDNN and DNN is 51.52% and 51.52%, respectively, and the error curve converges to 32.58% when epoch is 30, 17.57% when epoch is 90, and 18.01%. It can be seen that LSES-DNN performs well on test data, with the fastest convergence rate and the best age identification performance.
TABLE 2
Through the sparsity of network connection, the response sparsity and classification precision comparison of hidden neurons, the LSES-DNN has the sparsity of a topological structure and the response sparsity of the hidden neurons, and the sparse learning capability of the LSES-DNN is the best in the existing network. Further explaining data with the characteristic of high latitude small samples such as an MRI data set aiming at brain disease diagnosis and normal development, the deep neural network LSES-DNN based on the sparse enhancement BP algorithm has good diagnosis and identification effects, can effectively extract the hierarchical sparse characteristics of the data, further learn the real distribution of the brain disease data, lay a foundation for subsequent diagnosis, treatment and prediction according to the data, and further improve the treatment level.
Claims (4)
1. A brain disease data evaluation method based on an adaptive sparse deep neural network is characterized by comprising the following steps:
1) collecting documented brain disease data;
2) summarizing each individual and the corresponding data characteristics and the variation value thereof into a unit data in the acquired brain disease data, and then constructing a data matrix by using the unit data corresponding to each individual;
3) dividing the data matrix obtained in the step 2) into a training set and a test set;
4) establishing a deep neural network model based on a sparse enhancement BP algorithm;
5) training the deep neural network model based on the sparse enhancement BP algorithm established in the step 4) by using the training set and the test set obtained in the step 3) to obtain a trained deep neural network model based on the sparse enhancement BP algorithm;
6) and evaluating the brain disease data to be evaluated based on the trained deep neural network model based on the sparse enhancement BP algorithm.
2. The brain disease data evaluation method based on the adaptive sparse deep neural network of claim 1, wherein the deep neural network model based on the sparse enhancement BP algorithm comprises an input layer, five hidden layers and an output layer, wherein the five hidden layers are formed by stacking five restricted Boltzmann machines.
3. The adaptive sparse depth neural network-based brain disease data assessment method of claim 1, wherein log-sum functional form is added to the response and connection of hidden layer neurons after the loss function of sparse enhancement BP algorithm
The sparse penalty term of (1).
4. The brain disease data evaluation method based on the adaptive sparse deep neural network of claim 1, wherein the objective function of the sparse enhancement BP algorithm is:
wherein, tau1And τ2Represents the log-sum response penalty term and the log-sum join respectivelyThe penalty coefficient of the penalty term is set to,1and2is a given constant.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010653663.9A CN111916204A (en) | 2020-07-08 | 2020-07-08 | Brain disease data evaluation method based on self-adaptive sparse deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010653663.9A CN111916204A (en) | 2020-07-08 | 2020-07-08 | Brain disease data evaluation method based on self-adaptive sparse deep neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111916204A true CN111916204A (en) | 2020-11-10 |
Family
ID=73226868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010653663.9A Pending CN111916204A (en) | 2020-07-08 | 2020-07-08 | Brain disease data evaluation method based on self-adaptive sparse deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111916204A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113450921A (en) * | 2021-06-24 | 2021-09-28 | 西安交通大学 | Brain development data analysis method, system, equipment and storage medium |
| CN113505528A (en) * | 2021-06-24 | 2021-10-15 | 西安交通大学 | Method, system, device and storage medium for identifying brain neural development time-varying functional connection difference |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160174902A1 (en) * | 2013-10-17 | 2016-06-23 | Siemens Aktiengesellschaft | Method and System for Anatomical Object Detection Using Marginal Space Deep Neural Networks |
| US20170213000A1 (en) * | 2016-01-25 | 2017-07-27 | Shenzhen University | Metabolic mass spectrometry screening method for diseases based on deep learning and the system thereof |
| CN109192298A (en) * | 2018-07-27 | 2019-01-11 | 南京航空航天大学 | Deep brain medical diagnosis on disease algorithm based on brain network |
| CN109214416A (en) * | 2018-07-23 | 2019-01-15 | 华南理工大学 | A kind of multidimensional information fusion Diagnosis Method of Transformer Faults based on deep learning |
-
2020
- 2020-07-08 CN CN202010653663.9A patent/CN111916204A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160174902A1 (en) * | 2013-10-17 | 2016-06-23 | Siemens Aktiengesellschaft | Method and System for Anatomical Object Detection Using Marginal Space Deep Neural Networks |
| US20170213000A1 (en) * | 2016-01-25 | 2017-07-27 | Shenzhen University | Metabolic mass spectrometry screening method for diseases based on deep learning and the system thereof |
| CN109214416A (en) * | 2018-07-23 | 2019-01-15 | 华南理工大学 | A kind of multidimensional information fusion Diagnosis Method of Transformer Faults based on deep learning |
| CN109192298A (en) * | 2018-07-27 | 2019-01-11 | 南京航空航天大学 | Deep brain medical diagnosis on disease algorithm based on brain network |
Non-Patent Citations (2)
| Title |
|---|
| CHEN QIAO等: ""Log-sum enhanced sparse deep neural network"", 《NEUROCOMPUTING》 * |
| 汤宝平等: ""多共振分量融合CNN的行星齿轮箱故障诊断"", 《振动、测试与诊断》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113450921A (en) * | 2021-06-24 | 2021-09-28 | 西安交通大学 | Brain development data analysis method, system, equipment and storage medium |
| CN113505528A (en) * | 2021-06-24 | 2021-10-15 | 西安交通大学 | Method, system, device and storage medium for identifying brain neural development time-varying functional connection difference |
| CN113505528B (en) * | 2021-06-24 | 2024-04-05 | 西安交通大学 | Method, system, device and storage medium for identifying brain nerve development time-varying function connection difference |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111785329B (en) | Single-cell RNA sequencing clustering method based on countermeasure automatic encoder | |
| CN109389171B (en) | Medical image classification method based on multi-granularity convolution noise reduction automatic encoder technology | |
| CN111353373B (en) | Related alignment domain adaptive fault diagnosis method | |
| CN111738363B (en) | Alzheimer disease classification method based on improved 3D CNN network | |
| CN111368885A (en) | Aero-engine gas circuit fault diagnosis method based on deep learning and information fusion | |
| CN109993236A (en) | Few sample language of the Manchus matching process based on one-shot Siamese convolutional neural networks | |
| CN110880369A (en) | Gas marker detection method based on radial basis function neural network and application | |
| CN111009324A (en) | Mild cognitive impairment auxiliary diagnosis system and method based on brain network multi-feature analysis | |
| CN110940523A (en) | Unsupervised domain adaptive fault diagnosis method | |
| CN110110845B (en) | Learning method based on parallel multi-level width neural network | |
| CN111916204A (en) | Brain disease data evaluation method based on self-adaptive sparse deep neural network | |
| CN111985825A (en) | Crystal face quality evaluation method for roller mill orientation instrument | |
| CN111046961A (en) | Fault classification method based on bidirectional long-and-short-term memory unit and capsule network | |
| CN112541530B (en) | Data preprocessing method and device for clustering model | |
| CN117195027A (en) | Cluster weighted clustering integration method based on member selection | |
| CN115205599B (en) | Multi-age-segment child chest image pneumonia classification system based on domain generalization model | |
| Holz et al. | Relative feature importance: A classifier-independent approach to feature selection | |
| CN110941542A (en) | Sequence integration high-dimensional data anomaly detection system and method based on elastic network | |
| CN110673577A (en) | Distributed monitoring and fault diagnosis method for complex chemical production process | |
| CN115661498A (en) | Self-optimization single cell clustering method | |
| CN114330650A (en) | Small sample characteristic analysis method and device based on evolutionary element learning model training | |
| CN114037860A (en) | Image classification and feature selection method based on robust minimum two-regression framework | |
| CN108932550B (en) | Method for classifying images based on fuzzy dense sparse dense algorithm | |
| CN116226629B (en) | Multi-model feature selection method and system based on feature contribution | |
| RU2819348C1 (en) | Method for graphed neural network classification for absence or presence of major depressive disorder according to fmri data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |