CN114037853B - Depth image clustering method based on Laplace rank constraint - Google Patents
Depth image clustering method based on Laplace rank constraint Download PDFInfo
- Publication number
- CN114037853B CN114037853B CN202111354109.1A CN202111354109A CN114037853B CN 114037853 B CN114037853 B CN 114037853B CN 202111354109 A CN202111354109 A CN 202111354109A CN 114037853 B CN114037853 B CN 114037853B
- Authority
- CN
- China
- Prior art keywords
- network
- image data
- data set
- clustering
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 239000011159 matrix material Substances 0.000 claims abstract description 32
- 239000013598 vector Substances 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000001228 spectrum Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 9
- 238000012549 training Methods 0.000 abstract description 6
- 238000007781 pre-processing Methods 0.000 abstract description 3
- 230000003595 spectral effect Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 241000282472 Canis lupus familiaris Species 0.000 description 2
- 241000282994 Cervidae Species 0.000 description 2
- 241000282326 Felis catus Species 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 241000269350 Anura Species 0.000 description 1
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a depth image clustering method based on Laplace rank constraint. Firstly, preprocessing an image data set to obtain an expanded data set; then, introducing orthogonal constraint in the last layer of the spectrum embedded network to ensure that clustering indication vectors output by the network are mutually orthogonal, introducing a similarity matrix limited by Laplace matrix rank in a loss function, and training the network by using an image data set; and finally, processing the images to be classified by using the trained network to obtain a classification result. The method can obtain good low-dimensional data representation, is suitable for carrying out image data clustering processing of different scale scales, can efficiently process large-scale image data sets, and has good practical value.
Description
Technical Field
The invention belongs to the field of machine learning, and particularly relates to a depth image clustering method based on Laplace rank constraint.
Background
Clustering is a basic method in the machine learning field, and the use of the clustering in the big data age is also increasingly prominent, no matter the industries face massive data, the clustering is definitely the lowest-cost unsupervised data analysis method, and the clustering analysis is also a primary tool for data analysis in many fields including mathematics, computer science, statistics, biology, economy and other subjects. However, with the trend of diversification of data forms, the existing clustering method is somewhat caught in the process of processing multi-scale and complex manifold data.
There is a conventional clustering method Ulrike von Luxburg, which is a spectral clustering method proposed in the literature "A Tutorial on Spectral clustering. Statistics and Computing, vol.17, no.4,2007, pp.395-416," which characterizes the distance between data by using a similarity measure between different data, treating each data as a node, mapping the entire dataset, and clustering the data by cutting the map. Nie et al in document "The Constrained Laplacian Rank Algorithm for Graph-Based clustering. AAAI'16Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence,2016,pp.1969-1976," propose to introduce rank constraints on the Laplace matrix in graph-Based clustering methods to obtain a cluster-friendly representation of data, i.e., to learn a similarity matrix with a distinct block structure. Most of the methods focus on finding better similarity measurement methods and finding optimal neighbor data points to improve the accuracy of clustering tasks, but are limited by the constraints of the self calculation method architecture, have high complexity and are difficult to apply to large-scale data sets.
The deep learning based clustering algorithm is a deep embedding clustering method proposed by Xie, junyuan et al in the literature "Unsupervised Deep Embedding for Clustering analysis.ICML'16Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48,2016, pp.478-487. The deep clustering method greatly improves the processing capacity of a large-scale data set, but lacks the interpretability when acquiring the low-dimensional characterization information of the data, and is difficult to obtain good low-dimensional embedded representation suitable for clustering. Moreover, the methods are only suitable for data sets with specific scales, and the clustering method is poor in robustness.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a depth image clustering method based on Laplacian rank constraint. Firstly, preprocessing an image data set to obtain an expanded data set; then, introducing orthogonal constraint in the last layer of the spectrum embedded network to ensure that clustering indication vectors output by the network are mutually orthogonal, introducing a similarity matrix limited by Laplace matrix rank in a loss function, and training the network by using an image data set; and finally, processing the images to be classified by using the trained network to obtain a classification result. The method can acquire good low-dimensional representation of data, can flexibly process image data of different scale, and can solve the problems that the existing method cannot process a large-scale data set, is difficult to acquire good characteristic representation suitable for clustering, is difficult to process data in various different mode representation forms and the like.
A depth image clustering method based on Laplace rank constraint is characterized by comprising the following steps:
step 1: inputting an image data set, processing images in the image data set by utilizing rotation, scaling and color transformation respectively to obtain an expanded image data set, and performing size normalization processing to ensure that all the images keep consistent in size;
step 2: randomly selecting m images in the image data set obtained in the step 1 to input a spectrum into a network, carrying out orthogonal constraint on the last layer of the network, calculating network loss L, updating network weight theta through forward propagation until the loss converges, and obtaining a trained network;
wherein, the value range of m is more than or equal to 1 and less than or equal to n, and n is the total number of images contained in the image dataset; judging a loss convergence condition by adopting an early stopping mechanism;
the spectrum embedded network refers to a 5-layer fully connected network, the dimensions of the network of layers 1-5 are 1024, 512, 256, 128 and k respectively, and k represents the number of clusters; the loss function L (θ) of the network is set as follows:
wherein y is i Ith input image x representing network output i Corresponding low-dimensional embedded features; c i And c j To press respectivelyAnd->The i-th input image x obtained by calculation i And jth input image x j Degree of corresponding node, s i,j Representing the j-th column element of the ith row in the similarity matrix S, and calculating the similarity matrix S according to the following formula:
wherein s is i Representing the transpose of the vector of elements of row i of the matrix S, L S Representing a laplacian matrix constructed from the input batch of image data, gamma is the regularization parameter and, the value range is (0), ++ infinity];
The orthogonal constraint is instruction Y T Y=I k Wherein Y is a matrix formed by indicating vectors output by the last layer of the network according to columns, I k Representing a k-order identity matrix;
step 3: inputting the image data set to be processed into the spectrum embedded network trained in the step 2, outputting to obtain a corresponding low-dimensional embedded vector, and clustering the low-dimensional embedded vector by using a K-means method to obtain a final clustering result.
The beneficial effects of the invention are as follows: since the rank limit of the Laplace matrix is introduced to calculate the similarity matrix, the implicit semantic relation between image data can be fully mined, the problem of high calculation complexity of the existing image clustering algorithm on a large-scale image data set is solved, and the clustering precision is improved; the number of neighbors is selected in a self-adaptive manner in the similarity calculation process, so that the complexity of similarity matrix calculation can be reduced, and the method can maintain good clustering efficiency in processing a large-scale image data set; because the orthogonal constraint is introduced into the last layer of the network, the clustering indication vectors output by the network can be ensured to be mutually orthogonal, the characteristic decomposition process of the traditional spectral clustering is replaced, and the calculation time is reduced; because the self-adaptive Laplace matrix calculation is embedded into the neural network, better low-dimensional data representation can be obtained, and the high efficiency of the traditional similarity calculation method and the strong calculation advantage of the neural network are combined, so that the method can well process image data sets with different scale dimensions; because the network weights are continuously updated by utilizing the batch training capability of the neural network, the method can be suitable for cluster analysis of data sets with different scale dimensions, and particularly can well cope with large-scale data set cluster processing.
Drawings
Fig. 1 is a flowchart of a depth image clustering method based on laplacian rank constraint of the present invention.
Detailed Description
The invention will be further illustrated with reference to the following figures and examples, which include but are not limited to the following examples.
As shown in fig. 1, the invention provides a depth image clustering method based on laplacian rank constraint, which comprises the following specific implementation processes:
1. data set enhancement and preprocessing
The method comprises the steps of inputting an image data set, processing images in the image data set by rotation, scaling and color transformation respectively, expanding the data quantity, enhancing the expression capability of the data set, and performing size normalization processing to enable all the images to keep consistent sizes. Let the resulting image dataset be x= { X 1 ,x 2 ,…,x n X, where x i I=1, 2, …, n, n representing the total number of images contained in the expanded image dataset, for a total of k classes of images.
2. Constructing an adaptive similarity matrix
In order to fully mine the implicit semantic relation between image data, a Laplace matrix rank constraint is introduced to construct a similarity matrix S as follows:
wherein x is i Representing the ith input image data, x j Represents the j-th input image data, s i,j Represents the j-th row and column elements, S, in the similarity matrix S i Representing the transpose vector of the ith row element of matrix S, L S A laplace matrix representing the data set X, gamma is one regularization parameter introduced, the value range is (0), ++ infinity]。
The optimal solution for the above equation is:
wherein p represents the number of the set neighbors, and the value range is 1-p-n and d ij Calculated according to the following formula:
wherein lambda is an introduced super parameter, and the value range is (0, 1];e i And e j Respectively representing an ith row and a jth row of vectors of the embedded representation matrix E, wherein the embedded representation matrix E is calculated by the following extremum problem according to the attribute of the spectral cluster:
3. setting network and parameters
Randomly initializing a weight parameter theta of a spectrum embedded network, setting the size batch size of batch training data to be m, setting the value range of m to be more than or equal to 1 and less than or equal to n, and carrying out orthogonal constraint on the last layer of the network, namely enabling Y to be the same as the value range of m T Y=I k Wherein Y is a matrix formed by indicating vectors output by the last layer of the network according to columns, I k Representing a k-th order identity matrix.
The loss function L (θ) of the network is set as follows:
wherein y is i Ith input image x representing network output i Corresponding low-dimensional embedded features; c i And c j To press respectivelyAnd->The i-th input image x obtained by calculation i And jth input image x j Degree of the corresponding node.
The spectrum embedded network refers to a 5-layer fully connected network, the dimensions of the network of layers 1-5 are 1024, 512, 256, 128 and k respectively, and the network output is a low-dimensional embedded feature representation vector of the input data.
4. Network training
And (3) randomly selecting m image input spectrums from the image data set obtained in the step (1) to embed the image input spectrums into a network, calculating the network loss L, and updating the network weight theta through forward propagation until the loss converges (judged by adopting an early stopping mechanism), so as to obtain a trained network.
5. Image clustering
Inputting the image data set to be classified into the spectrum embedded network trained in the step 4, outputting to obtain a corresponding low-dimensional embedded vector, and clustering the low-dimensional embedded vector by using a K-means method to obtain a final clustering result.
The effects of the present invention can be further illustrated by the following experiments.
1. Experimental conditions
The invention is simulated on a ubuntu20.04 operating system with a CPU model i7-5930K, a GPU model TITAN X (Pascal) (12G) and a memory 64G by using the python language and related kits. The data set used in the experiment is MNIST, CIFAR-10 data set, MNIST is a picture data set of handwriting characters, the picture data set comprises 70000 handwriting character pictures, and the picture size is 28x28; CIFAR-10 is a small color image dataset used to identify pervasive objects. A total of 10 categories of RGB color pictures: aircraft (airland), automobiles (automatic), birds (bird), cats (cat), deer (deer), dogs (dog), frogs (frog), horses (horse), boats (ship), and trucks (truck). Each picture has a size of 32×32, 6000 images per category, and a total of 50000 training pictures and 10000 test pictures in the dataset.
2. Experimental details
The method is adopted to train a network, test is carried out on a test set, two quantization indexes of Accuracy (ACC) and Normalized Mutual Information (NMI) of different methods on each data set are calculated, clustering results are shown in table 1, and selected comparison methods include a Spectral Clustering (SC) method, a Deep Embedding Clustering (DEC) method, a cosine heterogeneous model (HOE) method and a denoising self-encoder (DAE) method. DEC is described in detail in the literature "J.Xie, R.Girshick and a.faradai," Unsupervised deep embedding for clustering analysis, "in International Conference on Machine Learning,2016, pp.478-487," HOE is described in detail in the literature "X.Peng, H.Zhu, J.Feng, C.Shen, H.Zhang and j.t. zhou," Deep clustering with sample-assignment invariance prior, "IEEE Transactions on Neural Networks and Learning Systems, vol.31, no.11, pp.4857-4868,2019," DAE is described in detail in the literature "P.Vincent, H.Larochelle, I.Lajoie, Y.Bengio, p. -a.manzagol, and l.boltou," Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion "Journal of Machine Learning Research, vol.11, no.12,2010.
TABLE 1
As can be seen from Table 1, the performance of the method of the invention on two data sets is greatly improved compared with that of the comparison method, and the clustering effect on CIFAR-10 is greatly improved, which indicates the robustness and the good generalization of the method of the invention on large-scale data sets.
Claims (1)
1. A depth image clustering method based on Laplace rank constraint is characterized by comprising the following steps:
step 1: inputting an image data set, processing images in the image data set by utilizing rotation, scaling and color transformation respectively to obtain an expanded image data set, and performing size normalization processing to ensure that all the images keep consistent in size;
step 2: randomly selecting m images in the image data set obtained in the step 1 to input a spectrum into a network, carrying out orthogonal constraint on the last layer of the network, calculating network loss L, updating network weight theta through forward propagation until the loss converges, and obtaining a trained network;
wherein, the value range of m is more than or equal to 1 and less than or equal to n, and n is the total number of images contained in the image dataset; judging a loss convergence condition by adopting an early stopping mechanism;
the spectrum embedded network refers to a 5-layer fully connected network, the dimensions of the network of layers 1-5 are 1024, 512, 256, 128 and k respectively, and k represents the number of clusters; the loss function L (θ) of the network is set as follows:
wherein y is i Ith input image x representing network output i Corresponding low-dimensional embedded features; c i And c j To press respectivelyAnd->The i-th input image x obtained by calculation i And the j-th input diagramImage x j Degree of corresponding node, s i,j Representing the j-th column element of the ith row in the similarity matrix S, and calculating the similarity matrix S according to the following formula:
wherein s is i Representing the transpose of the vector of elements of row i of the matrix S, L S Representing a laplacian matrix constructed from the input batch of image data, gamma is the regularization parameter and, the value range is (0), ++ infinity];
The orthogonal constraint is instruction Y T Y=I k Wherein Y is a matrix formed by indicating vectors output by the last layer of the network according to columns, I k Representing a k-order identity matrix;
step 3: inputting the image data set to be processed into the spectrum embedded network trained in the step 2, outputting to obtain a corresponding low-dimensional embedded vector, and clustering the low-dimensional embedded vector by using a K-means method to obtain a final clustering result.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111354109.1A CN114037853B (en) | 2021-11-11 | 2021-11-11 | Depth image clustering method based on Laplace rank constraint |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111354109.1A CN114037853B (en) | 2021-11-11 | 2021-11-11 | Depth image clustering method based on Laplace rank constraint |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114037853A CN114037853A (en) | 2022-02-11 |
CN114037853B true CN114037853B (en) | 2024-03-05 |
Family
ID=80144477
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111354109.1A Active CN114037853B (en) | 2021-11-11 | 2021-11-11 | Depth image clustering method based on Laplace rank constraint |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114037853B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118154447B (en) * | 2024-05-11 | 2024-08-20 | 国网安徽省电力有限公司电力科学研究院 | Image recovery method and system based on guide frequency loss function |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111144463A (en) * | 2019-12-17 | 2020-05-12 | 中国地质大学(武汉) | Hyperspectral image clustering method based on residual subspace clustering network |
CN111259917A (en) * | 2020-02-20 | 2020-06-09 | 西北工业大学 | Image feature extraction method based on local neighbor component analysis |
CA3119533A1 (en) * | 2019-08-05 | 2021-02-11 | Intuit Inc. | Finite rank deep kernel learning with linear computational complexity |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US12190231B2 (en) * | 2016-10-19 | 2025-01-07 | Samsung Electronics Co., Ltd | Method and apparatus for neural network quantization |
US11783198B2 (en) * | 2020-04-03 | 2023-10-10 | Baidu Usa Llc | Estimating the implicit likelihoods of generative adversarial networks |
-
2021
- 2021-11-11 CN CN202111354109.1A patent/CN114037853B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA3119533A1 (en) * | 2019-08-05 | 2021-02-11 | Intuit Inc. | Finite rank deep kernel learning with linear computational complexity |
CN111144463A (en) * | 2019-12-17 | 2020-05-12 | 中国地质大学(武汉) | Hyperspectral image clustering method based on residual subspace clustering network |
CN111259917A (en) * | 2020-02-20 | 2020-06-09 | 西北工业大学 | Image feature extraction method based on local neighbor component analysis |
Non-Patent Citations (3)
Title |
---|
朱恒东 ; 马盈仓 ; 杨婷 ; 张要 ; .基于ε-邻域和拉普拉斯矩阵秩约束的谱聚类算法.纺织高校基础科学学报.2020,(01),全文. * |
邱云飞 ; 潘博 ; 张睿 ; 王万里 ; 魏宪 ; .嵌入式深度神经网络高光谱图像聚类.中国图象图形学报.2020,(01),全文. * |
郑建炜 ; 李卓蓉 ; 王万良 ; 陈婉君 ; .联合Laplacian正则项和特征自适应的数据聚类算法.软件学报.2019,(12),全文. * |
Also Published As
Publication number | Publication date |
---|---|
CN114037853A (en) | 2022-02-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111860670B (en) | Domain adaptive model training method, image detection method, device, equipment and medium | |
WO2018010434A1 (en) | Image classification method and device | |
CN108776812A (en) | Multiple view clustering method based on Non-negative Matrix Factorization and various-consistency | |
CN113326390B (en) | Image retrieval method based on depth feature consistent Hash algorithm | |
CN112132145A (en) | Image classification method and system based on model extended convolutional neural network | |
CN109840518B (en) | Visual tracking method combining classification and domain adaptation | |
Tao et al. | Image semantic segmentation based on convolutional neural network and conditional random field | |
Wang et al. | The application of mixed-level model in convolutional neural networks for cashmere and wool identification | |
Zhao et al. | Feature and region selection for visual learning | |
CN106228027A (en) | A kind of semi-supervised feature selection approach of various visual angles data | |
CN114281985A (en) | Sample feature space enhancement method and device | |
CN114037853B (en) | Depth image clustering method based on Laplace rank constraint | |
CN111639686B (en) | Semi-supervised classification method based on dimension weighting and visual angle feature consistency | |
Hu et al. | An integrated classification model for incremental learning | |
Bi et al. | Critical direction projection networks for few-shot learning | |
Ayeni | Convolutional neural network (CNN): the architecture and applications | |
CN108388918B (en) | Data feature selection method with structure retention characteristics | |
CN113762005B (en) | Feature selection model training and object classification methods, devices, equipment and media | |
CN111310807B (en) | Feature subspace and affinity matrix joint learning method based on heterogeneous feature joint self-expression | |
CN111160398B (en) | Missing label multi-label classification method based on example level and label level association | |
CN114692809A (en) | Data processing method and device based on neural cluster, storage medium and processor | |
CN113850274B (en) | Image classification method based on HOG features and DMD | |
Jain et al. | Flynet–neural network model for automatic building detection from satellite images | |
CN113313153B (en) | Low-rank NMF image clustering method and system based on self-adaptive graph regularization | |
CN111461255B (en) | Siamese network image identification method and system based on interval distribution |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |