CN108090513A - Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension - Google Patents
Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension Download PDFInfo
- Publication number
- CN108090513A CN108090513A CN201711374940.7A CN201711374940A CN108090513A CN 108090513 A CN108090513 A CN 108090513A CN 201711374940 A CN201711374940 A CN 201711374940A CN 108090513 A CN108090513 A CN 108090513A
- Authority
- CN
- China
- Prior art keywords
- image
- fractal dimension
- biological characteristic
- typical correlation
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Abstract
The present invention relates to a kind of multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension, technical characteristics are：By the image that raw data set image reconstruction is 32*32；By convolution operation twice, feature selecting operation and full attended operation, by the one-dimensional characteristic vector of 120 dimensions of the image of pretreated 32*32；One-dimensional characteristic vector is analyzed using typical correlation fractal dimension, obtains fusion feature vector of the highest feature vector of the degree of association as multi-biological characteristic；Fusion feature vector is sent into ELM graders to classify.The present invention introduces particle swarm optimization algorithm and typical correlation fractal dimension based on convolutional neural networks structure, and different biometric image features is merged, finally obtains more complete biological characteristic set, so as to carry out effective authentication；The present invention has higher Stability and veracity, can be widely used for the fields such as image identification, security protection inspection.
Description
Technical field
It is especially a kind of to be associated based on particle cluster algorithm with typical case the invention belongs to biometric image identification technology field
The multi-biological characteristic blending algorithm of analytic approach.
Background technology
At present, biometrics identification technology plays more and more important application in field of identity authentication, and image co-registration skill
Art can obtain more biometric particulars and information, there is the promotion of bigger to recognition performance, therefore attract more and more
Researcher concern.Among these, multi-modal biological characteristic integration technology is exactly an important direction.Multi-modal biology
Fusion Features include following three level：Feature-level fusion refers to the plan merged after being extracted to biological characteristic
Slightly；Matching layer fusion is then to merge to obtain one group of new matching value and carry out identity to recognize by the matching value of different feature vector
Card；Decision-level fusion be to different biological characteristics respectively by it is respective carry feature, established model, identify after decision-making knot
Fruit is merged.But accuracy of the existing multi-modal biological characteristic integration technology when carrying out living things feature recognition and steady
It is qualitative that above there are still deficiencies.
The content of the invention
It is an object of the invention to overcome the deficiencies in the prior art, propose a kind of based on particle cluster algorithm and typical to associate point
The multi-biological characteristic blending algorithm of analysis method is combined together using the exclusive feature of a variety of biological characteristics, improves biological characteristic
The Stability and veracity of image identification.
The present invention solves its technical problem and following technical scheme is taken to realize：
A kind of multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension, comprises the following steps：
Step 1：Image preprocessing：By the image that raw data set image reconstruction is 32*32；
Step 2：Feature extraction：It, will be pretreated by convolution operation twice, feature selecting operation and full attended operation
The one-dimensional characteristic vector of 120 dimensions of the image of 32*32；
Step 3：Fusion Features：One-dimensional characteristic vector is analyzed using typical correlation fractal dimension, obtains the degree of association most
Fusion feature vector of the high feature vector as multi-biological characteristic；
Step 4：Fusion feature vector is sent into ELM graders to classify.
Further, the raw data set image includes facial image and refers to vein image, and the facial image is 112*
92, the finger vein image is 60*128.
Further, the implementation method of step 2 feature extraction：It is carried out for the first time using the convolution collecting image of 8 groups of 5x5
Convolution operation obtains the characteristic pattern of 8 groups of 28x28；Second of convolution collecting image using 20 groups of 5x5 carries out convolution operation, obtains
To the characteristic pattern of 20 groups of 5x5；Using once full attended operation, the one-dimensional characteristic for finally obtaining 120 dimensions is vectorial.
Further, the processing method of the convolution operation is：
W2=(w1+2*p-k)/s+1
H2=(h1+2*p-k)/s+1
C2=n
Wherein, w1, h1, c1 represent width, height and the depth of input picture respectively；W2, h2, c2 represent output figure respectively
Width, height and the depth of picture；In convolutional layer, n represents the number of convolution kernel, and k*k represents convolution kernel size, and p represents to expand
Edge, s represent convolution kernel step-length.
Further, step 2 feature selecting is realized using improved particle swarm optimization algorithm：For the first time from 28x28's
In characteristic pattern, 14x14 character subsets are selected；Second of character subset that 5x5 is selected from the characteristic pattern of 10x10.
Further, the calculation formula of the feature selecting is：
Fitness=∑s Dk (1<=k<=m)
Vx []=w*vx []+c1*rand () * (pbest []-p [])+c2*rand () * (gbest []-p [])
P []=p []+vx []
Wherein, Dk represents k-th of dimensional characteristics of current particle and the difference summation of surrounding features, and m represents population scale,
Pbest represents locally optimal solution, and gbest represents globally optimal solution, and v [] is the speed of particle, and w is inertia weight, and p [] is to work as
The position of preceding particle, rand () are the random numbers between (0,1), and c1, c2 are Studying factors.
Further, the implementation method of the step 3 is：Face characteristic is calculated using typical correlation fractal dimension and refers to vein spy
Correlativity between sign by the maps feature vectors of 120 dimensions to the sharing feature space of multidimensional, uses communal space feature weight
The multi-modal feature of structure obtains fusion feature vector.
The advantages and positive effects of the present invention are：
1st, the present invention is based on convolutional neural networks structure, introduces particle swarm optimization algorithm (PSO) and typical associates point
Different biometric image Fusion Features are obtained more complete biological characteristic set by analysis method (CCA), so as to carry out effectively
Authentication；It is of the invention that there is higher Stability and veracity compared with existing biological feather recognition method, it can be wide
It is general to be used for the fields such as image identification, security protection inspection.
2nd, the present invention is directed to the diversity and aggregation of characteristics of image, before fusion, using particle swarm optimization algorithm
(PSO) effective feature selecting is carried out, achievees the purpose that feature accumulation and dimensionality reduction.
3rd, the present invention is ensureing study precision using extreme learning machine (Extreme Learning Machine, ELM)
Under the premise of, improve pace of learning.
Description of the drawings
Fig. 1 is the general frame figure of the present invention；
Fig. 2 is the feature extracting method flow chart that the present invention uses；
Fig. 3 is the convolutional neural networks structural model that the present invention uses；
Fig. 4 be present invention introduces particle swarm optimization algorithm structure chart；
Fig. 5 is contrast and experiment figure of the present invention on single creature characteristic data set；
Fig. 6 is contrast and experiment figure of the present invention on multi-modal biological characteristic data set；
Fig. 7 is the ROC curve figure of present invention distinct methods on same data set.
Specific embodiment
The embodiment of the present invention is further described below in conjunction with attached drawing.
A kind of multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension, as shown in Figure 1, including
Following steps：
Step 1：Image preprocessing：Raw biometric image is become to the image of 32*32 by reconstruct.
In this step, it is necessary to by raw data set image (ORL facial images 112*92；Refer to vein image 60*128) weight
Structure is the image of 32*32.Image reconstructing method represents as follows：
I=imread (' IMG.jpg')；
I=imresize (I, [32,32])；
Wherein, IMG.jpg represents raw data set image, and I represents the graphical representation after reconstruct.
Step 2：Feature extraction, as shown in Fig. 2, by convolution operation twice, feature selecting operation and full attended operation, it will
The image of pretreated 32*32 is represented with the one-dimensional characteristic vector FeatureSet of 120 dimensions.
In this step, feature extraction uses convolution operation, as shown in Figure 3：For the first time using the convolution kernel pair of 8 groups of 5x5
Image carries out convolution operation, obtains the characteristic pattern of 8 groups of 28x28；Second of convolution collecting image using 20 groups of 5x5 carries out convolution
Operation, obtains the characteristic pattern of 20 groups of 5x5；Using once full attended operation, 120 dimensional vectors are finally obtained for character representation.
Feature extracting method represents as follows：
W2=(w1+2*p-k)/s+1
H2=(h1+2*p-k)/s+1
C2=n
Wherein, input picture w1*h1*c1, w1 represent width, and h1 represents height, and c1 represents depth, and output image is
w2*h2*c2；In convolutional layer, n represents the number of convolution kernel, and k*k represents convolution kernel size, and p represents to expand edge, and s represents volume
Product core step-length.
In this step, feature selecting uses improved particle swarm optimization algorithm, as shown in Figure 4：For the first time from 28x28's
In characteristic pattern, 14x14 character subsets are selected；Second of character subset that 5x5 is selected from the characteristic pattern of 10x10.Feature selecting
Calculation formula it is as follows：
Fitness=∑s Dk (1<=k<=m)
Vx []=w*vx []+c1*rand () * (pbest []-p [])+c2*rand () * (gbest []-p [])
P []=p []+vx []
Wherein, Dk represents k-th of dimensional characteristics of current particle and the difference summation of surrounding features, and m represents population scale,
Pbest represents locally optimal solution, and gbest represents globally optimal solution, and v [] is the speed of particle, and w is inertia weight, and p [] is to work as
The position of preceding particle, rand () are the random numbers between (0,1), and c1, c2 are Studying factors, usual c1=c2=2.
Step 3：Fusion Features：Feature vector obtained in the previous step is analyzed using typical correlation fractal dimension, is obtained
Fusion feature vector of the highest feature vector of the degree of association as multi-biological characteristic.
This step is to carry out CCA method fusions to the expression FeatureSet of multi-modal biological characteristic image, is merged
Feature FusionFeature.Specific method is：Two groups of variables are calculated using typical correlation fractal dimension (CCA) (face characteristic and to refer to
Vein pattern) between correlativity, by 120 dimension feature set be mapped to d dimension sharing feature space, the feature communal space
Feature has the correlation of height, and can reconstruct multi-modal feature, the most effective expression as multi-modal biological characteristic.
Step 4：Fusion feature vector is sent into ELM graders to classify.
In this step, go that grader is trained to improve its point using feature representation obtained in the previous step as the input of grader
Class ability.This step can also export experimental result, for being surveyed in test phase for performance using ELM as Decision Classfication device
Examination.
Further verification is done to the present invention below by experiment.
Fig. 5 gives performance of the different sorting algorithms on single creature characteristic data set and compares.This experiment is each
Database chooses 400 images, takes 80% to train, remaining is tested.As can be seen from the table, based on single creature feature
Recognition efficiency be not highly desirable, substantially 90% or so；It can reach highest recognition efficiency using CNN model trainings
(93.84%) but simultaneously it is also most to expend the time (83s)；ELM models on time (10s) and performance (92.67%) all
There is good performance, but still the space to make progress.Therefore, multi-biological characteristic fusion experiment be very it is necessary to.
Fig. 6 gives the comparison of distinct methods experimental performance on multi-modal data storehouse, this is tested each database and chooses
60000 images, take 50000 to train, remaining is tested.As can be seen from the table, even simple CCA fusions are to knowing
Other efficiency is obviously improved (5% or so)；The addition of deep learning model has recognition efficiency better promotion
(98.70%), but add simultaneously and calculate the time (866s)；It is proposed that method have part advantage (798s) in time,
It is more the promotion (98.89%) to experimental result.
Fig. 7 gives the ROC curve by comparing algorithms of different, it can be seen that curve of the invention is located at other algorithms song
The bottom of line, that is, the false acceptance rate and false rejection rate of the invention for image, all than relatively low, this is also from another side
Face demonstrates the advantage of the present invention.
It is emphasized that embodiment of the present invention is illustrative rather than limited, therefore present invention bag
The embodiment being not limited to described in specific embodiment is included, it is every by those skilled in the art's technique according to the invention scheme
The other embodiment drawn, also belongs to the scope of protection of the invention.
Claims (8)
1. a kind of multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension, it is characterised in that including with
Lower step：
Step 1：Image preprocessing：By the image that raw data set image reconstruction is 32*32；
Step 2：Feature extraction：By convolution operation twice, feature selecting operation and full attended operation, by pretreated 32*
The one-dimensional characteristic vector of 32 image, 120 dimensions；
Step 3：Fusion Features：One-dimensional characteristic vector is analyzed using typical correlation fractal dimension, it is highest to obtain the degree of association
Fusion feature vector of the feature vector as multi-biological characteristic；
Step 4：Fusion feature vector is sent into ELM graders to classify.
2. the multi-biological characteristic blending algorithm according to claim 1 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：The raw data set image include facial image and refer to vein image, the facial image be 112*92, institute
It states and refers to vein image as 60*128.
3. the multi-biological characteristic blending algorithm according to claim 1 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：The implementation method of step 2 feature extraction：For the first time convolution is carried out using the convolution collecting image of 8 groups of 5x5
Operation, obtains the characteristic pattern of 8 groups of 28x28；Second of convolution collecting image using 20 groups of 5x5 carries out convolution operation, obtains 20
The characteristic pattern of group 5x5；Using once full attended operation, the one-dimensional characteristic for finally obtaining 120 dimensions is vectorial.
4. the multi-biological characteristic blending algorithm according to claim 1 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：The processing method of the convolution operation is：
W2=(w1+2*p-k)/s+1
H2=(h1+2*p-k)/s+1
C2=n
Wherein, w1, h1, c1 represent width, height and the depth of input picture respectively；W2, h2, c2 represent output image respectively
Width, height and depth；In convolutional layer, n represents the number of convolution kernel, and k*k represents convolution kernel size, and p represents to expand edge,
S represents convolution kernel step-length.
5. the multi-biological characteristic blending algorithm according to claim 1 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：Step 2 feature selecting is realized using improved particle swarm optimization algorithm：For the first time from the feature of 28x28
In figure, 14x14 character subsets are selected；Second of character subset that 5x5 is selected from the characteristic pattern of 10x10.
6. the multi-biological characteristic blending algorithm according to claim 5 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：The calculation formula of the feature selecting is：
Fitness=∑s Dk (1<=k<=m)
Vx []=w*vx []+c1*rand () * (pbest []-p [])+c2*rand () * (gbest []-p [])
P []=p []+vx []
Wherein, Dk represents k-th of dimensional characteristics of current particle and the difference summation of surrounding features, and m represents population scale,
Pbest represents locally optimal solution, and gbest represents globally optimal solution, and v [] is the speed of particle, and w is inertia weight, and p [] is to work as
The position of preceding particle, rand () are the random numbers between (0,1), and c1, c2 are Studying factors.
7. the multi-biological characteristic blending algorithm according to claim 6 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：Described Studying factors c1, c2 are equal to 2.
8. the multi-biological characteristic blending algorithm according to claim 1 based on particle cluster algorithm and typical correlation fractal dimension,
It is characterized in that：The implementation method of the step 3 is：Using typical correlation fractal dimension calculate face characteristic and refer to vein pattern it
Between correlativity, by 120 dimension maps feature vectors to the sharing feature space of multidimensional, it is more using communal space feature reconstruction
Modal characteristics obtain fusion feature vector.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711374940.7A CN108090513A (en) | 2017-12-19 | 2017-12-19 | Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711374940.7A CN108090513A (en) | 2017-12-19 | 2017-12-19 | Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108090513A true CN108090513A (en) | 2018-05-29 |
Family
ID=62177170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711374940.7A Pending CN108090513A (en) | 2017-12-19 | 2017-12-19 | Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108090513A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046590A (en) * | 2019-04-22 | 2019-07-23 | 电子科技大学 | It is a kind of one-dimensional as recognition methods based on particle group optimizing deep learning feature selecting |
| CN112801163A (en) * | 2021-01-22 | 2021-05-14 | 安徽大学 | Multi-target feature selection method of mouse model hippocampal biomarker based on dynamic graph structure |
| CN112836630A (en) * | 2021-02-01 | 2021-05-25 | 清华大学深圳国际研究生院 | Attention detection system and method based on CNN |
| CN115830686A (en) * | 2022-12-13 | 2023-03-21 | 云指智能科技(广州)有限公司 | Biological recognition method, system, device and storage medium based on feature fusion |
| CN115984193A (en) * | 2022-12-15 | 2023-04-18 | 东北林业大学 | PDL1 expression level detection method fusing histopathology image and CT image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984919A (en) * | 2014-04-24 | 2014-08-13 | 上海优思通信科技有限公司 | Facial expression recognition method based on rough set and mixed features |
| CN104732249A (en) * | 2015-03-25 | 2015-06-24 | 武汉大学 | Deep learning image classification method based on popular learning and chaotic particle swarms |
| CN106022473A (en) * | 2016-05-23 | 2016-10-12 | 大连理工大学 | Construction method for gene regulatory network by combining particle swarm optimization (PSO) with genetic algorithm |
| CN106897538A (en) * | 2017-03-14 | 2017-06-27 | 中国人民解放军军械工程学院 | Geomagnetic chart direction suitability computational methods based on convolutional neural networks |
| CN107292256A (en) * | 2017-06-14 | 2017-10-24 | 西安电子科技大学 | Depth convolved wavelets neutral net expression recognition method based on secondary task |
-
2017
- 2017-12-19 CN CN201711374940.7A patent/CN108090513A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984919A (en) * | 2014-04-24 | 2014-08-13 | 上海优思通信科技有限公司 | Facial expression recognition method based on rough set and mixed features |
| CN104732249A (en) * | 2015-03-25 | 2015-06-24 | 武汉大学 | Deep learning image classification method based on popular learning and chaotic particle swarms |
| CN106022473A (en) * | 2016-05-23 | 2016-10-12 | 大连理工大学 | Construction method for gene regulatory network by combining particle swarm optimization (PSO) with genetic algorithm |
| CN106897538A (en) * | 2017-03-14 | 2017-06-27 | 中国人民解放军军械工程学院 | Geomagnetic chart direction suitability computational methods based on convolutional neural networks |
| CN107292256A (en) * | 2017-06-14 | 2017-10-24 | 西安电子科技大学 | Depth convolved wavelets neutral net expression recognition method based on secondary task |
Non-Patent Citations (2)
| Title |
|---|
| 祁昆仑: ""卷积神经网络中的感受野计算"", 《ZHUANLAN.ZHIHU.COM/P/26663577》 * |
| 薛冰霞: ""基于多模特征融合的人体跌倒检测算法研究"", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046590A (en) * | 2019-04-22 | 2019-07-23 | 电子科技大学 | It is a kind of one-dimensional as recognition methods based on particle group optimizing deep learning feature selecting |
| CN110046590B (en) * | 2019-04-22 | 2022-04-22 | 电子科技大学 | One-dimensional image identification method based on particle swarm optimization deep learning feature selection |
| CN112801163A (en) * | 2021-01-22 | 2021-05-14 | 安徽大学 | Multi-target feature selection method of mouse model hippocampal biomarker based on dynamic graph structure |
| CN112801163B (en) * | 2021-01-22 | 2022-10-04 | 安徽大学 | Multi-target feature selection method of mouse model hippocampal biomarker based on dynamic graph structure |
| CN112836630A (en) * | 2021-02-01 | 2021-05-25 | 清华大学深圳国际研究生院 | Attention detection system and method based on CNN |
| CN115830686A (en) * | 2022-12-13 | 2023-03-21 | 云指智能科技(广州)有限公司 | Biological recognition method, system, device and storage medium based on feature fusion |
| CN115984193A (en) * | 2022-12-15 | 2023-04-18 | 东北林业大学 | PDL1 expression level detection method fusing histopathology image and CT image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yan et al. | 'skimming-perusal'tracking: A framework for real-time and robust long-term tracking | |
| CN108090513A (en) | Multi-biological characteristic blending algorithm based on particle cluster algorithm and typical correlation fractal dimension | |
| CN103605972B (en) | Non-restricted environment face verification method based on block depth neural network | |
| Xu et al. | Deep learning of feature representation with multiple instance learning for medical image analysis | |
| Lim et al. | Sketch tokens: A learned mid-level representation for contour and object detection | |
| Feng et al. | Residual learning for salient object detection | |
| CN107316031A (en) | The image characteristic extracting method recognized again for pedestrian | |
| CN110414371A (en) | A kind of real-time face expression recognition method based on multiple dimensioned nuclear convolution neural network | |
| CN105956560A (en) | Vehicle model identification method based on pooling multi-scale depth convolution characteristics | |
| CN111126240B (en) | Three-channel feature fusion face recognition method | |
| CN110188708A (en) | A kind of facial expression recognizing method based on convolutional neural networks | |
| CN111027377B (en) | Double-flow neural network time sequence action positioning method | |
| CN106548149A (en) | The recognition methods of the micro- facial expression image sequence of face in monitor video sequence | |
| CN102194114A (en) | Method for recognizing iris based on edge gradient direction pyramid histogram | |
| CN103440471B (en) | The Human bodys' response method represented based on low-rank | |
| CN109325507A (en) | A kind of image classification algorithms and system of combination super-pixel significant characteristics and HOG feature | |
| CN109815920A (en) | Gesture identification method based on convolutional neural networks and confrontation convolutional neural networks | |
| CN103699874A (en) | Crowd abnormal behavior identification method based on SURF (Speed-Up Robust Feature) stream and LLE (Locally Linear Embedding) sparse representation | |
| Wang et al. | S 3 d: scalable pedestrian detection via score scale surface discrimination | |
| CN112861605A (en) | Multi-person gait recognition method based on space-time mixed characteristics | |
| CN103984954B (en) | Image combining method based on multi-feature fusion | |
| Sun et al. | Brushstroke based sparse hybrid convolutional neural networks for author classification of Chinese ink-wash paintings | |
| EP3642764A1 (en) | Learning unified embedding | |
| Shengtao et al. | Facial expression recognition based on global and local feature fusion with CNNs | |
| CN110969101A (en) | Face detection and tracking method based on HOG and feature descriptor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180529 |