CN108960069A - A method of the enhancing context for single phase object detector - Google Patents

A method of the enhancing context for single phase object detector Download PDF

Info

Publication number
CN108960069A
CN108960069A CN201810567939.4A CN201810567939A CN108960069A CN 108960069 A CN108960069 A CN 108960069A CN 201810567939 A CN201810567939 A CN 201810567939A CN 108960069 A CN108960069 A CN 108960069A
Authority
CN
China
Prior art keywords
network
sub
classification
single phase
detection block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810567939.4A
Other languages
Chinese (zh)
Inventor
庞彦伟
聂晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201810567939.4A priority Critical patent/CN108960069A/en
Publication of CN108960069A publication Critical patent/CN108960069A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/20Scenes; Scene-specific elements in augmented reality scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to a kind of methods of enhancing context for single phase object detector: the training dataset of selected object detection;Construct single phase object detector, main includes two parts: extracting the core network of feature and the sub-network for classifying and detection block returns, core network chooses ResNet50 network, it is improved for classification and the sub-network returned with detection block, design the sub-network that classification sub-network and detector based on enhancing context approach return, classification sub-network and detection block return sub-network and use identical design: constituting a submodule by three different convolution filters of expansion rate, then stack two sub- module composition sub-networks;Using the loss function of multitask;Training.

Description

A method of the enhancing context for single phase object detector
Technical field
It is the invention belongs to deep learning and computer vision field, in particular to a kind of for single phase object detector Enhance the method for context.
Background technique
Object detection is the extremely challenging project of computer vision field.Object detection based on deep learning achieves Significant achievement.Have been widely used for the various fields such as video monitoring, automatic Pilot and human-computer interaction.
The purpose of object detection is to discriminate between object and background, identifies object category, while being accurately located object with detection block Body.The object detection algorithms of mainstream are divided into two classes: the object detection algorithms of dual-stage and the object detection algorithms of single phase at present. The object detection algorithms [1] of dual-stage are that object detection problem is divided into two steps: the first step is to suggest that network generates with region Some candidate regions, second step classify and be adjusted candidate region location to these candidate regions.Single phase object Physical examination method of determining and calculating [2] is then from image directly positioning object itself.In contrast, dual-stage object detector has higher The detection speed of detection accuracy, single phase detector is high-efficient fastly.
The difficult point of object detection is that object is blocked with the presence of multiple scales and object, existing object detection algorithms benefit Issues On Multi-scales are solved with feature pyramid [3-5], are incorporated contextual information [4] and are handled occlusion issue
Bibliography:
[1]Ren S,HeK,GirshickR,etal.FasterR-CNN:towardsreal-time objectdetectionwith regionproposalnetworks[C]// InternationalConferenceonNeuralInformationProcessing Systems.MITPress,2015: 91-99.
[2]LiuW,AnguelovD,ErhanD,etal.SSD:Single ShotMultiBoxDetector[J] .2015:21-37.
[3]Lin TY,DollarP,GirshickR,etal.FeaturePyramidNetworks forObjectDetection[J].2016:936-944.
[4]CaiZ,FanQ,Feris RS,etal.AUnifiedMulti- scaleDeepConvolutionalNeuralNetwork forFastObjectDetection[C]// EuropeanConference onComputerVision.Springer,Cham,2016:354-370.
[5]Lin TY,GoyalP,GirshickR,etal.FocalLoss forDenseObjectDetection[J] .2017:2999-3007.
[6]Lin,Tsung-Yi,etal."Microsoftcoco:Commonobjectsincontext."European conference on computervision.Springer,Cham,2014
Summary of the invention
A kind of method of enhancing context for single phase object detector is provided it is an object of that present invention to provide a kind of, Single phase detector is set to incorporate more contextual informations, it is more efficient for the detection of wisp and the object that is blocked, It also contributes to reducing false-alarm simultaneously.Technical solution is as follows:
A method of the enhancing context for single phase object detector
1) training dataset of object detection is selected, data set includes picture and mark, marks the position containing detection block And object category;
2) construct single phase object detector, mainly include two parts: extract feature core network and for classify and Detection block return sub-network, core network choose ResNet50 network, for classification and with detection block return sub-network into Row improve, design based on enhancing context approach classification sub-network and detector return sub-network, classification sub-network with Detection block returns sub-network and uses identical design: constituting a submodule by three different convolution filters of expansion rate, so Two sub- module composition sub-networks are stacked afterwards;
3) loss function of multitask, the loss function including Classification Loss function and detection block precision, design grid are used The number and the final condition of convergence of network of network training loop iteration, and initialization network parameter;
4) being input to training data batch in the network is calculated and is trained, the specific steps are as follows:
A) training data is inputted in network, is sequentially inputted to core network and classification sub-network and detection block returns net Network calculates convolutional layer feature X ∈ RH×W×C, wherein H × W indicates the size of the characteristic pattern of output, and C indicates the characteristic pattern of output Port number;
B) it calculates and loses and carry out backpropagation, update network weight according to gradient descent method;
C) circulation step a)~b), after successive ignition, loss convergence obtains trained neural network model;
5) when input picture, position and the classification of the middle object of present image can be calculated by the model.
The present invention returns sub-network in the classification sub-network and detection block of single phase detector and introduces parallel expansion rate not With filter, the receptive field of these filters is of different sizes, to incorporate contextual information abundant, help detect wisp and There is the object blocked, realize simply, while guaranteeing detection efficiency, effectively promotes the detection performance of single phase detector, Facilitate wisp and has the object detection blocked.This patent method is applied in RetinaNet [5] network structure, replacement Classification and detection block originally returns sub-network, object detection experiment is carried out on COCO image data base [6], this patent is to small The accuracy rate of object detection improves 1 percentage point.
Detailed description of the invention
The network structure of the existing single phase object detector of Fig. 1
The network structure of single phase object detector of the Fig. 2 based on enhancing context
Fig. 3 expands the receptive field of convolution, and the expansion rate r of (a) (b) (c) is respectively 1,2,4
Specific embodiment
This patent is further described with reference to the accompanying drawing.
Fig. 1 describes the network structure of existing single phase object detector.In existing single phase object detector In network structure, sub-network of classifying and detection block return sub-network design simply, only the simple stacking of n convolutional layer, these The convolution kernel size of convolutional layer is fixed, and corresponding receptive field is also fixed, available Limited information.Classify sub-network it is defeated It is the size of characteristic pattern for W × H × KA, W and H out, K is the classification number of object, A and the pre-set default of single phase detector Box type number is consistent.Detection block return sub-network output be W × H × 4A, W and H be characteristic pattern size, 4 for default frame with Four offsets of real-world object position, A are characterized the number of types of the default frame of each position of figure.
Fig. 2 describes the network structure of the enhancing context for single phase detector of this patent proposition.Enhancing is up and down The operation of text is mainly reflected in classification sub-network and detection block returns in sub-network, and sub-network is no longer the convolution of fixed receptive field The stacking of layer, but the series connection of 2 submodules, detail are as follows:
(1) submodule of integrating context.As shown in Fig. 2, submodule is made of 3 parallel convolution blocks, C1, C2, C3。 The size of parallel convolution kernel is identical, but expansion rate difference (r1=1, r2=2, r3=4).The convolution kernel of different expansion rates is experienced It is wild different.It is 1 that common convolution, which is considered as expansion rate,.By taking one-dimensional expansion convolution as an example, when input is x, the power of convolution kernel Weight is w, and the expansion rate of convolution kernel is r, is exported as y, and the expression formula of one-dimensional expansion convolution is as follows:
Wherein, expansion rate is corresponding with the sampling span of input.As seen from Figure 3, the corresponding sense of different expansion rates It is of different sizes by open country, it is equally 3 × 3 convolution kernel, as r=2, receptive field is the 2.5 of r=12Times, the impression as r=4 5 when open country is r=12Times.The big receptive field of expansion rate is big, can incorporate more contextual informations.
(2) Fusion Features.The output of each parallel filter of submodule has different receptive fields, the big filtering of expansion rate The contextual information that the output of device obtains is abundanter, and the output characteristic pattern size of each parallel branch is identical, is added point by point, then It inputs in a common convolution filter together, i.e. C4 in Fig. 2, carries out Fusion Features, the feature extracted is made to have more robust Property.
(3) parameter sharing.Existing single phase detector [2] [5] can be to difference point in order to detect multiple dimensioned object The feature output of resolution carries out multistage detection output, then the filter of the corresponding configuration of multistage classification sub-network can be with parameter It is shared, parameter amount is reduced, similarly, detection blocks at different levels return sub-network can also be with parameter sharing.Thus in enhancing proposed in this paper The ginseng negligible amounts that network structure hereafter introduces, the influence to detection rates are little.
Technical solution of the present invention will be clearly and completely described below, it will be to single phase detector in description The method that RetinaNet [5] incorporates enhancing contextual information, it is clear that described embodiment is only that a part of the invention is real Example, rather than whole examples.
Apply the present invention in object detection task, mainly include three steps: preparing data set;It designs and trains base In the single phase detector of enhancing context approach;Test/apply detection model.It implements step and is described as follows:
Step 1: preparing data set.
(1) suitable object detection data set is selected.The data set of current more common object detection has PascalVOC With COCO etc., there is the label information of object category and object detection frame.Image size in data set is not fixed, can in training Image one side size is arranged and fixes, another side limits maximum length according to the demand of actual hardware condition and application.As A kind of example, we use COCO data set [6], and in the format that this data uses for picture short side a length of 800, long side is most very much not Color image format more than 1333x3, all images are by the data enhancing overturn at random and normalization operation.
(2) image set divides.COCO2014 data set includes training set, verifying collection and test set.We will use training set With the verifying collection single phase detector that training is enhanced based on context together, test set is follow-up test modelling effect or reality Using when use.
Step 2: designing and training the single phase detector based on context Enhancement Method.
Design the single phase detector based on context Enhancement Method.It is special that image is suitably extracted in entire design including selection The core network of sign, and classify, the number of the submodule for the integrating context information that detection block sub-network includes, submodule institute The number of parallel filter, the size and expansion rate of convolution kernel design the number of plies of convolutional layer, and melt for feature The convolution filter structure of conjunction, the number and the final condition of convergence of network of planned network training loop iteration, and initialize network Parameter.Using ResNet50 as core network in the present invention, while the thought of feature pyramid network [3] is also applied, adopted With from top to bottom with the structure of horizontal-associate come reinforce different resolution characteristic pattern semanteme, the design of sub-network selects 2 submodules Block series connection, there are three parallel convolution filters respectively for each submodule, and wherein their convolution kernel size is all 3 × 3, expansion Rate is respectively 1,2 and 4, and the output channel number of convolution filter is all 256.To reduce computation complexity, using 1 × 1 convolution Filter is as Fusion Features filter.Since COCO data set has 80 type objects, so K=81.Setting in same [5], I Set 9 for A, the combination of corresponding three kinds of different areas and Aspect Ratio.The area from 32 to 512 of default frame is evenly distributed to The outputs at different levels of feature pyramid network, the length-width ratio for defaulting frame have 1/2,1,2.
(1) the good single phase detector based on context Enhancement Method of first initialization design, core network are ResNet50 its weights initialisation in ImageNet data set classification based training, remaining network layer random initializtion.
(2) the single phase detector based on context Enhancement Method is then trained, training image batch is input to this In network, is calculated and is trained, the specific steps are as follows:
A) training image data are inputted in core network, extracts the feature of picture.
B) feature of calculating is passed into corresponding classification sub-network again and detection block returns sub-network.
1. characteristic pattern successively passes through two submodules of sub-network.Parallel convolution kernel expansion rate is different in submodule, right Receptive field of different sizes is answered, contextual information more abundant can be obtained, then, the output feature of parallel-convolution filter It is added, is input in the filter of Fusion Features pixel-by-pixel, using filter ω ∈ R1×1×256×256.Mixing operation introduces few Parameter to be learned is measured, computation complexity is reduced.
It is lost 2. the output for sub-network of classifying and true value are calculated using focal loss function, will test frame Recurrent networks Output and true value using Smooth L1 function calculate lose.
C) circulation step a)~b), after successive ignition, loss function convergence obtains trained neural network mould Type.
Step 3: the trained network model of test/reference.
(1) test set data are got out, call designed network structure and trained network parameter, and by test chart Piece batch or individual be input in trained model.
(2) image data is passed sequentially through core network by forward calculation, and sub-network of classifying and detection block return sub-network. Classification sub-network output is that detection block belongs to all kinds of probability, and class of the maximum classification of select probability as final detection block Not, what detection block returned sub-network output is the offset of opposite default frame, obtains more accurate detection frame knot by offset Fruit.After non-maxima suppression, class probability is regarded as into final testing result greater than given threshold.

Claims (1)

1. a kind of method of the enhancing context for single phase object detector, including the following steps:
1) training dataset of object detection is selected, data set includes picture and mark, marks the position containing detection block and object Body classification;
2) single phase object detector is constructed, mainly includes two parts: extracting the core network of feature and for classifying and detecting The sub-network that frame returns, core network choose ResNet50 network, are changed for classification and the sub-network returned with detection block Into designing the sub-network that classification sub-network and detector based on enhancing context approach return, classification sub-network and detection Frame returns sub-network and uses identical design: constituting a submodule by three different convolution filters of expansion rate, then heap Fold two sub- module composition sub-networks;
3) loss function of multitask, the loss function including Classification Loss function and detection block precision, planned network instruction are used Practice the number and the final condition of convergence of network of loop iteration, and initialization network parameter;
4) being input to training data batch in the network is calculated and is trained, the specific steps are as follows:
A) training data is inputted in network, is sequentially inputted to core network and classification sub-network and detection block Recurrent networks, Calculate convolutional layer feature X ∈ RH×W×C, wherein H × W indicates the size of the characteristic pattern of output, and C indicates the channel of the characteristic pattern of output Number;
B) it calculates and loses and carry out backpropagation, update network weight according to gradient descent method;
C) circulation step a)~b), after successive ignition, loss convergence obtains trained neural network model;
5) when input picture, position and the classification of the middle object of present image can be calculated by the model.
CN201810567939.4A 2018-06-05 2018-06-05 A method of the enhancing context for single phase object detector Pending CN108960069A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810567939.4A CN108960069A (en) 2018-06-05 2018-06-05 A method of the enhancing context for single phase object detector

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810567939.4A CN108960069A (en) 2018-06-05 2018-06-05 A method of the enhancing context for single phase object detector

Publications (1)

Publication Number Publication Date
CN108960069A true CN108960069A (en) 2018-12-07

Family

ID=64493163

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810567939.4A Pending CN108960069A (en) 2018-06-05 2018-06-05 A method of the enhancing context for single phase object detector

Country Status (1)

Country Link
CN (1) CN108960069A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109470701A (en) * 2018-12-18 2019-03-15 江苏镭博智能科技有限公司 A kind of method of the online biological monitoring of water process
CN109784194A (en) * 2018-12-20 2019-05-21 上海图森未来人工智能科技有限公司 Target detection network establishing method and training method, object detection method
CN109858451A (en) * 2019-02-14 2019-06-07 清华大学深圳研究生院 A kind of non-cooperation hand detection method
CN109948666A (en) * 2019-03-01 2019-06-28 广州杰赛科技股份有限公司 Image similarity recognition methods, device, equipment and storage medium
CN110020658A (en) * 2019-03-28 2019-07-16 大连理工大学 A kind of well-marked target detection method based on multitask deep learning
CN110070072A (en) * 2019-05-05 2019-07-30 厦门美图之家科技有限公司 A method of generating object detection model
CN110245655A (en) * 2019-05-10 2019-09-17 天津大学 A kind of single phase object detecting method based on lightweight image pyramid network
CN110348390A (en) * 2019-07-12 2019-10-18 创新奇智(重庆)科技有限公司 A kind of training method, computer-readable medium and the system of fire defector model
WO2021036013A1 (en) * 2019-08-30 2021-03-04 北京市商汤科技开发有限公司 Configuration method and apparatus for detector, electronic device, and storage medium
CN113129345A (en) * 2021-04-19 2021-07-16 重庆邮电大学 Target tracking method based on multi-feature map fusion and multi-scale expansion convolution
CN113537170A (en) * 2021-09-16 2021-10-22 北京理工大学深圳汽车研究院(电动车辆国家工程实验室深圳研究院) Intelligent traffic road condition monitoring method and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894012A (en) * 2016-03-29 2016-08-24 天津大学 Object identification method based on cascade micro neural network
CN107871134A (en) * 2016-09-23 2018-04-03 北京眼神科技有限公司 A kind of method for detecting human face and device
CN107958271A (en) * 2017-12-06 2018-04-24 电子科技大学 The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894012A (en) * 2016-03-29 2016-08-24 天津大学 Object identification method based on cascade micro neural network
CN107871134A (en) * 2016-09-23 2018-04-03 北京眼神科技有限公司 A kind of method for detecting human face and device
CN107958271A (en) * 2017-12-06 2018-04-24 电子科技大学 The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LIANG-CHIEH CHEN,AT EL.: ""Rethinking Atrous Convolution for Semantic Image Segmentation"", 《ARXIV》 *
TSUNG-YI LIN,AT EL.: ""Focal Loss for Dense Object Detection"", 《ARXIV》 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109470701A (en) * 2018-12-18 2019-03-15 江苏镭博智能科技有限公司 A kind of method of the online biological monitoring of water process
CN109784194A (en) * 2018-12-20 2019-05-21 上海图森未来人工智能科技有限公司 Target detection network establishing method and training method, object detection method
CN109784194B (en) * 2018-12-20 2021-11-23 北京图森智途科技有限公司 Target detection network construction method, training method and target detection method
CN109858451B (en) * 2019-02-14 2020-10-23 清华大学深圳研究生院 Non-matching hand detection method
CN109858451A (en) * 2019-02-14 2019-06-07 清华大学深圳研究生院 A kind of non-cooperation hand detection method
CN109948666A (en) * 2019-03-01 2019-06-28 广州杰赛科技股份有限公司 Image similarity recognition methods, device, equipment and storage medium
CN110020658A (en) * 2019-03-28 2019-07-16 大连理工大学 A kind of well-marked target detection method based on multitask deep learning
CN110070072A (en) * 2019-05-05 2019-07-30 厦门美图之家科技有限公司 A method of generating object detection model
CN110245655A (en) * 2019-05-10 2019-09-17 天津大学 A kind of single phase object detecting method based on lightweight image pyramid network
CN110245655B (en) * 2019-05-10 2023-06-06 天津大学 Single-stage object detection method based on lightweight image pyramid network
CN110348390A (en) * 2019-07-12 2019-10-18 创新奇智(重庆)科技有限公司 A kind of training method, computer-readable medium and the system of fire defector model
WO2021036013A1 (en) * 2019-08-30 2021-03-04 北京市商汤科技开发有限公司 Configuration method and apparatus for detector, electronic device, and storage medium
JP2022515274A (en) * 2019-08-30 2022-02-17 北京市商▲湯▼科技▲開▼▲発▼有限公司 Detector placement method, detector placement device and non-temporary computer readable storage medium
CN113129345A (en) * 2021-04-19 2021-07-16 重庆邮电大学 Target tracking method based on multi-feature map fusion and multi-scale expansion convolution
CN113537170A (en) * 2021-09-16 2021-10-22 北京理工大学深圳汽车研究院(电动车辆国家工程实验室深圳研究院) Intelligent traffic road condition monitoring method and computer readable storage medium

Similar Documents

Publication Publication Date Title
CN108960069A (en) A method of the enhancing context for single phase object detector
CN109447169A (en) The training method of image processing method and its model, device and electronic system
CN104392228B (en) Unmanned plane image object class detection method based on conditional random field models
KR101640998B1 (en) Image processing apparatus and image processing method
CN110826379B (en) Target detection method based on feature multiplexing and YOLOv3
CN110298298A (en) Target detection and the training method of target detection network, device and equipment
CN109522966A (en) A kind of object detection method based on intensive connection convolutional neural networks
CN110245655A (en) A kind of single phase object detecting method based on lightweight image pyramid network
CN109522967A (en) A kind of commodity attribute recognition methods, device, equipment and storage medium
CN109784386A (en) A method of it is detected with semantic segmentation helpers
CN110363201A (en) Weakly supervised semantic segmentation method and system based on Cooperative Study
CN106960195A (en) A kind of people counting method and device based on deep learning
CN109325395A (en) The recognition methods of image, convolutional neural networks model training method and device
CN106874894A (en) A kind of human body target detection method based on the full convolutional neural networks in region
CN109583483A (en) A kind of object detection method and system based on convolutional neural networks
CN110378348A (en) Instance of video dividing method, equipment and computer readable storage medium
Liu et al. SparseNet: A sparse DenseNet for image classification
CN108629369A (en) A kind of Visible Urine Sediment Components automatic identifying method based on Trimmed SSD
CN108921057A (en) Prawn method for measuring shape of palaemon, medium, terminal device and device based on convolutional neural networks
CN110349167A (en) A kind of image instance dividing method and device
CN108710893A (en) A kind of digital image cameras source model sorting technique of feature based fusion
CN110263731B (en) Single step human face detection system
CN109583455A (en) A kind of image significance detection method merging progressive figure sequence
CN107944403A (en) Pedestrian's attribute detection method and device in a kind of image
CN104699781B (en) SAR image search method based on double-deck anchor figure hash

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20181207