CN112070062A - Hadoop-based crop waterlogging image classification detection and implementation method - Google Patents
Hadoop-based crop waterlogging image classification detection and implementation method Download PDFInfo
- Publication number
- CN112070062A CN112070062A CN202011006864.6A CN202011006864A CN112070062A CN 112070062 A CN112070062 A CN 112070062A CN 202011006864 A CN202011006864 A CN 202011006864A CN 112070062 A CN112070062 A CN 112070062A
- Authority
- CN
- China
- Prior art keywords
- image
- hadoop
- crop
- algorithm
- waterlogging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/188—Vegetation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
A classification detection and realization method of crop waterlogging image based on Hadoop is characterized by comprising the following steps of a) collecting crop field waterlogging image, correcting and preprocessing the image, and analyzing the main components of the image; b) uploading the image matrix to a Hadoop computing platform, performing distributed storage, and compiling a parallel neural network algorithm; and c) carrying out algorithm modeling and prediction, and carrying out classification analysis on the crop waterlogging image information. The invention carries out distributed parallel classification analysis on the image data under crop disaster stress through the Hadoop frame, accelerates the modeling and predicting speed of a classification algorithm, and has more obvious advantages compared with a single-machine mode in the case of larger image data volume. A neural network algorithm is written by using a Scala language, and the algorithm is suitable for parallelization operation under a Hadoop framework. And the HDFS of the Hadoop frame is adopted for distributed storage of the image data, so that damage and loss of the image data are avoided.
Description
The technical field is as follows:
the invention belongs to the crossing field of image processing and computer distributed computing, and particularly relates to a Hadoop-based crop waterlogging image classification detection and implementation method.
Background art:
machine vision and image processing systems play an important role in modern agriculture and in the food processing industry. Machine vision and image analysis provide a powerful computer-aided image classification tool for crop production, and the identification accuracy can be greatly improved. In addition, machine vision techniques can improve and develop methods for referencing and building libraries of computer-aided analysis images of biological specimens of interest. The Hadoop can provide flexible and reliable data storage to form a large server cluster, and on the other hand, the Hadoop can also provide support for effective parallel computation in a distributed mode, so that the processing and analysis speed of the image is improved.
In the current crop disaster detection and analysis, crop disaster image information acquired by a machine vision system is stored in a computer, and modeling analysis processing is performed by using analysis software such as Matlab and SPSS. The accumulated crop image information will be more and the data volume will be larger and larger as time goes on. When the amount of image data is too large, the analysis processing speed is slower and slower when the traditional analysis processing method is used, and even the image analysis cannot be normally performed.
The invention content is as follows:
aiming at the development trend and the actual demand of the current precision agriculture and the intelligent agriculture and overcoming the defects in the field of the current agricultural image detection and analysis, the invention designs and realizes a crop image classification detection and realization method based on Hadoop. By collecting RGB images of the crop waterlogging stress, the main component analysis is adopted to extract the image characteristic information. And constructing a distributed storage and calculation platform based on a Hadoop frame, and storing the image matrix by adopting an HDFS (Hadoop distributed file system) mode. And realizing a parallel neural network classification algorithm by using a Scala language and an IntelliJ IDEA development environment, carrying out classification analysis on image information, and realizing the detection of the crop waterlogging image and the distributed storage of crop image data.
In order to solve the technical problem, the invention provides a Hadoop-based crop waterlogging image classification detection and implementation method, which comprises the following steps:
image acquisition: adopting a CCD image sensor to collect the image of the rape waterlogging stress leaf, correcting and preprocessing the collected image, converting an image color channel, and converting RGB into La*b*。
La*b*And the image is converted into a three-dimensional matrix, and the principal component analysis method and the extraction of image characteristic information are adopted, so that the data dimension is reduced, and the data calculation amount is reduced.
And performing Distributed storage and management on the image matrix by using an HDFS (Hadoop Distributed File System Hadoop Distributed storage System), wherein a Master node is used as a Namenode node to manage the whole HDFS File System and the directory tree, and one copy is stored on other 5 Datano nodes.
And (4) carrying out parallel neural network algorithm construction on the main node.
And by a Hadoop cooperative work mechanism, the HDFS distributed framework is used for distributing and scheduling the operation, and the data blocks are read by nodes and the operation task and calculation are completed.
And the main node sorts and combines the calculation results, and finally collects the calculation results to obtain the neural network parallel image classification result.
In order to avoid over (under) fitting of the neural network algorithm, the image data are used as a training set and a prediction set, wherein the data volume of the training set accounts for 70%, a neural network training model is established, and parameter tuning is carried out.
And inputting the remaining 30% of data sets after the algorithm training is finished, performing algorithm prediction, and performing model performance evaluation.
And obtaining a classification detection analysis result of the image data.
Compared with the prior art, the invention has the beneficial effects that:
(1) the distributed parallel classification analysis of the image data under the crop disaster stress is carried out through the Hadoop framework, the modeling and predicting speed of the classification algorithm is accelerated, and the advantages are obvious when the image data volume is larger compared with a single-machine mode.
(2) A neural network algorithm is written by using a Scala language, and the algorithm is suitable for parallelization operation under a Hadoop framework.
(3) And the HDFS of the Hadoop frame is adopted for distributed storage of the image data, so that damage and loss of the image data are avoided.
Description of the drawings:
the accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a flow chart of image principal component (dimensionality reduction).
Fig. 2 is a flow chart of a method.
The specific implementation mode is as follows:
the invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes and modifications of the present invention may be made by those skilled in the art after reading the teachings of the present invention, and such equivalents may fall within the scope of the invention as defined in the appended claims.
And acquiring the RGB image information of the field damaged crops by adopting a CCD sensor camera based on the CMOS.
Preprocessing image information, denoising and enhancing images, removing noise signals in the images, enhancing image recognition effect, and converting image color channel mode from RGB to La*b*And (4) format.
The image is converted into a three-dimensional matrix, and the image characteristic information is extracted by using a principal component analysis method, so that the data volume is reduced.
Calculating the mean and variance of the image matrix; standardizing the matrix and calculating a covariance matrix; solving an eigenvalue and an eigenvector of the covariance matrix; and calculating the cumulative variance contribution rate, wherein the contribution rate is more than 70%, and the cumulative variance contribution rate can be set as a principal component.
The Hadoop platform uses a standalon mode, wherein a Master Node is responsible for resource allocation and job scheduling, a Node1-Node5 is used as a Node to execute a calculation task, and the Master Node is also used as a Node to perform task processing.
And (4) carrying out parallel neural network algorithm construction on the main node.
The neural network algorithm comprises a three-layer structure, wherein the first layer is an input layer, and the number of nodes of the input layer is the characteristic number of a sample; the second layer is a hidden layer, and the number of nodes is manually set; the third layer is an output layer, and the number of nodes is the characteristic quantity of the sample target. In the forward propagation process of the signal, the input layer is used as the input of the hidden layer node to calculate the output of the hidden layer node, and meanwhile, the output of the hidden layer node is used as the input of the output layer to calculate the input of the output layer.
The parameters of the neural network comprise weights and bias of an input layer and a hidden layer, so that the connection weight between a first-layer unit and a second-layer unit is a bias term, and after signals are propagated forwards, the output function of an output layer node is as follows:
the middle layer uses Sigmoid function as activation function. By introducing nonlinear factors into the neurons, the neural network can arbitrarily approximate a nonlinear function. The output neuron calculates the output value of the whole neural network according to the input value and the activation function.
And (3) the back propagation of the error is carried out, the partial derivative of the objective function to the weight of each neuron is calculated layer by layer from the output layer, the weight and the threshold value of each layer are adjusted by using a gradient descent function according to the calculated partial derivative, and the weight value is updated until the final output value of the modified network is close to the expected value. Since the Sigmoid function is easy to have gradient vanishing when the Sigmoid function is reversely propagated, the Softmax function is used as a gradient descending function of a multi-classification problem when the Sigmoid function is reversely propagated.
The image data set is input with a step size, a priori (training) sample size, and iteration number.
Initializing network parameters;
For t∈{1,2,…n}
feed forward: the output of each neuron is computed at each connected layer.
Calculating an error according to the loss function to obtain a gradient function of the output layer;
inversely calculating the degree and gradient of error contribution of each neuron in the upper layer, calculating connection parameters and updating weights,
and (4) when the algorithm reaches a reverse input layer, iteratively updating the parameters and training a neural network.
Uploading image data to a Hadoop computing platform, managing a whole HDFS file system and a directory tree by adopting an HDFS storage form and taking a Master node as a Namenode node, and storing a copy of an image data set on other 5 Datanode nodes.
And (3) completing a neural network algorithm program by a Hadoop platform client (JobClient), and compiling and operating the program. Hadoop JobClient sends a task request to JobTracker, applies for free available computing resources (Job), and the JobTracker returns an available Job ID to the JobClient.
And after obtaining the Job ID, the JobClient copies the resource file required by running Job into a file system HDFS, and the copied file information is stored in a folder created by JobTracker.
After the resources are completely prepared, the submission task is started, and JobClient submits a Job task (Job) to JobTracker.
After receiving Job, JobTracker initializes Job, puts it into the Job queue, and waits for the Job scheduler to schedule it.
After initialization is completed, the JobTracker acquires input splits from the HDFS, creates a Map task for each partition according to a scheduling algorithm and partition information, and allocates the Map task to the TaskTracker to complete execution.
And the TaskTracker sends a heartbeat packet to the JobTracker according to a fixed interval time to inform the JobTracker of the running state, meanwhile, the heartbeat packet contains other information such as the progress of the completion of the current Map task and the like, and when the JobTracker receives the completion information of the operation task, the current operation state is set to be 'completed'. If the JobTracker does not receive the heartbeat packet returned by the TaskTracker within the fixed interval time, the JobTracker judges that the TaskTracker fails, and meanwhile, the job tasks of the TaskTracker are dispatched to other TaskTracker to be continuously executed.
The TaskTracker reads the job resource file from the HDFS. And after the TaskTracker obtains the tasks distributed by the JobTracker, acquiring Job resources from the HDFS.
After the resources are obtained, the TaskTracker starts the JVM subprocess and starts to run the task.
Job is divided into data blocks of the same size (64 MB by default) by the block size of the HDFS, and the user Job programs corresponding to the data blocks are executed.
The Master node (Master) finds and prepares available Map nodes for the job block and transfers the data block to the Map nodes. And meanwhile, the master node also finds and allocates available Reduce nodes for the data blocks and transmits the data blocks to the Reduce nodes.
The main node starts each Map node to execute a program, allocates a Map task to a TaskTracker containing a data block processed by the Map, and copies a JAR packet during program operation to the TaskTracker for operation, so that each Map node reads local data as much as possible to calculate.
And each Map node processes the read data blocks, combines and sorts the data, and stores the data on a local machine. And meanwhile, the master node is informed of the completion of the calculation task and the storage position of the intermediate result data of the master node, and after all Map nodes such as the master node and the like are calculated, the Reduce node is started to run. And the Reduce node acquires the position information of the intermediate result data from the main node and reads the data.
And the Reduce nodes sort and combine the calculation results, and finally collect to obtain an output file, namely the neural network classification result of the crop waterlogging image.
The method comprises the steps of adopting a 70% image matrix as input, operating a neural network algorithm, carrying out parallel algorithm training, establishing a neural network algorithm model, adjusting training parameters and preventing over (under) fitting of the algorithm.
And (4) performing algorithm prediction by taking the remaining 30% of image data as input, and performing crop stain image detection.
The technical means disclosed by the scheme of the invention are not limited to the technical means disclosed by the technical means, and also comprise the technical scheme formed by equivalent replacement of the technical features. The present invention is not limited to the details given herein, but is within the ordinary knowledge of those skilled in the art.
Claims (4)
1. A Hadoop-based crop waterlogging image classification detection and realization method is characterized by comprising the following steps:
step a), collecting a crop field waterlogging image, correcting and preprocessing the image, and analyzing the main components of the image;
b) uploading the image matrix to a Hadoop computing platform, performing distributed storage, and compiling a parallel neural network algorithm;
and c) carrying out algorithm modeling and prediction, and carrying out classification analysis on the crop waterlogging image information.
2. The Hadoop-based crop stain image classification detection and implementation method as claimed in claim 1, wherein the step a) comprises the following specific steps:
step a 1): collecting the information of the stain image in the crop field, converting the image into an image matrix, carrying out image correction and pretreatment, carrying out image denoising and image enhancement,
step a 2): analyzing image principal components, extracting image characteristic information, and performing image data dimension reduction;
step a 3): calculating the mean and variance of the image matrix;
step a 4): calculating matrix standardization and calculating a covariance matrix;
step a 5): and (3) calculating: solving an eigenvalue and an eigenvector of the covariance matrix;
step a 6): calculating the cumulative variance contribution rate; the contribution rate is more than 70 percent and can be used as a main component;
3. the Hadoop-based crop stain image classification detection and implementation method as claimed in claim 1, wherein the step b) comprises the following specific steps:
step b 1): uploading the image matrix data to a Hadoop computing platform, and performing distributed storage in an HDFS mode;
step b 2): the Master node serves as a Namenode node to manage the whole HDFS file system and the directory tree, and the remaining 5 Datanode nodes store copies of an image data set;
step b 3): and writing a neural network algorithm based on a Hadoop platform by adopting Scala.
4. The Hadoop-based crop stain image classification detection and implementation method as claimed in claim 1, wherein the step c) comprises the following specific steps:
step c 1): the neural network comprises three layers, namely an input layer, a hidden layer and an output layer, wherein the output function of the output layer node is as follows:
step c 2): the middle layer adopts a Sigmoid function as an activation function:
step c 3): the Softmax function is used as a gradient descent function for the multi-classification problem in the back propagation,
step c4), adopting 70% data to train the model, observing the training result, adjusting the training parameter according to the training result;
step c5) data model prediction of the remaining 30%, algorithm performance and efficiency evaluation,
and c6) carrying out the detection of the stain image of the crops.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011006864.6A CN112070062A (en) | 2020-09-23 | 2020-09-23 | Hadoop-based crop waterlogging image classification detection and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011006864.6A CN112070062A (en) | 2020-09-23 | 2020-09-23 | Hadoop-based crop waterlogging image classification detection and implementation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112070062A true CN112070062A (en) | 2020-12-11 |
Family
ID=73681987
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011006864.6A Pending CN112070062A (en) | 2020-09-23 | 2020-09-23 | Hadoop-based crop waterlogging image classification detection and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112070062A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627392A (en) * | 2021-09-02 | 2021-11-09 | 南京工业职业技术大学 | Rape waterlogging hyperspectral image detection method based on Spark platform and image acquisition device thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036286A (en) * | 2014-05-14 | 2014-09-10 | 武汉图歌信息技术有限责任公司 | Multispectral remote-sensing image classification processing method based on Hadoop |
| CN104270412A (en) * | 2014-06-24 | 2015-01-07 | 南京邮电大学 | Three-level caching method based on Hadoop distributed file system |
| CN104899561A (en) * | 2015-05-27 | 2015-09-09 | 华南理工大学 | Parallelized human body behavior identification method |
| CN106845343A (en) * | 2016-12-15 | 2017-06-13 | 中国科学院电子学研究所苏州研究院 | A kind of remote sensing image offshore platform automatic testing method |
| CN107122375A (en) * | 2016-12-12 | 2017-09-01 | 南京理工大学 | The recognition methods of image subject based on characteristics of image |
| CN107844780A (en) * | 2017-11-24 | 2018-03-27 | 中南大学 | A kind of the human health characteristic big data wisdom computational methods and device of fusion ZED visions |
-
2020
- 2020-09-23 CN CN202011006864.6A patent/CN112070062A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036286A (en) * | 2014-05-14 | 2014-09-10 | 武汉图歌信息技术有限责任公司 | Multispectral remote-sensing image classification processing method based on Hadoop |
| CN104270412A (en) * | 2014-06-24 | 2015-01-07 | 南京邮电大学 | Three-level caching method based on Hadoop distributed file system |
| CN104899561A (en) * | 2015-05-27 | 2015-09-09 | 华南理工大学 | Parallelized human body behavior identification method |
| CN107122375A (en) * | 2016-12-12 | 2017-09-01 | 南京理工大学 | The recognition methods of image subject based on characteristics of image |
| CN106845343A (en) * | 2016-12-15 | 2017-06-13 | 中国科学院电子学研究所苏州研究院 | A kind of remote sensing image offshore platform automatic testing method |
| CN107844780A (en) * | 2017-11-24 | 2018-03-27 | 中南大学 | A kind of the human health characteristic big data wisdom computational methods and device of fusion ZED visions |
Non-Patent Citations (3)
| Title |
|---|
| 孙文珺: "基于深度学习模型的感应电机故障诊断方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 林中琦等: "基于 Spark 的支持向量机在小麦病害图像识别中的应用", 《河南农业科学》 * |
| 沈俊灵: "基于Spark的高光谱图像分布式并行分类", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627392A (en) * | 2021-09-02 | 2021-11-09 | 南京工业职业技术大学 | Rape waterlogging hyperspectral image detection method based on Spark platform and image acquisition device thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Rafiki: Machine learning as an analytics service system | |
| CN109559320B (en) | Method and system for realizing visual SLAM semantic mapping function based on hole convolution deep neural network | |
| CN111598881A (en) | Image anomaly detection method based on variational self-encoder | |
| WO2019200735A1 (en) | Livestock feature vector acquisition method, apparatus, computer device and storage medium | |
| CN112949704B (en) | Tobacco leaf maturity state identification method and device based on image analysis | |
| CN112541458A (en) | Domain-adaptive face recognition method, system and device based on meta-learning | |
| CN110188812A (en) | A kind of multicore clustering method of quick processing missing isomeric data | |
| CN111179216A (en) | Crop disease identification method based on image processing and convolutional neural network | |
| CN114387499A (en) | Island coastal wetland waterfowl identification method, distribution query system and medium | |
| CN110598848A (en) | Migration learning acceleration method based on channel pruning | |
| WO2023236319A1 (en) | Convolutional neural network deployment and optimization method for microcontroller | |
| JP2022079947A (en) | Pruning management apparatus, pruning management system, and pruning management method | |
| CN115311483A (en) | Incomplete multi-view clustering method and system based on local structure and balance perception | |
| CN112308825A (en) | SqueezeNet-based crop leaf disease identification method | |
| CN112070062A (en) | Hadoop-based crop waterlogging image classification detection and implementation method | |
| CN113627240B (en) | Unmanned aerial vehicle tree species identification method based on improved SSD learning model | |
| CN114820435A (en) | Pepper leaf disease detection method based on improved AlexNet | |
| CN113421228A (en) | Thyroid nodule identification model training method and system based on parameter migration | |
| CN110363240B (en) | Medical image classification method and system | |
| CN117371511A (en) | Training method, device, equipment and storage medium for image classification model | |
| CN117058079A (en) | Thyroid imaging image automatic diagnosis method based on improved ResNet model | |
| KR20210086754A (en) | Method for autonomous diagnosis model of pests and diseases using deep learning | |
| CN111814853B (en) | Decorrelation clustering method and device under data selection deviation | |
| Sheoran et al. | Analysis on application of swarm-based techniques in processing remote sensed data | |
| CN113704528A (en) | Clustering center determination method, device and equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201211 |