CN112241693A - Illegal welding fire image identification method based on YOLOv3 - Google Patents
Illegal welding fire image identification method based on YOLOv3 Download PDFInfo
- Publication number
- CN112241693A CN112241693A CN202011022862.6A CN202011022862A CN112241693A CN 112241693 A CN112241693 A CN 112241693A CN 202011022862 A CN202011022862 A CN 202011022862A CN 112241693 A CN112241693 A CN 112241693A
- Authority
- CN
- China
- Prior art keywords
- flame
- model
- yolov3
- data set
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 13
- 238000003466 welding Methods 0.000 title claims abstract description 9
- 238000001514 detection method Methods 0.000 claims abstract description 43
- 238000003064 k means clustering Methods 0.000 claims abstract description 9
- 238000010276 construction Methods 0.000 claims abstract description 5
- 238000012544 monitoring process Methods 0.000 claims abstract description 5
- 238000013135 deep learning Methods 0.000 claims abstract description 4
- 238000002372 labelling Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims abstract description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000002485 combustion reaction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fire-Detection Mechanisms (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an illegal welding fire image recognition method based on YOLOv3, and belongs to the technical field of image recognition. It comprises the following steps: (1) arranging pictures containing flames collected from a construction site into a flame data set, and labeling flame areas in the flame data set; (2) obtaining an anchor parameter suitable for flame detection on a flame data set in a k-means clustering mode; (3) modifying a YOLOv3.cfg file in a YOLOv3 model, replacing the anchor parameter in the step (2), and debugging and modifying values of batch, filter and classes parameters; (4) debugging parameters such as learning rate and batch, and training to obtain the optimal model weight; (5) and loading the model weight by using a keras, constructing a fire model deep learning network, and detecting each frame of monitoring image transmitted from the rear end. According to the scheme, the anchor parameter is determined on the self-made flame data set in a k-means clustering mode to obtain a higher IOU score, the convergence rate of the model is increased, and the detection accuracy of the model is improved.
Description
Technical Field
The invention relates to an illegal welding fire image recognition method based on YOLOv3, and belongs to the technical field of image recognition.
Background
The traditional flame detection method benefits from the progress of image processing technology and the popularization of video monitoring equipment, and the flame detection technology based on computer vision is rapidly developed. The flame image features are various, and the traditional flame detection method mostly adopts the manually designed features to construct a detection model. Dimitropoulos and the like propose a detection method utilizing flame color, stroboscopic and other feature modeling, an SVM classifier is used for classification, and the calculation cost is high due to the fact that dynamic texture analysis is applied to a candidate region; celik et al propose a flame detector fusing flame foreground and color information, under the premise that the image input size is lower, the detection rate reaches 30fps, and the real-time performance is stronger; poobalan and the like adopt an RGB color model to detect the color of flame, and adopt a flame segmentation technology based on color to extract an interested region, so that the overall detection precision reaches 90 percent, and the method has certain practicability; gunn Qingtian and the like develop a new color recognition rule by utilizing an RGB model and a YCbCr model to establish a flame detection model, reduce interference caused by illumination background change and show stronger robustness under unfavorable illumination background conditions.
In recent years, with the improvement of hardware computing capability, more deep convolutional network models are applied to the field of target detection, and the deep convolutional network models are mainly divided into a two-step target detection algorithm based on an R-CNN series network and a single-step target detection algorithm based on a YOLO network and an SSD network. The former generates a series of sample candidate frames by a region candidate frame method, and then carries out sample classification by a convolutional neural network; the method integrates two tasks of extracting candidate frames and classifying into a network, and directly converts the target frame positioning problem into a regression problem for processing.
At present, a detection model with manually designed characteristics has certain limitations in the aspects of application scenes, detection precision and detection rate, and the detection accuracy of the existing model is not accurate enough. The flame, as a non-rigid body, changes its morphology during combustion, having a different aspect ratio. The initial stage of fire occurrence is a key period of flame detection, the flame form at the moment is mostly small flame, which requires that the model has higher detection capability for small targets, and more shallow features with high resolution are utilized to construct a high-level semantic feature map. And the YOLOv3 adopts multi-scale feature fusion, and the YOLOv3 model respectively fuses a high-resolution shallow feature, a relatively abstract middle-layer feature and a completely abstract high-layer feature in the network into one another to serve as 3 features to be output. Therefore, the characteristics of different flame forms in the initial stage, the middle stage and the later stage of the flame can be captured more conveniently, and the loss of information is reduced. However, the anchor dimension of YOLOv3 was determined based on the VOC2007 and VOC2012 data sets and was not universal. It is therefore particularly important to select a set of candidate boxes that fit into the flame data set. Aiming at the problem, the scheme determines the anchor parameter on the self-made flame data set by using a k-means clustering mode so as to obtain a higher IOU score and accelerate the convergence speed of the model. Therefore, an illegal welding fire image identification method based on YOLOv3 is designed to improve the problems.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the illegal welding fire image identification method based on YOLOv3 is provided, and the problem that the detection accuracy of a model is not accurate enough due to the limitation of a detection model with manually designed characteristics in the aspects of application scenes, detection accuracy and detection rate is solved.
The technical problem to be solved by the invention is realized by adopting the following technical scheme:
the illegal welding fire image recognition method based on YOLOv3 is characterized by comprising the following steps of:
(1) arranging pictures containing flames collected from a construction site into a flame data set, and labeling flame areas in the flame data set;
(2) obtaining an anchor parameter suitable for flame detection on a flame data set in a k-means clustering mode;
(3) modifying a YOLOv3.cfg file in a YOLOv3 model, replacing the anchor parameter in the step (2), and debugging and modifying values of batch, filter and classes parameters;
(4) debugging parameters such as learning rate and batch, and training to obtain the optimal model weight;
(5) and loading the model weight by using a keras, constructing a fire model deep learning network, detecting each frame of monitoring image transmitted from the rear end, judging whether the image contains flame or not, and returning a detection result signal to the rear end.
The invention has the beneficial effects that: according to the scheme, the anchor parameter is determined on the self-made flame data set in a k-means clustering mode to obtain a higher IOU score, the convergence rate of the model is increased, and the detection accuracy of the model is improved.
Drawings
FIG. 1 is a schematic representation of the steps of the present invention;
fig. 2 is a comparison of the performance of YOLOv3 on the VOC2012 data set with other models;
FIG. 3 is a network structure of the YOLOv3 model;
the IOU values are obtained on different anchors boxes in FIG. 4.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further explained below.
Examples
As shown in fig. 1, the illegal welding fire image recognition method based on YOLOv3 is characterized by comprising the following steps:
(1) arranging pictures containing flames collected from a construction site into a flame data set, and labeling flame areas in the flame data set;
(2) obtaining an anchor parameter suitable for flame detection on a flame data set in a k-means clustering mode;
(3) modifying a YOLOv3.cfg file in a YOLOv3 model, replacing the anchor parameter in the step (2), and debugging and modifying values of batch, filter and classes parameters;
(4) debugging parameters such as learning rate and batch, and training to obtain the optimal model weight;
(5) and loading the model weight by using a keras, constructing a fire model deep learning network, detecting each frame of monitoring image transmitted from the rear end, judging whether the image contains flame or not, and returning a detection result signal to the rear end.
According to the scheme, the anchor parameter is determined on the self-made flame data set in a k-means clustering mode to obtain a higher IOU score, the convergence rate of the model is increased, and the detection accuracy of the model is improved.
According to the scheme, a flame detection model based on a YOLOv3 network is mainly adopted, candidate frames are selected through clustering, and then the detection precision of the model is improved by applying a multi-scale feature fusion method. The model not only inherits the detection accuracy advantage of the double-step target detection model, but also has the detection speed advantage of the single-step target detection model, and can meet the speed and accuracy required by video detection.
As shown in fig. 2, the YOLOv3 is shown in comparison with other models on the VOC2012 data set, and it can be seen that the model has the best detection accuracy and detection speed.
As shown in fig. 3, is a network structure of YOLOv3 model. YOLOv3 employs 53-layer Darknet-53 as a feature extractor. Darknet-53 is mainly composed of 3 × 3 and 1 × 1 filters, with residual network connections. Darknet-53 has fewer billions of floating point operations than ResNet-152, but achieves the same classification accuracy, 2 times faster. Inside the entire v3 structure, there is no pooling layer and full connectivity layer. In the forward propagation process, the size of the tensor is transformed by changing the step size of the convolution kernel, for example, when the step size is 2(stride is 2,2), this is equivalent to reducing the side length of the image by half (i.e., reducing the area to 1/4). v3 also like v2, the backbone will narrow the output diagnostic images to 1/32 as input. Therefore, it is generally required that the input picture is a multiple of 32. YOLOv3 outputs features of 3 different scales, such as y1, y2, y3 shown in fig. 1. With reference to fpn (feature pyramid) networks, multiple scales are used to detect objects of different sizes, and the finer the bounding box is, the finer the object can be detected. In FIG. 3, the depths of y1, y2 and y3 are 255, and the side length is regular to be 13:26: 52.
The initial stage of fire occurrence is a key period of flame detection, the flame form at the moment is mostly small flame, which requires that the model has higher detection capability for small targets, and more shallow features with high resolution are utilized to construct a high-level semantic feature map. While YOLOv3 adopts multi-scale feature fusion, as can be seen from fig. 3, the YOLOv3 model fuses a high-resolution shallow feature, a relatively abstract middle-level feature and a fully abstract high-level feature in a network as 3 features to be output. Therefore, the characteristics of different flame forms in the initial stage, the middle stage and the later stage of the flame can be captured more conveniently, and the loss of information is reduced.
The flame, as a non-rigid body, changes its morphology during combustion, having a different aspect ratio. And the anchor dimension of YOLOv3 was determined based on the VOC2007 and VOC2012 datasets and was not universal. It is therefore particularly important to select a set of candidate boxes that fit into the flame data set. Aiming at the problem, the scheme determines the anchor parameter on the self-made flame data set by using a k-means clustering mode so as to obtain a higher IOU score and accelerate the convergence speed of the model.
As shown in fig. 4, different numbers of clusters are selected to obtain corresponding IOU values. It can be seen that the IOU value increases continuously and becomes flat as the number of clusters increases. In order to balance the model processing speed and the model processing precision, the Anchor value [19,19] [21,28] [31,24] [35,35] [40,54] [51,42] [58,60] [78,76] [124,125] with the cluster number of 9 is selected as the detection parameter of the fire model.
The actual realization effect is as follows: in a certain construction site, the algorithm is deployed by relying on an edge algorithm server, and in practical application, the behavior of illegal fire behavior of workers can be well detected.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (1)
1. The illegal welding fire image recognition method based on YOLOv3 is characterized by comprising the following steps of:
(1) arranging pictures containing flames collected from a construction site into a flame data set, and labeling flame areas in the flame data set;
(2) obtaining an anchor parameter suitable for flame detection on a flame data set in a k-means clustering mode;
(3) modifying a YOLOv3.cfg file in a YOLOv3 model, replacing the anchor parameter in the step (2), and debugging and modifying values of batch, filter and classes parameters;
(4) debugging parameters such as learning rate and batch, and training to obtain the optimal model weight;
(5) and loading the model weight by using a keras, constructing a fire model deep learning network, detecting each frame of monitoring image transmitted from the rear end, judging whether the image contains flame or not, and returning a detection result signal to the rear end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011022862.6A CN112241693A (en) | 2020-09-25 | 2020-09-25 | Illegal welding fire image identification method based on YOLOv3 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011022862.6A CN112241693A (en) | 2020-09-25 | 2020-09-25 | Illegal welding fire image identification method based on YOLOv3 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112241693A true CN112241693A (en) | 2021-01-19 |
Family
ID=74171708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011022862.6A Pending CN112241693A (en) | 2020-09-25 | 2020-09-25 | Illegal welding fire image identification method based on YOLOv3 |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112241693A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861737A (en) * | 2021-02-11 | 2021-05-28 | 西北工业大学 | Forest fire smoke detection method based on image dark channel and YoLov3 |
| CN113688921A (en) * | 2021-08-31 | 2021-11-23 | 重庆科技学院 | Fire operation identification method based on graph convolution network and target detection |
| CN113723300A (en) * | 2021-08-31 | 2021-11-30 | 平安国际智慧城市科技股份有限公司 | Artificial intelligence-based fire monitoring method and device and storage medium |
| CN114241420A (en) * | 2021-12-20 | 2022-03-25 | 国能(泉州)热电有限公司 | Fire operation detection method and device |
| CN117611928A (en) * | 2024-01-23 | 2024-02-27 | 青岛国实科技集团有限公司 | Illegal electric welding identification method, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165670A (en) * | 2018-07-12 | 2019-01-08 | 江南大学 | A kind of TS-RBF fuzzy neural network robust fusion algorithm applied to infra red flame identification |
| CN109829429A (en) * | 2019-01-31 | 2019-05-31 | 福州大学 | Security protection sensitive articles detection method under monitoring scene based on YOLOv3 |
| CN110102002A (en) * | 2019-05-15 | 2019-08-09 | 上海荷福人工智能科技（集团）有限公司 | The early warning of fire-fighting robot based on artificial intelligence and fire extinguishing system and method |
| CN110298292A (en) * | 2019-06-25 | 2019-10-01 | 东北大学 | Detection method is grabbed when the high-precision real of rule-based object polygon Corner Detection |
| CN110636715A (en) * | 2019-08-27 | 2019-12-31 | 杭州电子科技大学 | Self-learning-based automatic welding and defect detection method |
| CN111091072A (en) * | 2019-11-29 | 2020-05-01 | 河海大学 | YOLOv 3-based flame and dense smoke detection method |
| CN111598860A (en) * | 2020-05-13 | 2020-08-28 | 河北工业大学 | Lithium battery defect detection method based on yolov3 network embedded in self-attention door module |
-
2020
- 2020-09-25 CN CN202011022862.6A patent/CN112241693A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165670A (en) * | 2018-07-12 | 2019-01-08 | 江南大学 | A kind of TS-RBF fuzzy neural network robust fusion algorithm applied to infra red flame identification |
| CN109829429A (en) * | 2019-01-31 | 2019-05-31 | 福州大学 | Security protection sensitive articles detection method under monitoring scene based on YOLOv3 |
| CN110102002A (en) * | 2019-05-15 | 2019-08-09 | 上海荷福人工智能科技（集团）有限公司 | The early warning of fire-fighting robot based on artificial intelligence and fire extinguishing system and method |
| CN110298292A (en) * | 2019-06-25 | 2019-10-01 | 东北大学 | Detection method is grabbed when the high-precision real of rule-based object polygon Corner Detection |
| CN110636715A (en) * | 2019-08-27 | 2019-12-31 | 杭州电子科技大学 | Self-learning-based automatic welding and defect detection method |
| CN111091072A (en) * | 2019-11-29 | 2020-05-01 | 河海大学 | YOLOv 3-based flame and dense smoke detection method |
| CN111598860A (en) * | 2020-05-13 | 2020-08-28 | 河北工业大学 | Lithium battery defect detection method based on yolov3 network embedded in self-attention door module |
Non-Patent Citations (1)
| Title |
|---|
| 赵飞扬等: "基于改进YOLOv3的火焰检测", 中国科技论文, vol. 15, no. 7, pages 820 - 826 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861737A (en) * | 2021-02-11 | 2021-05-28 | 西北工业大学 | Forest fire smoke detection method based on image dark channel and YoLov3 |
| CN113688921A (en) * | 2021-08-31 | 2021-11-23 | 重庆科技学院 | Fire operation identification method based on graph convolution network and target detection |
| CN113723300A (en) * | 2021-08-31 | 2021-11-30 | 平安国际智慧城市科技股份有限公司 | Artificial intelligence-based fire monitoring method and device and storage medium |
| CN114241420A (en) * | 2021-12-20 | 2022-03-25 | 国能(泉州)热电有限公司 | Fire operation detection method and device |
| CN117611928A (en) * | 2024-01-23 | 2024-02-27 | 青岛国实科技集团有限公司 | Illegal electric welding identification method, electronic equipment and storage medium |
| CN117611928B (en) * | 2024-01-23 | 2024-04-09 | 青岛国实科技集团有限公司 | Illegal electric welding identification method, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112241693A (en) | Illegal welding fire image identification method based on YOLOv3 | |
| CN111553387B (en) | Personnel target detection method based on Yolov3 | |
| US20200285896A1 (en) | Method for person re-identification based on deep model with multi-loss fusion training strategy | |
| CN108304798B (en) | Street level order event video detection method based on deep learning and motion consistency | |
| CN102332092B (en) | Flame detection method based on video analysis | |
| CN111275688A (en) | Small target detection method based on context feature fusion screening of attention mechanism | |
| CN106325485B (en) | A kind of gestures detection recognition methods and system | |
| CN110135296A (en) | Airfield runway FOD detection method based on convolutional neural networks | |
| CN108229458A (en) | A kind of intelligent flame recognition methods based on motion detection and multi-feature extraction | |
| CN111611861B (en) | Image change detection method based on multi-scale feature association | |
| CN110569782A (en) | Target detection method based on deep learning | |
| CN110298297A (en) | Flame identification method and device | |
| CN112270331A (en) | Improved billboard detection method based on YOLOV5 | |
| CN108648211A (en) | A kind of small target detecting method, device, equipment and medium based on deep learning | |
| KR101449744B1 (en) | Face detection device and method using region-based feature | |
| CN107871315B (en) | Video image motion detection method and device | |
| CN117949942B (en) | Target tracking method and system based on fusion of radar data and video data | |
| CN113361475A (en) | Multi-spectral pedestrian detection method based on multi-stage feature fusion information multiplexing | |
| CN117372898A (en) | Unmanned aerial vehicle aerial image target detection method based on improved yolov8 | |
| CN111259736B (en) | Real-time pedestrian detection method based on deep learning in complex environment | |
| Mijić et al. | Traffic sign detection using YOLOv3 | |
| CN112233105A (en) | Road crack detection method based on improved FCN | |
| CN117011563A (en) | Road damage inspection cross-domain detection method and system based on semi-supervised federal learning | |
| CN113658129B (en) | Position extraction method combining visual saliency and line segment strength | |
| TWI696958B (en) | Image adaptive feature extraction method and its application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210119 |
|
| RJ01 | Rejection of invention patent application after publication |