CN111429546A - Large-size image compressed sensing reconstruction method based on neural network - Google Patents
Large-size image compressed sensing reconstruction method based on neural network Download PDFInfo
- Publication number
- CN111429546A CN111429546A CN202010298406.8A CN202010298406A CN111429546A CN 111429546 A CN111429546 A CN 111429546A CN 202010298406 A CN202010298406 A CN 202010298406A CN 111429546 A CN111429546 A CN 111429546A
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- data
- network
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 27
- 238000005070 sampling Methods 0.000 claims abstract description 17
- 230000006835 compression Effects 0.000 claims abstract description 16
- 238000007906 compression Methods 0.000 claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims abstract description 6
- 238000005259 measurement Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 2
- 230000011218 segmentation Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 3
- 238000012546 transfer Methods 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 abstract 1
- 230000001629 suppression Effects 0.000 abstract 1
- 230000006870 function Effects 0.000 description 8
- 230000000903 blocking effect Effects 0.000 description 3
- 238000004880 explosion Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 229960001948 caffeine Drugs 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- RYYVLZVUVIJVGH-UHFFFAOYSA-N trimethylxanthine Natural products CN1C(=O)N(C)C(=O)C2=C1N=CN2C RYYVLZVUVIJVGH-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention discloses a neural network-based large-size image compressed sensing reconstruction method. The method comprises the following steps: (1) carrying out block-dividing compression sampling on an original image through a measurement matrix to manufacture a training data set; (2) restoring the data from low dimensionality to high dimensionality by the compressed and sampled data through a full connection layer of a neural network; (3) splicing each block into an image according to the sequence of each block in the original image; (4) transferring the re-spliced image to an intensive connection convolution network consisting of convolution layers, and further reconstructing and recovering the image; (5) calculating the error between the reconstructed image and the original image by a loss function, and updating the parameters of the network through error reverse transfer; (6) and (5) repeating the steps (2) to (5) until the error between the reconstructed image and the original image is smaller, and finishing the training of the neural network. The method can complete the compression and reconstruction of larger images, and the accuracy of the reconstructed images and the suppression of image noise can both obtain better effects.
Description
Technical Field
The invention relates to the fields of computational photography and signal processing, in particular to a large-size image compressed sensing reconstruction method based on a neural network.
Background
Compressive sensing is a technique for finding sparse solutions to underdetermined linear systems, applied in electronic engineering, especially in signal processing, for acquiring and reconstructing sparse or compressible signals. In recent years, in order to adapt to the fifth generation mobile communication system which is about to be popularized, a compression sensing technology is also applied to a wireless communication system in a large amount, and a great deal of attention and research are paid.
The compressed sensing problem is a challenging problem in the field of signal processing, and since a neural network can sufficiently learn prior information in an image, in recent years, reconstructing a compressed sensing signal through the neural network becomes a popular method. When the compressed sensing signal is reconstructed through the neural network comprising the full connection layer, because the compressed signal with larger dimension number needs a large amount of parameters when being input into the network, the phenomenon of gradient explosion or gradient disappearance easily occurs to the large amount of parameters, and the reconstruction cannot be completed. Therefore, when the compressed sensing signal is reconstructed by the traditional neural network, the dimensionality of the signal input into the network is small, the number of parameters is not excessive, and the reconstruction of the signal is smoothly completed. In the method, because the signal with smaller dimension can be processed, when the method is applied to the image field, the size of the processed image is smaller, and the reconstructed smaller image can be obviously blocked after being spliced into the original large image, so that the reconstruction quality is reduced. The method of firstly blocking, reconstructing and then splicing based on neural network reconstruction compressed sensing has poor reconstruction effect and has great limitation in practical application.
Aiming at the problems, a novel reconstruction means is needed, so that not only is the influence of excessive network parameters on neural network training avoided, but also the blocking phenomenon generated by splicing small-size images into large-size original images after reconstruction is eliminated, and a better reconstruction effect is finally obtained.
Disclosure of Invention
Aiming at the defects existing in the conventional neural network reconstruction method, the invention aims to provide a large-size image compressed sensing reconstruction method based on a neural network.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a large-size image compressed sensing reconstruction method based on a neural network comprises the following steps:
step 1, dividing an original image x into a series of square images x with the same size1,x2,…,xiRespectively carrying out compression sampling processing on the segmented images by using a standard normally distributed measurement matrix to obtain data y after compression sampling1,y2,…,yiAs training data, training data y is created1,y2,…,yiTraining sets corresponding to the original images x one by one;
step 2, inputting the training data into a neural network, and training data y1,y2,…,yiRespectively through the first layer of the neural network: linear full connected layer, such that training data y1,y2,…,yiRecovery from Low dimensionality to Pre-sampling high dimensionality data x'1,x′2,…,x′i；
Step 3, the high-dimensional data x 'obtained in the step 2'1,x′2,…,x′iBlock image x according to its correspondence1,x2,…,xiSplicing the positions in the original image x into an original image in sequence to obtain a primarily reconstructed image x';
step 4, inputting the image x' preliminarily reconstructed in the step 3 into a dense connection convolution network consisting of convolution layers, and further reconstructing and restoring the image to obtain the output of the network, namely a final reconstructed image;
step 5, calculating the error between the reconstructed image obtained in the step 4 and the original real image through a loss function, reversely transmitting the error, and updating the parameters of the neural network;
and 6, repeating the steps 2-5 until the error of the loss function calculation reaches an expected value, and finishing the training of the neural network.
Further, theIn step 1, the processing mode of compressed sampling after original image segmentation is as follows: y isi＝ΦxiWherein xi∈ (N × 1) is data obtained by rearranging elements after dividing an original image x with the size of N × N into a small-size histogram, i is 1,2,3 and …, and a measurement matrix is phi ∈ (m × N, m < N), the elements of the measurement matrix are random numbers in standard normal distribution, N is the dimension of the original image data, and m is the dimension of the compressed and sampled image data.
Further, in the step 2, the basic network form of the linear full connection is as follows: the node number of the input layer is the value of the dimension m of the image data after compression sampling, the node number of the output layer is the value of the dimension n of the original image data, and no activation function exists.
Further, in the step 3, the concrete process of splicing the block images into the original image according to the position sequence of the block images in the original image is as follows: step 31, firstly, the data x 'obtained in the step 2'1,x′2,…,x′iElement arrangement transformation of
Step 32, according to (x)1,x′1),(x2,x′2),…,(xi,x′i) And (3) rearranging the results obtained in the step (2) by the one-to-one correspondence of the positions.
Further, in the step 4, the dense connection convolution network is composed of 4 same dense connection modules, each module outputs 64 channels as the input of the next dense connection module, the rear end of the dense connection convolution network is connected with a convolution layer with a convolution kernel size of 3 × 3, the input of the convolution layer is 64 channels, the output of the convolution layer is 1 channel, and the reconstructed image is obtained.
On the basis of solving the compressed sensing problem based on the neural network, the invention provides a novel solution in combination with the problem that training is difficult to complete due to the fact that too many parameters of the full connection layer in the neural network cause gradient explosion or gradient dispersion, so that the use of the parameters of the full connection layer is reduced, and the blocking phenomenon caused by direct reconstruction of images with smaller sizes is eliminated. Meanwhile, the invention adopts the intensive connection convolution network for training, improves the information transmission between networks, can effectively inhibit the noise of the reconstructed image and obtains a clearer image. Compared with the prior art, the method can obtain better image accuracy and lower image noise interference in the compression reconstruction of larger images, and can effectively deal with the challenge brought by the compression reconstruction of larger-size images in practical application.
Drawings
FIG. 1 is a schematic diagram of a neural network according to the present invention;
FIG. 2 is a flow chart of the method of the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and specific embodiments, which are illustrative and are not to be construed as limiting the invention.
Referring to fig. 1 and fig. 2, the method for reconstructing a compressed sensing of a large-size image based on a neural network according to the present embodiment includes the following specific steps:
step 1, dividing an original large-size image x ∈ (N × N) to be compressed and sampled into a series of square graphs with the same size and relatively small size
The elements are rearranged to N × 1, and the segmented small-size images are respectively compressed and sampled by using a measuring matrix phi ∈ (m × N, m < N) of standard normal distribution, wherein N is the dimensionality of signal data before compression sampling, and m is the dimensionality of signal data after compression sampling, in the embodiment, the original image size N is 128, the segmented image size N is 16, and the dimensionality m of the signal data after compression sampling is 4, namely:
yi＝Φxi,i＝1,2,…(Φ∈4×16；xi∈16×1)
obtaining the data y after compression sampling1,y2… training data y is created as training data1,y2… are training sets corresponding to the original images x one to one.
Step 2, inputting training data into the neural network, and compressing the sampled data y by blocks of the original image1,y2… through the first layer of the neural network: the structure of the linear full-connection layer is that the node number of the input layer is the dimension m of the signal data after compression sampling, the node number of the output layer is the dimension n of the original signal data, and no activation function exists. The original image is divided into blocks to compress the sampled data y1,y2… (4 × 1) reverts from low dimension to high dimension x 'before sampling'1,x′2… (16 × 1). the compressed reconstruction is carried out after the large-size image is divided into small blocks, namely, the use of parameters of a full connecting layer is reduced, gradient explosion or diffusion in the training process is avoided, and the reconstruction of the compressed image is also realized.
Step 3, the high-dimensional data x 'obtained in the step 2'1,x′2… Block image x according to its correspondence1,x2…, the original image x is sequentially stitched into the original image x to obtain the preliminary reconstructed image x', which includes the following steps:
step 31 x 'obtained in step 2'1,x′2… (16 × 1) element permutation is transformed to 4 × 4;
step 32 is according to (x)1,x′1),(x2,x′2) …, rearranging the results obtained in the step 2 by the one-to-one correspondence relationship of the positions, and completing the preliminary reconstruction of the large-size image compressed sensing.
Step 4, inputting the image x' preliminarily reconstructed in the step 3 into a dense connection convolution network consisting of convolution layers, and further reconstructing and recovering the image to obtain the output of the network, namely the final reconstructed image xoutputIn the present example, the dense connected convolution network is specifically structured as follows, the input of the first layer convolution of the dense connected convolution network composed of convolution layers is the image x' which is preliminarily reconstructed, the number of convolution kernels is 16, 16 feature maps are output, and batch normalization is performed, and then four groups of dense connected convolution modules are formed, wherein the dense connected convolution modules are structured as (Conv1+ BN + Re L U) - (Conv2+ BN + Re L U) - (Conv3+ BN + Re L U) - (Conv4+ BN + Re L U), and the dense connected convolution modules are all structured as followsThe number of convolution kernels connecting each layer of convolution is 16, 16 feature maps are output, and the output feature maps are input to each convolution layer behind the feature maps; after the intensive connection convolution module, a layer of convolution network is passed through, the number of convolution kernels is 1, 1 characteristic diagram is output, namely the reconstructed recovery image xoutputIn the example, the sizes of convolution kernels adopted by the convolution layers are all 3 × 3, and each layer of convolution is subjected to padding operation, so that the sizes of the images before and after convolution are not changed, and the sizes of the feature maps in the intensive connection module when the output feature maps are directly connected are the same.
And 5, calculating the error between the reconstructed image obtained in the step 4 and the original real image through a loss function, reversely transferring the error, updating a weight parameter w and a bias parameter bias of the neural network, wherein the adopted loss function is an MSE loss function, and the calculation mode is as follows:
MSE＝(xoutput-x)2
wherein xoutputFor reconstructing the image, x is the original real image, and the optimizer for error reverse transfer updating the network parameters is Adam.
And 6, after the parameters of the neural network are updated, repeating the steps 2-5 until the error of the loss function calculation reaches an expected value, and finishing the training of the neural network. In this example, the specific network hyper-parameters used are as follows: number of iterations (epoch): 100, respectively; learning rate (learning rate): 0.0002; number of samples of one training (batch size): 32, a first step of removing the first layer; the image training set used was: INRIA Holidays dataset.
Claims (5)
1. A large-size image compressed sensing reconstruction method based on a neural network is characterized by comprising the following steps:
step 1, dividing an original image x into a series of square images x with the same size1，x2，...，xiRespectively carrying out compression sampling processing on the segmented images by using a standard normally distributed measurement matrix to obtain data y after compression sampling1，y2，...，yiAs training data, training data y is created1，y2，...，yiTraining sets corresponding to the original images x one by one;
step 2, inputting the training data into a neural network, and training data y1，y2，...，yiRespectively through the first layer of the neural network: linear full connected layer, such that training data y1，y2，...，yiRecovery from Low dimensionality to Pre-sampling high dimensionality data x'1，x′2，...，x′i；
Step 3, the high-dimensional data x 'obtained in the step 2'1，x′2，...，x′iBlock image x according to its correspondence1，x2，...，xiSplicing the positions in the original image x into an original image in sequence to obtain a primarily reconstructed image x';
step 4, inputting the image x' preliminarily reconstructed in the step 3 into a dense connection convolution network consisting of convolution layers, and further reconstructing and restoring the image to obtain the output of the network, namely a final reconstructed image;
step 5, calculating the error between the reconstructed image obtained in the step 4 and the original real image through a loss function, reversely transmitting the error, and updating the parameters of the neural network;
and 6, repeating the steps 2-5 until the error of the loss function calculation reaches an expected value, and finishing the training of the neural network.
2. The method for compressed sensing reconstruction of large-size image based on neural network as claimed in claim 1, wherein in step 1, the processing mode of compressed sampling after segmentation of original image is:
yi＝Φxi
wherein xi∈ (N ×) is data of rearranged elements after an original image x with the size of N × N is cut into a small-size histogram, i is 1,2, 3.
3. The method for compressed sensing reconstruction of large size image based on neural network as claimed in claim 2, wherein in said step 2, the basic network form of linear full connection is: the node number of the input layer is the value of the dimension m of the image data after compression sampling, the node number of the output layer is the value of the dimension n of the original image data, and no activation function exists.
4. The method according to claim 3, wherein the step 3 of splicing the segmented images into the original image according to the position sequence of the segmented images in the original image comprises the following specific steps:
step 31, firstly, the data x 'obtained in the step 2'1，x′2，...，x′iElement arrangement transformation of
Step 32, according to (x)1，x′1)，(x2，x′2)，...，(xi，x′i) And (3) rearranging the results obtained in the step (2) by the one-to-one correspondence of the positions.
5. The method for compressed sensing reconstruction of a large-size image based on a neural network as claimed in claim 4, wherein in the step 4, the densely connected convolutional network is composed of 4 identical densely connected modules, each module outputs 64 channels as input of the next densely connected module, and the back end of the densely connected convolutional network is connected with a convolutional layer with a convolutional kernel size of 3 × 3, wherein the convolutional layer has 64 input channels and 1 output channel, and is the reconstructed image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010298406.8A CN111429546A (en) | 2020-04-16 | 2020-04-16 | Large-size image compressed sensing reconstruction method based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010298406.8A CN111429546A (en) | 2020-04-16 | 2020-04-16 | Large-size image compressed sensing reconstruction method based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111429546A true CN111429546A (en) | 2020-07-17 |
Family
ID=71556463
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010298406.8A Pending CN111429546A (en) | 2020-04-16 | 2020-04-16 | Large-size image compressed sensing reconstruction method based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111429546A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140185928A1 (en) * | 2012-12-28 | 2014-07-03 | Shai Ben NUN | Hardware-supported huffman coding of images |
| CN109102461A (en) * | 2018-06-15 | 2018-12-28 | 深圳大学 | Image reconstructing method, device, equipment and the medium of low sampling splits' positions perception |
| CN110033030A (en) * | 2019-03-27 | 2019-07-19 | 南京大学 | The method of compressed sensing problem based on Neural Networks Solution 0-1 calculation matrix |
-
2020
- 2020-04-16 CN CN202010298406.8A patent/CN111429546A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140185928A1 (en) * | 2012-12-28 | 2014-07-03 | Shai Ben NUN | Hardware-supported huffman coding of images |
| CN109102461A (en) * | 2018-06-15 | 2018-12-28 | 深圳大学 | Image reconstructing method, device, equipment and the medium of low sampling splits' positions perception |
| CN110033030A (en) * | 2019-03-27 | 2019-07-19 | 南京大学 | The method of compressed sensing problem based on Neural Networks Solution 0-1 calculation matrix |
Non-Patent Citations (1)
| Title |
|---|
| WUZHEN SHI等: "Deep networks for compressed image sensing", 《PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME) 2017》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111462013B (en) | Single-image rain removing method based on structured residual learning | |
| CN109118432B (en) | Image super-resolution reconstruction method based on rapid cyclic convolution network | |
| CN112435191B (en) | Low-illumination image enhancement method based on fusion of multiple neural network structures | |
| CN111028153B (en) | Image processing and neural network training method and device and computer equipment | |
| CN111862251B (en) | Method, device, storage medium and electronic equipment for medical image reconstruction technology | |
| CN108492249B (en) | Single-frame super-resolution reconstruction method based on small convolution recurrent neural network | |
| CN115953303B (en) | Multi-scale image compressed sensing reconstruction method and system combining channel attention | |
| CN112699844B (en) | Image super-resolution method based on multi-scale residual hierarchy close-coupled network | |
| CN111667444B (en) | Image compressed sensing reconstruction method based on multi-channel residual error network | |
| CN113902658B (en) | RGB image-to-hyperspectral image reconstruction method based on dense multiscale network | |
| CN115829834A (en) | Image super-resolution reconstruction method based on half-coupling depth convolution dictionary learning | |
| CN112036454A (en) | Image classification method based on multi-core dense connection network | |
| CN110033030B (en) | Method for solving compressed sensing problem of 0-1 measurement matrix based on neural network | |
| CN113222812A (en) | Image reconstruction method based on information flow reinforced deep expansion network | |
| CN111429546A (en) | Large-size image compressed sensing reconstruction method based on neural network | |
| CN115984949B (en) | Low-quality face image recognition method and equipment with attention mechanism | |
| CN116883265A (en) | Image deblurring method based on enhanced feature fusion mechanism | |
| CN115272131B (en) | Image mole pattern removing system and method based on self-adaptive multispectral coding | |
| CN111260562A (en) | Digital holographic image reconstruction method based on deep learning | |
| Zhou et al. | Multi-scale and attention residual network for single image dehazing | |
| CN113793267B (en) | Self-supervision single remote sensing image super-resolution method based on cross-dimension attention mechanism | |
| CN114529482B (en) | Image compressed sensing reconstruction method based on wavelet multichannel depth network | |
| CN115330930A (en) | Three-dimensional reconstruction method and system based on sparse to dense feature matching network | |
| Zhang et al. | CRA: A generic compression ratio adapter for end-to-end data-driven image compressive sensing reconstruction frameworks | |
| CN114862679A (en) | Single-image super-resolution reconstruction method based on residual error generation countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |