CN115293999A - Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution - Google Patents

Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution Download PDF

Info

Publication number
CN115293999A
CN115293999A CN202210852113.9A CN202210852113A CN115293999A CN 115293999 A CN115293999 A CN 115293999A CN 202210852113 A CN202210852113 A CN 202210852113A CN 115293999 A CN115293999 A CN 115293999A
Authority
CN
China
Prior art keywords
remote sensing
sensing image
matrix
cloud
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202210852113.9A
Other languages
Chinese (zh)
Inventor
李军杰
鞠尊洲
王丽媛
汪新潮
侯晨
王鸿泰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Geo Vision Tech Co ltd
Henan University of Urban Construction
Original Assignee
Beijing Geo Vision Tech Co ltd
Henan University of Urban Construction
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Geo Vision Tech Co ltd, Henan University of Urban Construction filed Critical Beijing Geo Vision Tech Co ltd
Priority to CN202210852113.9A priority Critical patent/CN115293999A/en
Publication of CN115293999A publication Critical patent/CN115293999A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038Image mosaicing, e.g. composing plane images from plane sub-images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/10Image enhancement or restoration using non-spatial domain filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration using local operators
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details
    • G06T2207/20032Median filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20056Discrete and fast Fourier transform, [DFT, FFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of remote sensing image cloud removing, and discloses a remote sensing image cloud removing method fusing multi-temporal information and sub-channel dense convolution, wherein the remote sensing image cloud removing system fusing multi-temporal information and sub-channel dense convolution comprises the following steps: the remote sensing image acquisition module, the remote sensing image enhancement module, the extraction module, the cloud removal module and the cloud removal image encryption module. According to the method, the remote sensing image enhancement module utilizes the remote sensing image enhancement neural network model to remove illumination change in the remote sensing image to be processed according to the input illumination layer, so that the enhancement effect of the remote sensing image is improved; meanwhile, the cloud image removing encryption module can effectively encrypt the plaintext remote sensing image, and the intermediate matrix is compressed and sensed through the half tensor, so that the compression processing of the intermediate matrix is realized; compared with the traditional remote sensing image encryption method, the method effectively reduces the storage and transmission amount of the remote sensing image data in the transmission process.

Description

Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution
Technical Field
The invention belongs to the technical field of cloud removal of remote sensing images, and particularly relates to a cloud removal method for remote sensing images, which integrates multi-temporal information and sub-channel dense convolution.
Background
Remote Sensing images (RS, remote Sensing Image) refer to films or photos recording electromagnetic waves of various ground features, and are mainly classified into aerial photos and satellite photos. The remotely sensed image processed by the computer must be a digital image. Analog images acquired in a photographic manner must be analog/digital (a/D) converted with an image scanner or the like; the digital data obtained by scanning must be transferred to general carriers such as CCT which can be read by general digital computer. Computer image processing is to be performed in an image processing system. An image processing system is composed of hardware (computer, display, digitizer, tape drive, etc.) and software (having data input, output, correction, transformation, classification, etc.) functions. The image processing contents mainly include correction, transformation, and classification. However, the existing remote sensing image cloud removing method which integrates multi-temporal information and sub-channel dense convolution has poor remote sensing image enhancement effect; meanwhile, the remote sensing image encryption mode processing process is complex, and a large amount of measurement data needs to be stored and transmitted.
Through the above analysis, the problems and defects of the prior art are as follows:
(1) The traditional remote sensing image cloud removing method which integrates multi-temporal information and sub-channel dense convolution has poor remote sensing image enhancement effect.
(2) The remote sensing image encryption mode has complex processing process and needs to store and transmit a large amount of measurement data.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution.
The invention is realized in this way, a remote sensing image cloud removing system which integrates multi-temporal information and sub-channel dense convolution comprises:
the remote sensing image acquisition module, the remote sensing image enhancement module, the extraction module, the cloud removal module and the cloud removal image encryption module;
the remote sensing image acquisition module is connected with the remote sensing image enhancement module and is used for acquiring a remote sensing image through remote sensing equipment;
the remote sensing image enhancement module is connected with the remote sensing image acquisition module and the extraction module and is used for enhancing the remote sensing image through an image enhancement program;
the extraction module is connected with the remote sensing image enhancement module and the cloud removal module and is used for extracting areas which are shaded by clouds in the remote sensing image;
the cloud removing module is connected with the extracting module and the cloud removing image encrypting module and is used for carrying out intersection operation on each shielded area of the remote sensing image to obtain a specific area and carrying out difference set operation and intersection operation on the specific area and the shielded area of the remote sensing image to be cloud removed so as to remove cloud;
the cloud removing image encryption module is connected with the cloud removing module 4 and used for encrypting the cloud removing remote sensing image.
The invention also aims to provide a remote sensing image cloud removing method fusing multi-temporal information and channel-division dense convolution, which comprises the following steps:
acquiring a remote sensing image by using remote sensing equipment through a remote sensing image acquisition module;
secondly, enhancing the remote sensing image by utilizing an image enhancement program through a remote sensing image enhancement module;
extracting the region shielded by the cloud in the remote sensing image through an extraction module;
performing intersection operation on all the shielded areas of the remote sensing image through a cloud removing module to obtain a specific area, and performing difference operation and intersection operation on the specific area and the shielded areas of the remote sensing image to be cloud removed to remove cloud;
and step five, encrypting the cloud-removed remote sensing image through a cloud-removed image encryption module.
Further, the remote sensing image enhancement module enhancement method comprises the following steps:
(1) Constructing an image database; storing the acquired remote sensing image into an image database; correcting the remote sensing image, constructing a remote sensing image enhanced neural network model, inputting the remote sensing image enhanced neural network model into a remote sensing image and an illumination layer corresponding to the remote sensing image, and outputting the remote sensing image after enhancement;
(2) Acquiring an illumination layer corresponding to a remote sensing image to be processed;
(3) Inputting the remote sensing image to be processed and an illumination layer corresponding to the remote sensing image to be processed into the remote sensing image enhanced neural network model, and outputting an enhanced remote sensing image;
the method for correcting the remote sensing image comprises the following steps:
extracting a plurality of collected remote sensing images; splicing a plurality of acquired remote sensing images in a GPS coordinate system to obtain a spliced acquired remote sensing image; converting the space vector of the spliced collected remote sensing image under a GPS coordinate system into a space vector under a WGS-84 coordinate system;
finding N points corresponding to the spliced acquired remote sensing images on the ground as standard points, wherein N is an integer greater than or equal to 2;
determining a check quaternion according to a first space vector group of the N standard points under a WGS-84 coordinate system corresponding to the GPS coordinates of the N standard points on the ground and a second space vector group of the N standard points under the WGS-84 coordinate system corresponding to the coordinates of the spliced acquired remote sensing images;
correcting all coordinate points in the spliced acquired remote sensing images according to the check quaternion;
each coordinate point (x ', y') in the acquired remote sensing image corresponds to geographical coordinate information (Long, lat), wherein Long represents longitude, and Lat represents latitude;
the vector conversion formula for converting the space vector of the spliced acquired remote sensing image under the GPS coordinate system into the space vector under the WGS-84 coordinate system is as follows:
Figure BDA0003755035820000031
v represents a space vector in the WGS-84 coordinate system;
the check quaternion is represented as:
Figure BDA0003755035820000041
ω represents a feature vector of a matrix obtained by orthogonal projection of the first space vector group and the second space vector group, and converts the check quaternion into a form of a transformation matrix, where the transformation matrix is represented as:
Figure BDA0003755035820000042
u = E · v, u representing the corrected space vector of the space vector v in the stitched acquired remote sensing image;
the formula for converting the corrected space vector u into longitude and latitude is as follows:
Lat adj =arcsinu(3),
Figure BDA0003755035820000043
where u is a 3-row 1-column vector, u (1) represents the elements of the first row of the spatial vector u, u (2) represents the elements of the second row of the spatial vector u, u (3) represents the elements of the third row of the spatial vector u, lat adj Indicating the latitude, long after conversion adj Indicating the converted longitude.
Further, the illumination layer corresponding to the remote sensing image is obtained by adopting the following method:
converting the remote sensing image from a first color space to a second color space containing first brightness information, and extracting the first brightness information of the remote sensing image;
filtering the first brightness information by adopting a filtering algorithm to obtain a gray level remote sensing image;
calculating the average value of second brightness information of all pixels of the gray remote sensing image;
and correcting the second brightness information of each pixel of the gray remote sensing image by adopting the average value to obtain an illumination layer, wherein the corrected brightness information of each pixel is equal to the average value subtracted from the second brightness information of each pixel.
Further, the constructing of the remote sensing image enhanced neural network model specifically includes:
constructing a convolutional neural network model, wherein the input of the convolutional neural network model is a remote sensing image and an illumination layer corresponding to the remote sensing image, and the output of the convolutional neural network model is an enhanced remote sensing image;
constructing a training set, wherein the training set comprises a plurality of groups of training data, and each group of training data comprises an original remote sensing image, an illumination layer corresponding to the original remote sensing image and an enhanced remote sensing image corresponding to the original remote sensing image;
and training the convolutional neural network model by using the training set to obtain the remote sensing image enhanced neural network model.
Further, the filtering algorithm is a median filtering algorithm.
Further, the convolutional neural network model is a full convolutional network model.
When the convolutional neural network is adopted to enhance the remote sensing image of the shot blackboard remote sensing image, the convolution kernel cannot capture illumination change in the remote sensing image, and the remote sensing image enhancement effect is poor.
Further, the cloud image removing encryption module encryption method comprises the following steps:
1) Obtaining a hash function value of a plaintext remote sensing image through an encryption program, and obtaining a first key corresponding to the plaintext remote sensing image according to the hash function value; carrying out sparse transformation processing on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image;
2) Scrambling the discrete matrix through the first secret key to obtain a scrambled intermediate matrix, wherein the intermediate matrix corresponds to the scrambled plaintext remote sensing image;
3) Measuring the intermediate matrix through half tensor compression sensing, and outputting a first encryption matrix, wherein the first encryption matrix corresponds to a plaintext remote sensing image after compression and encryption;
wherein said measuring said intermediate matrix by half tensor compressed sensing outputs a first encryption matrix comprising:
presetting a half tensor measurement model
Figure BDA0003755035820000052
Parameter alpha, parameter beta, measurement matrix in (1)
Figure BDA0003755035820000051
Auxiliary matrix
Figure BDA0003755035820000062
x is the input signal, y is the observed signal; wherein the measuring matrix
Figure BDA0003755035820000061
Is generated by a chaotic sequence which is generated by the chaotic sequence,
Figure BDA0003755035820000063
operating signs for half tensor products;
taking the intermediate matrix as an input signal of the semi-tensor measurement model, wherein an observation signal output by the semi-tensor measurement model forms a first encryption matrix;
in the conventional matrix multiplication, when the column number C (a) of the matrix a is equal to the row number R (B) of the matrix B, the matrix a and the matrix B may be multiplied; the matrix half tensor product corresponding to the half tensor measurement model comprises the step of popularizing the traditional matrix multiplication to the condition that the orders are not equal, namely C (A) ≠ R (B);
wherein, after the intermediate matrix is measured by half tensor compressed sensing and the first encryption matrix is output, the method further comprises the following steps:
scrambling the first encryption matrix through a logistic chaotic sequence to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image;
the chaotic scrambling of the first encryption matrix through locality to obtain a second encryption matrix comprises the following steps:
generating a corresponding logistic chaotic sequence by setting a chaotic initial value of the logistic chaotic sequence, setting a sampling initial position of the logistic chaotic sequence as s and a sampling interval as u, performing ascending arrangement on the sampled sequences according to the magnitude of numerical values, recording the position of the corresponding numerical value in the sampled ascending sequence at the logistic chaotic sequence through a sequence n, scrambling the first encryption matrix according to the sequence n to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image; and the chaotic initial value is also used as a second key, and the second key, the second encryption matrix and the first key are sent to a receiving end together.
Further, the obtaining a hash function value of the plaintext remote sensing image and obtaining a first key corresponding to the plaintext remote sensing image according to the hash function value includes:
processing the plaintext remote sensing image through an SHA-256 hash function, outputting an output numerical value with the length of 256 bits, dividing the 256-bit output numerical value into 256/k numerical values according to a preset digit k, and dividing the 256/k numerical values into 3 groups;
respectively adding 3 preset initial values into corresponding groups, carrying out XOR processing on the values in each group according to a preset rule to obtain 3 values, and taking the 3 values as a first key;
the sparse transformation processing is performed on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image, and the sparse transformation processing comprises the following steps:
and carrying out discrete wavelet transform or discrete Fourier transform on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image.
Further, the scrambling the discrete matrix by the first key includes:
scrambling said discrete matrix with an Arnold transform, the Arnold transform being: rearranging the values in the discrete matrix; and the 3 values in the first key are sequentially used as the scrambling round number of the Arnold transformation, the coefficient a and the coefficient b in the scrambling formula of the Arnold transformation.
In combination with the technical solutions and the technical problems to be solved, please analyze the advantages and positive effects of the technical solutions to be protected in the present invention from the following aspects:
first, aiming at the technical problems existing in the prior art and the difficulty in solving the problems, the technical problems to be solved by the technical scheme of the present invention are closely combined with results, data and the like in the research and development process, and some creative technical effects are brought after the problems are solved. The specific description is as follows:
according to the method, the remote sensing image enhancement module utilizes the remote sensing image enhancement neural network model to remove illumination change in the remote sensing image to be processed according to the input illumination layer, so that the enhancement effect of the remote sensing image is improved; meanwhile, the cloud image removing encryption module can effectively encrypt the plaintext remote sensing image, and the intermediate matrix is compressed and sensed through the half tensor, so that the compression processing of the intermediate matrix is realized; compared with the traditional remote sensing image encryption method, the method effectively reduces the storage and transmission amount of the remote sensing image data in the transmission process.
Secondly, considering the technical scheme as a whole or from the perspective of products, the technical effect and advantages of the technical scheme to be protected by the invention are specifically described as follows:
according to the method, the remote sensing image enhancement module utilizes the remote sensing image enhancement neural network model to remove illumination change in the remote sensing image to be processed according to the input illumination layer, so that the enhancement effect of the remote sensing image is improved; meanwhile, the cloud image removing encryption module can effectively encrypt the plaintext remote sensing image, and the intermediate matrix is compressed and sensed through the half tensor, so that the compression processing of the intermediate matrix is realized; compared with the traditional remote sensing image encryption method, the method effectively reduces the storage and transmission amount of the remote sensing image data in the transmission process.
Drawings
Fig. 1 is a flowchart of a remote sensing image cloud removing method fusing multi-temporal information and sub-channel dense convolution according to an embodiment of the present invention.
Fig. 2 is a structural block diagram of a remote sensing image cloud removing system which integrates multi-temporal information and sub-channel dense convolution according to an embodiment of the present invention.
Fig. 3 is a flowchart of a remote sensing image enhancement module enhancement method provided in an embodiment of the present invention.
Fig. 4 is a flowchart of an encryption method for a cloud-removed image encryption module according to an embodiment of the present invention.
In fig. 2: 1. a remote sensing image acquisition module; 2. a remote sensing image enhancement module; 3. an extraction module; 4. a cloud removal module; 5. and a cloud image removing encryption module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
1. Illustrative embodiments are explained. This section is an explanatory embodiment expanding on the claims so as to fully understand how the present invention is embodied by those skilled in the art.
As shown in fig. 1, the method for removing cloud from a remote sensing image by fusing multi-temporal information and sub-channel dense convolution, provided by the invention, comprises the following steps:
s101, acquiring a remote sensing image by using remote sensing equipment through a remote sensing image acquisition module;
s102, enhancing the remote sensing image by using an image enhancement program through a remote sensing image enhancement module;
s103, extracting the areas shaded by the cloud in the remote sensing image through an extraction module;
s104, performing intersection operation on all the occlusion areas of the remote sensing image through a cloud removing module to obtain a specific area, and performing difference operation and intersection operation on the specific area and the occlusion area of the remote sensing image to be cloud removed to remove cloud;
and S105, encrypting the cloud-removed remote sensing image through the cloud-removed image encryption module.
As shown in fig. 2, the remote sensing image cloud removing system fusing multi-temporal information and sub-channel dense convolution according to the embodiment of the present invention includes: the remote sensing image acquisition module 1, the remote sensing image enhancement module 2, the extraction module 3, the cloud removal module 4 and the cloud removal image encryption module 5.
The remote sensing image acquisition module 1 is connected with the remote sensing image enhancement module 2 and is used for acquiring a remote sensing image through remote sensing equipment;
the remote sensing image enhancement module 2 is connected with the remote sensing image acquisition module 1 and the extraction module 3 and is used for enhancing the remote sensing image through an image enhancement program;
the extraction module 3 is connected with the remote sensing image enhancement module 2 and the cloud removal module 4 and is used for extracting areas which are shaded by clouds in the remote sensing image;
the cloud removing module 4 is connected with the extracting module 3 and the cloud removing image encrypting module 5 and is used for carrying out intersection operation on each shielded area of the remote sensing image to obtain a specific area and carrying out difference operation and intersection operation on the specific area and the shielded area of the remote sensing image to be cloud removed so as to remove cloud;
the cloud removing image encryption module 5 is connected with the cloud removing module 4 and used for encrypting the cloud removing remote sensing image.
As shown in fig. 3, the remote sensing image enhancement module 2 provided by the present invention has the following enhancement method:
s201, constructing an image database; storing the acquired remote sensing image into an image database; correcting the remote sensing image, constructing a remote sensing image enhanced neural network model, inputting the remote sensing image enhanced neural network model into a remote sensing image and an illumination layer corresponding to the remote sensing image, and outputting the remote sensing image after enhancement;
s202, acquiring an illumination layer corresponding to the remote sensing image to be processed;
s203, inputting the remote sensing image to be processed and the illumination layer corresponding to the remote sensing image to be processed into the remote sensing image enhanced neural network model, and outputting the enhanced remote sensing image;
the method for correcting the remote sensing image comprises the following steps:
extracting a plurality of collected remote sensing images; splicing a plurality of acquired remote sensing images in a GPS coordinate system to obtain a spliced acquired remote sensing image; converting the space vector of the spliced collected remote sensing image under a GPS coordinate system into a space vector under a WGS-84 coordinate system;
finding N points corresponding to the spliced acquired remote sensing images on the ground as standard points, wherein N is an integer greater than or equal to 2;
determining a check quaternion according to a first space vector group of the N standard points under a WGS-84 coordinate system corresponding to the GPS coordinates of the ground and a second space vector group of the N standard points under the WGS-84 coordinate system corresponding to the coordinates of the spliced acquired remote sensing images;
correcting all coordinate points in the spliced collected remote sensing images according to the check quaternion;
each coordinate point (x ', y') in the acquired remote sensing image corresponds to geographical coordinate information (Long, lat), wherein Long represents longitude, and Lat represents latitude;
the vector conversion formula for converting the space vector of the spliced acquired remote sensing image under the GPS coordinate system into the space vector under the WGS-84 coordinate system is as follows:
Figure BDA0003755035820000101
v represents a space vector in a WGS-84 coordinate system;
the check quaternion is expressed as:
Figure BDA0003755035820000102
ω represents a feature vector of a matrix obtained by orthogonal projection of the first space vector group and the second space vector group, and converts the check quaternion into a form of a transformation matrix, where the transformation matrix is represented as:
Figure BDA0003755035820000103
u = E · v, u representing the corrected space vector of the space vector v in the stitched acquired remote sensing image;
the formula for converting the corrected space vector u into latitude and longitude is as follows:
Lat adj =arcsinu(3),
Figure BDA0003755035820000111
where u is a 3-row 1-column vector, u (1) denotes the elements of the first row of u, u (2) denotes the elements of the second row of u, u (3) denotes the elements of the third row of u, lat adj Indicating the latitude, long after conversion adj Indicating the converted longitude.
The illumination layer corresponding to the remote sensing image provided by the invention is obtained by adopting the following method:
converting the remote sensing image from a first color space to a second color space containing first brightness information, and extracting the first brightness information of the remote sensing image;
filtering the first brightness information by adopting a filtering algorithm to obtain a gray remote sensing image;
calculating the average value of second brightness information of all pixels of the gray remote sensing image;
and correcting the second brightness information of each pixel of the gray remote sensing image by adopting the average value to obtain an illumination layer, wherein the corrected brightness information of each pixel is equal to the average value subtracted from the second brightness information of each pixel.
The invention provides a method for constructing a remote sensing image enhanced neural network model, which specifically comprises the following steps:
constructing a convolutional neural network model, wherein the input of the convolutional neural network model is a remote sensing image and an illumination layer corresponding to the remote sensing image, and the output of the convolutional neural network model is an enhanced remote sensing image;
constructing a training set, wherein the training set comprises a plurality of groups of training data, and each group of training data comprises an original remote sensing image, an illumination image layer corresponding to the original remote sensing image and an enhanced remote sensing image corresponding to the original remote sensing image;
and training the convolutional neural network model by using the training set to obtain the remote sensing image enhanced neural network model.
The filtering algorithm provided by the invention is a median filtering algorithm.
The convolution neural network model provided by the invention is a full convolution network model.
When the convolutional neural network is adopted to enhance the remote sensing image of the shot blackboard remote sensing image, the illumination change in the remote sensing image cannot be captured by the convolutional kernel, and the remote sensing image enhancement effect is poor.
As shown in fig. 4, the cloud image removing encryption module 5 provided by the present invention has the following encryption method:
s301, obtaining a hash function value of a plaintext remote sensing image through an encryption program, and obtaining a first key corresponding to the plaintext remote sensing image according to the hash function value; carrying out sparse transformation processing on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image;
s302, scrambling the discrete matrix through the first key to obtain a scrambled intermediate matrix, wherein the intermediate matrix corresponds to the scrambled plaintext remote sensing image;
s303, measuring the intermediate matrix through half tensor compressed sensing, and outputting a first encryption matrix, wherein the first encryption matrix corresponds to a compressed and encrypted plaintext remote sensing image;
wherein said measuring said intermediate matrix by half tensor compressed sensing outputs a first encryption matrix comprising:
presetting a half tensor measurement model
Figure BDA0003755035820000124
Parameter alpha, parameter beta, measurement matrix
Figure BDA0003755035820000121
Auxiliary matrix
Figure BDA0003755035820000123
x is the input signal, y is the observed signal; wherein the measuring matrix
Figure BDA0003755035820000122
Is generated by the chaotic sequence and is used for generating a chaotic sequence,
Figure BDA0003755035820000125
operating signs for half tensor products;
taking the intermediate matrix as an input signal of the half tensor measurement model, wherein an observation signal output by the half tensor measurement model forms a first encryption matrix;
in the conventional matrix multiplication, when the column number C (a) of the matrix a is equal to the row number R (B) of the matrix B, the matrix a and the matrix B can be multiplied; the matrix half tensor product corresponding to the half tensor measurement model comprises the step of popularizing the traditional matrix multiplication to the condition that the orders are not equal, namely C (A) ≠ R (B);
wherein, after the intermediate matrix is measured by half tensor compressed sensing and the first encryption matrix is output, the method further comprises the following steps:
scrambling the first encryption matrix through a logistic chaotic sequence to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image;
the obtaining of the second encryption matrix by chaotic scrambling of the first encryption matrix by the logistic comprises:
generating a corresponding logistic chaotic sequence by setting a chaotic initial value of the logistic chaotic sequence, setting a sampling initial position of the logistic chaotic sequence as s and a sampling interval as u, performing ascending arrangement on the sampled sequence according to the magnitude of a numerical value, recording the position of a corresponding numerical value in the ascending sequence obtained by sampling through a sequence n, scrambling the first encryption matrix according to the sequence n to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image; and the chaotic initial value is also used as a second key, and the second key, the second encryption matrix and the first key are sent to a receiving end together.
The invention provides a method for obtaining a hash function value of a plaintext remote sensing image, which obtains a first key corresponding to the plaintext remote sensing image according to the hash function value, and comprises the following steps:
processing the plaintext remote sensing image through an SHA-256 hash function, outputting an output value with the length of 256 bits, dividing the 256-bit output value into 256/k values according to a preset number k, and dividing the 256/k values into 3 groups;
respectively adding 3 preset initial values into corresponding groups, carrying out XOR processing on the values in each group according to a preset rule to obtain 3 values, and taking the 3 values as a first key;
the sparse transformation processing is performed on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image, and the sparse transformation processing comprises the following steps:
and carrying out discrete wavelet transform or discrete Fourier transform on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image.
The scrambling processing of the discrete matrix by the first key provided by the invention comprises the following steps:
scrambling said discrete matrix with an Arnold transform, the Arnold transform being: rearranging the values in the discrete matrix; and the 3 values in the first key are sequentially used as the scrambling round number of the Arnold transformation, the coefficient a and the coefficient b in the scrambling formula of the Arnold transformation.
2. Application examples. In order to prove the creativity and the technical value of the technical scheme of the invention, the part is the application example of the technical scheme of the claims on specific products or related technologies.
According to the method, the remote sensing image enhancement module utilizes the remote sensing image enhancement neural network model to remove illumination change in the remote sensing image to be processed according to the input illumination layer, so that the enhancement effect of the remote sensing image is improved; meanwhile, the cloud image removing encryption module can effectively encrypt the plaintext remote sensing image, and the intermediate matrix is compressed and sensed through the half tensor, so that the compression processing of the intermediate matrix is realized; compared with the traditional remote sensing image encryption method, the method effectively reduces the storage and transmission amount of the remote sensing image data in the transmission process.
It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. The hardware portions may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided on a carrier medium such as a disk, CD-or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier, for example. The apparatus of the present invention and its modules may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, or software executed by various types of processors, or a combination of hardware circuits and software, e.g., firmware.
3. Evidence of the relevant effects of the examples. The embodiment of the invention has some positive effects in the process of research and development or use, and indeed has great advantages compared with the prior art, and the following contents are described by combining data, charts and the like in the test process.
According to the method, the remote sensing image enhancement module utilizes the remote sensing image enhancement neural network model to remove illumination change in the remote sensing image to be processed according to the input illumination layer, so that the enhancement effect of the remote sensing image is improved; meanwhile, the cloud image removing encryption module can effectively encrypt the plaintext remote sensing image, and the intermediate matrix is compressed and sensed through the half tensor, so that the compression processing of the intermediate matrix is realized; compared with the traditional remote sensing image encryption method, the method effectively reduces the storage and transmission amount of the remote sensing image data in the transmission process.
The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. The utility model provides a remote sensing image cloud system that goes that fuses multidate information and dense convolution of subchannel which characterized in that, remote sensing image cloud system that goes that fuses multidate information and dense convolution of subchannel includes:
the remote sensing image acquisition module, the remote sensing image enhancement module, the extraction module, the cloud removal module and the cloud removal image encryption module;
the remote sensing image acquisition module is connected with the remote sensing image enhancement module and is used for acquiring a remote sensing image through remote sensing equipment;
the remote sensing image enhancement module is connected with the remote sensing image acquisition module and the extraction module and is used for enhancing the remote sensing image through an image enhancement program;
the extraction module is connected with the remote sensing image enhancement module and the cloud removal module and is used for extracting areas which are shaded by clouds in the remote sensing image;
the cloud removing module is connected with the extracting module and the cloud removing image encrypting module and is used for carrying out intersection operation on each shielded area of the remote sensing image to obtain a specific area and carrying out difference set operation and intersection operation on the specific area and the shielded area of the remote sensing image to be cloud removed so as to remove cloud;
and the cloud removing image encryption module is connected with the cloud removing module and is used for encrypting the cloud removing remote sensing image.
2. The method for cloud removal of the remote sensing image fused with the multi-temporal information and the sub-channel dense convolution according to claim 1, wherein the method for cloud removal of the remote sensing image fused with the multi-temporal information and the sub-channel dense convolution comprises the following steps:
acquiring a remote sensing image by using remote sensing equipment through a remote sensing image acquisition module;
secondly, enhancing the remote sensing image by utilizing an image enhancement program through a remote sensing image enhancement module;
extracting the areas shaded by the cloud in the remote sensing image through an extraction module;
performing intersection operation on each shielded area of the remote sensing image through a cloud removing module to obtain a specific area, and performing difference operation and intersection operation on the specific area and the shielded area of the remote sensing image to be subjected to cloud removing to remove cloud;
and fifthly, encrypting the cloud-removed remote sensing image through a cloud-removed image encryption module.
3. The remote sensing image cloud removing system integrating multi-temporal information and sub-channel dense convolution according to claim 1 is characterized in that the remote sensing image enhancement module is enhanced as follows:
(1) Constructing an image database; storing the acquired remote sensing image into an image database; correcting the remote sensing image, constructing a remote sensing image enhanced neural network model, inputting the remote sensing image and an illumination layer corresponding to the remote sensing image into the remote sensing image enhanced neural network model, and outputting the remote sensing image after enhancement;
(2) Acquiring an illumination layer corresponding to a remote sensing image to be processed;
(3) Inputting the remote sensing image to be processed and the illumination layer corresponding to the remote sensing image to be processed into the remote sensing image enhanced neural network model, and outputting the enhanced remote sensing image;
the method for correcting the remote sensing image comprises the following steps:
extracting a plurality of collected remote sensing images; splicing a plurality of acquired remote sensing images in a GPS coordinate system to obtain a spliced acquired remote sensing image; converting the space vector of the spliced collected remote sensing image under a GPS coordinate system into a space vector under a WGS-84 coordinate system;
finding N points corresponding to the spliced acquired remote sensing images on the ground as standard points, wherein N is an integer greater than or equal to 2;
determining a check quaternion according to a first space vector group of the N standard points under a WGS-84 coordinate system corresponding to the GPS coordinates of the N standard points on the ground and a second space vector group of the N standard points under the WGS-84 coordinate system corresponding to the coordinates of the spliced acquired remote sensing images;
correcting all coordinate points in the spliced collected remote sensing images according to the check quaternion;
each coordinate point (x ', y') in the acquired remote sensing image corresponds to geographical coordinate information (Long, lat), wherein Long represents longitude, and Lat represents latitude;
the vector conversion formula for converting the space vector of the spliced collected remote sensing image under the GPS coordinate system into the space vector under the WGS-84 coordinate system is as follows:
Figure FDA0003755035810000021
v represents a space vector in the WGS-84 coordinate system;
the check quaternion is expressed as:
Figure FDA0003755035810000031
ω represents a feature vector of a matrix obtained by orthogonal projection of the first space vector group and the second space vector group, and converts the check quaternion into a form of a transformation matrix, where the transformation matrix is represented as:
Figure FDA0003755035810000032
u = E · v, u representing the corrected space vector of the space vector v in the stitched acquired remote sensing image;
the formula for converting the corrected space vector u into latitude and longitude is as follows:
Lat adj =arcsinu(3),
Figure FDA0003755035810000033
where u is a 3-row 1-column vector, u (1) represents the elements of the first row of the spatial vector u, u (2) represents the elements of the second row of the spatial vector u, u (3) represents the elements of the third row of the spatial vector u, lat adj Indicating the latitude, long after conversion adj Indicating the converted longitude.
4. The remote sensing image cloud removing system integrating the multi-temporal information and the sub-channel dense convolution according to claim 3 is characterized in that an illumination layer corresponding to the remote sensing image is obtained by the following method:
converting the remote sensing image from a first color space to a second color space containing first brightness information, and extracting the first brightness information of the remote sensing image;
filtering the first brightness information by adopting a filtering algorithm to obtain a gray remote sensing image;
calculating an average value of second brightness information of all pixels of the gray remote sensing image;
and correcting the second brightness information of each pixel of the gray remote sensing image by adopting the average value to obtain an illumination layer, wherein the corrected brightness information of each pixel is equal to the average value subtracted from the second brightness information of each pixel.
5. The remote sensing image cloud removing system fusing multi-temporal information and sub-channel dense convolution according to claim 3 is characterized in that the building of the remote sensing image enhanced neural network model specifically comprises:
constructing a convolutional neural network model, wherein the input of the convolutional neural network model is a remote sensing image and an illumination layer corresponding to the remote sensing image, and the output of the convolutional neural network model is an enhanced remote sensing image;
constructing a training set, wherein the training set comprises a plurality of groups of training data, and each group of training data comprises an original remote sensing image, an illumination layer corresponding to the original remote sensing image and an enhanced remote sensing image corresponding to the original remote sensing image;
and training the convolutional neural network model by using the training set to obtain the remote sensing image enhanced neural network model.
6. The remote sensing image cloud removing system fusing multi-temporal information and channel-division dense convolution according to claim 3, wherein the filtering algorithm is a median filtering algorithm.
7. The remote sensing image cloud removing system fusing multi-temporal information and channel-division dense convolution according to claim 5, wherein the convolution neural network model is a full convolution network model.
When the convolutional neural network is adopted to enhance the remote sensing image of the shot blackboard remote sensing image, the convolution kernel cannot capture illumination change in the remote sensing image, and the remote sensing image enhancement effect is poor.
8. The remote sensing image cloud removing system fusing multi-temporal information and sub-channel dense convolution according to claim 1 is characterized in that the cloud removing image encryption module encryption method is as follows:
1) Obtaining a hash function value of a plaintext remote sensing image through an encryption program, and obtaining a first key corresponding to the plaintext remote sensing image according to the hash function value; carrying out sparse transformation processing on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image;
2) Scrambling the discrete matrix through the first secret key to obtain a scrambled intermediate matrix, wherein the intermediate matrix corresponds to the scrambled plaintext remote sensing image;
3) Measuring the intermediate matrix through half tensor compressed sensing, and outputting a first encryption matrix, wherein the first encryption matrix corresponds to a compressed and encrypted plaintext remote sensing image;
wherein said measuring said intermediate matrix by semi-tensor compressed sensing, outputting a first encryption matrix, comprises:
presetting a half tensor measurement model y = alpha phi 1 ×x+βφ 2 Parameter alpha, parameter beta, measurement matrix
Figure FDA0003755035810000051
Auxiliary matrix
Figure FDA0003755035810000052
x is an input signal and y is an observation signal; wherein the measuring matrix
Figure FDA0003755035810000053
Is generated by a chaotic sequence which is generated by the chaotic sequence,
Figure FDA0003755035810000054
operating signs for half tensor products;
taking the intermediate matrix as an input signal of the half tensor measurement model, wherein an observation signal output by the half tensor measurement model forms a first encryption matrix;
in the conventional matrix multiplication, when the column number C (a) of the matrix a is equal to the row number R (B) of the matrix B, the matrix a and the matrix B may be multiplied; the matrix half tensor product corresponding to the half tensor measurement model comprises the step of popularizing the traditional matrix multiplication to the condition that the orders are not equal, namely C (A) ≠ R (B);
wherein, after measuring the intermediate matrix by half tensor compressed sensing and outputting a first encryption matrix, the method further comprises:
scrambling the first encryption matrix through a logistic chaotic sequence to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image;
the chaotic scrambling of the first encryption matrix through locality to obtain a second encryption matrix comprises the following steps:
generating a corresponding logistic chaotic sequence by setting a chaotic initial value of the logistic chaotic sequence, setting a sampling initial position of the logistic chaotic sequence as s and a sampling interval as u, performing ascending arrangement on the sampled sequences according to the magnitude of numerical values, recording the position of the corresponding numerical value in the sampled ascending sequence at the logistic chaotic sequence through a sequence n, scrambling the first encryption matrix according to the sequence n to obtain a second encryption matrix, wherein the second encryption matrix corresponds to a final ciphertext remote sensing image; and the chaotic initial value is also used as a second key, and the second key, the second encryption matrix and the first key are sent to a receiving end together.
9. The remote sensing image cloud removing system integrating multi-temporal information and dense channel convolution according to claim 8, wherein the obtaining of the hash function value of the plaintext remote sensing image and obtaining the first key corresponding to the plaintext remote sensing image according to the hash function value includes:
processing the plaintext remote sensing image through an SHA-256 hash function, outputting an output value with the length of 256 bits, dividing the 256-bit output value into 256/k values according to a preset number k, and dividing the 256/k values into 3 groups;
respectively adding 3 preset initial values into corresponding groups, carrying out XOR processing on the values in each group according to a preset rule to obtain 3 values, and taking the 3 values as a first key;
the sparse transformation processing is performed on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image, and the sparse transformation processing comprises the following steps:
and carrying out discrete wavelet transform or discrete Fourier transform on the plaintext remote sensing image to obtain a discrete matrix corresponding to the plaintext remote sensing image.
10. The remote sensing image cloud removing system integrating multi-temporal information and sub-channel dense convolution according to claim 8, wherein the scrambling of the discrete matrix through the first key includes:
scrambling said discrete matrix with an Arnold transform, the Arnold transform being: rearranging the numerical values in the discrete matrix; and the 3 values in the first key are sequentially used as the scrambling round number of the Arnold transformation, the coefficient a and the coefficient b in the scrambling formula of the Arnold transformation.
CN202210852113.9A 2022-07-20 2022-07-20 Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution Withdrawn CN115293999A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210852113.9A CN115293999A (en) 2022-07-20 2022-07-20 Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210852113.9A CN115293999A (en) 2022-07-20 2022-07-20 Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution

Publications (1)

Publication Number Publication Date
CN115293999A true CN115293999A (en) 2022-11-04

Family

ID=83823448

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210852113.9A Withdrawn CN115293999A (en) 2022-07-20 2022-07-20 Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution

Country Status (1)

Country Link
CN (1) CN115293999A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116108214A (en) * 2023-02-24 2023-05-12 中科星图数字地球合肥有限公司 Remote sensing image data processing method and device, computer equipment and storage medium
CN116342449A (en) * 2023-03-29 2023-06-27 银河航天(北京)网络技术有限公司 Image enhancement method, device and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116108214A (en) * 2023-02-24 2023-05-12 中科星图数字地球合肥有限公司 Remote sensing image data processing method and device, computer equipment and storage medium
CN116108214B (en) * 2023-02-24 2024-02-06 中科星图数字地球合肥有限公司 Remote sensing image data processing method and device, computer equipment and storage medium
CN116342449A (en) * 2023-03-29 2023-06-27 银河航天(北京)网络技术有限公司 Image enhancement method, device and storage medium
CN116342449B (en) * 2023-03-29 2024-01-16 银河航天(北京)网络技术有限公司 Image enhancement method, device and storage medium

Similar Documents

Publication Publication Date Title
CN115293999A (en) Remote sensing image cloud removing method integrating multi-temporal information and sub-channel dense convolution
Zhang Image processing
EP3481054A1 (en) Machine vision 3d line scan image acquisition and processing
CN111461998A (en) Environment reconstruction method and device
CN114708150A (en) Scanning data processing method and device, electronic equipment and medium
CN118097157B (en) Image segmentation method and system based on fuzzy clustering algorithm
EP3225024B1 (en) Method and system for determining parameters of an image processing pipeline of a digital camera
CN114758145A (en) Image desensitization method and device, electronic equipment and storage medium
US20130121558A1 (en) Point Selection in Bundle Adjustment
US12087033B2 (en) Information processing system, information processing method, image-capturing apparatus, and information processing apparatus
CN111722922A (en) Satellite image area network adjustment method and device for GPU parallel acceleration
CN114742707B (en) Multi-source remote sensing image splicing method and device, electronic equipment and readable medium
CN116385297A (en) Computer image processing system
CN114926491B (en) Matting method and device, electronic equipment and storage medium
CN113963000B (en) Image segmentation method, device, electronic equipment and program product
CN112308809B (en) Image synthesis method, device, computer equipment and storage medium
Yu et al. Efficient statistical validation of autonomous driving systems
CN114187367A (en) Camera external parameter calibration method and device, electronic equipment and storage medium
CN115797438A (en) Object positioning method, device, computer equipment, storage medium and program product
CN112487943B (en) Key frame de-duplication method and device and electronic equipment
CN110189279B (en) Model training method and device, electronic equipment and storage medium
CN114414067A (en) Thermal imaging data processing method and device, thermal imaging photographic equipment and storage medium
CN115643348B (en) Method and device for certifiable safety natural steganography based on reversible image processing network
US20230300385A1 (en) Image processing method and apparatus
CN114821325B (en) Method, device, equipment and medium for extracting vegetation canopy porosity based on model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20221104

WW01 Invention patent application withdrawn after publication