CN113393543B - Hyperspectral image compression method, device and equipment and readable storage medium - Google Patents
Hyperspectral image compression method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN113393543B CN113393543B CN202110662427.8A CN202110662427A CN113393543B CN 113393543 B CN113393543 B CN 113393543B CN 202110662427 A CN202110662427 A CN 202110662427A CN 113393543 B CN113393543 B CN 113393543B
- Authority
- CN
- China
- Prior art keywords
- hyperspectral image
- neural network
- convolutional neural
- representing
- image compression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000006835 compression Effects 0.000 title claims abstract description 86
- 238000007906 compression Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 60
- 230000009466 transformation Effects 0.000 claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 51
- 238000012360 testing method Methods 0.000 claims abstract description 27
- 238000013139 quantization Methods 0.000 claims abstract description 26
- 238000009826 distribution Methods 0.000 claims description 42
- 230000008569 process Effects 0.000 claims description 37
- 230000006870 function Effects 0.000 claims description 21
- 238000001228 spectrum Methods 0.000 claims description 15
- 238000013461 design Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
Abstract
The invention provides a hyperspectral image compression method, a hyperspectral image compression device, hyperspectral image compression equipment and a readable storage medium. The method comprises the following steps: training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model; and verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard. The method has better rate distortion performance for the compression of the hyperspectral image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a hyperspectral image compression method, a hyperspectral image compression device, hyperspectral image compression equipment and a readable storage medium.
Background
The hyperspectral image has rich and unique spectral information, and great convenience is brought to a plurality of applications based on the hyperspectral image, such as tasks of crop classification, quality detection, disaster prediction and the like. However, the advantages also restrict the further development of the hyperspectral image under the limited transmission bandwidth and storage capacity. Therefore, how to effectively solve various challenges brought by the large data volume of the hyperspectral image is a precondition and a key for the hyperspectral image to be widely applied.
In the hyperspectral image compression algorithm, transform coding is widely applied due to smaller computational complexity and good adaptability. The image compression algorithm based on the transformation coding comprises four parts: transform, quantization and entropy coding, inverse transform, implementing separately the coding process and the decorrelation process.
The transformation is to transform an image from a pixel domain to a more compact space in a certain way, the existing hyperspectral image compression method based on transform coding generally assumes that a hyperspectral image is a Gaussian source, pixels can be mapped into independent potential representations only through reversible linear transformation under the condition, and latent variables are compressed into code streams for storage and transmission through quantization and entropy coding. However, the hyperspectral image of the actual scene has obvious non-gaussian characteristics, so that linear transformation is not applicable any more, and the exploration of the nonlinear transformation provides a new method and thought for the problem. In recent years, the development of nonlinear transformation using artificial neural networks, especially deep learning, as a tool has changed the situation of traditional manual parameter setting for image compression. The existing image compression technology based on deep learning has great potential, and the performance exceeds the H.266/VVC (Versatile Video Coding, VVC) standard in the industry. However, these methods are mostly used to process three-band natural images, and the compression for hyperspectral images is relatively small.
The transformation process enables quantization and entropy coding to be performed in a compact space, and spectra of the hyperspectral images have stronger correlation compared with RGB natural images, so that potential representations obtained by the hyperspectral images have different statistical properties from RGB images through the same transformation process. Upon quantization, the latent variable becomes a discrete form, which is then encoded based on an entropy encoding algorithm. The entropy coding process depends on a probability distribution model of latent variables, and the designed entropy model is closer to the real latent variable distribution, the code rate is smaller, and the solution obtained in the entropy rate optimization process is closer to the optimal solution.
In combination with the above analysis, the current compression technology based on deep learning needs to further design a more flexible and accurate entropy model according to the characteristics of the hyperspectral image so as to reduce the mismatch between the entropy model and the real latent variable distribution, thereby achieving the optimal rate distortion performance.
Disclosure of Invention
In order to solve the technical problems, the invention provides a hyperspectral image compression method, a hyperspectral image compression device, hyperspectral image compression equipment and a readable storage medium.
In a first aspect, the present invention provides a hyperspectral image compression method, including:
training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model;
and verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard.
Optionally, before the step of training the convolutional neural network through the training set, the method further includes:
cutting a sample hyperspectral image into a plurality of cubic blocks with fixed sizes in a spatial dimension;
and dividing the cuboids with fixed sizes into a training set and a testing set according to a preset proportion.
Optionally, the nonlinear transformation module performs forward nonlinear transformation on the space and spectrum dimensions of the hyperspectral image to obtain a latent variable; the quantization module quantizes the latent variable by adding uniform noise; the entropy model is used for obtaining the probability distribution of the latent variable, so that the code word allocated to each element in the latent variable is determined based on the probability distribution during entropy coding.
Optionally, the convolutional neural network training process is constrained based on a rate-distortion criterion, and is used to determine parameter values in the nonlinear transformation module and the entropy model.
Optionally, the nonlinear transformation includes a forward transformation: g represents Ya(WaX+ba) And (3) inverse transformation:
wherein the content of the first and second substances,
representing the input hyperspectral image,
representing a reconstructed image, divided into H, W, BRespectively corresponding to the number of rows, columns and wave bands of the hyperspectral image,
representing the latent variables, h, w, N respectively corresponding to the number of rows, columns and filters of the latent variables,
and
which is indicative of a network parameter being converted,
and
representing inverse transform network parameters, ga(.) represents a nonlinear forward transformation function, gs(.) represents a non-linear inverse transformation function.
Optionally, the function for quantizing the latent variable by adding uniform noise is expressed as follows:
wherein training represents a training process, and testing represents a testing process,
indicating a unit of uniform noise, round indicates a rounding operation,
representing the quantized latent vector.
Optionally, the statistical characteristics of the latent variables are introduced into the design of the entropy model, and meanwhile, an additional variable is introduced to construct a condition model, so as to improve the accuracy of the entropy model.
In a second aspect, the present invention also provides a hyperspectral image compression apparatus comprising:
the training module is used for training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model;
and the processing module is used for verifying the compression performance of the trained convolutional neural network by using the test set, and when the compression performance of the trained convolutional neural network reaches the standard, the hyperspectral image is compressed by the trained convolutional neural network.
In a third aspect, the present invention further provides a hyperspectral image compression apparatus comprising a processor, a memory, and a hyperspectral image compression program stored on the memory and executable by the processor, wherein the hyperspectral image compression program, when executed by the processor, implements the steps of the hyperspectral image compression method as described above.
In a fourth aspect, the present invention further provides a readable storage medium, on which a hyperspectral image compression program is stored, where the hyperspectral image compression program, when executed by a processor, implements the steps of the hyperspectral image compression method as described above.
In the invention, a convolutional neural network is trained through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model; and verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard. The method has better rate distortion performance for the compression of the hyperspectral image.
Drawings
FIG. 1 is a schematic diagram of a hardware structure of a hyperspectral image compression device according to an embodiment of the invention;
FIG. 2 is a schematic flow chart of an embodiment of a hyperspectral image compression method according to the invention;
FIG. 3 is a functional block diagram of an embodiment of the hyperspectral image compression apparatus of the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In a first aspect, an embodiment of the present invention provides a hyperspectral image compression apparatus, where the hyperspectral image compression apparatus may be an apparatus with a data processing function, such as a Personal Computer (PC), a notebook computer, and a server.
Referring to fig. 1, fig. 1 is a schematic diagram of a hardware structure of a hyperspectral image compression device according to an embodiment of the present invention. In an embodiment of the present invention, the hyperspectral image compression apparatus may include a processor 1001 (e.g., a Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used for realizing connection communication among the components; the user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WI-FI interface, WI-FI interface); the memory 1005 may be a Random Access Memory (RAM) or a non-volatile memory (non-volatile memory), such as a magnetic disk memory, and the memory 1005 may optionally be a storage device independent of the processor 1001. Those skilled in the art will appreciate that the hardware configuration depicted in FIG. 1 is not intended to be limiting of the present invention, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
With continued reference to FIG. 1, the memory 1005 of FIG. 1, which is one type of computer storage medium, may include an operating system, a network communication module, a user interface module, and a hyper-spectral image compression program. The processor 1001 may call a hyperspectral image compression program stored in the memory 1005, and execute the hyperspectral image compression method provided by the embodiment of the present invention.
In a second aspect, an embodiment of the present invention provides a hyperspectral image compression method.
In an embodiment, referring to fig. 2, fig. 2 is a schematic flowchart of a hyperspectral image compression method according to an embodiment of the invention. As shown in fig. 2, the hyperspectral image compression method includes:
step S10, training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model;
in this embodiment, a training set is pre-constructed, and a convolutional neural network is trained through the training set, where the convolutional neural network includes a nonlinear transformation module, a quantization module, and an entropy model.
Further, in an embodiment, before step S10, the method further includes:
cutting a sample hyperspectral image into a plurality of cubic blocks with fixed sizes in a spatial dimension; and dividing the cuboids with fixed sizes into a training set and a testing set according to a preset proportion.
In this embodiment, before training, a data set, including a training set and a test set, needs to be prepared, and a hyper-parameter of the convolutional neural network needs to be set. For example, 27 KAIST data sets (30) with a size of 2704 × 3376 × 31 and 28 CAVE data (32) with a size of 512 × 512 × 31 are randomly cropped into image blocks with a size of 128 × 128 × 31. The experiment adopts tensierflow frame to train model, sends cut 128 × 128 × 31 image block batch (batch 32) into the built network, iterates 500000 times, and the loss function in training is as follows:
wherein, the approximate posterior
Using a unit uniform density function of the full decomposition, thus losing the first term of the function
Expressing a distortion item, performing distortion measurement by adopting Mean Square Error (MSE) loss with a parameter lambda in a training process, wherein the value of lambda is from 0.00001 to 0.01, and the range of bppp (the number of bits occupied by each pixel of each band to be coded) can be controlled to be between 0.1 and 2 (the larger the lambda is, the larger the bppp is);
representing the total number of coded bits.
Further, in one embodiment, the nonlinear transformation module performs forward nonlinear transformation on the space and spectrum dimensions of the hyperspectral image to obtain a latent variable; the quantization module quantizes the latent variable by adding uniform noise; the entropy model is used for obtaining the probability distribution of the latent variable, so that the code word distributed to each element in the latent variable is determined based on the probability distribution during entropy coding.
In the embodiment, a training set is input into a convolutional neural network, and a nonlinear transformation module in the convolutional neural network carries out forward nonlinear transformation on the space and spectrum dimensions of a hyperspectral image, so that the image is mapped to a compact latent space from a pixel domain to obtain a latent variable; then, a quantization module in the convolutional neural network quantizes the latent variable by adding uniform noise, the gradient of back propagation is 0 due to quantization in the training process, in order to ensure smooth training, the quantization process is replaced by adding uniform noise in the embodiment, so that the quantization is conducted, and the test process is directly rounded; then, a probability distribution of the latent variable is obtained based on the entropy model, so that when entropy coding is carried out, the code word assigned to each element in the latent variable (namely, each element uses several characters) is determined based on the probability distribution.
Further, in one embodiment, the convolutional neural network training process is constrained based on a rate-distortion criterion to determine the nonlinear transformation module and the parameter values in the entropy model.
In this embodiment, a rate distortion criterion is adopted to solve parameter values in the nonlinear transformation module and the entropy model, a concept of variation inference is combined with rate distortion in an optimization process, and a rate distortion optimization process is explained from a probability perspective:
wherein, the first and the second end of the pipe are connected with each other,
the reconstructed image is represented, d (·) represents a distortion measurement criterion, a peak signal-to-noise ratio PSNR is generally adopted in the hyperspectral image, the structural similarity SSIM is used for carrying out distortion measurement on pixels, the larger the value is, the better the reconstruction effect of the pixels is, a spectrum angle SAM is used for measuring the reconstruction accuracy of the spectrum, and lambda represents a Lagrange multiplier.
In order to optimize the loss function, the thought of variation inference is adopted, parameter solution is carried out by designing an approximate posterior to approximate to a real posterior, KL divergence is adopted to measure two posteriors, and a calculation formula is as follows:
wherein the content of the first and second substances,
representing an approximate a posteriori, can be represented by any simple distribution, typically represented in compression as a uniform distribution of units of a full decomposition, so as to remove this term from the loss function, while the remaining three terms except the constant term const,
corresponding distortion
Corresponding code rate
Corresponding additional information
Further, in an embodiment, the non-linear transform comprises a forward transform: g represents Ya(WaX+ba) And (3) inverse transformation:
wherein the content of the first and second substances,
representing the input hyperspectral image,
representing the reconstructed image, H, W and B respectively correspond to the row, column and wave band number of the hyperspectral image,
representing the latent variables, h, w, N respectively corresponding to the number of rows, columns and filters of the latent variables,
and
representing the parameters of the network being transformed,
and
representing inverse transform network parameters, ga(.) represents a nonlinear forward transformation function, gs(.) represents a non-linear inverse transformation function.
Further, in one embodiment, the function for quantizing the latent variable by adding uniform noise is expressed as follows:
wherein training represents a training process, and testing represents a testing process,
indicating unity uniform noise, round indicates a rounding operation,
representing the quantized latent vector.
In this embodiment, the latent variable is quantized, unit uniform noise approximation is adopted during training, and an rounding mode is adopted during testing.
Further, in one embodiment, statistical properties of latent variables are introduced into the design of the entropy model, and at the same time, additional variables are introduced to construct a condition model, so as to improve the accuracy of the entropy model.
In the embodiment, the statistical characteristics of the latent variables are introduced into the design of the entropy model to reduce the difference between the entropy model and the distribution of the real latent variables, and the smaller the difference between the entropy model and the real latent variables is, the smaller the obtained code rate is, and a certain priori cognition can be added into the latent layer representation to improve the precision of the entropy model. Here, a condition model is constructed by introducing additional variables, and the calculation formula is as follows:
wherein the content of the first and second substances,
a representation of the quantized latent layer is represented,
a conditional entropy model is represented by a model of conditional entropy,
representing additional variables
As a priori information of the entropy model,
representing the true distribution of the latent representation.
In the design of the entropy model, the statistic prior represented by a potential layer is added, and the parameter is solved by using a convolutional neural network:
wherein f represents a distribution capable of describing the statistical properties of the subsurface, which can be represented as a gaussian distribution if the gaussian properties are more obvious, or a T distribution, a laplacian distribution, etc. if the non-gaussian properties are more obvious; the choice of f is determined by the statistical properties of the latent layer representation. The parameter of f is obtained by convolutional neural network learning, namely, on the premise of determining the type of f, the parameter information of f distribution is learned by using variables.
After the hyperspectral image is subjected to nonlinear transformation of the convolutional neural network, the distribution of latent variables has obvious non-Gaussian characteristic, so that the prior information needs to be added during design of an entropy model, and the characteristic that t distribution can well capture is found in the experimental process, so that the t distribution is selected to model the latent variables of the hyperspectral image.
In order to make the whole compression process microminiaturible, the quantization process adopts additive unit uniform noise approximation; in order to make the entropy model more fit with the posterior distribution, when the entropy model is designed, the convolution is uniformly distributed by one unit, and the formula is as follows:
wherein eta isiA scale parameter representing the t-distribution (similar but not equal to the variance), and v represents a degree of freedom, the shape of the t-distribution can be adjusted. c represents a summary of the entropy modelAnalytic form of the rate distribution. Using additional variables for parametric variables of probability distributions
Through the prior network.
Entropy coding adopts arithmetic coding, in the process of arithmetic coding, an entropy model provides probability distribution for the processes of arithmetic coding and arithmetic decoding, the size of a code word distributed to each element (namely, each symbol occupies a plurality of bits) is determined according to the probability distribution of each element in a latent variable, and after entropy coding, the latent variable is changed into a binary code stream for storage or transmission.
And step S20, verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard.
In this embodiment, a uniform noise approximate quantization process is used for training, and an rounding mode is directly used for testing. The entropy coding adopts common arithmetic coding, and minimizes the rate distortion loss training model until convergence. The whole large graph is directly put in during testing, and on the CAVE data set, when the degree of freedom is 21, the effects that the bppp is 0.1219, the PSNR is 36.74dB, the SSIM is 0.9175 and the SAM is 0.2137 can be achieved. At 20 degrees of freedom, on the KAIST data set, the effects of the PSNR being 39.99dB, the SSIM being 0.9524 and the SAM being 0.2331 can be achieved when the bppp is 0.0885.
If the user needs to use the image information, the binary code stream can be reduced into latent variables through arithmetic decoding, then the latent variables are input into an inverse transformation network consisting of two space and spectrum modules, in the inverse transformation network, the space and spectrum modules are connected by IGDN, and the up-sampling is recovered to the original image size. The compression frame is divided into four parts including transformation network, quantization, entropy coding and inverse transformation network.
In this embodiment, for the anisotropy of the hyperspectral image, a spatial and spectral convolution module (SS module, including SS module _ down (for constructing an encoding network) and SS module _ up (for constructing a decoding network)) is proposed, and the SS module _ down and the SS module _ up are connected by a GDN. In SS module _ down, for an image tensor with spectral dimension B (B × H × W), a first layer of down-sampling is followed by inputting a filter of 5 × B, generating N feature representations; after GDN and one down sampling, 5 × N convolution layers are input to generate B feature representations, and N feature representations are generated through 1 × B convolution layers. The process of SS module _ up is similar to SS module _ down, but downsampling is replaced with upsampling. When the spectrum module is designed, the spectrum dimension of the hyperspectral image is introduced into the network, the rearrangement of the spectrum information is realized, and therefore the correlation among the spectra is reduced. Aiming at the non-Gaussian characteristics represented by the latent layer of the hyperspectral image, the assumption of traditional Gaussian distribution is not adopted during the design of the entropy model, and the matching degree of the entropy model and the latent layer representation statistical distribution is improved by introducing some non-Gaussian distributions as statistical prior of the latent layer representation. According to the fitting of latent variables of the hyperspectral data sets CAVE and KAIST data sets, the t distribution is found to be good in performance, meanwhile, the distribution shape can be flexibly changed by adjusting the degree of freedom, and when the degree of freedom tends to be infinite, the t distribution and Gaussian distribution can be equivalent. This property enables the t-distribution to capture both the non-gaussian property of the latent layer representation and the universality of the gaussian distribution.
In this embodiment, a convolutional neural network is trained through a training set, wherein the convolutional neural network includes a nonlinear transformation module, a quantization module, and an entropy model; and verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard. Through the embodiment, the method has better rate distortion performance for the compression of the hyperspectral image.
In a third aspect, an embodiment of the present invention further provides a hyperspectral image compression apparatus.
In an embodiment, referring to fig. 3, fig. 3 is a functional module schematic diagram of an embodiment of a hyperspectral image compression device according to the invention. As shown in fig. 3, the hyperspectral image compression apparatus includes:
the training module 10 is used for training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model;
and the processing module 20 is configured to verify the compression performance of the trained convolutional neural network by using the test set, and compress the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network meets the standard.
Further, in an embodiment, the hyperspectral image compression apparatus further includes a construction module configured to:
cutting a sample hyperspectral image into a plurality of cubic blocks with fixed sizes in a spatial dimension;
and dividing the cuboids with fixed sizes into a training set and a testing set according to a preset proportion.
Further, in one embodiment, the nonlinear transformation module performs forward nonlinear transformation on the space and spectrum dimensions of the hyperspectral image to obtain a latent variable; the quantization module quantizes the latent variable by adding uniform noise; the entropy model is used for obtaining the probability distribution of the latent variable, so that the code word allocated to each element in the latent variable is determined based on the probability distribution during entropy coding.
Further, in one embodiment, the convolutional neural network training process is constrained based on a rate-distortion criterion to determine the nonlinear transformation module and the parameter values in the entropy model.
Further, in an embodiment, the non-linear transform comprises a forward transform: g represents Ya(WaX+ba) And (3) inverse transformation:
wherein the content of the first and second substances,
representing the input hyperspectral image,
representing the reconstructed image, H, W and B respectively correspond to the row, column and wave band number of the hyperspectral image,
representing the latent variables, h, w, N respectively corresponding to the number of rows, columns and filters of the latent variables,
and
representing the parameters of the network being transformed,
and
representing inverse transform network parameters, ga(.) represents a nonlinear forward transformation function, gs(.) represents a nonlinear inverse transformation function.
Further, in one embodiment, the function for quantizing the latent variable by adding uniform noise is expressed as follows:
wherein training represents a training process, and testing represents a testing process,
indicating unity uniform noise, round indicates a rounding operation,
representing the quantized latent vector.
Further, in one embodiment, statistical properties of latent variables are introduced into the design of the entropy model, and at the same time, additional variables are introduced to construct a condition model, so as to improve the accuracy of the entropy model.
The function implementation of each module in the hyperspectral image compression device corresponds to each step in the hyperspectral image compression method embodiment, and the functions and the implementation process are not described in detail herein.
In a fourth aspect, the embodiment of the present invention further provides a readable storage medium.
The readable storage medium of the present invention stores a hyper-spectral image compression program, wherein the hyper-spectral image compression program, when executed by a processor, implements the steps of the hyper-spectral image compression method as described above.
The method implemented when the hyper-spectral image compression program is executed may refer to each embodiment of the hyper-spectral image compression method of the present invention, and details are not repeated here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for causing a terminal device to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (6)
1. A hyperspectral image compression method is characterized by comprising the following steps:
cutting a sample hyperspectral image into a plurality of cubic blocks with fixed sizes in a spatial dimension;
dividing the cuboids with fixed sizes into a training set and a testing set according to a preset proportion;
training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model; the nonlinear transformation module carries out forward nonlinear transformation on the space and spectrum dimensions of the hyperspectral image to obtain a latent variable; the quantization module quantizes the latent variable by adding uniform noise; the entropy model is used for obtaining the probability distribution of the latent variable so as to determine the code word distributed to each element in the latent variable based on the probability distribution during entropy coding; constraining the training process of the convolutional neural network based on a rate-distortion criterion to determine parameter values in a nonlinear transformation module and an entropy model; the non-linear transformation comprises a forward transformation: g represents Ya(WaX+ba) And (3) inverse transformation:
wherein, the first and the second end of the pipe are connected with each other,
representing the input hyperspectral image,
representing the reconstructed image, H, W and B respectively correspond to the row, column and wave band number of the hyperspectral image,
representing the latent variables, h, w, N respectively corresponding to the number of rows, columns and filters of the latent variables,
and
representing the parameters of the network being transformed,
and
representing inverse transform network parameters, ga(.) represents a nonlinear forward transformation function, gs(.) representing a nonlinear inverse transformation function;
and verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard.
2. The hyperspectral image compression method according to claim 1, wherein the function for quantizing the latent variable by adding uniform noise is expressed as follows:
wherein training represents a training process, and testing represents a testing process,
indicating unity uniform noise, round indicates a rounding operation,
representing the quantized latent vector.
3. The hyperspectral image compression method according to claim 2, wherein the statistical properties of latent variables are introduced into the design of the entropy model, and simultaneously, additional variables are introduced to construct a conditional model so as to improve the accuracy of the entropy model.
4. A hyperspectral image compression apparatus, characterized in that the hyperspectral image compression apparatus comprises:
a build module to:
cutting a sample hyperspectral image into a plurality of cubic blocks with fixed sizes in a space dimension;
dividing the cuboids with fixed sizes into a training set and a testing set according to a preset proportion;
the training module is used for training a convolutional neural network through a training set, wherein the convolutional neural network comprises a nonlinear transformation module, a quantization module and an entropy model; the nonlinear transformation module carries out forward nonlinear transformation on the space and spectrum dimensions of the hyperspectral image to obtain a latent variable; the quantization module quantizes the latent variable by adding uniform noise; the entropy model is used for obtaining the probability distribution of the latent variable so as to determine the code word distributed to each element in the latent variable based on the probability distribution during entropy coding; constraining the training process of the convolutional neural network based on a rate-distortion criterion to determine parameter values in a nonlinear transformation module and an entropy model; the non-linear transform comprises a forward transform: g represents Ya(WaX+ba) And (3) inverse transformation:
wherein, the first and the second end of the pipe are connected with each other,
representing the input hyperspectral image,
representing the reconstructed image, H, W and B respectively correspond to the row, column and wave band number of the hyperspectral image,
representing the latent variables, h, w, N respectively corresponding to the number of rows, columns and filters of the latent variables,
and
representing the parameters of the network being transformed,
and
representing inverse transform network parameters, ga(.) represents a nonlinear forward transformation function, gs(.) represents a non-linear inverse transformation function;
and the processing module is used for verifying the compression performance of the trained convolutional neural network by using the test set, and compressing the hyperspectral image by using the trained convolutional neural network when the compression performance of the trained convolutional neural network reaches the standard.
5. A hyperspectral image compression apparatus comprising a processor, a memory, and a hyperspectral image compression program stored on the memory and executable by the processor, wherein the hyperspectral image compression program when executed by the processor implements the steps of the hyperspectral image compression method according to any of claims 1 to 3.
6. A readable storage medium having stored thereon a hyper-spectral image compression program, wherein the hyper-spectral image compression program when executed by a processor implements the steps of the hyper-spectral image compression method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110662427.8A CN113393543B (en) | 2021-06-15 | 2021-06-15 | Hyperspectral image compression method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110662427.8A CN113393543B (en) | 2021-06-15 | 2021-06-15 | Hyperspectral image compression method, device and equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113393543A CN113393543A (en) | 2021-09-14 |
| CN113393543B true CN113393543B (en) | 2022-07-01 |
Family
ID=77621105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110662427.8A Expired - Fee Related CN113393543B (en) | 2021-06-15 | 2021-06-15 | Hyperspectral image compression method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113393543B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2632161A1 (en) * | 2012-02-24 | 2013-08-28 | Raytheon Company | Hyperspectral image compression |
| CN110348487A (en) * | 2019-06-13 | 2019-10-18 | 武汉大学 | A kind of method for compressing high spectrum image and device based on deep learning |
| EP3611700A1 (en) * | 2018-08-14 | 2020-02-19 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| CN110880194A (en) * | 2019-12-03 | 2020-03-13 | 山东浪潮人工智能研究院有限公司 | Image compression method based on convolutional neural network |
| CN111683250A (en) * | 2020-05-13 | 2020-09-18 | 武汉大学 | Generation type remote sensing image compression method based on deep learning |
| CN112734867A (en) * | 2020-12-17 | 2021-04-30 | 南京航空航天大学 | Multispectral image compression method and system based on space spectrum feature separation and extraction |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8432974B2 (en) * | 2009-07-10 | 2013-04-30 | The United States Of America As Represented By The Secretary Of The Navy | Method and system for compression of hyperspectral or multispectral imagery with a global optimal compression algorithm (GOCA) |
| US20200160565A1 (en) * | 2018-11-19 | 2020-05-21 | Zhan Ma | Methods And Apparatuses For Learned Image Compression |
| CN110097051A (en) * | 2019-04-04 | 2019-08-06 | 平安科技（深圳）有限公司 | Image classification method, device and computer readable storage medium |
| CN112149652A (en) * | 2020-11-27 | 2020-12-29 | 南京理工大学 | Space-spectrum joint depth convolution network method for lossy compression of hyperspectral image |
-
2021
- 2021-06-15 CN CN202110662427.8A patent/CN113393543B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2632161A1 (en) * | 2012-02-24 | 2013-08-28 | Raytheon Company | Hyperspectral image compression |
| EP3611700A1 (en) * | 2018-08-14 | 2020-02-19 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| CN110348487A (en) * | 2019-06-13 | 2019-10-18 | 武汉大学 | A kind of method for compressing high spectrum image and device based on deep learning |
| CN110880194A (en) * | 2019-12-03 | 2020-03-13 | 山东浪潮人工智能研究院有限公司 | Image compression method based on convolutional neural network |
| CN111683250A (en) * | 2020-05-13 | 2020-09-18 | 武汉大学 | Generation type remote sensing image compression method based on deep learning |
| CN112734867A (en) * | 2020-12-17 | 2021-04-30 | 南京航空航天大学 | Multispectral image compression method and system based on space spectrum feature separation and extraction |
Non-Patent Citations (3)
| Title |
|---|
| Global Consistent Graph Convolutional Network for Hyperspectral Image Classification;Yun Ding;《IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT》;20210124;第70卷;第1-16页 * |
| 基于块稀疏表达模式的高光谱图像压缩;种衍文等;《华中科技大学学报(自然科学版)》;20171231(第12期);第65-70、77页 * |
| 基于深度学习的图像压缩算法研究综述;于恒等;《计算机工程与应用》;20200618(第15期);第21-29页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113393543A (en) | 2021-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11153566B1 (en) | Variable bit rate generative compression method based on adversarial learning | |
| Nash et al. | Generating images with sparse representations | |
| Cai et al. | Efficient variable rate image compression with multi-scale decomposition network | |
| US11221990B2 (en) | Ultra-high compression of images based on deep learning | |
| US20220215595A1 (en) | Systems and methods for image compression at multiple, different bitrates | |
| CN112149652A (en) | Space-spectrum joint depth convolution network method for lossy compression of hyperspectral image | |
| CN114449276B (en) | Super prior side information compensation image compression method based on learning | |
| Ahanonu | Lossless image compression using reversible integer wavelet transforms and convolutional neural networks | |
| Han et al. | Toward variable-rate generative compression by reducing the channel redundancy | |
| CN115361559A (en) | Image encoding method, image decoding method, image encoding device, image decoding device, and storage medium | |
| Wang et al. | A customized deep network based encryption-then-lossy-compression scheme of color images achieving arbitrary compression ratios | |
| CN113393543B (en) | Hyperspectral image compression method, device and equipment and readable storage medium | |
| WO2023118317A1 (en) | Method and data processing system for lossy image or video encoding, transmission and decoding | |
| Ororbia et al. | Learned iterative decoding for lossy image compression systems | |
| Zhuang et al. | A robustness and low bit-rate image compression network for underwater acoustic communication | |
| Lyu et al. | Statistically and perceptually motivated nonlinear image representation | |
| Thakker et al. | Lossy Image Compression-A Comparison Between Wavelet Transform, Principal Component Analysis, K-Means and Autoencoders | |
| CN115361555A (en) | Image encoding method, image encoding device, and computer storage medium | |
| Sinha et al. | Self-supervised variable rate image compression using visual attention | |
| Sahu et al. | Image compression methods using dimension reduction and classification through PCA and LDA: A review | |
| Aidini et al. | Tensor decomposition learning for compression of multidimensional signals | |
| Kamisli | Learned lossless image compression through interpolation with low complexity | |
| Rajakumar et al. | Lossy image compression using multiwavelet transform for wireless transmission | |
| Byju et al. | Generative Adversarial Networks for Spatio-Spectral Compression of Hyperspectral Images | |
| Al Falahi et al. | Comparitive Analysis and Findings on Dct & Lbg Compression Techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220701 |