WO2022217746A1 - 一种高分辨率高光谱计算成像方法、系统及介质 - Google Patents

一种高分辨率高光谱计算成像方法、系统及介质 Download PDF

Info

Publication number
WO2022217746A1
WO2022217746A1 PCT/CN2021/101763 CN2021101763W WO2022217746A1 WO 2022217746 A1 WO2022217746 A1 WO 2022217746A1 CN 2021101763 W CN2021101763 W CN 2021101763W WO 2022217746 A1 WO2022217746 A1 WO 2022217746A1
Authority
WO
WIPO (PCT)
Prior art keywords
hyperspectral
image
module
hyperspectral image
initial
Prior art date
Application number
PCT/CN2021/101763
Other languages
English (en)
French (fr)
Inventor
李树涛
佃仁伟
郭安静
康旭东
孙斌
方乐缘
卢婷
Original Assignee
湖南大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 湖南大学 filed Critical 湖南大学
Priority to US17/871,943 priority Critical patent/US20220366536A1/en
Publication of WO2022217746A1 publication Critical patent/WO2022217746A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the invention relates to high-resolution hyperspectral imaging technology, in particular to a high-resolution hyperspectral computational imaging method, system and medium.
  • Hyperspectral imaging technology can obtain image information in dozens or hundreds of spectral bands at the same time, and rich spectral information helps to accurately identify substances in the scene.
  • the cost of spectral imaging equipment is expensive, which greatly limits the application of hyperspectral imaging.
  • existing imaging systems can obtain high-resolution RGB images and the cost of RGB cameras is low. Obtaining high-resolution hyperspectral images from RGB images is an economical and efficient approach, and the process is often referred to as spectral super-resolution.
  • Model optimization-based methods assume that RGB images can be downsampled from hyperspectral images. This class of methods estimates hyperspectral images by combining image imaging models and given image prior information through maximum a posteriori estimation. However, these pre-given prior information often cannot describe the characteristics of the image well, and it is easy to cause distortion of spectral and spatial information.
  • Deep convolutional neural networks can effectively learn the prior information of images.
  • Data-driven deep convolutional neural networks have been widely used in spectral super-resolution. This kind of method pre-trains the network through RGB images and corresponding hyperspectral images. , so as to obtain the best parameters.
  • spectral super-resolution This kind of method pre-trains the network through RGB images and corresponding hyperspectral images. , so as to obtain the best parameters.
  • such methods often ignore the imaging model in spectral super-resolution, which limits the performance of convolutional neural networks.
  • the technical problem to be solved by the present invention aiming at the above-mentioned problems of the prior art, a high-resolution hyperspectral computational imaging method, system and medium are provided, and the present invention can effectively realize the reconstruction of RGB images to high-resolution hyperspectral images , which has the advantages of high reconstruction accuracy, high computational efficiency, low memory consumption, and strong generalization ability.
  • the technical scheme adopted in the present invention is:
  • a high-resolution hyperspectral computational imaging method comprising:
  • step 1) spectral upsampling is performed on the input RGB image Y to obtain the function expression of the initial hyperspectral image X 0 :
  • the deep convolutional neural network guided by the imaging model in step 2) is composed of multiple modules with the same structure, and the multiple modules are cascade-connected, and the input of each module includes the initial hyperspectral image X. , the previous module or the up-sampling result of the initial hyperspectral image X 0 , and the hyperspectral image X is obtained from the output of the last module.
  • the module is composed of a hyperspectral prior learning module HPL and an imaging model guidance module IMG, and the hyperspectral prior learning module HPL is used to learn the previous module or the upsampling result of the initial hyperspectral image X 0 .
  • the imaging model guidance module IMG is configured to optimize the learned features according to the imaging model based on the input initial hyperspectral image X 0 and the prior features output by the hyperspectral prior learning module HPL.
  • the hyperspectral prior learning module HPL is a 3 ⁇ 3 first convolutional layer, a nonlinear modified linear unit, a channel attention mechanism, a 3 ⁇ 3 second convolutional layer, and a spatial attention mechanism in sequence.
  • the channel attention mechanism includes a convolution operation with a size of 1 ⁇ 1, a nonlinear normalization unit, a linear operation operation, and multiple modified linear units, and a 1 ⁇ 1 convolution operation,
  • the nonlinear normalization unit, the linear operation operation, and a plurality of modified linear units are connected in sequence.
  • step 2) includes:
  • the number of initialization iterations t is equal to 1, and the parameter values in the deep convolutional neural network guided by the imaging model and the value of the penalty factor ⁇ t of the t-th iteration are initialized;
  • the hyperspectral prior learning module HPL in the t-th module learns the prior feature of the previous module or the upsampling result of the initial hyperspectral image X 0 , and then guides the module IMG through the imaging model based on the input initial hyperspectral image Image X 0 , the prior features output by the hyperspectral prior learning module HPL, optimize the learned features according to the imaging model, and update the hyperspectral image X obtained by the t-th iteration;
  • the hyperspectral prior learning module HPL learns the prior feature of the previous module or the upsampling result of the initial hyperspectral image X 0 means: introducing a variable G, and executing formulas (3) to (4) to update the variable G Finish learning the prior features of the previous module or the upsampling result of the initial hyperspectral image X 0 ;
  • G t+1 is the value of the introduced variable in the t+1 round of iteration
  • G is the introduced variable
  • ⁇ is the weight parameter
  • (G) is the regular term of the introduced variable G
  • ⁇ t is the penalty factor of the t-th iteration
  • ⁇ t+1 is the penalty factor of the t+1-th iteration
  • X t is the hyperspectral image obtained by the t-th iteration
  • X t+1 is the hyperspectral image obtained by the t+1 round iteration
  • Y is the RGB image
  • F is the spectral response function
  • X is the hyperspectral image
  • ⁇ is the penalty factor update coefficient.
  • the imaging model guidance module IMG is used to optimize the learned features based on the input initial hyperspectral image X 0 and the prior features output by the hyperspectral prior learning module HPL, and the function expression of the learned features is optimized according to the imaging model:
  • X t+1 is the result obtained by optimizing the learned features according to the imaging model
  • F is the spectral response function
  • I is the identity matrix
  • Y is the input RGB image
  • ⁇ t is the penalty factor for the t-th iteration
  • G t+1 is the value of the introduced variable in the t+1 iteration.
  • the present invention also provides a high-resolution hyperspectral computational imaging system comprising an interconnected microprocessor and a memory, the microprocessor being programmed or configured to perform the steps of the high-resolution hyperspectral computational imaging method, Or the microprocessor is programmed or configured together with a neural network acceleration processor to perform the steps of the high resolution hyperspectral computational imaging method.
  • the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to execute the high-resolution hyperspectral computational imaging method.
  • the present invention has the following advantages:
  • the present invention performs spectral upsampling on the input RGB image Y to obtain the initial hyperspectral image X 0 , and inputs the initial hyperspectral image X 0 into the deep convolutional neural network guided by the pre-trained imaging model, and obtains the high spectral density through iterative solution.
  • the spectral image X the relationship between the RGB image Y and the hyperspectral image X is established through the deep convolutional neural network guided by the imaging model, and the hyperspectral image X is obtained by iterative solution.
  • the present invention can effectively realize the RGB image to high resolution.
  • the reconstruction of hyperspectral images can effectively realize the direct acquisition of high spatial resolution hyperspectral images from high spatial resolution RGB images, and has the advantages of high reconstruction accuracy, high computational efficiency, low memory consumption, and strong generalization ability.
  • the present invention establishes the relationship between the RGB image Y and the hyperspectral image X through the deep convolutional neural network guided by the imaging model, adopts the imaging model to guide the learning process of the deep convolutional neural network, and significantly reduces the neural network. parameters and improve the learning performance of the neural network.
  • the present invention does not need to change the structure and parameters of the network when imaging different types of hyperspectral, and has strong universality and robustness.
  • FIG. 1 is a schematic diagram of a basic flow of a method according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of the principle of a hyperspectral prior learning module proposed in an embodiment of the present invention.
  • FIG. 3 is a comparison diagram of the results of the implementation method of the present invention and four imaging methods on Harvard hyperspectral images.
  • FIG. 4 is a comparison diagram of the results of the implementation method of the present invention and four imaging methods on the CAVE hyperspectral image.
  • FIG. 5 is a comparison diagram of objective performance indicators of the implementation method of the present invention and four imaging methods on the CAVE data set.
  • FIG. 6 is a comparison diagram of objective performance indicators of the implementation method of the present invention and four imaging methods in the Harvard data set.
  • the high-resolution hyperspectral computational imaging method in this embodiment includes:
  • the generalized inverse upsampling of the spectral response function is used for the input RGB image Y to obtain the initial hyperspectral image X 0
  • the function expression for obtaining the initial hyperspectral image X 0 by spectrally upsampling the input RGB image Y is as follows: :
  • the deep convolutional neural network guided by the imaging model in step 2) is composed of multiple modules with the same structure, and the multiple modules are connected in cascade, and the input of each module includes the initial hyperspectral image.
  • X 0 the previous module or the up-sampling result of the initial hyperspectral image X 0
  • the hyperspectral image X is obtained from the output of the last module.
  • the deep convolutional neural network has the advantage of lightweight model.
  • the module consists of a hyperspectral prior learning module HPL and an imaging model guidance module IMG.
  • the hyperspectral prior learning module HPL is used to learn the prior features of the previous module or the upsampling results of the initial hyperspectral image X 0 , and the imaging model
  • the guidance module IMG is used to optimize the learned features according to the imaging model based on the input initial hyperspectral image X 0 and the prior features output by the hyperspectral prior learning module HPL.
  • the hyperspectral prior learning module HPL is used to learn the prior features of the previous module or the upsampling result of the initial hyperspectral image X 0 .
  • the hyperspectral prior learning module HPL in this embodiment is a 3 ⁇ 3 first convolutional layer, a nonlinear correction linear unit, a channel attention mechanism, and a 3 ⁇ 3 second
  • the convolutional layer and the spatial attention mechanism (Spatial Attention) are connected in turn to form a five-layer structure.
  • the channel attention mechanism includes a convolution operation with a size of 1 ⁇ 1, a nonlinear normalization unit, a linear operation operation, and multiple A modified linear unit is connected, and a 1 ⁇ 1 convolution operation, a nonlinear normalization unit, a linear operation operation, and a plurality of modified linear units are connected in sequence.
  • the third layer is the channel attention mechanism, which is used to learn the spectral characteristics of hyperspectral images;
  • the last layer is the spatial attention mechanism, which is used to learn the spatial characteristics of hyperspectral images.
  • the deep convolutional neural network guided by the aforementioned imaging model is the hyperspectral imaging model established in this embodiment, which quantitatively describes the relationship between the RGB image and the hyperspectral image, and uses the maximum a posteriori probability theory to solve the hyperspectral imaging problem. It is decomposed into two sub-problems to be solved alternately, and the two sub-problems are solved by designing a hyperspectral prior learning module and an imaging model guidance module, which can effectively realize the reconstruction of RGB images to hyperspectral images and reduce the acquisition cost of hyperspectral images. .
  • the step of obtaining hyperspectral image X by iterative solution in step 2) includes:
  • the number of initialization iterations t is equal to 1, and the parameter values in the deep convolutional neural network guided by the imaging model and the value of the penalty factor ⁇ t of the t-th iteration are initialized;
  • the hyperspectral prior learning module HPL in the t-th module learns the prior feature of the previous module or the upsampling result of the initial hyperspectral image X 0 , and then guides the module IMG through the imaging model based on the input initial hyperspectral image Image X 0 , the prior features output by the hyperspectral prior learning module HPL, optimize the learned features according to the imaging model, and update the hyperspectral image X obtained by the t-th iteration;
  • the hyperspectral prior learning module HPL learns the prior feature of the previous module or the upsampling result of the initial hyperspectral image X 0 , which refers to: introducing a variable G, and executing formulas (3) to (4) to update the variables G completes learning the prior features of the previous module or the upsampling result of the initial hyperspectral image X 0 ;
  • G t+1 is the value of the introduced variable in the t+1 iteration
  • G is the introduced variable
  • ⁇ is the weight parameter
  • (G) is the regular term of the introduced variable G
  • ⁇ t is the penalty factor of the t-th iteration
  • ⁇ t+1 is the penalty factor of the t+1-th iteration
  • X t is the hyperspectral image obtained by the t-th iteration
  • X t+1 is the hyperspectral image obtained by the t+1 round iteration
  • Y is the RGB image
  • F is the spectral response function
  • X is the hyperspectral image
  • ⁇ is the penalty factor update coefficient.
  • Hyperspectral Prior Learning Module HPL Hyperspectral images contain rich spectral and spatial information, so we propose a channel attention mechanism to learn the spectral information of hyperspectral images, and use a spatial attention mechanism to learn the spatial information of hyperspectral images.
  • the imaging model guidance module IMG is used to optimize the learned features based on the input initial hyperspectral image X 0 and the a priori features output by the hyperspectral prior learning module HPL, and the function expression for optimizing the learned features according to the imaging model is:
  • X t+1 is the result obtained by optimizing the learned features according to the imaging model
  • F is the spectral response function
  • I is the identity matrix
  • Y is the input RGB image
  • ⁇ t is the penalty factor for the t-th iteration
  • G t+1 is the value of the introduced variable in the t+1th iteration.
  • Update X t+1 according to Equation (3) to be the hyperspectral image obtained by the t+1th round of iteration.
  • Equation (3) is regarded as a strongly convex problem with an analytical solution, as shown in Equation (5).
  • Analytical solution the imaging model guidance module IMG uses matrix multiplication to perform the aforementioned formula (5) to obtain an analytical solution to the strongly convex problem.
  • This embodiment also includes the steps of pre-establishing a plurality of sub-problem solving models shown in equations (3) to (4):
  • F is the spectral response matrix
  • Y) is the probability that X occurs under the condition that Y occurs
  • X) is the probability that Y occurs under the condition that X occurs
  • P(X) is the prior probability of X.
  • ⁇ is the variance of the noise
  • ⁇ is the weight parameter (greater than 0)
  • ⁇ (X) is the regular term of the hyperspectral image X to be estimated.
  • ⁇ is the penalty factor
  • images with 31 bands and a spatial size of 512 ⁇ 512 in the CAVE public data set and images with 31 bands and a spatial size of 1392 ⁇ 1040 in the Harvard public data set are used for verification experiments.
  • the image is taken as a high-resolution hyperspectral image, and the corresponding RGB image is taken as the input image, which is obtained by downsampling the spectral response function.
  • 20 hyperspectral images were randomly selected as the training set and 12 hyperspectral images as the test set in the CAVE dataset; in the Harvard dataset, 35 hyperspectral images were selected as the training set and 15 hyperspectral images were test set.
  • FIG. 3 is a comparison diagram of the results of the implementation method of the present invention and four typical imaging methods on Harvard hyperspectral images
  • FIG. 4 is the implementation method of the present invention and four typical imaging methods. Comparison of the results of the method on the CAVE hyperspectral image.
  • SAM spectral angle
  • RMSE root mean square error
  • UIQI unified image quality indicator
  • SSIM structural similarity
  • the table shown in Figure 5 shows the objective evaluation indicators of four typical fusion methods (Arad, HSCNN-R, DFMN, AWAN+) and the method proposed in this example (SSRNet) on the CAVE dataset.
  • the table shown in Figure 6 shows the objective evaluation indicators of the four typical fusion methods (Arad, HSCNN-R, DFMN, AWAN+) and the method proposed in this example (SSRNet) on the Harvard dataset. It can be seen from both Figures 5 and 6 that all objective evaluation indicators of the method in this embodiment (SSRNet) are better than other methods. This is because the method in this embodiment (SSRNet uses an image-guided model unit, which can better optimize the model). Parameters. More importantly, the deep convolutional neural network used can learn the prior knowledge of the image well and save the spatial details of the image.
  • the method of this embodiment utilizes the powerful learning ability of the deep neural convolutional network and the imaging model of spectral super-resolution, which can improve the imaging accuracy and efficiency at the same time.
  • the RGB image is up-sampled by the Moore-Penrose pseudo-inverse method, and the dense fusion strategy is used to superimpose the up-sampled image and the RGB image as input to guide the high correlation between spectral bands and the low-rank characteristic of the spectral dimension. Therefore, the channel attention block is firstly used to obtain the correlation between the spectral bands of the hyperspectral image.
  • the non-local similarity is used to obtain the non-local spatial similarity of the hyperspectral images and then reconstruct the hyperspectral images.
  • the above is referred to as the spectral prior learning module in this embodiment.
  • the image features learned by the spectral prior learning module are optimized, which can make full use of the prior information learned from the image.
  • the estimation of the entire hyperspectral image adopts the semi-quadratic splitting algorithm to iterate continuously, and finally a high-resolution hyperspectral image is obtained.
  • the hyperspectral image obtained by the fast hyperspectral imaging method in this embodiment has better quality, and the method in this paper consumes less memory and requires lower hardware.
  • the present embodiment also provides a high-resolution hyperspectral computational imaging system, comprising a microprocessor and a memory connected to each other, the microprocessor being programmed or configured to perform the steps of the aforementioned high-resolution hyperspectral computational imaging method, Alternatively, the microprocessor is programmed or configured together with the neural network acceleration processor to perform the steps of the aforementioned high-resolution hyperspectral computational imaging method.
  • the present embodiment also provides a computer-readable storage medium, where a computer program programmed or configured to execute the aforementioned high-resolution hyperspectral computational imaging method is stored in the computer-readable storage medium.
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • the present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application.
  • each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams can be implemented by computer program instructions.
  • These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
  • These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

一种高分辨率高光谱计算成像方法、系统及介质，所述方法包括：对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X0；将初始高光谱图像X0输入到预先完成训练的成像模型引导的深度卷积神经网络，通过迭代求解得到高光谱图像X，该深度卷积神经网络由多个具有相同结构的模块级联构成，每一个模块由高光谱先验学习模块HPL和成像模型引导模块IMG构成，高光谱先验学习模块HPL用于学习上一个模块或者初始高光谱图像X0的上采样结果的先验特征。能够有效实现RGB图像到高分辨率的高光谱图像的重构，具有重构精度高、计算效率高、内存消耗小、泛化能力强的优点。

Description

一种高分辨率高光谱计算成像方法、系统及介质 技术领域
本发明涉及高分辨高光谱成像技术，具体涉及一种高分辨率高光谱计算成像方法、系统及介质。
背景技术
高光谱成像技术能够同时获得几十上百个光谱波段的图像信息，丰富的光谱信息有助于对场景内物质的准确识别，因此高光谱成像技术被广泛地应用于对地观测、军事监测、环境监测、地质勘探、医学检测和人脸识别等多个领域。但是由于光学成像系统的限制，现有光学成像系统难以直接获取高分辨率高光谱图像。同时，光谱成像设备的造价昂贵，极大地限制了高光谱图像的应用。另一方面，现有的成像系统可获得高分辨率的RGB图像且RGB相机成本低。通过RGB图像获得高分辨率高光谱图像是一种经济且有效的途径，该过程通常被称为光谱超分辨率。
目前流行的光谱超分辨率方法可以分为基于模型优化的方法和基于深度卷积神经网络的方法。基于模型优化的方法假设RGB图像可以由高光谱图像下采样得到。该类方法通过最大后验概率估计，将图像成像模型和给定的图像先验信息结合起来估计高光谱图像。而这些预先给定的先验信息往往并不能很好的描述图像的特性，容易造成光谱和空间信息的失真。
深度卷积神经网络能够有效地学习图像的先验信息，基于数据驱动的深度卷积神经网络已被广泛应用于光谱超分辨，该类方法通过RGB图像和相应的高光谱图像对网络进行预训练，从而获得最佳参数。然而该类方法往往忽略了光谱超分辨中的成像模型，这限制了卷积神经网络的性能。
发明内容
本发明要解决的技术问题：针对现有技术的上述问题，提供一种高分辨率高光谱计算成像方法、系统及介质，本发明能够有效实现RGB图像到高分辨率的高光谱图像的重构，具有重构精度高、计算效率高、内存消耗小、泛化能力强的优点。
为了解决上述技术问题，本发明采用的技术方案为：
一种高分辨率高光谱计算成像方法，包括：
1)对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0；
2)将初始高光谱图像X 0输入到预先完成训练的成像模型引导的深度卷积神经网络，通过迭 代求解得到高光谱图像X。
可选地，步骤1)中对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0的函数表达式为：
Figure PCTCN2021101763-appb-000001
上式中，
Figure PCTCN2021101763-appb-000002
为光谱响应函数R的广义逆。
可选地，步骤2)中的成像模型引导的深度卷积神经网络由多个具有相同结构的模块构成，且多个模块之间级联连接，每一个模块的输入包括初始高光谱图像X 0、上一个模块或者初始高光谱图像X 0的上采样结果，且由最后一个模块输出得到高光谱图像X。
可选地，所述模块由高光谱先验学习模块HPL和成像模型引导模块IMG构成，所述高光谱先验学习模块HPL用于学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，所述成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征。
可选地，所述高光谱先验学习模块HPL为3×3的第一卷积层、非线性修正线性单元、通道注意力机制、3×3的第二卷积层以及空间注意力机制依次相连构成的五层结构，所述通道注意力机制包括大小为1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元，且1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元之间依次相连。
可选地，步骤2)中通过迭代求解得到高光谱图像X的步骤包括：
2.1)初始化迭代次数t等于1，初始化成像模型引导的深度卷积神经网络中的参数值和第t轮迭代的惩罚因子μ t的取值；
2.2)首先通过第t个模块中的高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，然后通过成像模型引导模块IMG基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征，更新第t轮迭代得到的高光谱图像X；
2.3)判断迭代次数t等于预设阈值T是否成立，如果成立则将第t轮迭代得到的高光谱图像X作为最终结果输出；否则，将迭代次数t加1，跳转执行步骤2.2)继续进行迭代。
可选地，高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征是指：引入变量G，并执行式(3)～(4)更新变量G完成学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征；
Figure PCTCN2021101763-appb-000003
Figure PCTCN2021101763-appb-000004
μ t+1＝γμ t  (4)
上式中，G t+1为引入的变量在第t+1轮迭代时的取值，G为引入的变量，λ为权重参数，
Figure PCTCN2021101763-appb-000005
(G)为引入的变量G的正则项，μ t为第t轮迭代的惩罚因子，μ t+1为第t+1轮迭代的惩罚因子，X t为第t轮迭代得到的高光谱图像，X t+1为第t+1轮迭代得到的高光谱图像，Y表示RGB图像，F为光谱响应函数，X表示高光谱图像，γ为惩罚因子更新系数。
可选地，成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征的函数表达式为：
X t+1＝(F TF+μ tI) -1(F TY+μ tG t+1)  (5)
上式中，X t+1为根据成像模型来优化学习到的特征得到的结果，F为光谱响应函数，I为单位矩阵，Y为输入的RGB图像，μ t为第t轮迭代的惩罚因子，G t+1为引入的变量在第t+1轮迭代时的取值。
此外，本发明还提供一种分辨率高光谱计算成像系统，包括相互连接的微处理器和存储器，所述微处理器被编程或配置以执行所述高分辨率高光谱计算成像方法的步骤，或者所述微处理器与神经网络加速处理器共同被编程或配置以执行所述高分辨率高光谱计算成像方法的步骤。
此外，本发明还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有被编程或配置以执行所述高分辨率高光谱计算成像方法的计算机程序。
和现有技术相比，本发明具有下述优点：
1、本发明对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0，将初始高光谱图像X 0输入到预先完成训练的成像模型引导的深度卷积神经网络，通过迭代求解得到高光谱图像X，通过成像模型引导的深度卷积神经网络建立了RGB图像Y、高光谱图像X之间的关系并通过迭代求解得到高光谱图像X，本发明能够有效实现RGB图像到高分辨率的高光谱图像的重构，能够有效实现从高空间分辨率的RGB图像直接得到高空间分辨率的高光谱图像，具有重构精度高、计算效率高、内存消耗小、泛化能力强的优点。
2、本发明通过成像模型引导的深度卷积神经网络建立了RGB图像Y、高光谱图像X之间的关系，采用了成像模型来引导深度卷积神经网络的学习过程，显著降低了神经网络的参数量，并且提升了神经网络的学习性能。
3、本发明在对不同类型的高光谱成像时，不需要改变网络的结构和参数，具有很强的普适性和鲁棒性。
附图说明
图1为本发明实施例方法的基本流程示意图。
图2为本发明实施例中所提出的高光谱先验学习模块原理示意图。
图3为本发明实施方法与四种成像方法在Harvard高光谱图像上的结果对比图。
图4为本发明实施方法与四种成像方法在CAVE高光谱图像上的结果对比图。
图5为本发明实施方法与四种成像方法在CAVE数据集上的客观性能指标对比图。
图6为本发明实施方法与四种成像方法在Harvard数据集的客观性能指标对比图。
具体实施方式
如图1所述，本实施例高分辨率高光谱计算成像方法包括：
1)对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0；
2)将初始高光谱图像X 0输入到预先完成训练的成像模型引导的深度卷积神经网络，通过迭代求解得到高光谱图像X。
本实施例中，对输入的RGB图像Y采用光谱响应函数的广义逆上采样得到初始高光谱图像X 0，对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0的函数表达式为：
Figure PCTCN2021101763-appb-000006
上式中，
Figure PCTCN2021101763-appb-000007
为光谱响应函数R的广义逆。
如图1所示，步骤2)中的成像模型引导的深度卷积神经网络由多个具有相同结构的模块构成，且多个模块之间级联连接，每一个模块的输入包括初始高光谱图像X 0、上一个模块或者初始高光谱图像X 0的上采样结果，且由最后一个模块输出得到高光谱图像X。该深度卷积神经网络具有模型轻量化的优点。其中，模块由高光谱先验学习模块HPL和成像模型引导模块IMG构成，高光谱先验学习模块HPL用于学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征。
高光谱先验学习模块HPL用于学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征。如图2所示，本实施例中的高光谱先验学习模块HPL为3×3的第一卷积层、非线性修正线性单元、通道注意力机制(Channel Attention)、3×3的第二卷积层以及空间注意力机制(Spatial Attention)依次相连构成的五层结构，所述通道注意力机制包括大小为1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元，且1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元之间依次相连。其中，第三层为通道注意力机制(Channel Attention)用来学习高光谱图像的光谱特性；最后一层为 空间注意力机制(Spatial Attention)，用来学习高光谱图像的空间特性。
前述成像模型引导的深度卷积神经网络即为本实施例中建立的高光谱成像模型，其定量描述了RGB图像和高光谱图像之间的关系，利用最大后验概率理论，把高光谱成像问题分解为交替解决的两个子问题，分别通过设计高光谱先验学习模块和成像模型引导模块来求解这两个子问题，能够有效实现RGB图像到高光谱图像重构，降低了高光谱图像的获取成本。本实施例中，步骤2)中通过迭代求解得到高光谱图像X的步骤包括：
2.1)初始化迭代次数t等于1，初始化成像模型引导的深度卷积神经网络中的参数值和第t轮迭代的惩罚因子μ t的取值；
2.2)首先通过第t个模块中的高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，然后通过成像模型引导模块IMG基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征，更新第t轮迭代得到的高光谱图像X；
2.3)判断迭代次数t等于预设阈值T是否成立，如果成立则将第t轮迭代得到的高光谱图像X作为最终结果输出；否则，将迭代次数t加1，跳转执行步骤2.2)继续进行迭代。
本实施例中，高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征是指：引入变量G，并执行式(3)～(4)更新变量G完成学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征；
Figure PCTCN2021101763-appb-000008
Figure PCTCN2021101763-appb-000009
μ t+1＝γμ t  (4)
上式中，G t+1为引入的变量在第t+1轮迭代时的取值，G为引入的变量，λ为权重参数，
Figure PCTCN2021101763-appb-000010
(G)为引入的变量G的正则项，μ t为第t轮迭代的惩罚因子，μ t+1为第t+1轮迭代的惩罚因子，X t为第t轮迭代得到的高光谱图像，X t+1为第t+1轮迭代得到的高光谱图像，Y表示RGB图像，F为光谱响应函数，X表示高光谱图像，γ为惩罚因子更新系数。根据式(2)更新变量G时具体是指将式(3)视为从图像先验信息正则化的图像去噪问题，本实施例采用深度卷积神经网络解决该问题，该模块被称作高光谱先验学习模块HPL。高光谱图像包含丰富的光谱和空间信息，因而我们提出通道注意力机制来学习高光谱图像的光谱信息，采用空间注意力机制来学习高光谱图像空间信息。
本实施例中，成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征的函数表达式为：
X t+1＝(F TF+μ tI) -1(F YY+μ tG t+1)  (5)
上式中，X t+1为根据成像模型来优化学习到的特征得到的结果，F为光谱响应函数，I为单位矩阵，Y为输入的RGB图像，μ t为第t轮迭代的惩罚因子，G t+1为引入的变量在第t+1轮迭代时的取值。根据式(3)更新X t+1为第t+1轮迭代得到的高光谱图像，具体是指将式(3)视为具有解析解的强凸问题，具有如式(5)所示的解析解。本实施例中，成像模型引导模块IMG采用矩阵乘法执行前述式(5)得到强凸问题的解析解。
本实施例中还包括预先建立式(3)～(4)所示多个子问题求解模型的步骤：
S1)建立高光谱图像X、传统的RGB图像Y之间的线性映射关系：
(6)上式中，F为光谱响应矩阵。
S2)根据贝叶斯公式和最大后验，将对高光谱图像的估计问题转化为下式的基础模型:
X＝argmax XP(X|Y)  (7)
Figure PCTCN2021101763-appb-000011
上式中，P(X|Y)为X在Y发生的条件下发生的概率，P(Y|X)为Y在X发生的条件下发生的概率，P(X)为X的先验概率，σ为噪声的方差，λ为权重参数(大于0)，φ(X)为待估计的高光谱图像X的正则项。
S3)引入变量G＝X，X为待估计的高光谱图像，建立需要优化的外罚函数L(X,G)；
Figure PCTCN2021101763-appb-000012
上式中，μ为惩罚因子。
S4)将需要优化的外罚函数L(X,G)转换分解，即可得到式(3)～(4)所示多个子问题求解模型。
为了对本实施例方法进行验证，本实施例中利用CAVE公开数据集中波段数为31、空间尺寸为512×512的图像以及Harvard公开数据集中波段为31、空间尺寸为1392×1040的图像进行验证实验。在实验中，把该图像当作高分辨率高光谱图像，与之对应的RGB图像当作输入图像，是通过光谱响应函数下采样得到。在实际过程中，在CAVE数据集中随机选择20个高光谱图像作为训练集，12个高光谱图像作为测试集；在Harvard数据集中，选择35个高光谱图像作为训练集，15个高光谱图像作为测试集。并对比了4种典型的高光谱 成像方法，图3为本发明实施方法与四种典型的成像方法在Harvard高光谱图像上的结果对比图，图4为本发明实施方法与四种典型的成像方法在CAVE高光谱图像上的结果对比图。其中融合图像的评价指标有4种，分别是光谱角(SAM)、均方根误差(RMSE)、统一图像质量指标(UIQI)和结构相似度(SSIM)。其中UIQI和SSIM的值越大，图像质量越好，SAM和RMSE的值越大代表高分辨率图像的质量越差。图5所示表格展示了4种典型的融合方法(Arad,HSCNN-R,DFMN,AWAN+)和本实施例提出的方法(SSRNet)在CAVE数据集上成像实验的客观评价指标。图6所示表格展示了4种典型的融合方法(Arad,HSCNN-R,DFMN,AWAN+)和本实施例提出的方法(SSRNet)在Harvard数据集上成像实验的客观评价指标。从图5和图6均可以看出，本实施例方法(SSRNet)的所有客观评价指标都优于其它方法，这是因为本实施例方法(SSRNet采用了图像指导模型单元，更好的优化模型参数。更重要的是利用的深度卷积神经网络能很好的学习图像先验知识，保存图像的空间细节。
综上所述，本实施例方法利用了深度神经卷积网络的强大学习能力与光谱超分辨率的成像模型，能够同时提升成像精度和效率。首先通过Moore-Penrose伪逆法对RGB图像进行上采样，采用密集融合策略将上采样的图像与RGB图像进行叠加作为输入，来指导由于谱带之间相关性高且光谱维度具有低秩特性，因此首先利用通道注意块来获取高光谱图像光谱波段之间的相关性。然后利用光谱图像的空间相似性的先验知识，采用非局部相似性来获取高光谱图像的非局部空间相似性进而重建高光谱图像。上面在本实施例中称为光谱先验学习模块。之后基于建立的光谱成像模型和深度卷积神经网络来优化由光谱先验学习模块学习到的图像特征，这样能够充分利用从图像中学习到的先验信息。整个高光谱图像的估计采用半二次方分裂算法不断迭代，最终获得高分辨率高光谱图像。通过与其它高性能的快速高光谱成像方法相比，本实施例快速高光谱成像方法得出来的高光谱图像具有更好的质量，并且本文方法的内存消耗小，对硬件要求更低。
此外，本实施例还提供一种分辨率高光谱计算成像系统，包括相互连接的微处理器和存储器，所述微处理器被编程或配置以执行前述高分辨率高光谱计算成像方法的步骤，或者所述微处理器与神经网络加速处理器共同被编程或配置以执行前述高分辨率高光谱计算成像方法的步骤。
此外，本实施例还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有被编程或配置以执行前述高分辨率高光谱计算成像方法的计算机程序。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序 产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可读存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅是本发明的优选实施方式，本发明的保护范围并不仅局限于上述实施例，凡属于本发明思路下的技术方案均属于本发明的保护范围。应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理前提下的若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (9)

  1. 一种高分辨率高光谱计算成像方法，其特征在于，包括：
    1)对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0；
    2)将初始高光谱图像X 0输入到预先完成训练的成像模型引导的深度卷积神经网络，通过迭代求解得到高光谱图像X；步骤2)中的成像模型引导的深度卷积神经网络由多个具有相同结构的模块构成，且多个模块之间级联连接，每一个模块的输入包括初始高光谱图像X 0、上一个模块或者初始高光谱图像X 0的上采样结果，且由最后一个模块输出得到高光谱图像X。
  2. 根据权利要求1所述的高分辨率高光谱计算成像方法，其特征在于，步骤1)中对输入的RGB图像Y进行光谱上采样得到初始高光谱图像X 0的函数表达式为：
    Figure PCTCN2021101763-appb-100001
    上式中，
    Figure PCTCN2021101763-appb-100002
    为光谱响应函数R的广义逆。
  3. 根据权利要求1所述的高分辨率高光谱计算成像方法，其特征在于，所述模块由高光谱先验学习模块HPL和成像模型引导模块IMG构成，所述高光谱先验学习模块HPL用于学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，所述成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征。
  4. 根据权利要求3所述的高分辨率高光谱计算成像方法，其特征在于，所述高光谱先验学习模块HPL为3×3的第一卷积层、非线性修正线性单元、通道注意力机制、3×3的第二卷积层以及空间注意力机制依次相连构成的五层结构，所述通道注意力机制包括大小为1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元，且1×1的卷积操作、非线性归一化单元、线性运算操作以及多个修正线性单元之间依次相连。
  5. 根据权利要求3所述的高分辨率高光谱计算成像方法，其特征在于，步骤2)中通过迭代求解得到高光谱图像X的步骤包括：
    2.1)初始化迭代次数t等于1，初始化成像模型引导的深度卷积神经网络中的参数值和第t轮迭代的惩罚因子μ t的取值；
    2.2)首先通过第t个模块中的高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征，然后通过成像模型引导模块IMG基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征，更新第t轮迭代得到的高光谱图像X；
    2.3)判断迭代次数t等于预设阈值T是否成立，如果成立则将第t轮迭代得到的高光谱图像 X作为最终结果输出；否则，将迭代次数t加1，跳转执行步骤2.2)继续进行迭代。
  6. 根据权利要求3所述的高分辨率高光谱计算成像方法，其特征在于，高光谱先验学习模块HPL学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征是指：引入变量G，并执行式(3)～(4)更新变量G完成学习上一个模块或者初始高光谱图像X 0的上采样结果的先验特征；
    Figure PCTCN2021101763-appb-100003
    Figure PCTCN2021101763-appb-100004
    μ t+1＝γμ t  (4)
    上式中，G t+1为引入的变量在第t+1轮迭代时的取值，G为引入的变量，λ为权重参数，
    Figure PCTCN2021101763-appb-100005
    (G)为引入的变量G的正则项，μ t为第t轮迭代的惩罚因子，μ t+1为第t+1轮迭代的惩罚因子，X t为第t轮迭代得到的高光谱图像，X t+1为第t+1轮迭代得到的高光谱图像，Y表示RGB图像，F为光谱响应函数，X表示高光谱图像，γ为惩罚因子更新系数。
  7. 根据权利要求6所述的高分辨率高光谱计算成像方法，其特征在于，成像模型引导模块IMG用于基于输入的初始高光谱图像X 0、高光谱先验学习模块HPL输出的先验特征，根据成像模型来优化学习到的特征的函数表达式为：
    X t+1＝(F TF+μ tI) -1(F TY+μ tG t+1)  (5)
    上式中，X t+1为根据成像模型来优化学习到的特征得到的结果，F为光谱响应函数，I为单位矩阵，Y为输入的RGB图像，μ t为第t轮迭代的惩罚因子，G t+1为引入的变量在第t+1轮迭代时的取值。
  8. 一种分辨率高光谱计算成像系统，包括相互连接的微处理器和存储器，其特征在于，所述微处理器被编程或配置以执行权利要求1～7中任意一项所述高分辨率高光谱计算成像方法的步骤，或者所述微处理器与神经网络加速处理器共同被编程或配置以执行权利要求1～7中任意一项所述高分辨率高光谱计算成像方法的步骤。
  9. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有被编程或配置以执行权利要求1～7中任意一项所述高分辨率高光谱计算成像方法的计算机程序。
PCT/CN2021/101763 2021-04-13 2021-06-23 一种高分辨率高光谱计算成像方法、系统及介质 WO2022217746A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/871,943 US20220366536A1 (en) 2021-04-13 2022-07-24 High-resolution hyperspectral computational imaging method and system and medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110395614.4A CN112801881B (zh) 2021-04-13 2021-04-13 一种高分辨率高光谱计算成像方法、系统及介质
CN202110395614.4 2021-04-13

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/871,943 Continuation US20220366536A1 (en) 2021-04-13 2022-07-24 High-resolution hyperspectral computational imaging method and system and medium

Publications (1)

Publication Number Publication Date
WO2022217746A1 true WO2022217746A1 (zh) 2022-10-20

Family

ID=75816951

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/101763 WO2022217746A1 (zh) 2021-04-13 2021-06-23 一种高分辨率高光谱计算成像方法、系统及介质

Country Status (3)

Country Link
US (1) US20220366536A1 (zh)
CN (1) CN112801881B (zh)
WO (1) WO2022217746A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115760670A (zh) * 2023-01-06 2023-03-07 之江实验室 基于网络隐式先验的无监督高光谱融合方法及装置

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801881B (zh) * 2021-04-13 2021-06-22 湖南大学 一种高分辨率高光谱计算成像方法、系统及介质
CN114998109B (zh) * 2022-08-03 2022-10-25 湖南大学 基于双rgb图像融合的高光谱成像方法、系统及介质
CN116188774B (zh) * 2022-12-13 2024-03-22 湖南大学 高光谱图像实例分割方法及建筑实例分割方法
CN116862819B (zh) * 2023-05-29 2024-04-09 武汉工程大学 基于液晶可调谐滤波器的近红外高光谱成像方法及系统
CN116433551B (zh) * 2023-06-13 2023-08-22 湖南大学 基于双光路rgb融合的高分辨率高光谱成像方法及装置
CN116993584A (zh) * 2023-06-21 2023-11-03 西北工业大学 一种面向多域图像的光谱跨域迁移超分重建方法
CN117765402B (zh) * 2024-02-21 2024-05-17 山东科技大学 一种基于注意力机制的高光谱图像匹配检测方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110322402A (zh) * 2019-04-30 2019-10-11 武汉理工大学 基于稠密混合注意力网络的医学图像超分辨率重建方法
CN111192193A (zh) * 2019-11-26 2020-05-22 西安电子科技大学 一种基于1维-2维卷积神经网络高光谱单图超分辨方法
CN111223046A (zh) * 2019-11-20 2020-06-02 中国科学院遥感与数字地球研究所 一种图像超分辨率重建方法及装置
US20200302249A1 (en) * 2019-03-19 2020-09-24 Mitsubishi Electric Research Laboratories, Inc. Systems and Methods for Multi-Spectral Image Fusion Using Unrolled Projected Gradient Descent and Convolutinoal Neural Network
CN112561799A (zh) * 2020-12-21 2021-03-26 江西师范大学 一种红外图像超分辨率重建方法
CN112801881A (zh) * 2021-04-13 2021-05-14 湖南大学 一种高分辨率高光谱计算成像方法、系统及介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826160B (zh) * 2010-03-31 2012-11-14 北京航空航天大学 基于免疫进化策略的高光谱图像分类方法
US10832094B2 (en) * 2018-04-10 2020-11-10 International Business Machines Corporation Generating hyperspectral image database by machine learning and mapping of color images to hyperspectral domain
CN110458777B (zh) * 2019-08-05 2021-10-15 湖南大学 一种基于自适应秩校正的高光谱图像去噪方法、系统及介质
CN111160300B (zh) * 2019-12-31 2022-06-28 北京理工大学重庆创新中心 一种结合全局先验的深度学习高光谱图像显著性检测算法
CN111579506B (zh) * 2020-04-20 2021-04-09 湖南大学 基于深度学习的多摄像头高光谱成像方法、系统及介质
CN111678599B (zh) * 2020-07-07 2022-11-22 安徽大学 基于深度学习优化s-g滤波的激光光谱降噪方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200302249A1 (en) * 2019-03-19 2020-09-24 Mitsubishi Electric Research Laboratories, Inc. Systems and Methods for Multi-Spectral Image Fusion Using Unrolled Projected Gradient Descent and Convolutinoal Neural Network
CN110322402A (zh) * 2019-04-30 2019-10-11 武汉理工大学 基于稠密混合注意力网络的医学图像超分辨率重建方法
CN111223046A (zh) * 2019-11-20 2020-06-02 中国科学院遥感与数字地球研究所 一种图像超分辨率重建方法及装置
CN111192193A (zh) * 2019-11-26 2020-05-22 西安电子科技大学 一种基于1维-2维卷积神经网络高光谱单图超分辨方法
CN112561799A (zh) * 2020-12-21 2021-03-26 江西师范大学 一种红外图像超分辨率重建方法
CN112801881A (zh) * 2021-04-13 2021-05-14 湖南大学 一种高分辨率高光谱计算成像方法、系统及介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115760670A (zh) * 2023-01-06 2023-03-07 之江实验室 基于网络隐式先验的无监督高光谱融合方法及装置

Also Published As

Publication number Publication date
CN112801881B (zh) 2021-06-22
US20220366536A1 (en) 2022-11-17
CN112801881A (zh) 2021-05-14

Similar Documents

Publication Publication Date Title
WO2022217746A1 (zh) 一种高分辨率高光谱计算成像方法、系统及介质
Reehorst et al. Regularization by denoising: Clarifications and new interpretations
Arjomand Bigdeli et al. Deep mean-shift priors for image restoration
US11645835B2 (en) Hypercomplex deep learning methods, architectures, and apparatus for multimodal small, medium, and large-scale data representation, analysis, and applications
US20210365710A1 (en) Image processing method, apparatus, equipment, and storage medium
Patel et al. Latent space sparse and low-rank subspace clustering
US9858529B2 (en) Systems and methods for multi-task Bayesian optimization
CN111369487A (zh) 一种高光谱和多光谱图像融合方法、系统及介质
WO2024027095A1 (zh) 基于双rgb图像融合的高光谱成像方法、系统及介质
Lefkimmiatis et al. Nonlocal structure tensor functionals for image regularization
US11574198B2 (en) Apparatus and method with neural network implementation of domain adaptation
Ni et al. Example-driven manifold priors for image deconvolution
CN110969089B (zh) 噪声环境下的轻量级人脸识别系统及识别方法
CN115311187B (zh) 基于内部和外部先验的高光谱融合成像方法、系统及介质
CN113222998B (zh) 基于自监督低秩网络的半监督图像语义分割方法及装置
Maggu et al. Greedy deep transform learning
Seibert et al. Learning co-sparse analysis operators with separable structures
Altinel et al. Deep structured energy-based image inpainting
CN112270650B (zh) 基于稀疏自编码器的图像处理方法、系统、介质、设备
Su et al. High-Similarity-Pass Attention for Single Image Super-Resolution
Budiman et al. Adaptive convolutional ELM for concept drift handling in online stream data
US20230132630A1 (en) Apparatus and method with neural network training based on knowledge distillation
US11657282B2 (en) Efficient inferencing with fast pointwise convolution
Huang et al. Bayesian robust tucker decomposition for multiway data analysis
Yoshida et al. Kernelized Back-Projection Networks for Blind Super Resolution

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21936618

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21936618

Country of ref document: EP

Kind code of ref document: A1