WO2021135074A1 - 一种快照型解色散模糊的高光谱成像方法 - Google Patents

一种快照型解色散模糊的高光谱成像方法 Download PDF

Info

Publication number
WO2021135074A1
WO2021135074A1 PCT/CN2020/095002 CN2020095002W WO2021135074A1 WO 2021135074 A1 WO2021135074 A1 WO 2021135074A1 CN 2020095002 W CN2020095002 W CN 2020095002W WO 2021135074 A1 WO2021135074 A1 WO 2021135074A1
Authority
WO
WIPO (PCT)
Prior art keywords
dispersion
image
matrix
spectral
wavelength
Prior art date
Application number
PCT/CN2020/095002
Other languages
English (en)
French (fr)
Inventor
曹汛
黄烨
华夏
李晓雯
Original Assignee
南京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京大学 filed Critical 南京大学
Priority to US17/757,367 priority Critical patent/US20230021358A1/en
Publication of WO2021135074A1 publication Critical patent/WO2021135074A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/12—Generating the spectrum; Monochromators
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J2003/283—Investigating the spectrum computer-interfaced
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G06T2207/10036—Multispectral image; Hyperspectral image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30128—Food products
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the invention belongs to the field of spectral imaging, and in particular relates to a snapshot-type hyperspectral imaging method for dispersive blurring.
  • Hyperspectral imaging has broad application prospects in the fields of automatic segmentation and matching of materials, material identification, and geological material monitoring. Therefore, it is very important in the fields of physical chemistry, agriculture, and even military.
  • traditional hyperspectral imaging systems are usually complex in structure, require customized hardware for optical path settings, and are bulky and expensive.
  • professional engineering skills are needed to process and adjust these devices in real time. For ordinary users, such a system cannot afford the expensive price, is inconvenient to use, and is not practical.
  • the purpose of the present invention is to provide a snapshot-type hyperspectral imaging method for dispersive blurring, which can use a simple, portable, and low-cost system to achieve high-precision hyperspectral imaging.
  • a snapshot-type hyperspectral imaging method for de-dispersion blurring The device used includes a device and a sensor that can generate planar dispersion.
  • the sensor is used to collect images after the device that can generate planar dispersion.
  • the method includes the following steps: S1, Select the reference wavelength, calibrate the dispersion of the reference wavelength, and select the center wavelength; S2, estimate the relative dispersion of all reconstructed wavelengths and the center wavelength; S3, generate a dispersion matrix based on the estimation result of step S2, and use the spectral response curve of the sensor to generate Spectral response matrix; S4, collecting a dispersion-blurred image; S5, using the dispersion matrix and spectral response matrix generated in step S3 to de-dispersion-blur the image collected in step S4 to obtain spectral data aligned with each channel image; S6, step S5 The obtained aligned spectral data is projected into the imaging space, the foreground image is extracted by the threshold method, and the dispersion image obtained in step
  • the method for calibrating the dispersion of the reference wavelength is: when calibrating the dispersion of a certain wavelength, place a filter of that wavelength in front of the light source to allow only light of that wavelength to pass, and then mark the Imaging location.
  • the method for estimating the relative dispersion of all reconstructed wavelengths and the center wavelength is: according to the calibrated dispersion position of the reference wavelength and the selected center wavelength, obtain the relative dispersion of all the reference wavelengths and the center wavelength, and interpolate Obtain the relative dispersion of other reconstructed wavelengths and the center wavelength.
  • the image of the center wavelength channel has no translation.
  • the relative translation of the images of other wavelength channels is consistent with the result obtained in step S2.
  • the matrix S and the spatial hyperspectral data i can construct the dispersion matrix ⁇ .
  • step S5 is: solving Where i aligned is the obtained spectral data of the alignment of each channel, ⁇ is the dispersion matrix, ⁇ is the spectral response matrix, i is the spatial hyperspectral data, j is the actually collected image with dispersion, Means to find the gradient in the plane, Represents the gradient of the spectral direction, ⁇ 1 and ⁇ 1 are the coefficients of the constraint terms respectively.
  • step S6 is:
  • ⁇ i aligned to get the dispersively blurred image set an appropriate threshold to extract the foreground image i front , the size is x ⁇ y, and it is a binary matrix, x and y represent the horizontal and vertical dimensions of the plane image; in each pixel as the center, the dispersion of the blurred image S4 are taken in accordance with direction of the dispersion sample, as a priori values for each channel of the spectra of the pixels, the finally obtained spectrum priori i prior to all the pixels, the size of x ⁇ y ⁇ ⁇ ; Solution Among them, i recons is the final hyperspectral data, W is the weight matrix, which means to constrain the foreground images of all channels, its size is x ⁇ y ⁇ , and each dimension is i front , ⁇ , ⁇ 2 , ⁇ 2 Is the adjustment factor.
  • the hyperspectral imaging method of the present invention only needs a simple sensor and a device that can generate planar dispersion, including but not limited to a super lens, a prism, etc., to perform low-cost hyperspectral video imaging.
  • the scanning system does not need to separate the wavelengths of the measurement, so it is faster and simplifies the imaging model and system. It is convenient to use software to realize the reconstruction of hyperspectral data, and use strong a priori as a constraint , To ensure the accuracy of the reconstruction results.
  • Fig. 1 is a schematic diagram of the structure of the device used in the present invention, in which, 1-sample, 2-device capable of generating planar dispersion, and 3-sensor.
  • Figure 2 is a flow chart of the method of the present invention.
  • Figure 3 is the result of hyperspectral reconstruction of simulated data using the method of the present invention, (a) original image; (b) simulated dispersion image; (c) reconstruction result; (d) randomly selected original spectrum of the first pixel The comparison between the curve and the reconstructed spectral curve, (e) the comparison between the original spectral curve of the second pixel point randomly selected and the reconstructed spectral curve.
  • Fig. 4 is a dispersion map taken by the device of the present invention and the reconstruction result using the method of the present invention, (a) the dispersion map taken; (b) the reconstruction result; (c) the comparison between the reconstructed 22 channel spectral curve and the original spectral curve
  • the horizontal axis is the wavelength in nm
  • the vertical axis is the normalized pixel value.
  • the imaging device used in this embodiment is shown in Figure 1. It includes a device 2 that can produce planar dispersion and a sensor 3.
  • the sample 1 is placed in front of the device 2 that can produce planar dispersion, and the sensor 3 is placed behind the device 2 that can produce planar dispersion.
  • the sensor 3 collects the sample image after the dispersion of the device.
  • the device 2 that can generate planar dispersion includes but is not limited to a super lens, a prism, and the like.
  • the specific steps are as follows :
  • the reference wavelengths can be selected as 450nm, 500nm, 550nm, 600nm, 650nm, and calibrate the dispersion of these reference wavelengths.
  • the light source Place the filter of this wavelength before, and only allow the light of this wavelength to pass, and then mark the imaging position of the reference object to obtain the center coordinates of the reference object of all wavelengths. These coordinates fall approximately on a straight line.
  • Select the center wavelength, 550nm can be selected as the center wavelength.
  • the center wavelength is selected as 550nm
  • the relative displacement of other reference wavelengths relative to the imaging center of 550nm can be obtained, such as 650nm relative displacement + 13 pixels, 600nm relative Displacement +6 pixels, 500nm relative displacement -4 pixels, 450nm relative displacement -8 pixels, interpolate to obtain the relative dispersion of other reconstruction wavelengths and the center wavelength, such as 550-600nm relative displacement of 6 pixels
  • you can calibrate the middle five pixel displacement representative Wavelength value of 550nm, 550+50/6nm, 550+50/6*2nm, 550+50/6*3nm, 550+50/6*4nm, 550+50/6*5nm, 550+50/6 *6 600nm, of course, the more wavelengths that are calibrated, the more wavelength channels are reconstructed, and the more difficult it is to solve.
  • the center wavelength channel is assumed that a 0, b 0, c 0 , d 0, e 0, then the relative displacement of the pixel channel values -1 b -1, c - 1 , d -1 , e -1 , 0, the value of the channel with a relative displacement of +1 pixel is 0, a 1 , b 1 , c 1 , d 1 , and so on, the spectrum matrix after dispersion can be obtained.
  • S According to the relationship between S and i, ⁇ can be constructed.
  • S4 use the same light source as S1 to collect the image. Due to the device 2 that can produce planar dispersion, the image has in-plane dispersion and blur.
  • i aligned is the obtained spectral data of each channel aligned
  • j is the actually collected dispersion image
  • Means to find the gradient in the plane Indicates that the gradient of the spectrum direction is obtained
  • ⁇ 1 and ⁇ 1 are the coefficients of the constraint terms respectively.
  • the calculation of the gradient can be expressed by a matrix operation.
  • the first term of the equation is a data term to reduce the mean square error between the model result and the actual data.
  • the last two terms are a priori terms, and the second term is a commonly used variable differential operator.
  • the third term guarantees cross-band alignment, and the coefficients of the latter two need to be actually adjusted.
  • the ADMM algorithm iteratively optimizes each variable separately, namely:
  • u 1 and u 2 are Langrangian multipliers, and i k+1 is solved with only l 2 terms, which can be solved by the conjugate gradient method.
  • the introduced variables z 1 and z 2 are solved by the soft threshold operator, as follows:
  • the Langrangian multipliers u 1 and u 2 are updated using the gradient ascent method.
  • S6 first project the aligned spectral data obtained in S5 to the imaging space (RGB space or grayscale space, depending on whether the sensor is color or grayscale), that is, perform matrix operation ⁇ i aligned to obtain a dispersively blurred image, and then Set a suitable threshold to extract the foreground image i front , the size is x ⁇ y, and it is a binary matrix, and then take each pixel as the center, and sample the dispersion blurred image taken by S4 according to the dispersion direction, because each point is sampled It can be considered as the result of less channel aliasing, so it can be used as a priori of the spectral value of each channel of the pixel, and finally the spectral a priori i prior of all pixels is obtained, the size is x ⁇ y ⁇ , and the final solution Among them, i recons is the final hyperspectral data, W is the weight matrix, which means to constrain the foreground images of all channels, its size is x ⁇ y ⁇ , and each dimension is
  • the first two items of this optimization problem can be considered as data items.
  • ⁇ represents the credibility of a strong prior and can be set to a higher value.
  • the third and fourth items are the same as above.
  • the value of ⁇ is 1e -3 ⁇ 1e -1
  • the value of ⁇ is about 1e-5, which needs to be adjusted according to the actual situation.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Spectrometry And Color Measurement (AREA)

Abstract

一种快照型解色散模糊的高光谱成像方法，步骤为：S1，选择参考波长，标定参考波长的色散，并选定中心波长；S2，估算所有重建波长与中心波长的相对色散；S3，生成色散矩阵，利用传感器3的光谱响应曲线，生成光谱响应矩阵；S4，采集色散模糊的图像；S5，利用S3生成的色散矩阵、光谱响应矩阵，对S4采集的图像解色散模糊，得到各个通道图像对齐的光谱数据；S6，将S5得到的对齐的光谱数据投影到成像空间，通过阈值法提取前景图像，对S4得到的色散图像采样，作为前景图像像素值的强先验约束，重建精确的空间高光谱数据，实现高光谱成像。成像方法利用简单、低成本的系统，实现精准地高光谱成像。

Description

一种快照型解色散模糊的高光谱成像方法 技术领域
本发明属于光谱成像领域，尤其涉及一种快照型解色散模糊的高光谱成像方法。
背景技术
高光谱成像在自动分割匹配材料、材料识别、地质材料监测等方向具有广泛的应用前景，因此在物理化学、农业、甚至军事等领域都十分重要。但是，传统的高光谱成像系统通常结构复杂，需要定制的硬件进行光路设置，笨重且昂贵。而且，使用的时候需要专业的工程技能才能实时处理、调整这些器件，这样的系统对于普通用户来说，既负担不起昂贵的价格，也不方便使用，更不具实用性。
传统的扫描式系统利用滤波器对每个波长分离测量，过程十分缓慢、并且光谱分辨率受所用滤波器的类型和数量所限制。一些其他的光谱成像技术，如编码孔径快照光谱成像(CASSI)，系统使用编码孔径掩膜来编码、使用棱镜产生色散，利用编码、色散后的压缩图像重建光谱信息，加上由于重建技术利用了空间不变的色散模型，需要光路的准直设置，系统也相对复杂。
发明内容
针对以上现有技术中存在的缺陷，本发明的目的在于提供一种快照型解色散模糊的高光谱成像方法，该方法可以使用简单、轻便、低成本的系统，实现高精度的高光谱成像。
实现本发明目的的技术解决方案为：
一种快照型解色散模糊的高光谱成像方法，所用的装置包括可产生平面色散的器件和传感器，传感器用于采集经可产生平面色散的器件色散后的图像，该方法包括如下步骤：S1，选择参考波长，标定参考波长的色散，并选定中心波长；S2，估算所有重建波长与中心波长的相对色散；S3，根据步骤S2的估算结果，生成色散矩阵，利用传感器的光谱响应曲线，生成光谱响应矩阵；S4，采集色散模糊的图像；S5，利用步骤S3生成的色散矩阵和光谱响应矩阵，对步骤S4采集的图像解色散模糊，得到各个通道图像对齐的光谱数据；S6，将步骤S5得到的对齐的光谱数据投影到成像空间，通过阈值法提取前景图像，对步骤S4得到的色散图像采样，作为前景图像像素值的强先验约束，重建精确的空间高光谱数据，实现高光谱成像。
进一步地，所述步骤S1中，标定参考波长的色散的方法为：在标定某波长的色散情况时，在光源前放置该波长的滤波片，只允许该波长的光通过，随后标记参考物的成像位置。
进一步地，所述步骤S2中，估算所有重建波长与中心波长的相对色散的方法为：根据标定的参考波长的色散位置和选定的中心波长，得到所有参考波长与中心波长的相对色散，插值得到其他重建波长与中心波长的相对色散。
进一步地，步骤S3中，生成色散矩阵的方法为：令空间高光谱数据为i，其大小为xyΛ×1，其中x、y表示平面的图像横向尺寸和纵向尺寸，Λ为光谱通道数，令色散矩阵为Ω，其大小为xyΛ×xyΛ，得到色散后的光谱矩阵S＝Ωi，中心波长通道的图像无平移，其他波长通道的图像的相对平移与步骤S2所得结果一致，根据色散后的光谱矩阵S与空间高光谱数据i，可构建出色散矩阵Ω。
进一步地，步骤S3中，生成光谱响应矩阵的方法为：令光谱响应矩阵为Φ，其大小为xyN×xyΛ，x、y表示平面的图像横向尺寸和纵向尺寸，Λ为光谱通道数，N为3或1，将空间高光谱数据i投影到对应的色彩空间得到R＝Φi，大小为xyN×1，查询传感器的响应曲线并采样φ＝N×Λ，得到传感器在重建光谱波长处的响应系数，根据i、R、φ可构建出光谱响应矩阵Φ。
进一步地，步骤S5的具体做法为：求解
Figure PCTCN2020095002-appb-000001
Figure PCTCN2020095002-appb-000002
其中i aligned为求得的各通道对齐的光谱数据，Ω为色散矩阵，Φ为光谱响应矩阵，i为空间高光谱数据，j为实际采集到的带色散的图像，
Figure PCTCN2020095002-appb-000003
表示求平面内梯度，
Figure PCTCN2020095002-appb-000004
表示对光谱方向求梯度，α 1、β 1分别为约束项的系数。
进一步地，步骤S6的具体做法为：
计算Φi aligned得到解色散模糊的图像，设定合适阈值提取前景图像i front，大小为x×y，并且为二值矩阵，x、y表示平面的图像横向尺寸和纵向尺寸；以每个像素点为中心，分别对S4拍摄的色散模糊图像按照色散方向采样，作为该像素点的各通道光谱值的先验，最终得到所有像素的光谱先验i prior，大小为x×y×Λ；求解
Figure PCTCN2020095002-appb-000005
Figure PCTCN2020095002-appb-000006
其中i recons为最后得到的高光谱数据，W为权值矩阵，表示对所有通道的前景图像作约束，其大小为x×y×Λ，每一维是i front，γ、α 2、β 2为各项调节系数。
本发明的显著优点在于：
(1)本发明的高光谱成像方法只需要简单的传感器和一个可产生平面色散的器件，包括但不局限于超透镜、棱镜等，就可进行低成本的高光谱视频成像。
(2)利用本发明的方法，无需扫描式系统各波长分离测量因而更加快速，简化成像模型和系统，可方便地只利用软件即可实现高光谱数据重建，并且利用强先验做约束的方法，保证重建结果的精确。
附图说明
图1是本发明采用的装置结构示意图，其中，1-样本，2-可产生平面色散的器件，3-传感器。
图2是本发明方法的流程图。
图3是采用本发明方法对仿真数据进行高光谱重建的结果，(a)原始图像；(b)仿真的色散图像；(c)重建结果；(d)随机选取第一个像素点的原始光谱曲线和重建光谱曲线的对比，(e)随机选取第二个像素点的原始光谱曲线和重建光谱曲线的对比。
图4是采用本发明装置拍摄的色散图及采用本发明方法的重建结果，(a)拍摄的色散图；(b)重建结果；(c)重建的22个通道的光谱曲线与原始光谱曲线对比图，横轴为波长，单位nm，纵轴为归一化后的像素值。
具体实施方式
本实施例采用的成像装置如图1所示，包括可产生平面色散的器件2和传感器3，样本1放置于可产生平面色散的器件2前方，传感器3放置于可产生平面色散的器件2后方，传感器3采集经过器件色散后的样本图像。其中，可产生平面色散的器件2包括但不局限于超透镜、棱镜等。
参见图2，本发明基于色散的高光谱成像方法，首先利用提出的色散模型，即j＝ΩΦi，其中j为传感器采集到的的色散图像，Ω为光谱响应矩阵，Φ为空间色散矩阵，i为空间高光谱数据，以及梯度稀疏约束和各通道数据对齐的约束，得到对齐的空间光谱数据，解空间色散模糊，然后利用采样的强先验约束像素值 恢复精确的光谱数据值，具体步骤如下：
S1，选择参考波长，假设光源为450-650nm的白光，可选择参考波长为450nm、500nm、550nm、600nm、650nm，标定这些参考波长处的色散情况，在标定某波长的色散情况时，在光源前放置该波长的滤波片，只允许该波长的光通过，随后标记参考物的成像位置，得到所有波长的参考物的中心坐标，这些坐标近似落于一条直线。选定中心波长，可选定550nm为中心波长。
S2，根据S1的结果，估算所有重建波长与中心波长的相对色散：选定了中心波长为550nm，可得到其他参考波长相对于550nm成像中心的相对位移，如650nm相对位移+13像素，600nm相对位移+6像素，500nm相对位移-4像素，450nm相对位移-8像素，插值得到其他重建波长与中心波长的相对色散，如550-600nm相对位移6像素，即可标定出中间五个像素位移代表的波长值，即550nm、550+50/6nm、550+50/6*2nm、550+50/6*3nm、550+50/6*4nm、550+50/6*5nm、550+50/6*6＝600nm，当然标定出越多的波长代表重建波长通道越多，也代表了求解难度的增大。
S3，根据S2的结果，生成色散矩阵：令空间高光谱数据为i，其大小为xyΛ×1，其中x、y表示平面的图像尺寸，Λ为光谱通道数，原始的数据为x×y×Λ，将其先按列再按波长通道重塑为列向量。令色散矩阵为Ω，其大小为xyΛ×xyΛ，运算得到色散后的光谱向量S＝Ωi，中心波长通道的图像无平移，其他波长通道的图像的相对平移与S2所得结果一致，以一维数据举例(假设图像尺寸1×5)，假设中心波长通道的值为a 0、b 0、c 0、d 0、e 0，那么相对位移为-1像素的通道的值为b -1、c -1、d -1、e -1、0，相对位移为+1像素的通道的值为0、a 1、b 1、c 1、d 1，以此类推可得到色散后的光谱矩阵，将其先按列再按波长通道重塑为列向量得到S。根据S与i的关系，可构建出Ω。生成响应矩阵的方法为：令响应矩阵为Φ，其大小为xyN×xyΛ，N为3或1(取决于传感器是彩色的或是灰度的)，将空间光谱数据投影到对应的色彩空间得到R＝Φi，大小为xyN×1，查询传感器的响应曲线φ＝N×Λ，根据i、R、φ的关系可构建出响应矩阵Φ。
S4，利用同S1一致的光源，采集图像，由于可产生平面色散的器件2，图像发生平面内色散，并产生模糊。
S5，利用S3生成的色散矩阵、光谱响应矩阵，对S4采集的图像解色散模糊，得到各个通道图像对齐的光谱数据，具体地：
求解
Figure PCTCN2020095002-appb-000007
其中i aligned为求得的各通道对齐的光谱数据，j为实际采集到的色散图像，
Figure PCTCN2020095002-appb-000008
表示求平面内梯度，
Figure PCTCN2020095002-appb-000009
表示对光谱方向求梯度，α 1、β 1分别为约束项的系数。求梯度的运算均可用矩阵运算表示，方程的第一项为数据项，减小模型结果与实际数据的均方差，后两项为先验项，第二项是常用的变微分算子，减小平面内梯度伪影，第三项保证跨波段的对齐，后两项的系数需要进行实际调整。此优化问题可利用ADMM算法求解，将上述问题分裂为三个子问题：
Figure PCTCN2020095002-appb-000010
则目标优化问题转化内：
Figure PCTCN2020095002-appb-000011
Figure PCTCN2020095002-appb-000012
ADMM算法迭代地对每个变量分别进行优化，即：
Figure PCTCN2020095002-appb-000013
Figure PCTCN2020095002-appb-000014
Figure PCTCN2020095002-appb-000015
Figure PCTCN2020095002-appb-000016
Figure PCTCN2020095002-appb-000017
其中，u 1和u 2是朗格朗日乘子，i k+1求解只有l 2项，可用共轭梯度法求解，引入的变量z 1和z 2通过软阈值算子求解，如下：
Figure PCTCN2020095002-appb-000018
Figure PCTCN2020095002-appb-000019
Figure PCTCN2020095002-appb-000020
而朗格朗日乘子u 1和u 2利用梯度上升法进行更新。
S6，首先将S5得到的对齐的光谱数据投影到成像空间(RGB空间或灰度空间，取决于传感器是彩色的或是灰度的)，即进行矩阵运算Φi aligned得到解色散 模糊的图像，然后设定合适阈值提取前景图像i front，大小为x×y，并且为二值矩阵，接着以每个像素点为中心，分别对S4拍摄的色散模糊图像按照色散方向采样，因为采样的每个点可认为是较少通道混叠后的结果，所以可以作为该像素点的各通道光谱值的先验，最终得到所有像素的光谱先验i prior，大小为x×y×Λ，最后求解
Figure PCTCN2020095002-appb-000021
Figure PCTCN2020095002-appb-000022
其中i recons为最后得到的高光谱数据，W为权值矩阵，表示对所有通道的前景图像作约束，其大小为x×y×Λ，每一维是i front，γ、α 2、β 2为各项调节系数。此优化问题前两项可认为是数据项，γ表示强先验的可信度，可以设置较高的值，第三项和第四项同上，一般β的取值1e -3～1e -1，而α的取值在1e-5左右，需要根据实际情况调整。

Claims (7)

  1. 一种快照型解色散模糊的高光谱成像方法，所用的装置包括可产生平面色散的器件和传感器，传感器用于采集经可产生平面色散的器件色散后的图像，其特征在于，该方法包括如下步骤：
    S1，选择参考波长，标定参考波长的色散，并选定中心波长；
    S2，估算所有重建波长与中心波长的相对色散；
    S3，根据步骤S2的估算结果，生成色散矩阵，利用传感器的光谱响应曲线，生成光谱响应矩阵；
    S4，采集色散模糊的图像；
    S5，利用步骤S3生成的色散矩阵和光谱响应矩阵，对步骤S4采集的图像解色散模糊，得到各个通道图像对齐的光谱数据；
    S6，将步骤S5得到的对齐的光谱数据投影到成像空间，通过阈值法提取前景图像，对步骤S4得到的色散图像采样，作为前景图像像素值的强先验约束，重建精确的空间高光谱数据，实现高光谱成像。
  2. 根据权利要求1所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，所述步骤S1中，标定参考波长的色散的方法为：在标定某波长的色散情况时，在光源前放置该波长的滤波片，只允许该波长的光通过，随后标记参考物的成像位置。
  3. 根据权利要求1所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，所述步骤S2中，估算所有重建波长与中心波长的相对色散的方法为：根据标定的参考波长的色散位置和选定的中心波长，得到所有参考波长与中心波长的相对色散，插值得到其他重建波长与中心波长的相对色散。
  4. 根据权利要求1所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，步骤S3中，生成色散矩阵的方法为：令空间高光谱数据为i，其大小为xyΛ×1，其中x、y表示平面的图像横向尺寸和纵向尺寸，Λ为光谱通道数，令色散矩阵为Ω，其大小为xyΛ×xyΛ，得到色散后的光谱矩阵S＝Ωi，中心波长通道的图像无平移，其他波长通道的图像的相对平移与步骤S2所得结果一致，根据色散后的光谱矩阵S与空间高光谱数据i，可构建出色散矩阵Ω。
  5. 根据权利要求1所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，步骤S3中，生成光谱响应矩阵的方法为：令光谱响应矩阵为Φ，其大小为xyN×xyΛ，x、y表示平面的图像横向尺寸和纵向尺寸，Λ为光谱通道数，N 为3或1，将空间高光谱数据i投影到对应的色彩空间得到R＝Φi，大小为xyN×1，查询传感器的响应曲线并采样φ＝N×Λ，得到传感器在重建光谱波长处的响应系数，根据i、R、φ可构建出光谱响应矩阵Φ。
  6. 根据权利要求1所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，步骤S5的具体做法为：
    求解
    Figure PCTCN2020095002-appb-100001
    其中i aligned为求得的各通道对齐的光谱数据，Ω为色散矩阵，Φ为光谱响应矩阵，i为空间高光谱数据，j为实际采集到的带色散的图像，
    Figure PCTCN2020095002-appb-100002
    表示求平面内梯度，
    Figure PCTCN2020095002-appb-100003
    表示对光谱方向求梯度，α 1、β 1分别为约束项的系数。
  7. 根据权利要求6所述的一种快照型解色散模糊的高光谱成像方法，其特征在于，步骤S6的具体做法为：
    计算Φi aligned得到解色散模糊的图像，设定合适阈值提取前景图像i front，大小为x×y，并且为二值矩阵，x、y表示平面的图像横向尺寸和纵向尺寸；
    以每个像素点为中心，分别对S4拍摄的色散模糊图像按照色散方向采样，作为该像素点的各通道光谱值的先验，最终得到所有像素的光谱先验i prior，大小为x×y×Λ；
    求解
    Figure PCTCN2020095002-appb-100004
    Figure PCTCN2020095002-appb-100005
    其中i recons为最后得到的高光谱数据，W为权值矩阵，表示对所有通道的前景图像作约束，其大小为x×y×Λ，每一维是i front，γ、α 2、β 2为各项调节系数。
PCT/CN2020/095002 2020-01-03 2020-06-09 一种快照型解色散模糊的高光谱成像方法 WO2021135074A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/757,367 US20230021358A1 (en) 2020-01-03 2020-06-09 Snapshot hyperspectral imaging method with de-blurring dispersed images

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010006728.0 2020-01-03
CN202010006728.0A CN111174912B (zh) 2020-01-03 2020-01-03 一种快照型解色散模糊的高光谱成像方法

Publications (1)

Publication Number Publication Date
WO2021135074A1 true WO2021135074A1 (zh) 2021-07-08

Family

ID=70656056

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/095002 WO2021135074A1 (zh) 2020-01-03 2020-06-09 一种快照型解色散模糊的高光谱成像方法

Country Status (3)

Country Link
US (1) US20230021358A1 (zh)
CN (1) CN111174912B (zh)
WO (1) WO2021135074A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111174912B (zh) * 2020-01-03 2021-02-23 南京大学 一种快照型解色散模糊的高光谱成像方法
CN112465922A (zh) * 2020-10-09 2021-03-09 浙江大学 一种结合色差模糊成像与图像重建技术的高光谱成像系统
CN113008371B (zh) * 2021-03-05 2022-02-08 南京大学 一种深度学习解色散模糊的高光谱成像方法
CN112950736B (zh) * 2021-03-05 2024-04-09 南京大学 一种基于无监督网络解色散模糊的高光谱成像方法
CN115950837B (zh) * 2023-03-10 2023-05-23 湖南师范大学 基于即插即用先验的快照式光谱成像方法、系统及介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6965108B2 (en) * 2001-07-30 2005-11-15 Euro-Celtique, S.A. Method and apparatus for three dimensional imaging using infrared radiation
CN107421640A (zh) * 2017-08-29 2017-12-01 南京大学 基于色差扩大原理的多光谱光场成像系统及方法
CN108291800A (zh) * 2015-07-30 2018-07-17 科技创新动量基金（以色列）有限责任合伙公司 光谱成像方法和系统
CN108961392A (zh) * 2018-06-13 2018-12-07 清华大学深圳研究生院 一种三维样本的图像重构方法
CN109196333A (zh) * 2016-05-27 2019-01-11 威里利生命科学有限责任公司 用于4-d高光谱成像的系统和方法
US20190096049A1 (en) * 2017-09-27 2019-03-28 Korea Advanced Institute Of Science And Technology Method and Apparatus for Reconstructing Hyperspectral Image Using Artificial Intelligence
CN110501072A (zh) * 2019-08-26 2019-11-26 北京理工大学 一种基于张量低秩约束的快照式光谱成像系统的重构方法
CN111174912A (zh) * 2020-01-03 2020-05-19 南京大学 一种快照型解色散模糊的高光谱成像方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102855609B (zh) * 2012-07-30 2014-10-01 南京大学 集成高光谱数据和稀疏声纳数据的浅水水下地形构建方法
CN105338326B (zh) * 2015-11-26 2019-01-01 南京大学 一种嵌入式高空间高光谱分辨率视频采集系统
CN105842173B (zh) * 2016-06-06 2018-05-08 南京大学 一种高光谱材质鉴别方法
CN106791318B (zh) * 2016-12-30 2019-06-25 南京大学 一种便携式高光谱视频实时采集和处理装置及其方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6965108B2 (en) * 2001-07-30 2005-11-15 Euro-Celtique, S.A. Method and apparatus for three dimensional imaging using infrared radiation
CN108291800A (zh) * 2015-07-30 2018-07-17 科技创新动量基金（以色列）有限责任合伙公司 光谱成像方法和系统
CN109196333A (zh) * 2016-05-27 2019-01-11 威里利生命科学有限责任公司 用于4-d高光谱成像的系统和方法
CN107421640A (zh) * 2017-08-29 2017-12-01 南京大学 基于色差扩大原理的多光谱光场成像系统及方法
US20190096049A1 (en) * 2017-09-27 2019-03-28 Korea Advanced Institute Of Science And Technology Method and Apparatus for Reconstructing Hyperspectral Image Using Artificial Intelligence
CN108961392A (zh) * 2018-06-13 2018-12-07 清华大学深圳研究生院 一种三维样本的图像重构方法
CN110501072A (zh) * 2019-08-26 2019-11-26 北京理工大学 一种基于张量低秩约束的快照式光谱成像系统的重构方法
CN111174912A (zh) * 2020-01-03 2020-05-19 南京大学 一种快照型解色散模糊的高光谱成像方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG FENG, LUO JIAN-JUN, TANG XING-JIA, LI LI-BO, HU BIN-LIANG: "Super Resolution Optic Three-dimensional Imaging Based on Compressed Sensing", ACTA PHOTONICA SINICA, vol. 44, no. 3, 31 March 2015 (2015-03-31), XP055826054, DOI: 10.3788/gzxb20154403.0328001 *

Also Published As

Publication number Publication date
US20230021358A1 (en) 2023-01-26
CN111174912B (zh) 2021-02-23
CN111174912A (zh) 2020-05-19

Similar Documents

Publication Publication Date Title
WO2021135074A1 (zh) 一种快照型解色散模糊的高光谱成像方法
Arad et al. Sparse recovery of hyperspectral signal from natural RGB images
US10274420B2 (en) Compact multifunctional system for imaging spectroscopy
EP3832359B1 (en) Method and device for imaging of lensless hyperspectral image
He et al. Fast hyperspectral image recovery of dual-camera compressive hyperspectral imaging via non-iterative subspace-based fusion
CN109146787B (zh) 一种基于插值的双相机光谱成像系统的实时重建方法
EP3460427A1 (en) Method for reconstructing hyperspectral image using prism and system therefor
CN108288256A (zh) 一种多光谱马赛克图像复原方法
KR20190035476A (ko) 프리즘을 이용한 초분광 영상 재구성 방법 및 시스템
Jagatap et al. Sample efficient fourier ptychography for structured data
Zeng et al. Robust reconstruction with deep learning to handle model mismatch in lensless imaging
Saragadam et al. KRISM—Krylov subspace-based optical computing of hyperspectral images
Ramirez et al. Spectral image classification from multi-sensor compressive measurements
Shoeiby et al. PIRM2018 challenge on spectral image super-resolution: methods and results
Xu et al. Hyperspectral image reconstruction based on the fusion of diffracted rotation blurred and clear images
CN107170013B (zh) 一种rgb相机光谱响应曲线的标定方法
Qi et al. A super-resolution fusion video imaging spectrometer based on single-pixel camera
CN114022364A (zh) 基于光谱库优化学习的多光谱图像光谱超分方法和系统
Lv et al. Turn a silicon camera into an ingaas camera
Zhang et al. Learnable Reconstruction Methods from RGB Images to Hyperspectral Imaging: A Survey
Zhao et al. Coded aperture snapshot hyperspectral light field tomography
Yu et al. High Spectral Resolution Imaging Based on Dual-Camera System With Filter Wheel
Zhang et al. Sub-pixel Dispersion Model for Coded Aperture Snapshot Spectral Imaging
Chen et al. Hyperspectral Neural Radiance Fields
CN112950507B (zh) 基于深度学习提高散射环境下单像素彩色成像性能的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20909590

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20909590

Country of ref document: EP

Kind code of ref document: A1