CN112985601B - Long-focus spectral coding imaging system and method based on diffraction - Google Patents

Long-focus spectral coding imaging system and method based on diffraction Download PDF

Info

Publication number
CN112985601B
CN112985601B CN202110157367.4A CN202110157367A CN112985601B CN 112985601 B CN112985601 B CN 112985601B CN 202110157367 A CN202110157367 A CN 202110157367A CN 112985601 B CN112985601 B CN 112985601B
Authority
CN
China
Prior art keywords
diffraction element
wavelength
spectral
image
diffraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110157367.4A
Other languages
Chinese (zh)
Other versions
CN112985601A (en
Inventor
徐之海
胡海泉
周浩
李奇
冯华君
陈跃庭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202110157367.4A priority Critical patent/CN112985601B/en
Publication of CN112985601A publication Critical patent/CN112985601A/en
Application granted granted Critical
Publication of CN112985601B publication Critical patent/CN112985601B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/12—Generating the spectrum; Monochromators
    • G01J3/18—Generating the spectrum; Monochromators using diffraction elements, e.g. grating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Abstract

The invention discloses a diffraction-based tele spectral coding imaging system and method. The method comprises the following steps: 1) obtaining a coded diffraction element by using a constructive interference wavelength arrangement mode; 2) selecting the magnification gamma of the telescopic system, and calculating the internal parameters of the telescopic system to obtain the telescopic system; 3) constructing an imaging system for calibrating the point diffusion function, and performing a spectrum calibration experiment by using the imaging system for calibrating the point diffusion function to obtain point diffusion functions of different wave bands; 4) constructing a long-focus spectral coding imaging system, and performing a spectral image acquisition experiment by using the long-focus spectral coding imaging system to obtain spectral aliasing images of different wave bands; 5) and processing the spectrum aliasing image by utilizing a hyperspectral image reconstruction network algorithm to obtain hyperspectral images under different wave bands. The invention is beneficial to the effect of the reconstruction network algorithm and the processing of the diffraction element; the invention is beneficial to the increase of the system focal length and the protection of the diffraction element; the hyperspectral image is accurate and clear, the spectrum range is wide, and the robustness is high.

Description

Long-focus spectral coding imaging system and method based on diffraction
Technical Field
The invention relates to a spectral imaging system and a method in the technical field of spectral imaging, in particular to a diffraction-based long-focus spectral coding imaging system and a diffraction-based long-focus spectral coding imaging method.
Background
Spectral imaging is a novel technique for acquisition and processing of spectral data, image display and interpretation of analysis using single or multiple spectral channels. The method mainly aims to obtain a large number of target narrow-band continuous spectrum images and obtain almost continuous spectrum data of each pixel, and the existing spectral imaging technology is mainly applied to the fields of hyperspectral aviation and aerospace remote sensing.
The existing spectral imaging system mainly faces the following technical difficulties: first, the push-broom and swing-broom systems are too bulky and complex to meet the requirements of miniaturization and light weight in the aerospace field. Secondly, due to the limitation of the system, time is required for both push scanning and swing scanning, so that a spectral image at the same time cannot be obtained, and the timeliness is poor. Third, in actual detection, in order to improve the detection capability and increase the detection distance, the imaging system needs a long focal length.
Baek et al in 2017 proposed the first compact snapshot spectral imaging method, which captures spectral information using dispersion at the edges of the image by installing a prism in front of a single lens reflex camera. However, his method is based on the optimization of the conventional method, while prism splitting is difficult to quantify.
In 2019, Kim et al propose compact snapshot spectral imaging based on rotation diffraction, and compared with a snapshot-type hyperspectral system with a prism as a spectral device, a system with a diffraction element as the spectral device has more regular and clear spectral distinction. However, the structure of the diffraction element design is more complex, which is not beneficial to processing, and the processing error can affect the reconstruction effect of the image. Meanwhile, the system needs a large space and an excellent darkroom environment for calibrating the point spread function, the difficulty of calibrating the point spread function is improved, and the calibration accuracy is also influenced. Moreover, the spectrum range in the text is from 420nm to 660nm, and the spectrum band is not wide enough.
The diffraction element has the problems of small size and small light flux, which causes the system to decrease the frame rate of spectral imaging in order to increase the exposure time. Meanwhile, in the spectral imaging based on diffraction, only one diffraction element is used for imaging and light splitting, and no protective glass exists, so that the diffraction element is easily damaged in practical application.
Disclosure of Invention
Aiming at solving the technical problems of complex structure, low timeliness for acquiring spectral information, long focal length required for deep space detection, small size of a diffraction element in rotary diffraction, small light flux amount, complex structure, difficulty in processing, easiness in damage, space improvement of a reconstruction effect, limitation of point spread function calibration, insufficient wide spectral imaging range and the like of the traditional spectral imaging system, the invention adopts a diffraction-based long-focus spectral coding imaging system and method.
The invention aims to use a diffraction element additionally provided with a telescope system to carry out special coding to replace an optical lens and a light splitting element of a traditional spectral imaging system, simultaneously record the spectrum and the light intensity information thereof in a target scene on a two-dimensional focal plane within one exposure time, obtain the point spread function of each spectral band by calibrating the spectrum of the imaging system, and obtain the image and the spectral information of the target scene through image deconvolution and neural network iterative reconstruction.
In order to achieve the above purpose, the invention adopts the following technical scheme:
long-focus spectral coding imaging system based on diffraction
The system comprises a telescopic system, a coded diffraction element, a detector, an optical bracket and an optical guide rail; a telescopic system, a coded diffraction element and a detector are sequentially arranged on an optical guide rail, the telescopic system is connected with the optical guide rail, the coded diffraction element and the detector are respectively connected with the optical guide rail through respective optical supports, and the centers of the telescopic system, the coded diffraction element and the detector are all on the same horizontal plane.
The coded diffraction element is obtained by processing the diffraction element in a constructive interference wavelength arrangement mode;
the constructive interference wavelength arrangement mode specifically comprises the following steps:
selecting a desired wavelength range [ lambda ] in the wavelength range in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element;
then, the desired wavelength range [ lambda ] is measuredmin,λmax]Sampling at intervals by taking spectral resolution as an interval to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircle are the same and are sequentially decreased or increased progressively along the circumferential direction, the same sampling wavelength in the semicircle is a sector area, and the diameter of the sector area isThe other semicircle of the circle of L and the semicircle of the circle with the diameter of L are symmetrical about the center of the circle, and a constructive interference wavelength arrangement mode is formed.
Second, a long focus spectrum coding imaging method based on diffraction
The method comprises the following steps:
(1) obtaining a coded diffraction element by using a constructive interference wavelength arrangement mode;
(2) selecting the magnification gamma of the telescopic system, and calculating the intrinsic parameters of the telescopic system according to the encoded diffraction element and the magnification gamma to obtain the telescopic system;
(3) constructing an imaging system for calibrating the point diffusion function, and performing a spectrum calibration experiment by using the imaging system for calibrating the point diffusion function to obtain point diffusion functions of different wave bands;
(4) constructing a long-focus spectral coding imaging system by using an imaging system of a calibration point diffusion function and a telescopic system 8, and carrying out a spectral image acquisition experiment by using the long-focus spectral coding imaging system to obtain spectral aliasing images of different wave bands;
(5) and processing the spectrum aliasing image by utilizing a hyperspectral image reconstruction network algorithm according to the point spread functions of different wave bands to obtain hyperspectral images under different wave bands.
The step (1) is specifically as follows:
(1.1) selecting a desired wavelength range [ lambda ] among wavelength ranges in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element;
(1.2) converting the desired wavelength range [ lambda ]min,λmax]Sampling at intervals by taking the spectral resolution as an interval to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircles are the same and are sequentially decreased or increased in a descending or increasing manner along the circumferential direction, the same sampling wavelength in the semicircles is a sector area, and the other semicircle of the circle with the diameter of L and the semicircle of the circle with the diameter of L are symmetrical about the center of a circle to form a constructive interference wavelength arrangement mode;
and (1.3) encoding the diffraction element by utilizing the principle of diffraction imaging according to the constructive interference wavelength arrangement mode to obtain the encoded diffraction element.
The step (2) is specifically as follows:
(2.1) selecting the magnification gamma of the telescopic system in the magnification range of the telescopic system;
(2.2) calculating intrinsic parameters of a telescopic system by utilizing a thin lens imaging relation according to the magnification factor gamma and the diffraction element coded in the step (1), wherein the intrinsic parameters comprise the focal length f of the objective lens1', eyepiece focal length f'2Exit pupil distance D, objective lens diameter D1Diameter D of eyepiece2And an exit pupil diameter D'.
The step (3) is specifically as follows:
(3.1) the imaging system for calibrating the point spread function comprises a point light source, a collimator, an optical filter, a coded diffraction element, a detector, an optical bracket and an optical guide rail; the point light source, the collimator, the optical filter, the coded diffraction element and the detector are arranged on the optical guide rail in sequence, the centers of the point light source, the collimator, the optical filter, the coded diffraction element and the detector are all on the same horizontal plane, and the point light source, the collimator, the optical filter, the coded diffraction element and the detector are connected with the optical guide rail through respective optical brackets;
and (3.2) replacing optical filters with different wave bands, and performing a spectrum calibration experiment by using an imaging system for calibrating the point spread function to obtain respective point spread functions of the different wave bands.
The step (4) is specifically as follows:
(4.1) replacing a point light source, a collimator and a light filter in an imaging system of the calibration point diffusion function with a telescopic system to form a long-focus spectrum coding imaging system, wherein the telescopic system is connected with an optical guide rail;
and (4.2) acquiring the spectral image by using the long-focus spectral coding imaging system to complete a spectral image acquisition experiment, and acquiring a spectral aliasing image.
The step (5) is specifically as follows:
the iterative optimization of the spectrum aliasing image under the hyperspectral image reconstruction network algorithm is set through the following formula:
Figure BDA0002934239100000041
Figure BDA0002934239100000042
wherein, Iλ (l+1)Represents a spectrally aliased restored image, V, after the l +1 th iteration at wavelength λλ (l)Denotes the spectral optimization variable, # after the first iteration at the wavelength lambdaλRepresenting the point spread function at the wavelength lambda,
Figure BDA0002934239100000043
which represents a convolution operation, the operation of the convolution,
Figure BDA0002934239100000044
indicating that the image is restored by spectral aliasing after the l-th iteration at wavelength lambda,
Figure BDA0002934239100000045
represents the value of x when f (x) takes the minimum value;
Figure BDA0002934239100000046
which represents the square of the two-norm,
Figure BDA0002934239100000047
is a penalty factor, jλRepresents a spectrally aliased sub-image at wavelength λ, J represents a spectrally aliased image;
restoring the image by aliasing the spectrum after the (l + 1) th iteration under the wavelength lambda
Figure BDA0002934239100000048
As a hyperspectral image of the wave band of the wavelength lambda, the image is restored by spectrum aliasing after the 1 +1 iteration under the wavelength lambda
Figure BDA0002934239100000049
Comprises the following steps:
Figure BDA00029342391000000410
where ε is the step size of the gradient descent, T denotes the transposition of the matrix, Iλ (l)Representing spectrally aliased restored images, V, after the first iteration at wavelength λλ (l)Representing the intermediate variable after the l iteration at the wavelength lambda;
wherein the spectral optimization variable V after the first iteration at the wavelength lambdaλ (l)The acquisition specifically comprises the following steps: restoring the image by aliasing the spectrum after the first iteration under the wavelength lambda
Figure BDA00029342391000000411
Inputting neural network training, and taking the output of the neural network as a spectrum optimization variable V after the first iteration under the wavelength lambdaλ (l)。
The arrangement mode of the constructive interference wavelengths in the step (1.2) is set by the following formula:
Figure BDA00029342391000000412
where λ (r, θ) represents the wavelength of the diffraction element in polar coordinates of the polar diameter r and the polar angle θ, r represents the polar diameter, θ represents the polar angle, λminAnd λmaxIs the desired wavelength range [ lambda ]min,λmax]L is the diameter of the diffraction element, round [. cndot. ]]Representing a rounding integer operation.
The step (1.3) of encoding the diffraction element by using the principle of diffraction imaging is mainly set by the following formula:
Δφ＝2πn
Figure BDA00029342391000000413
where Δ φ represents the incidence of light from different positions on the diffractive elementOptical path difference h converged at the center of the image plane0Is the initial height, Δ η, of the diffraction elementλIs the difference between the refractive index of the diffractive element and the refractive index of air at the wavelength λ; f is the focal length of the diffractive element; n is an integer, Δ h (r, θ) is a change in height of the diffraction element at r pole angle of the radial dimension, h (r, θ) is a height of the diffraction element after encoding at r pole angle of the radial dimension, r is a radial dimension, and θ is a pole angle.
The invention has the beneficial effects that:
the whole structure of the method only needs a small telescopic system, a diffraction element with codes and a color CMOS detector, and the miniaturization and the light weight of the spectral imaging system are realized to a great extent.
The method of the invention can complete the acquisition of the image and the spectrum information within one exposure time, and has timeliness.
According to the invention, discrete sampling wavelengths are distributed on the semi-circle, and compared with the sampling wavelengths distributed on a 120-degree fan or a 90-degree fan, the difference between the calibrated psfs is larger, so that the reconstruction effect is more favorably improved; meanwhile, the design of the central symmetry of the discrete sampling wavelength is also beneficial to the robustness of reconstruction.
The discretized distribution of the invention eliminates the continuously changed slight difference, and is more beneficial to the processing and manufacturing of the coding diffraction element.
The invention ensures that the clear aperture is large enough, and simultaneously applies the telescope system to lead the focal length of the whole spectral imaging system to be changed into gamma times, thereby having great advantages in deep space detection. In addition, the advent of telescopic systems, which are a great protection for the diffractive element, has resulted in an improvement in the robustness and robustness of the overall imaging system.
The maximum spectrum range of the whole system can be 380nm-900nm, the spectrum range is wider, and the application range is wider.
Drawings
FIG. 1 is an overall schematic diagram of a tele spectrally encoded imaging system.
FIG. 2 is a schematic diagram of an imaging system for scaling a point spread function.
FIG. 3 is a schematic flow chart of the framework of the present invention.
FIG. 4 is a schematic flow chart of the present invention.
FIG. 5 is a schematic diagram of a discrete constructive interference wavelength arrangement.
FIG. 6 is a schematic view of an encoded diffraction element.
FIG. 7 is a calibrated point spread function image.
Fig. 8 is a schematic diagram of a neural network structure of the Unet.
FIG. 9 is a schematic diagram of a hyperspectral image reconstruction network algorithm.
Fig. 10 is an exemplary diagram of a spectrally aliased restored image.
In the figure: 1. the system comprises a coded diffraction element 2, a detector 3, a point light source 4, a collimator 5, a filter 6, an optical bracket 7, an optical guide rail 8 and a telescopic system.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Aiming at solving the technical problems of complex structure, low timeliness for acquiring spectral information, long focal length required for deep space detection, small size of a diffraction element in rotary diffraction, small light flux amount, complex structure, difficulty in processing, easiness in damage, space improvement of a reconstruction effect, limitation of point spread function calibration, insufficient wide spectral imaging range and the like of the traditional spectral imaging system, the invention adopts a diffraction-based long-focus spectral coding imaging system and method. The method comprises the steps of utilizing a diffraction element additionally provided with a telescope system to carry out special coding to replace an optical lens and a light splitting element of a traditional spectrum imaging system, simultaneously recording the spectrum and the light intensity information thereof in a target scene on a two-dimensional focal plane within one-time exposure time, obtaining a point spread function of each spectral band by calibrating the spectrum of the imaging system, and obtaining the image and the spectrum information of the target scene through image deconvolution and neural network iterative reconstruction.
As shown in fig. 1, the system comprises a telescopic system 8, a coded diffraction element 1, a detector 2, an optical support 6 and an optical guide 7; a telescopic system 8, a coded diffraction element 1 and a detector 2 are sequentially arranged on an optical guide rail 7, the telescopic system 8 is connected with the optical guide rail 7, the coded diffraction element 1 and the detector 2 are respectively connected with the optical guide rail 7 through respective optical supports 6, and the centers of the telescopic system 8, the coded diffraction element 1 and the detector 2 are all on the same horizontal plane.
The coded diffraction element 1 is obtained by processing the diffraction element in a constructive interference wavelength arrangement mode;
the constructive interference wavelength arrangement mode is as follows:
selecting a desired wavelength range [ lambda ] in the wavelength range in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element;
then, the desired wavelength range [ lambda ] is measuredmin,λmax]Sampling at intervals by taking spectral resolution as intervals to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircle are the same and are sequentially decreased or increased progressively along the circumferential direction, the same sampling wavelengths in the semicircle are in a sector area, and the other semicircle of the circle with the diameter of L and the semicircle of the circle with the diameter of L are symmetrical about the center of a circle to form a constructive interference wavelength arrangement mode.
As shown in fig. 3 and 4, the method includes the steps of:
(1) obtaining a coded diffraction element by using a constructive interference wavelength arrangement mode;
the step (1) is specifically as follows:
(1.1) selecting a desired wavelength range [ lambda ] among wavelength ranges in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element; the wavelength range in which the diffraction element operates is 380-900 nm.
(1.2) converting the desired wavelength range [ lambda ]min,λmax]Sampling at intervals by taking spectral resolution as an interval to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircle are the same and are sequentially decreased or increased progressively along the circumferential direction, the same sampling wavelength in the semicircle is a sector area, and the circle with the diameter of LThe other semicircle of (b) is symmetrical with a semicircle of a circle with a diameter of L about the center of the circle to form a constructive interference wavelength arrangement mode, as shown in FIG. 5;
the constructive interference wavelength arrangement mode in the step (1.2) is set by the following formula:
Figure BDA0002934239100000071
where λ (r, θ) represents the wavelength of the diffraction element in polar coordinates of the polar diameter r and the polar angle θ, r represents the polar diameter, θ represents the polar angle, λminAnd λmaxIs the desired wavelength range [ lambda ]min,λmax]L is the diameter of the diffraction element, round [. cndot. ]]Representing a rounding integer operation.
(1.3) according to the constructive interference wavelength arrangement mode, the diffraction element is coded by using the principle of diffraction imaging, and a coded diffraction element 1 is obtained, as shown in fig. 6.
The encoding of the diffraction element in the step (1.3) by using the principle of diffraction imaging is mainly set by the following formula:
Δφ＝2πn
Figure BDA0002934239100000072
wherein, delta phi represents the optical path difference of the light rays which are incident from different positions of the diffraction element and converged at the central position of the image plane, h0Is the initial height of the diffraction element, i.e. the height of the diffraction element before encoding; Δ ηλIs the difference between the refractive index of the diffractive element and the refractive index of air at the wavelength λ, which will vary with the wavelength of the incident light; f is the focal length of the diffractive element; n is an integer, Δ h (r, θ) is a change in height of the diffraction element at r pole angle of the radial dimension, h (r, θ) is a height of the diffraction element after encoding at r pole angle of the radial dimension, r is a radial dimension, and θ is a pole angle.
(2) Selecting the magnification gamma of the telescopic system 8, and calculating the intrinsic parameters of the telescopic system 8 according to the encoded diffraction element and the magnification gamma to obtain the telescopic system 8;
the step (2) is specifically as follows:
(2.1) selecting the magnification gamma of the telescopic system 8 in the magnification range of the telescopic system 8, wherein the magnification gamma is not more than 8 times;
(2.2) calculating intrinsic parameters of the telescopic system 8 by utilizing the thin lens imaging relation according to the magnification factor gamma and the diffraction element coded in the step (1), wherein the intrinsic parameters comprise the focal length f of the objective lens1', eyepiece focal length f'2Exit pupil distance D, objective lens diameter D1Diameter D of eyepiece2And an exit pupil diameter D'.
In a specific implementation, the telescopic system 8 is a keplerian telescopic system. Determining the focal length f of the objective lens of the telescopic system 8 by the magnification gamma1'and eyepiece focal length f'2Ocular focal length f 'is typically chosen'210mm or 20mm, using magnification gamma and focal length f of objective lens1'and eyepiece focal length f'2Calculating the pupil distance D and the objective diameter D1Diameter D of eyepiece2And exit pupil diameter D':
Γ＝f1'/f'2
D'＝L
d＝f'2(f1'+f'2)/(f1'+2f'2)
D1＝(f1'+f'2)L/d＝(Γ+2)L
D2＝D1/Γ
where L is the diameter of the encoded diffraction element.
(3) As shown in fig. 2, an imaging system for calibrating the point spread function is set up, and a spectrum calibration experiment is performed by using the imaging system for calibrating the point spread function to obtain point spread functions of different wave bands;
the step (3) is specifically as follows:
(3.1) the imaging system for calibrating the point spread function comprises a point light source 3, a collimator 4, a filter 5, a coded diffraction element 1, a detector 2, an optical bracket 6 and an optical guide rail 7; the point light source 3, the collimator 4, the optical filter 5, the coded diffraction element 1 and the detector 2 are sequentially placed on the optical guide rail 7, the centers of the point light source 3, the collimator 4, the optical filter 5, the coded diffraction element 1 and the detector 2 are all on the same horizontal plane, and the point light source 3, the collimator 4, the optical filter 5, the coded diffraction element 1 and the detector 2 are respectively connected with the optical guide rail 7 through respective optical supports 6;
(3.2) replacing the optical filters 5 with different wave bands, and carrying out a spectrum calibration experiment by using an imaging system for calibrating the point spread function to obtain respective point spread functions of different wave bands. The optical filter 5 is used for screening a required waveband, the length of the required waveband is 10nm, the wavelength interval screened by the optical filter 5 is 10nm, a point spread function image is displayed on the detector 2, and respective point spread functions of different wavebands are obtained, as shown in fig. 7. As seen in fig. 7, the point spread functions corresponding to different bands are similar in shape, but differ in tilt angle and attitude. After the operation of the step 1), point spread functions corresponding to different wave bands can be realized, and further spectral information of different wavelengths can be obtained;
the imaging system of the calibration point spread function is formed by improving a point light source device, wherein the point light source device comprises a point light source 3, a collimator 4 and a detector 2; the point light source 3, the collimator 4 and the detector 2 are sequentially placed on the optical guide rail 7, the point light source 3, the collimator 4 and the optical guide rail 7 are respectively connected through respective optical brackets 6, the centers of the point light source 3, the collimator 4 and the detector 2 are all on the same horizontal plane, and in specific implementation, the detector is a CMOS sensor; the point light source 1 forms a light spot on the detector 2 after passing through the collimator 4, and the size of the light spot is smaller than one pixel, and the pixel size is 5 μm by 5 μm.
(4) As shown in fig. 1, a tele spectral coding imaging system is constructed by using an imaging system of a calibration point diffusion function and a telescopic system 8, and a spectral image acquisition experiment is performed by using the tele spectral coding imaging system to obtain spectrally aliased images of different wave bands;
the step (4) is specifically as follows:
(4.1) replacing a point light source 3, a collimator 4 and a filter 5 in an imaging system of the calibration point diffusion function with a telescopic system 8 to form a long-focus spectrum coding imaging system, wherein the telescopic system 8 is connected with an optical guide rail 7, and the center of the telescopic system, the center of the coded diffraction element 1 and the center of the detector 2 are on the same horizontal plane;
and (4.2) acquiring the spectral image by using the long-focus spectral coding imaging system to complete a spectral image acquisition experiment, and acquiring the spectral aliasing images of different wave bands.
(5) According to the point spread functions of different wave bands, the hyperspectral image reconstruction network algorithm is used for processing the spectrum aliasing image to obtain hyperspectral images under different wave bands, as shown in fig. 10. The hyperspectral image is a spectral image with spectral resolution of 10 nm.
As shown in fig. 9, the step (5) specifically includes:
the iterative optimization of the spectrum aliasing image under the hyperspectral image reconstruction network algorithm is set through the following formula:
Figure BDA0002934239100000091
Figure BDA0002934239100000092
when the iteration number l is 0, the image is restored by aliasing the spectrum after the 0 th iteration under the wavelength lambda
Figure BDA0002934239100000093
For the initially assigned image, the assignment principle is: under the wavelength lambda, an aliasing image under a channel with the maximum spectral response in the three channels of the R channel, the G channel and the B channel is used as an image of the initial assignment; i isλ (l+1)Represents a spectrally aliased restored image, V, after the l +1 th iteration at wavelength λλ (l)Denotes the spectral optimization variable, # after the first iteration at the wavelength lambdaλRepresenting the point spread function at the wavelength lambda,
Figure BDA0002934239100000094
which represents a convolution operation, the operation of the convolution,
Figure BDA00029342391000000911
indicating that the image is restored by spectral aliasing after the l-th iteration at wavelength lambda,
Figure BDA0002934239100000095
represents the value of x when f (x) takes the minimum value;
Figure BDA0002934239100000096
which represents the square of a two-norm, i.e. the sum of the squares of the absolute values of each element,
Figure BDA0002934239100000097
is a penalty factor, jλThe spectrally aliased sub-image at wavelength λ is shown, J represents the spectrally aliased image, and in a specific implementation l is 4. Wherein the wavelength λ is in the end point of a certain band; when the wavelength lambda is at the left end point of the waveband, the wavelength lambda should be in the last waveband of the waveband, and the point spread function corresponding to the wavelength lambda is the point spread function of the last waveband of the waveband; when the wavelength lambda is at the right end point of the waveband, the wavelength lambda belongs to the waveband, and the point spread function corresponding to the wavelength lambda is the point spread function of the waveband;
restoring the image by aliasing the spectrum after the (l + 1) th iteration under the wavelength lambda
Figure BDA0002934239100000098
The optimal solution is taken as the spectrum aliasing image under the wavelength lambda, namely the hyperspectral image of the wave band where the wavelength lambda is, and the spectrum aliasing image is restored after the l +1 iteration under the wavelength lambda
Figure BDA0002934239100000099
Comprises the following steps:
Figure BDA00029342391000000910
where ε is the step size of the gradient descent, T denotes the transposition of the matrix, Iλ (l)Indicating that the image is restored by spectral aliasing after the l-th iteration at wavelength lambda,Vλ (l)representing the intermediate variable after the l iteration at the wavelength lambda;
wherein the spectral optimization variable V after the first iteration at the wavelength lambdaλ (l)The acquisition specifically comprises the following steps: restoring the image by aliasing the spectrum after the first iteration under the wavelength lambda
Figure BDA0002934239100000101
Inputting neural network training, and taking the output of the neural network as a spectrum optimization variable V after the first iteration under the wavelength lambdaλ (l)(ii) a The specific structure of the neural network is a pre-trained Unet network structure by utilizing a hyperspectral data set.
In specific implementation, a formula for iterative optimization of a spectral aliasing image in a hyperspectral image reconstruction network algorithm is obtained through the following steps:
the point spread functions under different wave bands are obtained through calibration, and the problem of reconstructing the spectrum aliasing restoration image from the obtained point spread functions can be resolved into two problems: 1) the image restored by spectral aliasing of a specific waveband is a non-blind restoration problem of a known fixed fuzzy kernel. 2) Aiming at the full-wave band signal, the problem of disassembling the superposed image is solved.
Firstly, establishing an objective function of spectrum restoration:
Figure BDA0002934239100000102
Figure BDA0002934239100000103
wherein the content of the first and second substances,
Figure BDA0002934239100000104
indicating an optimal spectrally aliased restored image at wavelength lambda,
Figure BDA0002934239100000105
means that when f (x) takes the minimum value, the value of x,
Figure BDA0002934239100000106
representing the square of a two-norm, i.e. the sum of the squares of the absolute values of each element, R () is a priori of the spectral image, unlike the explicit image priors, where implicit priors involve more broadly and are also more flexible, R (I)λ) Representing the image IλA priori processing is performed, jλDenotes a spectrally aliased sub-image at wavelength λ, J denotes a spectrally aliased image, #λRepresenting the point spread function at the wavelength lambda,
Figure BDA0002934239100000107
represents the convolution of IλRepresenting a spectrally aliased restored image at wavelength lambda,
Figure BDA0002934239100000108
for data fidelity terms, R (I)λ) The second term is a regularization term.
Then introducing a spectrum optimization variable V by a Half-Quadratic Splitting Half Quadratic Splitting (HQS) methodλConverting the spectrum-restored objective function solution problem into a spectrum-restored objective function and spectrum optimization variable convex function optimization problem:
Figure BDA0002934239100000109
wherein the content of the first and second substances,
Figure BDA00029342391000001010
an objective function representing spectral restitution with the addition of spectral optimization variables,
Figure BDA00029342391000001011
is the penalty factor for the number of bits in the block,
Figure BDA00029342391000001012
indicating an optimal spectrally aliased restored image at wavelength lambda,
Figure BDA00029342391000001013
representing a restored image spectrally optimized at wavelength lambda, which, in the ideal case,
Figure BDA0002934239100000111
denotes the value of x and y when f (x, y) takes the minimum value,
Figure BDA0002934239100000112
representing the square of a two-norm, i.e. the sum of the squares of the absolute values of each element, R (V)λ) Representation pair image VλAnd carrying out prior processing.
The formula of the l +1 th iteration optimization of the target function of spectrum restoration added with the spectrum optimization variable is as follows:
Figure BDA0002934239100000113
Figure BDA0002934239100000114
wherein, Iλ (l+1)Represents the spectrally aliased restored image after the l +1 th iteration at wavelength λ, Iλ (l)Representing a spectrally aliased restored image after the l-th iteration at wavelength lambda,
Figure BDA00029342391000001113
is a penalty factor, Vλ (l+1)Denotes the spectral optimization variable, V, after the l +1 th iteration at wavelength λλ (l)Represents the spectrally optimized variable after the l-th iteration at wavelength lambda,
Figure BDA0002934239100000115
means that when f (x) takes the minimum value, the value of x,
Figure BDA0002934239100000116
which represents the square of a two-norm, i.e. the sum of the squares of the absolute values of each element,
Figure BDA0002934239100000117
representing the optimized variable for the spectrum after the l-th iteration at wavelength lambda
Figure BDA0002934239100000118
And carrying out prior processing.
Spectral optimization variable V after the first iteration at wavelength λλ (l)The acquisition specifically comprises the following steps: restoring the image by aliasing the spectrum after the first iteration under the wavelength lambda
Figure BDA0002934239100000119
Inputting neural network training, and taking the output of the neural network as a spectrum optimization variable V after the first iteration under the wavelength lambdaλ (l)(ii) a The specific structure of the neural network is a pre-trained Unet network structure using a hyperspectral data set, as shown in FIG. 8.
Finally, solving the objective function of spectrum restoration added with the spectrum optimization variable after the (l + 1) th iteration by using a gradient descent method to obtain the spectrum aliasing restoration image after the (l + 1) th iteration under the wavelength lambda
Figure BDA00029342391000001110
Figure BDA00029342391000001111
Where ε is the step size of the gradient descent, T denotes the transposition of the matrix, Iλ (l)Representing spectrally aliased restored images, V, after the first iteration at wavelength λλ (l)Represents the spectrally optimized variable after the l-th iteration at wavelength lambda,
Figure BDA00029342391000001112
the image of the initial assignment.

Claims (6)

1. A diffraction-based tele spectrally encoded imaging system, comprising: comprises a telescope system (8), a coded diffraction element (1), a detector (2), an optical bracket (6) and an optical guide rail (7); a telescope system (8), a coded diffraction element (1) and a detector (2) are sequentially arranged on an optical guide rail (7), the telescope system (8) is connected with the optical guide rail (7), the coded diffraction element (1) and the detector (2) are respectively connected with the optical guide rail (7) through respective optical supports (6), and the centers of the telescope system (8), the coded diffraction element (1) and the detector (2) are all on the same horizontal plane;
the coded diffraction element (1) is obtained by processing the diffraction element in a constructive interference wavelength arrangement mode;
the constructive interference wavelength arrangement mode specifically comprises the following steps:
selecting a desired wavelength range [ lambda ] in the wavelength range in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element;
then, the desired wavelength range [ lambda ] is measuredmin,λmax]Sampling at intervals by taking spectral resolution as intervals to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircle are the same and are sequentially decreased or increased progressively along the circumferential direction, the same sampling wavelengths in the semicircle are in a sector area, and the other semicircle of the circle with the diameter of L and the semicircle of the circle with the diameter of L are symmetrical about the center of a circle to form a constructive interference wavelength arrangement mode.
2. A diffraction-based tele spectral encoding imaging method is characterized in that: the method comprises the following steps:
(1) obtaining a coded diffraction element by using a constructive interference wavelength arrangement mode;
(2) selecting a magnification gamma of the telescopic system (8), and calculating the intrinsic parameters of the telescopic system (8) according to the encoded diffraction element and the magnification gamma to obtain the telescopic system (8);
(3) constructing an imaging system for calibrating the point diffusion function, and performing a spectrum calibration experiment by using the imaging system for calibrating the point diffusion function to obtain point diffusion functions of different wave bands;
(4) constructing a long-focus spectral coding imaging system by using an imaging system of a calibration point diffusion function and a telescopic system (8), and carrying out a spectral image acquisition experiment by using the long-focus spectral coding imaging system to obtain spectral aliasing images of different wave bands;
(5) processing the spectrum aliasing image by utilizing a hyperspectral image reconstruction network algorithm according to point spread functions of different wave bands to obtain hyperspectral images under different wave bands;
the step (1) is specifically as follows:
(1.1) selecting a desired wavelength range [ lambda ] among wavelength ranges in which the diffraction element operatesmin,λmax]The focal length f of the diffractive element, the diameter L of the diffractive element;
(1.2) converting the desired wavelength range [ lambda ]min,λmax]Sampling at intervals by taking the spectral resolution as an interval to obtain discrete sampling wavelengths, wherein the discrete sampling wavelengths are uniformly distributed on a semicircle of a circle with the diameter of L, the sampling wavelengths on one radius in the semicircles are the same and are sequentially decreased or increased in a descending or increasing manner along the circumferential direction, the same sampling wavelength in the semicircles is a sector area, and the other semicircle of the circle with the diameter of L and the semicircle of the circle with the diameter of L are symmetrical about the center of a circle to form a constructive interference wavelength arrangement mode;
(1.3) according to the constructive interference wavelength arrangement mode, encoding the diffraction element by using the diffraction imaging principle to obtain an encoded diffraction element (1);
the step (3) is specifically as follows:
(3.1) the imaging system for calibrating the point spread function comprises a point light source (3), a collimator (4), a filter (5), a coded diffraction element (1), a detector (2), an optical bracket (6) and an optical guide rail (7); the point light source (3), the collimator (4), the optical filter (5), the coded diffraction element (1) and the detector (2) are sequentially arranged on the optical guide rail (7), the centers of the point light source (3), the collimator (4), the optical filter (5), the coded diffraction element (1) and the coded detector (2) are all on the same horizontal plane, and the point light source (3), the collimator (4), the optical filter (5), the coded diffraction element (1) and the coded detector (2) are respectively connected with the optical guide rail (7) through respective optical supports (6);
(3.2) replacing optical filters (5) with different wave bands, and performing a spectrum calibration experiment by using an imaging system for calibrating point spread functions to obtain respective point spread functions of different wave bands;
the step (4) is specifically as follows:
(4.1) replacing a point light source (3), a collimator tube (4) and a filter (5) in an imaging system of the calibration point diffusion function with a telescopic system (8) to form a long-focus spectral coding imaging system, wherein the telescopic system (8) is connected with an optical guide rail (7);
and (4.2) acquiring the spectral image by using the long-focus spectral coding imaging system to complete a spectral image acquisition experiment, and acquiring a spectral aliasing image.
3. The diffraction-based tele spectrally encoded imaging method of claim 2, wherein: the step (2) is specifically as follows:
(2.1) selecting the magnification gamma of the telescopic system (8) in the magnification range of the telescopic system (8);
(2.2) calculating intrinsic parameters of the telescopic system (8) by utilizing the thin lens imaging relation according to the magnification factor gamma and the diffraction element coded in the step (1), wherein the intrinsic parameters comprise the focal length f of the objective lens1' ocular focal length f2', exit pupil distance D, objective lens diameter D1Diameter D of eyepiece2And an exit pupil diameter D'.
4. The diffraction-based tele spectrally encoded imaging method of claim 2, wherein: the step (5) is specifically as follows:
the iterative optimization of the spectrum aliasing image under the hyperspectral image reconstruction network algorithm is set through the following formula:
Figure FDA0003397997110000031
Figure FDA0003397997110000032
wherein, Iλ (l+1)Represents a spectrally aliased restored image, V, after the l +1 th iteration at wavelength λλ (l)Denotes the spectral optimization variable, # after the first iteration at the wavelength lambdaλRepresenting the point spread function at the wavelength lambda,
Figure FDA0003397997110000033
which represents a convolution operation, the operation of the convolution,
Figure FDA0003397997110000034
indicating that the image is restored by spectral aliasing after the l-th iteration at wavelength lambda,
Figure FDA0003397997110000035
represents the value of x when f (x) takes the minimum value;
Figure FDA0003397997110000036
which represents the square of the two-norm,
Figure FDA0003397997110000037
is a penalty factor, jλRepresents a spectrally aliased sub-image at wavelength λ, J represents a spectrally aliased image;
restoring the image by aliasing the spectrum after the (l + 1) th iteration under the wavelength lambda
Figure FDA0003397997110000038
As a hyperspectral image of the wave band of the wavelength lambda, the image is restored by spectrum aliasing after the 1 +1 iteration under the wavelength lambda
Figure FDA0003397997110000039
Comprises the following steps:
Figure FDA00033979971100000310
where ε is the step size of the gradient descent, T denotes the transposition of the matrix, Iλ (l)Representing spectrally aliased restored images, V, after the first iteration at wavelength λλ (l)Representing the intermediate variable after the ith iteration at the wavelength lambda, and I represents an identity matrix;
wherein the spectral optimization variable V after the first iteration at the wavelength lambdal (l)The acquisition specifically comprises the following steps: restoring the image by aliasing the spectrum after the first iteration under the wavelength lambda
Figure FDA00033979971100000311
Inputting neural network training, and taking the output of the neural network as a spectrum optimization variable V after the first iteration under the wavelength lambdaλ (l)。
5. The diffraction-based tele spectrally encoded imaging method of claim 2, wherein: the arrangement mode of the constructive interference wavelengths in the step (1.2) is set by the following formula:
Figure FDA00033979971100000312
where λ (r, θ) represents the wavelength of the diffraction element in polar coordinates of the polar diameter r and the polar angle θ, r represents the polar diameter, θ represents the polar angle, λminAnd λmaxIs the desired wavelength range [ lambda ]min,λmax]L is the diameter of the diffraction element, round [. cndot. ]]Representing a rounding integer operation.
6. The diffraction-based tele spectrally encoded imaging method of claim 2, wherein: the step (1.3) of encoding the diffraction element by using the principle of diffraction imaging is mainly set by the following formula:
Δφ＝2πn
Figure FDA00033979971100000313
wherein, delta phi represents the optical path difference of the light rays which are incident from different positions of the diffraction element and converged at the central position of the image plane, h0Is the initial height of the diffraction element, Δ hlIs the difference between the refractive index of the diffractive element and the refractive index of air at the wavelength λ; f is the focal length of the diffractive element; n is an integer, Δ h (r, θ) is a change in height of the diffraction element at r pole diameter and θ pole angle, h (r, θ) is a height of the diffraction element after encoding at r pole diameter and θ pole angle, r is the pole diameter, θ is the pole angle, and λ (r, θ) is the wavelength of the diffraction element in the polar coordinates of r pole diameter and θ pole angle.
CN202110157367.4A 2021-02-04 2021-02-04 Long-focus spectral coding imaging system and method based on diffraction Active CN112985601B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110157367.4A CN112985601B (en) 2021-02-04 2021-02-04 Long-focus spectral coding imaging system and method based on diffraction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110157367.4A CN112985601B (en) 2021-02-04 2021-02-04 Long-focus spectral coding imaging system and method based on diffraction

Publications (2)

Publication Number Publication Date
CN112985601A CN112985601A (en) 2021-06-18
CN112985601B true CN112985601B (en) 2022-02-11

Family

ID=76347193

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110157367.4A Active CN112985601B (en) 2021-02-04 2021-02-04 Long-focus spectral coding imaging system and method based on diffraction

Country Status (1)

Country Link
CN (1) CN112985601B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113847986B (en) * 2021-10-11 2022-08-30 浙江大学 Co-optical-axis dual-image-plane spectral coding imaging system and method based on diffraction

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3274673A4 (en) * 2015-03-24 2019-05-01 University of Utah Research Foundation Imaging device with image dispersing to create a spatially coded image
CN105676331B (en) * 2016-03-24 2018-03-20 电子科技大学 Angle tuning filters chip arrays and its method
WO2018142295A1 (en) * 2017-02-03 2018-08-09 Gamaya Sa Wide-angle computational imaging spectroscopy method and apparatus
CN111415303B (en) * 2020-02-14 2022-09-13 清华大学 Zone plate coding aperture imaging method and device based on deep learning
CN111458023B (en) * 2020-04-02 2021-05-25 华南理工大学 High-speed large-dynamic-range spectral measurement method and system based on radio frequency coding

Also Published As

Publication number Publication date
CN112985601A (en) 2021-06-18

Similar Documents

Publication Publication Date Title
CN112985600B (en) Spectrum coding imaging system and method based on diffraction
CN104457708B (en) A kind of compact multispectral camera
Lantz et al. SNIFS: a wideband integral field spectrograph with microlens arrays
US7894058B2 (en) Single-lens computed tomography imaging spectrometer and method of capturing spatial and spectral information
EP2495540A2 (en) Design of filter modules for aperture-coded, multiplexed imaging systems
CN106017676A (en) Infrared imaging spectral measurement system based on gradual filter
US20230177655A1 (en) System and method for digital optical aberration correction and spectral imaging
US9343491B2 (en) Spectral imaging sensors and methods
Wang et al. Light-guide snapshot imaging spectrometer for remote sensing applications
CN112985601B (en) Long-focus spectral coding imaging system and method based on diffraction
CN105548032A (en) Compact high-resolution wide-view-field spectral imaging system
CN113155284B (en) Refraction-diffraction mixed spectral coding imaging system and method
Hubold et al. Ultra-compact micro-optical system for multispectral imaging
CN113847986B (en) Co-optical-axis dual-image-plane spectral coding imaging system and method based on diffraction
Xue et al. Compact, UAV-mounted hyperspectral imaging system with automatic geometric distortion rectification
CN108896179B (en) DMD space dimension coding symmetric Offner dispersion medium wave infrared spectrum imaging device
CN109781260B (en) Ultra-compact snapshot type polarization spectrum imaging detection device and detection method
Fisher et al. Survey and analysis of fore-optics for hyperspectral imaging systems
Dittrich et al. Measurement principle and arrangement for the determination of spectral channel-specific angle dependencies for multispectral resolving filter-on-chip CMOS cameras
Yang Compact static infrared broadband snapshot imaging spectrometer
CN109239914B (en) Imaging method for realizing high space bandwidth product
CN109884776B (en) Large-view-field, low-distortion and high-spectrum optical system based on pixel-level optical filter
CN109839190B (en) Snapshot type hyperspectral imaging device
CN113390508B (en) Spectrum-space resolution image quality optimized short wave infrared imaging method and device
Zhou et al. A broadband spherical prism imaging spectrometer based on a single integrated module

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant