CN109683343B - Design method of super-resolution imaging system - Google Patents

Abstract

The invention discloses a design method of a super-resolution imaging system. First, the detectors in the super-resolution imaging system are selected, the pixel size p is determined, and the focal length f of the optical lens is obtained; the super-resolution imaging system is determined according to requirements. Resolution increase by multiples

The performance factor η is obtained, and the F _# number of the optical lens in the super-resolution imaging system is obtained; the imaging signal-to-noise ratio SNR of the super-resolution imaging system is calculated, and the F _# number is verified to meet the requirements of the imaging signal-to-noise ratio index; if it meets the index requirements, then obtain the aperture D of the optical lens in the super-resolution imaging system, and then obtain the design parameters of the optical lens; then use the super-resolution imaging system to observe the target to be measured for many times, and form the image sequence y _k of the target to be measured; The image sequence y _k is substituted into the super-resolution reconstruction equation, and the super-resolution reconstruction equation is solved according to the minimum relative entropy, and the super-resolution imaging result is obtained. This method enables the system to give full play to the advantages of super-resolution imaging and achieve system-level optimization design.

Description

A design method of super-resolution imaging system

technical field

The invention relates to the technical field of photoelectric imaging systems, in particular to a design method of a super-resolution imaging system.

Background technique

Geometric resolution is an important performance index of photoelectric imaging systems. In traditional imaging systems, imaging geometric resolution is directly related to the focal length of the optical lens of the system and the pixel size of the imaging detector. The longer the focal length, the higher the resolution and the smaller the pixel size. higher rate. However, with the continuous pursuit of imaging quality, the development of high geometric resolution imaging systems is restricted by optical lenses and detectors, which are mainly reflected in: first, increasing the focal length increases the resolution and also increases the system volume and weight, which not only limits the application of the system, but also increases the manufacturing cycle and development cost of the system; secondly, due to the limitations of electronic technology and semiconductor technology, reducing the size of the detector pixel will greatly reduce the luminous flux of each pixel of the detector, seriously affect image quality.

With the rapid development of signal processing technology, super-resolution imaging technology provides a more direct solution to the above contradictions. It uses the sub-pixel non-redundant information between multiple frames of images to expand the spectral reconstruction of the detector. High-resolution images, breaking through the limitations of imaging devices on system resolution. The design of the super-resolution imaging system consists of two parts. One is the algorithm design. The super-resolution reconstruction algorithm is a mathematical inverse problem and has a strong ill-conditioned nature. In order to obtain high-precision reconstruction results, it is necessary to suppress the registration error and The propagation of calculation noise; the second is the optimal design of camera parameters. The super-resolution algorithm is closely related to the parameters of the optical system and detector. If only the algorithm is optimized without the design of the camera, then from the perspective of the super-resolution imaging system, the Only the local optimum of the super-resolution enhancement effect can be obtained, and the system-level optimal solution can be obtained only by constraining the design parameters of the optical system and detector around the essence and requirements of the super-resolution algorithm. In the design of traditional super-resolution imaging systems, the insufficient optimization of the reconstruction algorithm design for the inverse problem leads to the propagation and amplification of computational noise. In addition, the traditional design method does not consider the coupling relationship between the detector parameters, optical lens parameters, and reconstruction algorithm parameters. It makes it difficult for the reconstruction algorithm to achieve the desired performance.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a design method of a super-resolution imaging system, which fully considers the constraints of the ill-conditioned solution of the inverse problem, takes into account the reconstruction of image edge texture information and noise suppression, and simultaneously integrates the main parameters of the detector and the main optical lens. The parameters and algorithm performance are unified into the same design model, so that the system can give full play to the advantages of super-resolution imaging and achieve system-level optimization design.

The purpose of this invention is to realize through the following technical solutions:

A design method of a super-resolution imaging system, the method comprising:

其中，L表示成像距离，GSD表示像元分辨率；Step 1. Select the detector in the super-resolution imaging system, determine the pixel size p, and obtain the focal length f of the optical lens according to the following formula:

Among them, L represents the imaging distance, and GSD represents the pixel resolution;

再结合具体应用场景、安装尺寸、制造成本确定超分辨成像系统的性能因子η，并得到超分辨成像系统中光学镜头的F_#数；Step 2. Determine the resolution improvement ratio of the super-resolution imaging system according to the requirements

The performance factor η of the super-resolution imaging system is determined in combination with the specific application scene, installation size, and manufacturing cost, and the F _# number of the optical lens in the super-resolution imaging system is obtained;

Step 3, calculating and obtaining the imaging signal-to-noise ratio SNR of the super-resolution imaging system, and verifying whether the F _# number obtained in step 2 meets the requirements of the imaging signal-to-noise ratio index;

Step 4, if the obtained signal-to-noise ratio meets the index requirements, then according to the formula F _# =f/D, the aperture D of the optical lens in the super-resolution imaging system is obtained, and then the design parameters of the optical lens are obtained;

Step 5. If the obtained signal-to-noise ratio does not meet the index requirements, repeat the operations of steps 2 and 3 until the signal-to-noise ratio meets the index requirements, and determine the F _# number, and then determine the super-resolution imaging according to the formula F _# =f/D The aperture D of the optical lens in the system;

Step 6. Then use the super-resolution imaging system to observe the object to be measured multiple times, so that there are different sub-pixel displacements between each two observation images, and the results of the multiple observations constitute the image sequence y _k of the object to be measured. ;

Step 7. Substitute the image sequence y _k into the super-resolution reconstruction equation, establish a regularization model that adaptively changes according to the image gradient information, constrain the edge area and flat area of the observed image of the target to be measured, and solve according to the minimum relative entropy. The super-resolution reconstruction equation is used to obtain super-resolution imaging results.

It can be seen from the technical solutions provided by the present invention that the above method fully considers the constraints of the ill-conditioned solution of the inverse problem, and takes into account the reconstruction of image edge texture information and noise suppression; The performance is unified into the same design model to realize the decoupling between the camera hardware parameters and the algorithm parameters, so that the system can give full play to the advantages of super-resolution imaging, and then realize the optimal design of the system level.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic flowchart of a design method of a super-resolution imaging system according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of sub-pixel displacement provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. FIG. 1 is a schematic flowchart of a design method of a super-resolution imaging system provided by an embodiment of the present invention, and the method includes:

Step 1. Select the detector in the super-resolution imaging system, determine the pixel size p, and obtain the focal length f of the optical lens according to the following formula:

Among them, L represents the imaging distance, and GSD represents the pixel resolution.

再结合具体应用场景、安装尺寸、制造成本确定超分辨成像系统的性能因子η，并得到超分辨成像系统中光学镜头的F_#数；Step 2. Determine the resolution improvement ratio of the super-resolution imaging system according to the requirements

The performance factor η of the super-resolution imaging system is determined in combination with the specific application scene, installation size, and manufacturing cost, and the F _# number of the optical lens in the super-resolution imaging system is obtained;

的范围为

其中，

表示超分辨重建算法能实现的分辨率提高倍数最大值；In this step, the resolution is increased by a factor of

The range is

in,

Represents the maximum value of the resolution increase that can be achieved by the super-resolution reconstruction algorithm;

其中，ω_S表示探测器截止频率，ω_O表示光学截止频率；The performance factor η of the super-resolution imaging system is expressed as

where ω _S is the detector cut-off frequency, and ω _O is the optical cut-off frequency;

and satisfy the relationship

Among them, λ represents the wavelength of the incident light; p is the determined pixel size; F _# is the parameter of the optical lens in the super-resolution imaging system.

For example, an optical lens can usually be optimized as a diffraction-limited system, where the modulation transfer function of the optical lens is approximated as:

Among them, ω _O is the optical cut-off frequency, which represents the limit capability of the optical lens, and the relationship between ω _O and the diameter D of the optical system can be expressed as:

In the above formula, D represents the diameter of the aperture of the optical system, and λ represents the wavelength of the incident light.

The modulation transfer function of the detector is:

为探测器像元的相对宽度，与填充因子有关；σ表示探测器的角距，与探测器的截止频率ω_S之间是互为倒数的关系，ω_S也代表了探测器的极限能力。in,

is the relative width of the detector pixel, which is related to the filling factor; σ represents the angular distance of the detector, which is inversely related to the cut-off frequency ω _S of the detector, and ω _S also represents the limit capability of the detector.

则超分辨率成像系统的截止频率ω_S可用如下不等式表示：In a super-resolution imaging system, if the reconstruction algorithm can achieve a maximum resolution increase of

Then the cut-off frequency ω _S of the super-resolution imaging system can be expressed by the following inequality:

Express the performance of the imaging system as the ratio of the detector cutoff frequency _ωS to the optical cutoff frequency _ωO :

In the above formula, η is the performance factor of the super-resolution imaging system, further:

The above formula links the main parameters of the optical lens of the imaging system, the main parameters of the detector, and the parameters of the reconstruction algorithm, and realizes the overall optimization design of the parameters of the super-resolution imaging system.

Step 3, calculating and obtaining the imaging signal-to-noise ratio SNR of the super-resolution imaging system, and verifying whether the F _# number obtained in step 2 meets the requirements of the imaging signal-to-noise ratio index;

In this step, first calculate the imaging signal-to-noise ratio SNR according to the following formula:

Among them, E _img (λ) is the radiation energy collected by the lens at any position on the detector target surface, hυ is the single-photon energy, QE is the detector quantum efficiency, V _NSH is the shot noise, and V _floor is the detector noise floor Including charge transfer noise, dark current noise, readout noise and quantization noise, V _PRNU represents the response non-uniformity noise of the detector, and SNR _target represents the signal-to-noise ratio index that the super-resolution imaging system needs to achieve;

The relationship between the above E _img (λ) and the F _# number is expressed as:

Among them, T _opt is the transmittance of the optical system, L _λ (λ) is the entrance pupil radiance of the optical lens, and θ _tilt is the tilt angle of the optical axis;

Then, according to the above relationship, it is checked whether the F _# number obtained in step 2 meets the requirements of the imaging signal-to-noise ratio index.

Step 4, if the obtained signal-to-noise ratio meets the index requirements, then according to the formula F _# =f/D, the aperture D of the optical lens in the super-resolution imaging system is obtained, and then the design parameters of the optical lens are obtained;

Step 5. If the obtained signal-to-noise ratio does not meet the index requirements, repeat the operations of steps 2 and 3 until the signal-to-noise ratio meets the index requirements, and determine the F _# number, and then determine the super-resolution imaging according to the formula F _# =f/D The aperture D of the optical lens in the system;

Step 6. Then use the super-resolution imaging system to observe the object to be measured multiple times, so that there are different sub-pixel displacements between each two observation images, and the results of the multiple observations constitute the image sequence y _k of the object to be measured. ;

In this step, FIG. 2 is a schematic diagram of sub-pixel displacement provided by an embodiment of the present invention, and the sub-pixel displacement refers to: in the image coordinate system OXY, a point on the ground object is represented by a solid line in Figure B _The corresponding pixel O ₁ in the dashed line corresponds to the pixel O ₂ in Figure _A represented by the dotted line, and the number of pixels that differ from each other in the OX and OY directions is not an integer. If the affine transformation of Figure A and Figure B also has rotation in addition to translation, the two images need to be transformed to the same plane through the rotation transformation.

Step 7. Substitute the image sequence y _k into the super-resolution reconstruction equation, establish a regularization model that adaptively changes according to the image gradient information, constrain the edge area and flat area of the observed image of the target to be measured, and solve according to the minimum relative entropy. The super-resolution reconstruction equation is used to obtain super-resolution imaging results.

The specific implementation process of this step is as follows:

In this step, y _k represents the K times of observed image data of the same target scene, then the super-resolution reconstruction equation can be expressed as:

y _k =AHC(s _k )x+n _k ,k=1,2,...,K

Among them, x represents the target scene, y _k represents the image sequence obtained by observing the target K times, and H represents the optical point spread function. If the imaging result of the first observation is used as the reference image, the kth observation image relative to the reference image Rotational displacement θ _k , horizontal displacement _ck , vertical displacement d _k , denoted as s _k =(θ _k ,c _k ,d _k ) ^T ;

C( _sk ) is the displacement matrix related to the motion vector _sk ; A is the detector down-sampling function, which is related to the magnification of the pixel number of the super-resolution reconstruction; n _k represents the additive noise.

Further, for each frame of observed image data y _k , the matrix C(s _k ) and the noise n _k are different, and the matrices A, H, C(s _k ) and x are convolutional relationships. In terms of resolution reconstruction, the probability form of _sk and x, y _k should be simultaneously decoupled from the convolution relationship of these four:

In the above formula, the symbol {·} represents the set, and if the probability p(x,{s _k }|{y _k }) is required, the posterior probability distribution p({y _k }|x,{s _k } ) and the prior probability distribution p(x) are modeled separately, the posterior probability is also called the least squares term, and the prior probability is also called the regularization term.

Let the noise obey a Gaussian distribution with zero mean, then

The inverse of the variance σ _k follows a Gamma distribution:

先验协方差

的高斯分布：In order to improve the accuracy of super-resolution reconstruction, let the registration parameter _sk obey the prior mean

prior covariance

The Gaussian distribution of :

Combine the above formula to get the expression of posterior probability or least squares term p({y _k }|x,{s _k }):

The second factor that affects the accuracy of super-resolution reconstruction is the regularization term introduced in image restoration, that is, the prior probability distribution p(x), whose function is to constrain the solution space of the equation, which is reflected in the reconstructed image result as the reconstruction of edge information. However, edge reconstruction and noise filtering are contradictory, and a trade-off is needed to enhance the robustness of the algorithm. Therefore, the modeling of the regularization term is particularly important for the design of the super-resolution reconstruction algorithm. In this example, the regularization term can be The modeling design is as follows: A or B:

1) Regularization term design form A:

The meanings and expressions of p(x|α _wB ) and p(α _wB ) in the above formula are described below.

p _wB (x|α _wB )=(α _wB ) ^P2 exp[-α _wB U _wB (x)]

In the above formula, the expression of U _wB (x) is:

Among them, _Δhi and _Δvi represent the difference operators in the horizontal and vertical directions, respectively, and the "arrow" symbol indicates that the first-order difference is taken along the direction to the pixel point closest to the pixel point _xi , which reflects the gradient distribution of the image, λ The expression for _wB (x) is:

λ _S represents the constraint strength factor, and the symbol norm represents the normalization operator.

In the above formula, α _wB obeys Gamma distribution:

2) Design form B of the regularization term:

The meanings and expressions of the items on the right side of the equal sign are explained below:

表示参数

的集合，则

可以记为

其表达式为：make

Indicates parameters

collection, then

can be recorded as

Its expression is:

的表达式：in,

expression:

λ _wh (x) and λ _wv (x) are:

Among them, λ _h and λ _v are constant parameters, and the relationship between them is:

的表达式为：

The expression is:

中先验概率分布p(x)，即正则化项的设计。So far, complete the equation

The prior probability distribution p(x), the design of the regularization term.

使正则化项能够按照图像梯度信息自适应的对图像中的边缘区域和平坦区域进行约束，在边缘区域进行边缘纹理信息的重建，在平坦区域进行噪声抑制，从而能够保证获得较好的超分辨率重建结果。Further, the advantages of the above regularization term Form A and Form B design are: by U _wB (x) or

The regularization term can adaptively constrain the edge area and flat area in the image according to the image gradient information, reconstruct the edge texture information in the edge area, and perform noise suppression in the flat area, so as to ensure better super-resolution. rate reconstruction results.

即：Finally, the equation is solved by using the minimum relative entropy between the distribution q(Θ) and the distribution p(Θ|{y _k })

which is:

Θ represents the set of all unknowns, Θ={x,{s _k },α,{β _k }}, and the specific solution process of this formula can be understood in combination with the prior art.

It should be noted that the content not described in detail in the embodiments of the present invention belongs to the prior art known to those skilled in the art.

The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. Substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (3)

1. A method of designing a super-resolution imaging system, the method comprising:

step 1, carrying out model selection on a detector in a super-resolution imaging system, determining a pixel size p, and obtaining a focal length f of an optical lens according to the following formula:

wherein L represents an imaging distance, and GSD represents a pixel resolution;

step 2, determining the resolution improvement multiple of the super-resolution imaging system according to the requirement

And determining a performance factor eta of the super-resolution imaging system by combining specific application scenes, installation sizes and manufacturing cost to obtain F of an optical lens in the super-resolution imaging system_#Counting;

step 3, calculating to obtain the SNR of the imaging signal to noise ratio of the super-resolution imaging system, and checking the F obtained in the step 2_#Whether the number meets the requirement of the imaging signal-to-noise ratio index or not is determined by the following specific process:

first, the imaging signal-to-noise ratio SNR is calculated according to the following formula:

wherein E is_img(lambda) is the radiation energy of the lens on any position on the target surface of the detector, h upsilon is the single photon energy, QE is the quantum efficiency of the detector, V_NSHFor shot noise, V_floorRepresenting the detector noise floor, V_PRNURepresenting the response inhomogeneity noise, SNR, of the detector_targetThe signal-to-noise ratio index required to be achieved by the super-resolution imaging system is represented;

above E_img(lambda) and F_#The relationship of numbers is expressed as:

wherein, T_optIs the optical system transmittance, L_λ(λ) represents the optical lens entrance pupil radiance, θ_tiltIs the optical axis inclination angle;

checking F obtained in step 2 according to the relation_#Whether the number meets the requirements of the imaging signal-to-noise ratio index;

step 4, if the obtained signal-to-noise ratio meets the index requirement, according to a formula F_#Obtaining the caliber D of an optical lens in the super-resolution imaging system and further obtaining the design parameters of the optical lens;

step 5, if the obtained signal-to-noise ratio does not meet the index requirement, repeating the operations of the steps 2 and 3 until the signal-to-noise ratio meets the index requirement, and determining F_#Number, and further according to formula F_#Determining the caliber D of an optical lens in the super-resolution imaging system as f/D;

and 6, observing the target to be detected for multiple times by using the super-resolution imaging system, so that different sub-pixel displacements exist between every two observed images, and the result of the multiple observation forms an image sequence y of the target to be detected_k；

Step 7, the image sequence y is re-processed_kSubstituting the image gradient information into a super-resolution reconstruction equation, establishing a regularization model which changes adaptively according to the image gradient information, constraining the marginal area and the flat area of the observed image of the target to be detected, and solving the super-resolution reconstruction equation according to the minimum relative entropy to obtain a super-resolution imaging result.

2. The design method of the super-resolution imaging system according to claim 1, wherein, in step 2,

resolution improvement factor

In the range of

Wherein,

the maximum value of the resolution improvement factor which can be realized by the super-resolution reconstruction algorithm is shown;

the performance factor eta of the super-resolution imaging system is expressed as

Wherein, ω is_SRepresenting the detector cut-off frequency, omega_ORepresents the optical cut-off frequency;

and satisfy the relationship

Wherein λ represents the wavelength of the incident light; p is the determined pixel size; f_#The parameters of the optical lens in the super-resolution imaging system.

3. The method for designing a super-resolution imaging system according to claim 1, wherein, in step 7,

y_krepresenting K times of observation image data of the same target scene, the super-resolution reconstruction equation is expressed as:

y_k＝AHC(s_k)x+n_k,k＝1,2,...,K

where x denotes the target scene, y_kRepresenting that K times of target observation are carried out to obtain an image sequence; a is a down-sampling function of the detector and is related to the pixel number amplification rate of super-resolution reconstruction; c(s)_k) Is a sum motion vector s_kA matrix of related displacements; h represents an optical point spread function; n is_kRepresenting additive noise.

Images