CN111416980A - High-resolution camera imaging method based on compressed coded aperture - Google Patents

Abstract

The invention discloses a high-resolution camera imaging method based on a compressed coded aperture, which is used for solving the technical problem of lower imaging resolution of the existing high-resolution imaging method. The technical scheme includes that a set of frequency domain complementary coding aperture templates which enable the evaluation index to be maximum is searched by constructing the evaluation index of the frequency domain complementary coding aperture diaphragm and utilizing a template sequence to perform the processes of selection, intersection and variation, and the templates are mutually complementary among frequency domains, so that images shot through different designed coding aperture diaphragms are guaranteed to retain high-frequency detail information of different components in a scene, and support is provided for detail information recovery. The invention optimally designs a group of coding aperture combinations with complementary frequency domains to replace a single Gaussian coding form at the aperture diaphragm of the traditional optical system, thereby not only widening the spectral range of the aperture of the camera, but also removing redundant information of the coding aperture in frequency response, realizing the acquisition of the maximum information image at the sensing stage and improving the imaging resolution of the image.

Description

High-resolution camera imaging method based on compressed coded aperture

Technical Field

The invention relates to a high-resolution imaging method, in particular to a high-resolution camera imaging method based on a compressed coded aperture.

Background

In the camera imaging process, the point spread function formed by the traditional camera aperture is generally of a gaussian-like shape, which shows that high-frequency components are lost in a frequency domain and more points with zero frequency domain are contained. Scene information is incident on the detector through the aperture, and due to the low-pass characteristic of the aperture frequency domain, high-frequency information in the scene can be filtered out, so that the resolution capability of acquiring image details is reduced. However, only the high-frequency information lost in the image capturing stage is recovered by the back-end reconstruction method, which often injects unreal information into the reconstructed image, even causes reconstruction errors such as information confusion. Therefore, it is necessary to effectively design the aperture diaphragm of the camera and an effective reconstruction method, and to recover more high-frequency detail information of the scene by a computational imaging means. At present, the mainstream camera coding aperture design mainly obtains a relatively excellent coding aperture template through random or subjective judgment, and widens the spectrum range of a filter, however, frequency domain response of the coding aperture designed through random or subjective judgment has more redundant information, mutual frequency complementarity is not considered, and the retention capability of scene high-frequency information cannot be maximized. In addition, image reconstruction through sparse representation of an external dictionary generally requires a large amount of external data to train the dictionary, and correlation information between images inside the images and images of coded sequences is not considered, so that reconstruction accuracy is reduced.

The document 'Robust All-in-focus Super-Resolution for Focal Stack Photography [ J ]. IEEE Transactions on Image Processing,2016: 1-1.' discloses a focus Stack-based full-focus high-Resolution imaging method, which is based on the data requirement of the rear end and proposes to acquire a scene sequence Image by utilizing complementary information among different sizes of camera aperture frequency domains, utilizes triple interpolation parametric fuzzy kernel projection and applies randon transformation to reconstruct a defocusing fuzzy kernel with any depth, and simultaneously uses L1 norm suppression noise to perform high-Resolution Image reconstruction on the sequence Image, thereby effectively improving the Image Resolution.

Disclosure of Invention

In order to overcome the defect that the imaging resolution of the conventional high-resolution imaging method is low, the invention provides a high-resolution camera imaging method based on a compressed coded aperture. The method comprises the steps of constructing a frequency domain complementary coding aperture diaphragm evaluation index, utilizing a template sequence to carry out selection, intersection and variation processes, searching a group of frequency domain complementary coding aperture template sets which enable the evaluation index to be maximum, wherein the templates are mutually complementary among frequency domains, so that images shot through different designed coding aperture diaphragms are guaranteed to retain high-frequency detail information of different components in a scene, detail information contained in different image sequences are mutually complementary, and support is provided for detail information recovery. The invention optimally designs a group of coding aperture combinations with complementary frequency domains to replace a single Gaussian coding form at the aperture diaphragm of the traditional optical system, thereby not only widening the spectral range of the aperture of the camera, but also removing redundant information of the coding aperture in frequency response and realizing the acquisition of the maximum information image at the sensing stage. The method fully utilizes the correlation between different coded images and the correlation of image internal blocks to establish a back-end high-resolution image self-similarity reconstruction model, thereby improving the reconstruction precision. The limitation of physical imaging in the aspects of resolution ratio and the like is broken through, and the imaging resolution ratio of the image is improved.

The technical scheme adopted by the invention for solving the technical problems is as follows: a high-resolution camera imaging method based on compressed coded aperture is characterized by comprising the following steps:

step one, designing a coded aperture template.

Randomly initializing M L× N binary coding template sequences, stretching each L× N binary coding template into line vectors with the length of L× N according to a line sequence to obtain k_MThen repeating the iteration steps 1) -3) G times, after the iteration treatment is finished, selecting L× N coding templates with the maximum complementarity and the number of 0 in the binary codes being more than 50% of the total number as an optimal coding template combination K according to the step 1)_MHere, M is 4000, L is 4, N is 169, and G is 1000.

1) And (4) selecting a template sequence.

For each L× N binary code line vector k of input_MIt is first decomposed into L vectors k of size 1 × N_MiTransforming it to frequency domain by Fourier transformation to obtain K_MiPairs of L K_MiMethod for calculating frequency domain complementary size R (K) of coding template by using formula (1)_M) Wherein, σ is the noise term value of 0.001, and A is the 1/f image frequency domain prior obtained by carrying out average statistics on the frequency domains of the T images.

From the calculated M R (K)_M) The first P coding template combinations K corresponding to the binary codes with the maximum frequency domain complementarity and the number of 0 being more than 50 percent of the total number are selected_P400, the P coding template combinations are transformed into the space domain by inverse fourier transform, resulting in k_P。

2) The template sequences are crossed.

Combining k with the P coding templates selected in the step 1)_PRandomly pick 2 sequences k from them_i、k_jAligning the two sequences according to bits, performing operations on the two sequences bit by bit from left to right, firstly generating a random number r1 of 0-1 before each bit is operated, exchanging the numerical value of the bit of the two sequences if the random number r1 is less than q1, and otherwise, operating the next bit. After each bit of the two sequences is operated from left to right, the operation result is reserved. Repetition ofThe sequence crossing process in the step 2) is carried out until the number of sequences is increased from P to M. Q1 is taken to be 0.2.

3) Template sequence variation.

For M sequences k obtained in step 2)_MEach bit in the sequence is processed from left to right, a random number r2 of 0-1 is generated before each bit is processed, if r2 is less than q2, the bit is inverted, otherwise, the bit is not processed and is transferred to the next bit for processing until the whole sequence is processed. Q2 is taken to be 0.05.

And step two, processing the space-time coding camera system.

According to the optimal L× N coding template aperture set K obtained in the step one_MPerforming inverse Fourier transform on the k-space to obtain k_MIt is decomposed into L coding templates k with N length_MiL, each encoding template k_MiIs converted into

Two-dimensional matrix H of_iL, cutting copper sheets with the same size according to the radius V of the aperture diaphragm of the camera lens, and carrying out encoding on the copper sheets according to L designed encoding aperture templates H_iThe pattern is precisely processed, and a template H_iPunching the copper sheet at the point with the median value of 1, wherein the side length of each hole is

And fixing the processed copper sheets at the aperture diaphragm of the L camera lens to form the coded aperture camera.

The whole set of space-time coding camera system comprises a coded aperture camera at a collecting end and a main control computer at a processing end. The operation process of the system is to transmit the compressed coded image collected by the processed coded aperture camera to a main control computer at the rear end, and the coded image is subjected to perception decoding reconstruction through the main control computer to obtain a high-resolution image.

And step three, collecting compressed coded data.

Fixing the encoding camera on the tripod, adjusting the relevant shooting parameters of the camera and keeping the shooting parameters unchangedAnd (3) L coded lenses of the processed z, wherein the z is 1, the z is used for shooting a real scene by using a coding camera, the scene is marked as I, and a coding template adopted by the z coded lens is k_zThen the image acquisition process is

Where D is the down-sampling matrix of the camera, B_zIn order to encode the image for the compression that is acquired,

representing the convolution operation and n is the sensor noise.

At each shooting, respectively using L encoding lenses to collect L compressed and encoded image sequences B_zL, then the sequence of images acquired and the set k of encoding templates employed are combined_zAnd transmitting the data to a main control computer for decoding and reconstructing.

And fourthly, high-resolution perception reconstruction.

For compressed coded image sequence B transmitted to main control computer_zAnd (5) carrying out step 4) -6) iteration processing, wherein the iteration number is Q-40.

4) And (4) PCA dictionary learning.

For image sequence B_zExtracting image blocks with the size of n × n, clustering the image blocks into C classes by using a K-means algorithm, and establishing a PCA dictionary base psi for each class_C. Wherein C is 70.

5) Non-local block sparse prior.

To B_zThe image is processed to extract image blocks ρ of the image size n × n_tFinding an image block ρ_tIn clustering, using PCA dictionary base psi of the class in which the cluster is located_CFinding corresponding sparse representation coefficients

Ψ_C' representing dictionary base Ψ_CThe transposing of (1). For image block rho_tAccording to exp (- | ρ) in the image where it is located_t-ρ_t,qI | l) find q, q ═ 1,., 12 similar image blocks ρ_t,qForm a set omega^t. For q are similarImage block ρ_t,qCorresponding sparsity α^t,qWeighted summation is carried out according to the formulas (2) and (3) to obtain non-local sparse prior β^t. Wherein n is 7.

6) And solving sparse coefficients.

For each image B_zSolving sparse representation coefficients α according to equation (4) optimization_ZBased on the obtained coefficient α_ZThen, the image is reconstructed

If the number of iterations<Q, then go to step 4) and order

Wherein phi_ZFor coding a template k_zThe convolution is converted into a matrix multiplication expression, and lambda is equal to gamma is equal to 0.5.

The invention has the beneficial effects that: the method comprises the steps of constructing a frequency domain complementary coding aperture diaphragm evaluation index, utilizing a template sequence to carry out selection, intersection and variation processes, searching a group of frequency domain complementary coding aperture template sets which enable the evaluation index to be maximum, wherein the templates are mutually complementary among frequency domains, so that images shot through different designed coding aperture diaphragms are guaranteed to retain high-frequency detail information of different components in a scene, detail information contained in different image sequences are mutually complementary, and support is provided for detail information recovery. The invention optimally designs a group of coding aperture combinations with complementary frequency domains to replace a single Gaussian coding form at the aperture diaphragm of the traditional optical system, thereby not only widening the spectral range of the aperture of the camera, but also removing redundant information of the coding aperture in frequency response and realizing the acquisition of the maximum information image at the sensing stage. The method fully utilizes the correlation between different coded images and the correlation of image internal blocks to establish a back-end high-resolution image self-similarity reconstruction model, thereby improving the reconstruction precision. The limitation of physical imaging in the aspects of resolution ratio and the like is broken through, and the imaging resolution ratio of the image is improved.

The present invention will be described in detail with reference to the following embodiments.

Detailed Description

The high-resolution camera imaging method based on the compressed coded aperture specifically comprises the following steps:

step one, designing a coded aperture template.

Randomly initializing M L× N binary (only containing 0,1) coding template sequences, stretching each L× N binary coding template into L× N-length row vectors according to a row sequence to obtain k_MThen repeating the iteration steps 1) -3) G times, after the iteration treatment is finished, selecting L× N coding templates with the maximum complementarity and the number of 0 in the binary codes being more than 50% of the total number as an optimal coding template combination K according to the step 1)_MHere, M is 4000, L is 4, N is 169, and G is 1000.

1) And (4) selecting a template sequence.

For each L× N binary code line vector k of input_MIt is first decomposed into L vectors k of size 1 × N_MiTransforming it to frequency domain by Fourier transformation to obtain K_MiPairs of L K_MiMethod for calculating frequency domain complementary size R (K) of coding template by using formula (1)_M) Wherein, σ is the noise term value of 0.001, and A is the 1/f image frequency domain prior obtained by carrying out average statistics on the frequency domains of the T images.

From the calculated M R (K)_M) The first P coding template combinations K corresponding to the binary codes with the maximum frequency domain complementarity and the number of 0 being more than 50 percent of the total number are selected_P(P1.., 400), converting the P coding template combinations into a spatial domain by inverse fourier transformTo obtain k_P(P＝1,...,400)。

2) The template sequences are crossed.

Combining k with the P coding templates selected in the step 1)_P(P1.., 400), from which 2 sequences k were randomly chosen_i、k_jAligning the two sequences according to bits, performing operations on the two sequences bit by bit from left to right, firstly generating a random number r1 of 0-1 before each bit is operated, exchanging the numerical value of the bit of the two sequences if the random number r1 is less than q1, and otherwise, operating the next bit. After each bit of the two sequences is operated from left to right, the operation result is reserved. Repeating the sequence crossing process in the step 2) until the number of sequences is increased from P to M. Q1 is taken to be 0.2.

3) Template sequence variation.

For M sequences k obtained in step 2)_MEach bit in the sequence is processed from left to right, a random number r2 of 0-1 is generated before each bit is processed, if r2 is less than q2, the bit is inverted, otherwise, the bit is not processed and is transferred to the next bit for processing until the whole sequence is processed. Q2 is taken to be 0.05.

And step two, processing the space-time coding camera system.

According to the optimal L× N coding template aperture set K obtained in the step one_MPerforming inverse Fourier transform on the k-space to obtain k_MIt is decomposed into L coding templates k with N length_Mi(i ═ 1.. L), each encoding template k is coded_Mi(i ═ 1.. L) to

Two-dimensional matrix H of_i(i-1.. L.) copper sheets with the same size are cut according to the radius V of the aperture diaphragm of the camera lens, and the copper sheets are subjected to L designed coded aperture templates H_iPattern (i ═ 1.. L) was subjected to precision machining, and a template H was used_i(i-1.. L) punching the copper sheet at a point with a median of 1, wherein each hole has a side length of one side

Square of (2). Respectively fixing the processed copper sheetsAt L camera lens aperture stops, a coded aperture camera is constructed.

The whole set of space-time coding camera system comprises a coded aperture camera at a collecting end and a main control computer at a processing end. The operation process of the system is to transmit the compressed coded image collected by the processed coded aperture camera to a main control computer at the rear end, and the coded image is subjected to perception decoding reconstruction through the main control computer to obtain a high-resolution image.

And step three, collecting compressed coded data.

Fixing the encoding camera on a tripod, adjusting relevant shooting parameters of the camera and keeping the camera unchanged, replacing a processed z (z is 1,.. L) encoding lens, and shooting a real scene by using the encoding camera, wherein the scene is I, and an encoding template adopted by the z (z is 1,.. L) encoding lens is k_zThen the image acquisition process is

Where D is the down-sampling matrix of the camera, B_zIn order to encode the image for the compression that is acquired,

representing the convolution operation and n is the sensor noise.

At each shooting, respectively using L encoding lenses to collect L compressed and encoded image sequences B_z(z 1.. L), and then the sequence of images acquired and the set k of encoding templates employed_zAnd transmitting the data to a main control computer for decoding and reconstructing.

And fourthly, high-resolution perception reconstruction.

For compressed coded image sequence B transmitted to main control computer_z(z-1.. L) performing the steps 4) -6) of iteration processing, wherein the number of iterations is Q-40.

4) And (4) PCA dictionary learning.

For image sequence B_z(z ═ 1.. L) image blocks of size n × n are extracted, the image blocks are grouped into classes C using the K-means algorithm, and a PCA dictionary base Ψ is established for each class_C. Wherein C is 70.

5) Non-local block sparse prior.

To B_z(z 1.. L) image processing, and extracting image block ρ of n × n image size_tFinding an image block ρ_tIn clustering, using PCA dictionary base psi of the class in which the cluster is located_CFinding corresponding sparse representation coefficients

Ψ_C' representing dictionary base Ψ_CThe transposing of (1). For image block rho_tAccording to exp (- | ρ) in the image where it is located_t-ρ_t,q| | l) find q (q ═ 1.., 12) similar image blocks ρ_t,qForm a set omega^t. For q (q ═ 1.., 12) similar image blocks ρ_t,qCorresponding sparse α^t,qWeighted summation is carried out according to the formulas (2) and (3) to obtain non-local sparse prior β^t. Wherein n is 7.

6) And solving sparse coefficients.

For each image B_z(z 1.. L) optimizing and solving sparse representation coefficients α according to equation (4)_ZBased on the obtained coefficient α_ZThen, the image is reconstructed

If the number of iterations<Q, then go to step 4) and order

Wherein phi_ZFor coding a template k_zThe convolution is converted into a matrix multiplication expression, and lambda is equal to gamma is equal to 0.5.

Claims (1)

1. A high resolution camera imaging method based on compressed coded aperture, comprising the steps of:

designing a coded aperture template;

randomly initializing M L× N binary coding template sequences, stretching each L× N binary coding template into line vectors with the length of L× N according to a line sequence to obtain k_MThen repeating the iteration steps 1) -3) G times, and after the iteration treatment is finished, selecting L× N coding templates with the maximum complementarity and the number of 0 in the binary codes being more than 50 percent of the total number as an optimal coding template combination K according to the step 1)_MWherein, M is 4000, L is 4, N is 169, G is 1000;

1) selecting a template sequence;

for each L× N binary code line vector k of input_MIt is first decomposed into L vectors k of size 1 × N_MiTransforming it to frequency domain by Fourier transformation to obtain K_MiL pairs of K_MiMethod for calculating frequency domain complementary size R (K) of coding template by using formula (1)_M) Wherein, the value of the noise term is 0.001 for sigma, and the frequency domain prior of the 1/f image obtained by carrying out average statistics on the frequency domains of the T images is A;

from the calculated M R (K)_M) The first P coding template combinations K corresponding to the binary codes with the maximum frequency domain complementarity and the number of 0 being more than 50 percent of the total number are selected_P400, the P coding template combinations are transformed into the space domain by inverse fourier transform, resulting in k_P；

2) Crossing template sequences;

combining k with the P coding templates selected in the step 1)_PRandomly pick 2 sequences k from them_i、k_jAligning the two sequences according to bits, operating the two sequences bit by bit from left to right, firstly generating a random number r1 of 0-1 before each bit is operated, and exchanging the number of the bit of the two sequences if the random number r1 is less than q1Value, otherwise, operating on the next bit; after each bit of the two sequences is operated from left to right, the operation result is reserved; repeating the sequence crossing process in the step 2) until the number of sequences is increased from P to M; taking q1 as 0.2;

3) template sequence variation;

for M sequences k obtained in step 2)_MProcessing each bit from left to right, generating a random number r2 of 0-1 before each bit is processed, if r2 is less than q2, inverting the bit, otherwise, not processing the bit, and switching to the next bit for processing until the whole sequence is processed; taking q2 as 0.05;

step two, processing the space-time coding camera system;

according to the optimal L× N coding template aperture set K obtained in the step one_MPerforming inverse Fourier transform on the k-space to obtain k_MIt is decomposed into L coding templates k with N length_MiL, each encoding template k_MiIs converted into

Two-dimensional matrix H of_iAnd i is equal to 1.. L, cutting copper sheets with the same size according to the radius V of the aperture diaphragm of the camera lens, and carrying out design on the copper sheets according to L coded aperture templates H_iThe pattern is precisely processed, and a template H_iPunching the copper sheet at the point with the median value of 1, wherein the side length of each hole is

Fixing the processed copper sheets at the aperture diaphragm of the L camera lens to form a coded aperture camera;

the whole set of space-time coding camera system comprises a coded aperture camera at an acquisition end and a main control computer at a processing end; the operation flow of the system is that the compressed coded image collected by the processed coded aperture camera is transmitted to a main control computer at the rear end, and the coded image is subjected to perception decoding reconstruction through the main control computer to obtain a high-resolution image;

thirdly, collecting compressed coded data;

fixing a coding camera on a tripod, adjusting relevant shooting parameters of the camera and keeping the camera unchanged, replacing processed z-th and z-1.. L coding lenses, and shooting a real scene by using the coding camera, wherein the scene is marked as I, and a coding template adopted by the z-th coding lens is k_zThen the image acquisition process is

Where D is the down-sampling matrix of the camera, B_zIn order to encode the image for the compression that is acquired,

representing a convolution operation, n is sensor noise;

at each shooting, respectively using L encoding lenses to collect L compressed and encoded image sequences B_zL, then the sequence of images acquired and the set k of encoding templates employed are combined_zTransmitting to a main control computer for decoding and reconstruction processing;

step four, high-resolution perception reconstruction;

for compressed coded image sequence B transmitted to main control computer_zCarrying out step 4) -6) iteration processing, wherein the iteration number is Q-40;

4) learning a PCA dictionary;

for image sequence B_zExtracting image blocks with the size of n × n, clustering the image blocks into C classes by using a K-means algorithm, and establishing a PCA dictionary base psi for each class_C(ii) a Wherein C is 70;

5) non-local block sparse prior;

to B_zThe image is processed to extract image blocks ρ of the image size n × n_tFinding an image block ρ_tIn clustering, using PCA dictionary base psi of the class in which the cluster is located_CFinding corresponding sparse representation coefficients

Ψ_C' representing dictionary base Ψ_CTransposing; for image block rho_tIn the image where it is locatedIn accordance with exp (- | ρ)_t-ρ_t,qI | l) find q, q ═ 1,., 12 similar image blocks ρ_t,qForm a set omega^t(ii) a For q similar image blocks ρ_t,qCorresponding sparsity α^t,qWeighted summation is carried out according to the formulas (2) and (3) to obtain non-local sparse prior β^t(ii) a Wherein n is 7;

6) solving sparse coefficients;

for each image B_zSolving sparse representation coefficients α according to equation (4) optimization_ZBased on the obtained coefficient α_ZThen, the image is reconstructed

If the number of iterations<Q, then go to step 4) and order

L, where Φ_ZFor coding a template k_zConverting from convolution to a representation of matrix multiplication, λ ═ γ ═ 0.5;