CN110660022A - Image super-resolution reconstruction method based on surface fitting - Google Patents

Image super-resolution reconstruction method based on surface fitting Download PDF

Info

Publication number
CN110660022A
CN110660022A CN201910845540.2A CN201910845540A CN110660022A CN 110660022 A CN110660022 A CN 110660022A CN 201910845540 A CN201910845540 A CN 201910845540A CN 110660022 A CN110660022 A CN 110660022A
Authority
CN
China
Prior art keywords
resolution
low
image
pixel
neighborhood
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910845540.2A
Other languages
Chinese (zh)
Inventor
曾向荣
周典乐
刘衍
孙博良
钟志伟
龙鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN201910845540.2A priority Critical patent/CN110660022A/en
Publication of CN110660022A publication Critical patent/CN110660022A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Abstract

The invention combines the neighborhood expansion and the surface fitting technology to complete the image super-resolution reconstruction method, which comprises the steps of constructing a series of nested neighborhoods for each high-resolution grid node; then eliminating abnormal low-resolution pixels from each neighborhood according to the pixel intensity value; and performing surface fitting on the reserved low-resolution pixels to obtain a sampling value of the position of the high-resolution grid node to be estimated, and finally estimating the high-resolution pixel value of the grid node for the series of sampling values by using the maximum posterior probability. The method does not need iteration and does not consider the convergence problem, thereby effectively reducing the calculation complexity.

Description

Image super-resolution reconstruction method based on surface fitting
Technical Field
The invention belongs to the field of computer image processing, relates to an image super-resolution reconstruction method applied to aviation and satellite images, and particularly relates to an image super-resolution reconstruction method based on curved surface fitting.
Background
The image resolution is an index critical to the evaluation of image quality, and is simply a measure of the image detail resolution capability of the imaging system, and is also an index of the fineness of an object in an image, which indicates the detail degree of scene information. However, many current imaging systems, such as infrared imagers and CCD cameras, cannot achieve high image resolution due to the inherent density of the sensor array during image acquisition; meanwhile, the undersampling effect can cause the frequency spectrum overlapping of the images, so that the obtained images are degraded due to the deformation effect. On the other hand, due to atmospheric disturbance, relative motion between the subject and the camera, etc., blurring of the image is also generated, reducing the resolution of the image. Increasing the sampling density of the sensor array to improve image resolution and eliminate distortion effects can be costly or faced with difficult technical difficulties.
The purpose of multi-image super-resolution reconstruction is to obtain a high-resolution image by using a plurality of low-resolution observation images, and in most cases, the motion displacement amount between the images is unknown, so that the displacement relation between the low-resolution images needs to be accurately estimated. The process of estimating the displacement relationship is called motion estimation, or image registration. Motion estimation between low-resolution images is the key of super-resolution reconstruction of sequence images, and the reconstruction effect is directly influenced by the motion estimation precision. However, in some practical applications, it is difficult to acquire image sequences of the same scene, for example, under local war conditions, the battlefield environment changes constantly, which puts higher demands on military reconnaissance, and in such a case, it is very difficult to acquire sequence images of the same scene. On the other hand, since the multi-image super-resolution algorithm usually has high requirements on motion displacement between low-resolution image sequences (motion displacement at the sub-pixel level is required), in many practical applications, even if image sequences of the same scene are obtained, due to the fact that overlapping portions between different images are few (such as aerial and satellite images and the like), the prerequisite requirement of slight difference between the images cannot be met, and therefore the sequence image super-resolution algorithm is not feasible under the circumstances.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide a super-resolution reconstruction technique applied to images with few overlapping portions, such as aerial and satellite images.
In order to achieve the purpose, the invention adopts the following technical scheme that the image super-resolution reconstruction method based on surface fitting comprises the following steps:
(1) acquiring a Low-resolution (LR) image sequence, and acquiring m Low-resolution images y by shooting through a camera array system p1, 2.. m, using an image imaging model of: y isp＝DHpWpu+e p1,2,.. m, wherein m is a positive integer, ypFor the pth low resolution image, u is the high resolution image to be estimated, Wp、HpAnd D are respectively a deformation matrix, a fuzzy matrix and a down-sampling matrix, epIs additive noise;
(2) image registration, namely solving motion estimation parameters between a series of low-resolution images and a reference frame by adopting an SURF (speeded Up Robust features) -based image registration algorithm, and then performing interpolation mapping on the series of low-resolution images to a high-resolution grid by utilizing the motion estimation parameters;
(3) and (2) reconstructing a super-resolution image, namely searching a series of nested neighborhoods for the nodes on the high-resolution grid by utilizing neighborhood expansion, estimating a listed sampling value by utilizing a surface fitting technology after an abnormal point is eliminated in each neighborhood, and finally estimating a high-resolution pixel value by utilizing the maximum posterior probability to obtain the high-resolution image.
Preferably, the image registration comprises the following steps:
21) selecting a reference image from the low resolution image y p1,2, 1, mSelecting an image with the maximum peak signal-to-noise ratio (PSNR) as a reference frame;
22) presetting a registration rule, and extracting a feature point set by using an SURF algorithm to represent as follows:
Figure BDA0002196960090000031
wherein n ismFor low resolution images ypP 1,2, m,
Figure BDA0002196960090000032
k＝1,2,…,m，j＝1,2,…,max(n1,n2,…,nm) And calculate P1To P from each feature point2,…,PmEuclidean distance of each feature point in (1):
Figure BDA0002196960090000033
wherein j is1＝1,2,…,n1，j2＝1,2,…,n2，jm＝1,2,…,nm；
Judging whether the feature points are matched according to a preset matching rule, wherein the preset matching rule is as follows:
Figure BDA0002196960090000034
wherein r ═ d1/d2Eta is a predetermined threshold value, d1Is the nearest Euclidean distance, d2The next nearest Euclidean distance; when r is>When eta, the feature point matching is successful, otherwise, the feature point matching is failed; obtaining P1And P2,…,PmAnd P is compared with the characteristic point of the matching1And P2,…,PmThe matching feature points between are expressed as:
wherein q ism,nSolving the nth matching characteristic point of the mth low-resolution image and the number of the n matching characteristic points by an affine transformation formula based on the matching characteristic points0Affine transformation parameter in between, i.e. rotation angle thetapDisplacement Δ x of abscissa and ordinatepAnd Δ ypThe affine transformation formula is:
Qp＝RpQ1+ΔDp (5)
wherein
Figure BDA0002196960090000041
p＝2,3,...,m，
After SURF image registration, obtaining an image y to be registeredp1,2, m and a reference image y0Motion estimation parameters;
23) and interpolating and amplifying the low-resolution image by using a bicubic interpolation technology according to the size of the scaling factor, and then mapping the low-resolution image onto a high-resolution grid according to the motion estimation parameters to obtain a non-uniformly distributed spatial sampling map.
Preferably, the super-resolution image reconstruction includes the following steps:
31) constructing a series of nested fields, taking a high-resolution grid node as a center, searching an initial neighborhood with the size of 1 multiplied by 1 block, and initializing a value b10.5, then 0.1 in step l, and b is set as the maximum search value of the radius of the enlarged search2The final neighborhood size is 3 × 3, and the number of neighborhoods is b ═ 1.5 (b ═ 3)2-b1)/l+1；
32) Representing the neighborhood searched as NBi(i ═ 1,2, …, b), search neighborhood NBiInner low resolution pixels, noted as: LPij,i＝1,2,…,b；j＝1,2,…,biWherein b isiIs a neighborhood NBiThe number of inner low resolution pixels;
33) establishing a pixel intensity plane, constructing a coordinate system by taking an XOY plane as an image plane and taking the size of a pixel value as a Z axis, and aligning the neighborhood NBiEach low-resolution pixel LP of (i ═ 1,2, …, b)ij,i＝1,2,…,b；j＝1,2,…,biEstablishing a pixel intensity plane, which is marked as P, by taking the pixel value as high and making the plane parallel to the XOY coordinate planeijPlane in which PijThe plane is represented as:
Pij＝p(fNBi(LPij)) (6)
wherein
Figure BDA0002196960090000042
Is LPijP (-) is a function of the generating plane;
34) eliminating abnormal data; judging the domain NB according to a preset judgment criterioni(i ═ 1,2, …, b) whether the low resolution pixel is abnormal data or not and acquiring a low resolution pixel set after the abnormal data point is removed:
Figure BDA0002196960090000043
wherein M isi(Mi≤bi) Is the neighborhood NBiThe number of remaining low-resolution pixels;
35) for neighborhood NBi(i-1, 2, …, b) the above MiA low resolution pixel LPij(j＝1,2,…,Mi) Using the general quadratic surface equation at LPijCorresponding coordinate (x) ofij,yij) Fitting a curved surface:
Figure BDA0002196960090000051
fitting a surface to each neighborhood, LP for each low resolution pixelijBy passing
Figure BDA0002196960090000052
Estimate their intensity values, wherein
Figure BDA0002196960090000053
Is a neighborhood NBiMiddle and low resolution pixel LPijThe estimated intensity value of (a); xi (xi)ij0, the following system of equations can be obtained:
Figure BDA0002196960090000054
as can be seen from equation (8), as long as M is presentiIs large enough to solve the parameter c0,c1,…,c5The value of (1) is equivalent to solving an over-determined equation set, the equation set is solved by adopting a least square method, and after surface fitting, each low-resolution pixel LP can be obtained by calculation through a formula (7)ijError xi ofijThen solving the mean square error of the surface and estimating the high resolution pixel HP value at the high resolution grid node:
Figure BDA0002196960090000055
Figure BDA0002196960090000056
finally, a maximum a posteriori probability model is used to calculate the high resolution pixel values:
Figure BDA0002196960090000057
wherein q (-) represents a probability density function;
according to the gaussian assumption, equation (11) is equivalent to:
Figure BDA0002196960090000058
wherein f is0(HP) is an a priori estimate of f (HP), obtained by B-spline interpolation, and λ is an empirical parameter. Let the gradient value of equation (12) be 0, the maximum a posteriori estimate of the high-resolution pixel HP, i.e., the final high-resolution pixel estimate, is obtained
Figure BDA0002196960090000061
Figure BDA0002196960090000062
WhereinLet λ be 0, equation (13) is actually a weighted sum function.
Preferably, the removing the abnormal data includes:
pixel LPikTo the pixel LPijIs defined as plane PikTo plane PijCalculating the neighborhood NBiEach pixel LP inikTo LPijThe distance of (a) is:
Figure BDA0002196960090000063
calculate each Low resolution LPij,j＝1,2,…,biRoot mean square error of (d):
Figure BDA0002196960090000064
the set of root mean square errors is noted as:
Figure BDA0002196960090000065
to WiPerforming ascending arrangement to obtain a set:
Figure BDA0002196960090000066
for W after sequencingi' Adjacent elements are differed to obtain a set
Wherein:
Figure BDA0002196960090000068
suppose that
Figure BDA0002196960090000069
To obtain HiMiddle and largest element
Figure BDA00021969600900000610
Index n ofiThen extracting the set Wi' n th ofiAn element
Figure BDA00021969600900000611
Then, a judgment criterion is defined to judge the neighborhood NBiMiddle and low resolution pixel LPijWhether it is abnormal data.
The criteria are as follows:
Figure BDA0002196960090000071
finally, obtaining a neighborhood NBiThe low-resolution pixel set after the abnormal points are removed internally is as follows:
Figure BDA0002196960090000072
wherein M isi(Mi≤bi) Is the neighborhood NBiThe number of remaining low resolution pixels.
Drawings
FIG. 1 is a block diagram of a method implementation of the present invention.
Detailed Description
In order to more clearly describe the technical contents of the present invention, the following further description is given in conjunction with specific embodiments.
An image super-resolution reconstruction method based on surface fitting is shown in the attached figure 1 and comprises the following steps:
(1) acquiring a Low-resolution (LR) image sequence, and acquiring m Low-resolution images y by shooting through a camera array system p1,2, 1, m, is minedThe image imaging model used was: y isp＝DHpWpu+e p1,2,.. m, wherein m is a positive integer, ypFor the pth low resolution image, u is the high resolution image to be estimated, Wp、HpAnd D are respectively a deformation matrix, a fuzzy matrix and a down-sampling matrix, epIs additive noise;
(2) image registration, namely solving motion estimation parameters between a series of low-resolution images and a reference frame by adopting an SURF (speeded Up Robust features) -based image registration algorithm, and then performing interpolation mapping on the series of low-resolution images to a high-resolution grid by utilizing the motion estimation parameters;
(3) and (3) super-resolution image reconstruction, searching a plurality of nested neighborhoods at each high-resolution grid node by using a neighborhood expansion method, and estimating the high-resolution pixel value of the high-resolution grid node by using a surface fitting method for each neighborhood. The method comprises the steps of firstly selecting an initial neighborhood for a high-resolution node position, expanding a neighborhood range according to the step size, searching a series of nested equal-step-size neighborhoods, eliminating abnormal data in each neighborhood due to the fact that abnormal constants (offset from a point at the center of the neighborhood, or high or low) possibly exist in the searched neighborhoods and influence on the effect of surface fitting, then using quadratic surface fitting to each neighborhood to obtain a series of estimated high-resolution sampling values, and finally estimating the sampling values by using a maximum posterior probability method to obtain a final high-resolution pixel estimation value. The specific implementation process is as follows:
to estimate the high resolution pixel HP at the high resolution grid node Hg, a series of nested fields were first constructed. Firstly, using high-resolution grid node as centre, searching an initial neighborhood with 1X 1 block size, and making initial value b10.5. Then, according to the step length l being 0.1, the maximum search value of the enlarged search radius is set as b21.5. The final neighborhood size is 3 × 3, and the number of neighborhoods is b ═ b2-b1)/l+1。
Suppose the neighborhood of the search is denoted NBi(i ═ 1,2, …, b), search neighborhood NBiInner low resolution pixels, noted as: LPij,i＝1,2,…,b,j＝1,2,…,bi，biIs a neighborhood NBiThe size of the number of inner low resolution pixels.
Establishing a pixel intensity plane, constructing a coordinate system by taking an XOY plane as an image plane and taking the size of a pixel value as a Z axis, and aligning the neighborhood NBiEach low-resolution pixel LP of (i ═ 1,2, …, b)ij,i＝1,2,…,b,j＝1,2,…,biEstablishing a pixel intensity plane, which is marked as P, by taking the pixel value as high and making the plane parallel to the XOY coordinate planeijPlane, the following formula:
Figure BDA0002196960090000081
here, theIs LPijIs a function of the generated plane.
Eliminating abnormal data and LP pixelsikTo the pixel LPijThe distance of (d) is defined as: plane PikTo plane PijThe distance of (c). Computing neighborhood NBiEach pixel LP inikTo LPijThe distance of (a) is:
Figure BDA0002196960090000083
calculate each Low resolution LPij,j＝1,2,…,biRoot mean square error of (d):
Figure BDA0002196960090000084
the set of root mean square errors is noted as:
Figure BDA0002196960090000091
to WiPerforming ascending arrangement to obtain a set:
Figure BDA0002196960090000092
for W after sequencingi' Adjacent elements are differed to obtain a set
Figure BDA0002196960090000093
Wherein:
Figure BDA0002196960090000094
suppose that
To obtain HiMiddle and largest element
Figure BDA0002196960090000096
Index n ofiThen extracting the set Wi' n th ofiAn element
Figure BDA0002196960090000097
Then, a judgment criterion is defined to judge the neighborhood NBiMiddle and low resolution pixel LPijWhether it is abnormal data.
The criteria are as follows:
finally we get the neighborhood NBiThe low-resolution pixel set after the abnormal points are removed internally is as follows:
Figure BDA0002196960090000099
wherein M isi(Mi≤bi) Is the neighborhood NBiThe number of remaining low resolution pixels.
For neighborhood NBi(i-1, 2, …, b) inner MiA low resolutionPixel LPij(j＝1,2,…,Mi) Using the general quadratic surface equation at LPijCorresponding coordinate (x) ofij,yij) Fitting a curved surface:
Figure BDA00021969600900000910
fitting a surface to each neighborhood, LP for each low resolution pixelijBy passing
Figure BDA00021969600900000911
Their intensity values are estimated. WhereinIs a neighborhood NBiMiddle and low resolution pixel LPijThe estimated intensity value of (a). Xi (xi)ij0, the following system of equations can be obtained:
Figure BDA0002196960090000101
as can be seen from equation (8), as long as M is presentiIs large enough to solve the parameter c0,c1,…,c5Is equivalent to solving an overdetermined system of equations where we solve such a system of equations using the least squares method.
For better accuracy of the fitted surface, a sufficiently large number of low resolution pixels are required, so the initial neighborhood value b1It cannot be too small. However, too large a neighborhood introduces more noisy data, which also affects the accuracy of the fitted surface, so the maximum neighborhood value b2Nor too large. Choose reasonable b1And b2The quality of the final reconstructed image will be directly affected. After the surface is fitted, the surface can pass through a formulaCalculating to obtain each low resolution pixel LPijError xi ofijThen solving the mean square error of the surface
Figure BDA0002196960090000103
And estimating the high resolution pixel HP values at the high resolution grid nodes:
Figure BDA0002196960090000105
one high resolution pixel is associated with a plurality of low resolution pixels in three ways. The first is the spatial distribution of low resolution pixels on the high resolution grid, which can be represented by a series of nested neighborhoods per high resolution grid node. The second is the correlation between low resolution pixels, which can be represented by surface fitting. Thirdly, the high resolution grid nodes search for the intensity of the low resolution pixels in the neighborhood, which can be represented by the pixel value.
Finally, the high resolution pixel values can be calculated using a maximum a posteriori probability model:
Figure BDA0002196960090000106
where q (-) denotes the probability density function.
According to the gaussian assumption, equation (11) is equivalent to:
Figure BDA0002196960090000111
wherein f is0(HP) is an a priori estimate of f (HP), obtained by B-spline interpolation. λ is an empirical parameter. Let the gradient value of equation (12) be 0, the maximum posterior estimate of the high resolution pixel HP is obtained
Figure BDA0002196960090000114
Figure BDA0002196960090000112
Wherein
Figure BDA0002196960090000113
Let λ be 0, equation (13) is actually a weighted sum function.
Through the above description of the embodiments, it is obvious for those skilled in the art that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. With this understanding in mind, portions of the above-described technical solutions that substantially or otherwise contribute to the prior art may be embodied in the form of a software product that can be stored on a computer readable and writable medium, such as a usb-disk, a removable hard disk, a ROM, a RAM, a magnetic or optical disk, or the like. Including instructions for causing a computing device (e.g., a personal computer, server, or network device, etc.) to perform the methods described in the method embodiments or portions of the method embodiments above.
The foregoing is a more detailed description of the invention in connection with specific/preferred embodiments and is not intended to limit the practice of the invention to those descriptions. It will be apparent to those skilled in the art that various substitutions and modifications can be made to the described embodiments without departing from the spirit of the invention, and these substitutions and modifications should be considered to fall within the scope of the invention.

Claims (5)

1. An image super-resolution reconstruction method based on surface fitting is characterized by comprising the following steps:
(1) acquiring a low-resolution image sequence of the same scene, and shooting by a camera array system to obtain m low-resolution images yp1, 2.. m, using an image imaging model of: y isp＝DHpWpu+ep,p＝1,2,...,m，
Wherein m is a positive integer, ypFor the pth low resolution image, u is the high resolution image to be estimated, Wp、HpAnd D are eachDeformation, blur and downsampling matrices, epIs additive noise;
(2) image registration, namely solving motion estimation parameters between a series of low-resolution images and a preset reference frame by adopting an SURF (speeded Up Robust features) -based image registration algorithm, and then utilizing the motion estimation parameters to map the series of low-resolution images onto a high-resolution grid in an interpolation manner;
(3) and (2) reconstructing a super-resolution image, namely searching a series of nested neighborhoods for the nodes on the high-resolution grid by utilizing neighborhood expansion, estimating a listed sampling value by utilizing a surface fitting technology after an abnormal point is eliminated in each neighborhood, and finally estimating a high-resolution pixel value by utilizing the maximum posterior probability to obtain the high-resolution image.
2. The surface fitting-based image super-resolution reconstruction method according to claim 1, wherein the image registration comprises the following steps:
21) selecting a reference image from the low resolution image ypSelecting an image with the maximum peak signal-to-noise ratio (PSNR) from m as a reference frame;
22) presetting a registration rule, and extracting a feature point set by using an SURF algorithm to represent as follows:
Figure FDA0002196960080000011
wherein n ismFor low resolution images ypP 1,2, m,
Figure FDA0002196960080000012
k＝1,2,…,m，j＝1,2,…,max(n1,n2,…,nm) And calculate P1To P from each feature point2,…,PmEuclidean distance of each feature point in (1):
Figure FDA0002196960080000021
wherein j is1＝1,2,…,n1，j2＝1,2,…,n2，jm＝1,2,…,nm；
Judging whether the feature points are matched according to a preset matching rule, wherein the preset matching rule is as follows:
wherein r ═ d1/d2Eta is a predetermined threshold value, d1Is the nearest Euclidean distance, d2The next nearest Euclidean distance; when r is>When eta, the feature point matching is successful, otherwise, the feature point matching is failed; obtaining P1And P2,…,PmAnd P is compared with the characteristic point of the matching1And P2,…,PmThe matching feature points between are expressed as:
Figure FDA0002196960080000023
wherein q ism,nSolving the nth matching characteristic point of the mth low-resolution image and the number of the n matching characteristic points by an affine transformation formula based on the matching characteristic points0Affine transformation parameter in between, i.e. rotation angle thetapDisplacement Δ x of abscissa and ordinatepAnd Δ ypThe affine transformation formula is:
Qp＝RpQ1+ΔDp (5)
whereinp＝2,3,...,m，
After SURF image registration, obtaining a low-resolution image y to be registeredp1,2, m and a reference image y0Motion estimation parameters;
23) and interpolating and amplifying the low-resolution image by using a bicubic interpolation technology according to the size of the scaling factor, and then mapping the low-resolution image onto a high-resolution grid according to the motion estimation parameters to obtain a non-uniformly distributed spatial sampling map.
3. The method for reconstructing the super-resolution image based on the surface fitting according to claim 1, wherein the method for reconstructing the super-resolution image comprises the following steps:
31) constructing a series of nested fields, taking a high-resolution grid node as a center, searching an initial neighborhood with the size of 1 multiplied by 1 block, and initializing a value b10.5, then 0.1 in step l, and b is set as the maximum search value of the radius of the enlarged search2The final neighborhood size is 3 × 3, and the number of neighborhoods is b ═ 1.5 (b ═ 3)2-b1)/l+1；
32) Representing the neighborhood searched as NBi(i ═ 1,2, …, b), search neighborhood NBiInner low resolution pixels, noted as: LPij,i＝1,2,…,b；j＝1,2,…,biWherein b isiIs a neighborhood NBiThe number of inner low resolution pixels;
33) establishing a pixel intensity plane, constructing a coordinate system by taking an XOY plane as an image plane and taking the size of a pixel value as a Z axis, and aligning the neighborhood NBiEach low-resolution pixel LP of (i ═ 1,2, …, b)ij,i＝1,2,…,b；j＝1,2,…,biEstablishing a pixel intensity plane, which is marked as P, by taking the pixel value as high and making the plane parallel to the XOY coordinate planeijPlane in which PijThe plane is represented as:
Figure FDA0002196960080000031
whereinIs LPijP (-) is a function of the generating plane;
34) eliminating abnormal data, and judging domain NB according to preset judgment criteriai(i ═ 1,2, …, b) whether the low resolution pixel is abnormal data or not and acquiring a low resolution pixel set after the abnormal data point is removed:
Figure FDA0002196960080000033
wherein M isi(Mi≤bi) Is the neighborhood NBiThe number of remaining low-resolution pixels;
35) for neighborhood NBi(i-1, 2, …, b) the above MiA low resolution pixel LPij(j＝1,2,…,Mi) Using the general quadratic surface equation at LPijCorresponding coordinate (x) ofij,yij) Fitting a curved surface:
Figure FDA0002196960080000034
fitting a surface to each neighborhood, LP for each low resolution pixelijBy passing
Figure FDA0002196960080000035
1≤i≤b,1≤j≤MiEstimate their intensity values, wherein
Figure FDA0002196960080000036
Is a neighborhood NBiMiddle and low resolution pixel LPijThe estimated intensity value of (a); xi (xi)ij0, the following system of equations can be obtained:
Figure FDA0002196960080000041
as can be seen from equation (8), as long as M is presentiIs large enough to solve the parameter c0,c1,…,c5The value of (1) is equivalent to solving an overdetermined equation set, the equation set is solved by adopting a least square method, and after surface fitting, each low-resolution pixel LP is obtained by calculation through a formula (7)ijError xi ofijThen solving the mean square error of the surface
Figure FDA0002196960080000042
And estimating the high resolution pixel HP values at the high resolution grid nodes:
Figure FDA0002196960080000043
Figure FDA0002196960080000044
finally, a maximum a posteriori probability model is used to calculate the high resolution pixel values:
Figure FDA0002196960080000045
wherein q (-) represents a probability density function;
according to the gaussian assumption, equation (11) is equivalent to:
Figure FDA0002196960080000046
where f is0(HP) is an a priori estimate of f (HP), obtained by B-spline interpolation, and λ is an empirical parameter. Let the gradient value of equation (12) be 0, the maximum a posteriori estimate of the high-resolution pixel HP, i.e., the final high-resolution pixel estimate, is obtained
Figure FDA0002196960080000048
Wherein
Figure FDA0002196960080000049
Let λ be 0, equation (13) is actually a weighted sum function.
4.The image super-resolution reconstruction method based on surface fitting according to claim 3, wherein the step of eliminating abnormal data comprises the following steps: pixel LPikTo the pixel LPijIs defined as plane PikTo plane PijCalculating the neighborhood NBiEach pixel LP inikTo LPijThe distance of (a) is:
Figure FDA0002196960080000051
calculate each low resolution pixel LPij,j＝1,2,…,biRoot mean square error of (d):
Figure FDA0002196960080000052
the set of root mean square errors is noted as:
Figure FDA0002196960080000053
to WiPerforming ascending arrangement to obtain a set:
Figure FDA0002196960080000054
for W after sequencingi' Adjacent elements are differed to obtain a set
Figure FDA0002196960080000055
Wherein:
Figure FDA0002196960080000056
suppose that
Figure FDA0002196960080000057
To obtain HiMiddle and largest elementIndex n ofiThen extracting the set Wi' n th ofiAn element
Figure FDA0002196960080000059
Then, a judgment criterion is defined to judge the neighborhood NBiMiddle and low resolution pixel LPijWhether it is abnormal data, the criteria are as follows:
Figure FDA00021969600800000510
finally, obtaining a neighborhood NBiThe low-resolution pixel set after the abnormal points are removed internally is as follows:
wherein M isi(Mi≤bi) Is the neighborhood NBiThe number of remaining low resolution pixels.
5. A computer-readable write medium, on which a computer program is stored, characterized in that the program, when executed, carries out the steps of the method according to any one of claims 2-4.
CN201910845540.2A 2019-09-10 2019-09-10 Image super-resolution reconstruction method based on surface fitting Pending CN110660022A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910845540.2A CN110660022A (en) 2019-09-10 2019-09-10 Image super-resolution reconstruction method based on surface fitting

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910845540.2A CN110660022A (en) 2019-09-10 2019-09-10 Image super-resolution reconstruction method based on surface fitting

Publications (1)

Publication Number Publication Date
CN110660022A true CN110660022A (en) 2020-01-07

Family

ID=69036819

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910845540.2A Pending CN110660022A (en) 2019-09-10 2019-09-10 Image super-resolution reconstruction method based on surface fitting

Country Status (1)

Country Link
CN (1) CN110660022A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953297A (en) * 2022-12-27 2023-04-11 二十一世纪空间技术应用股份有限公司 Remote sensing image super-resolution reconstruction and enhancement method and device
CN116805283A (en) * 2023-08-28 2023-09-26 山东大学 Submicron super-resolution microscopic imaging reconstruction method and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933678A (en) * 2015-06-30 2015-09-23 西安理工大学 Image super-resolution reconstruction method based on pixel intensity
CN109308683A (en) * 2018-07-23 2019-02-05 华南理工大学 A kind of method of flexible integration circuit substrate image super-resolution rebuilding

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933678A (en) * 2015-06-30 2015-09-23 西安理工大学 Image super-resolution reconstruction method based on pixel intensity
CN109308683A (en) * 2018-07-23 2019-02-05 华南理工大学 A kind of method of flexible integration circuit substrate image super-resolution rebuilding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIAOFENG WANG等: "Super-resolution image reconstruction using surface fitting with hierarchical structure", 《JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION》, vol. 53, pages 65 - 75 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953297A (en) * 2022-12-27 2023-04-11 二十一世纪空间技术应用股份有限公司 Remote sensing image super-resolution reconstruction and enhancement method and device
CN115953297B (en) * 2022-12-27 2023-12-22 二十一世纪空间技术应用股份有限公司 Remote sensing image super-resolution reconstruction and enhancement method and device
CN116805283A (en) * 2023-08-28 2023-09-26 山东大学 Submicron super-resolution microscopic imaging reconstruction method and system
CN116805283B (en) * 2023-08-28 2023-11-24 山东大学 Submicron super-resolution microscopic imaging reconstruction method and system

Similar Documents

Publication Publication Date Title
CN103190078B (en) For estimating the signal processor and method of the conversion between signal
CN110675347B (en) Image blind restoration method based on group sparse representation
CN110310310B (en) Improved method for aerial image registration
CN104933678B (en) A kind of image super-resolution rebuilding method based on image pixel intensities
CN108288256B (en) Multispectral mosaic image restoration method
CN112686935B (en) Airborne sounding radar and multispectral satellite image registration method based on feature fusion
CN106169173B (en) Image interpolation method
Su et al. Super-resolution without dense flow
CN110660022A (en) Image super-resolution reconstruction method based on surface fitting
CN110060209B (en) MAP-MRF super-resolution image reconstruction method based on attitude information constraint
CN108876829B (en) SAR high-precision registration method based on nonlinear scale space and radial basis function
Hadar et al. Image motion restoration from a sequence of images
CN109615584B (en) SAR image sequence MAP super-resolution reconstruction method based on homography constraint
Ghosh et al. Robust spatial-domain based super-resolution mosaicing of CubeSat video frames: Algorithm and evaluation
Barnard et al. High-resolution iris image reconstruction from low-resolution imagery
CN107767342B (en) Wavelet transform super-resolution image reconstruction method based on integral adjustment model
Li et al. Imaging simulation and learning-based image restoration for remote sensing time delay and integration cameras
Ghosh et al. Super-resolution mosaicing of unmanned aircraft system (UAS) surveillance video frames
Mohan Adaptive super-resolution image reconstruction with lorentzian error norm
WO2023138764A1 (en) Device and method for super resolution kernel estimation
CN108681988B (en) Robust image resolution enhancement method based on multiple images
Zhan et al. HDR synthesis technology for spaceborne CMOS cameras based on virtual digital TDI
Wang et al. Single Image Super-Resolution Based on Cascaded Recursive Residual Convolutional Neural Network
Webster et al. Radial deblurring with ffts
CN113822823B (en) Point neighbor restoration method and system for aerodynamic optical effect image space-variant fuzzy core

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200107