CN113506212A - Improved POCS-based hyperspectral image super-resolution reconstruction method - Google Patents
Improved POCS-based hyperspectral image super-resolution reconstruction method Download PDFInfo
- Publication number
- CN113506212A CN113506212A CN202110558431.XA CN202110558431A CN113506212A CN 113506212 A CN113506212 A CN 113506212A CN 202110558431 A CN202110558431 A CN 202110558431A CN 113506212 A CN113506212 A CN 113506212A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- images
- iteration
- reference frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012937 correction Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000006731 degradation reaction Methods 0.000 claims description 5
- 238000001228 spectrum Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims 2
- 238000003384 imaging method Methods 0.000 description 10
- 230000003595 spectral effect Effects 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an improved super-resolution reconstruction method of a hyperspectral image based on POCS, which comprises the steps of randomly selecting one gray image from gray images of a first wave band of a sequence low-resolution hyperspectral image, and obtaining an initial reference frame through bicubic interpolation, so that the problem of edge blurring of a reconstructed image is relieved to a certain extent; then, the remaining gray level images of the first wave band are corrected according to a projection formula with a relaxation operator, and burrs in a smooth area of a reconstructed image are suppressed; after iteration is carried out for more than two times, the condition of exiting iteration is taken as whether the mean square error between the iteration reconstructed images of the previous iteration and the next iteration is smaller than a certain threshold value, so that the iteration process is self-adaptive, and the subjectivity of manually setting the iteration times is avoided; and finally, repeating the process on the gray level image of each wave band of the hyperspectral image to obtain the hyperspectral image with improved spatial resolution. The method can be used as an effective means for improving the spatial resolution of the hyperspectral image.
Description
Technical Field
The invention relates to the technical field of remote sensing image processing, in particular to a super-resolution reconstruction method of a hyperspectral image based on POCS (point of care computing).
Background
The spectral resolution of hyperspectral images is extremely high, but this high spectral resolution comes at the expense of spatial resolution. When the hyperspectral image is shot, the exposure time of the camera is fixed, the number of the obtained photons is also fixed, and the higher the spectral resolution is, the lower the number of the photons which are distributed to each wave band and used for imaging is, so that the low spatial resolution of the hyperspectral image is caused. The mutual restriction of the spectral resolution and the spatial resolution limits the application of the hyperspectral image, so how to improve the spatial resolution of the hyperspectral image becomes a problem which needs to be solved urgently.
Considering that starting from the perspective of hardware, that is, improving the performance of a hyperspectral camera is subject to the double limitations of cost and technical process, the existing methods for solving the problem of low spatial resolution of a hyperspectral image are based on the perspective of software, and the image is reconstructed by using a super-resolution reconstruction algorithm so as to improve the spatial resolution of the hyperspectral image.
In the super-resolution reconstruction method of the sequence image spatial domain, a convex set projection algorithm (POCS) is favored by many scholars due to the characteristics of simple principle, good reconstruction effect and the like. The theory relied on by the POCS algorithm is set theory, the method mainly comprises the steps that a plurality of priori information of an image is used as the basis, each priori information forms a closed convex set, and the intersection of the closed convex sets is a solution space for POCS super-resolution reconstruction. However, the images reconstructed by the algorithm have the problems of edge blurring, burr generation in a smooth area, over-subjective iteration times and the like.
Disclosure of Invention
The invention discloses an improved POCS-based hyperspectral image super-resolution reconstruction method, which comprises the following steps of:
s1: selecting a gray level image of a certain wave band from the high spectrum images of the sequence low resolution to obtain a sequence low resolution gray level image of the wave band;
s2: randomly selecting any one of the sequential low-resolution gray level images and carrying out interpolation processing on the selected image in a bicubic interpolation mode to obtain an initial reference frame;
s3: calculating a gradient map of the image, performing Gaussian filtering processing on the gradient map, acquiring a relaxation operator according to the gradient map, and adding the relaxation operator into a projection formula to obtain an improved projection formula;
s4: selecting one image from the rest low-resolution images, and finding out the corresponding position of a pixel point on the low-resolution image on the initial reference frame through motion estimation;
s5: simulating a degradation process by adopting a point spread function PSF, reducing the initial reference frame image to the same size as the low-resolution image, solving the residual error of the initial reference frame image and the low-resolution image, and modifying the initial reference frame to optimize the initial reference frame of the wave band by utilizing an improved projection formula according to the residual error;
s6: judging whether all the sequence low-resolution images are used for optimizing the initial reference frame, if so, entering S7, otherwise, returning to S4;
s7: calculating the mean square error of the high-resolution image of the iterative reconstruction and the high-resolution image of the iterative reconstruction last time, comparing the mean square error with a set threshold, if the mean square error is less than the set threshold, entering S8, and if the mean square error is greater than the set threshold, returning to S4;
s8: and repeating the steps from S1 to S7 until the gray level images of all wave bands of the hyperspectral image are reconstructed.
The following method is specifically adopted in S2: the gray values of 16 points around the point to be sampled are used for weighted superposition, thereby not only considering the gray influence of 4 directly adjacent points, but also considering the influence of the change rate of the gray values between the adjacent points. The method specifically adopts the following steps:
s21: calculating the corresponding position (X + u, Y + v) of the point (X, Y) in the magnified image in the original image according to the multiple k of the original image A and the magnified image B;
s22: finding 16 pixel points closest to (x + u, y + v) in the original image;
s23: calculating the horizontal weight and the vertical weight of the 16 pixel points according to a bicubic interpolation basis function, wherein a bicubic interpolation basis function calculation formula is as follows:
s24: the pixel values of 16 pixel points are combined with the horizontal and vertical weights thereof to carry out weighted superposition to obtain the pixel value of a point (X, Y) in the amplified image, and the calculation formula is as follows:
the following method is specifically adopted in S3: measuring the difference degree between the current pixel and the surrounding pixels by using a gradient map, wherein the larger the gradient is, the larger the difference is, and the smaller the gradient is, the smaller the difference is;
the gradient map is acquired as follows:
let original image F ═ F (m)1,m2)]M1×M2Has a size of M1×M2And defining the weighting gradient of the image according to the neighborhood distribution of the current pixel as shown in the formula (2).
In the formula: g1，g2，g3，g4Respectively represented by formula (3), formula (4), formula (5) and formula (6);
in the formula: g1，g2，g3，g4Respectively is the weighted gradient value of the current pixel point in each direction;
performing Gaussian filtering processing on the gradient map obtained by calculation;
the relaxation operator calculated according to the gradient map has the same property with the gradient map, the gradient value is large near a strong edge, the gradient value is small near a weak edge, an improved projection formula can be obtained by adding the relaxation operator into the projection formula, and the improved projection formula can well distinguish an edge region and a smooth region when the initial reference frame is corrected to carry out correction in different degrees.
The relaxation operator is obtained by adopting the following method:
wherein min (g) represents the minimum value of the gradient map, k is an adjustment coefficient, and the value range is [ -1,1 ];
the improved projection formula is obtained by adopting the following method:
where x is the initial reference frame, λ is the relaxation operator, r(y)As residual, h is the PSF template, δ0Is a noise related quantity.
The following method is specifically adopted in S7: the mean square error of the high-resolution images iteratively reconstructed in the previous and subsequent times is used for measuring the similarity between the two images, if the mean square error is small enough, the two images iteratively reconstructed in the adjacent two times are considered to be very similar, the difference between the image iteratively reconstructed in the current time and the image iteratively reconstructed in the previous time is very small, the need of continuing iteration is avoided, the algorithm is converged, otherwise, the algorithm is not converged, and the iteration needs to be continued, and the specific mode is as follows:
calculating the mean square error of the two iteration reconstructed images according to the following formula:
and judging the relative magnitude of the mean square error and the set threshold, if the mean square error is small, exiting the iteration, otherwise, returning to S4.
Based on the technical scheme, the invention provides an improved POCS-based hyperspectral image super-resolution reconstruction method. The method comprises the steps of firstly, randomly selecting one gray image of a first wave band of a sequence low-resolution high-spectrum image, obtaining an initial reference frame through bicubic interpolation, and relieving the problem of edge blurring of a reconstructed image to a certain extent; then, the remaining gray level images of the first wave band are corrected according to a projection formula with a relaxation operator, and burrs in a smooth area of a reconstructed image are suppressed; after iteration is carried out for more than two times, the condition of exiting iteration is taken as whether the mean square error between the iteration reconstructed images of the previous iteration and the next iteration is smaller than a certain threshold value, so that the iteration process is self-adaptive, and the subjectivity of manually setting the iteration times is avoided; and finally, repeating the process on the gray level image of each wave band of the hyperspectral image to obtain the hyperspectral image with improved spatial resolution. The method can be used as an effective means for improving the spatial resolution of the hyperspectral image, and the hyperspectral image with high spatial resolution is a premise that the hyperspectral image is applied to the work of classification, detection and the like, so the method has important application value.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of an improved POCS-based hyperspectral image super-resolution reconstruction method provided by the invention;
FIGS. 2 a-2 b are schematic diagrams of a low resolution AVIRIS Indian pins data set used in the present invention and the results after super resolution reconstruction;
FIGS. 3 a-3 b are schematic diagrams of a low resolution ROSIS University of Pavia data set and super-resolution reconstruction results used in the present invention.
Fig. 4 is an explanatory diagram of pixel point planning in the embodiment of the method.
Detailed Description
In order to make the technical solutions and advantages of the present invention clearer, the following describes the technical solutions in the embodiments of the present invention clearly and completely with reference to the drawings in the embodiments of the present invention:
an improved POCS-based hyperspectral image super-resolution reconstruction method shown in fig. 1 specifically includes the following steps:
assume an original hyperspectral image as
Wherein
b represents the total number of bands and N represents the total number of picture elements of the image.
Step 101: selecting a gray level image of a certain wave band in the sequence low-resolution high-spectrum image to obtain a sequence low-resolution gray level image of the wave band;
step 102: randomly selecting any one of the sequence low-resolution images to obtain an initial reference frame through bicubic interpolation;
assuming that the original data A is M × N in size, the target data B is obtained by amplifying the image A by k times, and the size of B is M × N, the original data A is M × N
To find out the value of the pixel point (X, Y) in the target image B, the corresponding pixel (X, Y) of the point in the original image a is found first, the method is to find 16 pixel points closest to the pixel (X, Y) in the image a, the weight of the 16 points is solved by using a bicubic interpolation basis function, then the pixel value of (X, Y) in the image B is equal to the weighted sum of the 16 pixel points, and the point (X, Y) in the image a can also be called as a point to be sampled. The bicubic interpolation basis function calculation formula is as follows:
wherein a is generally-0.5 or-0.75.
From the correspondence between two points in the A, B diagram, a proportional relationship can be obtained
Further, the position of the point (X, Y) on the map a corresponding to the coordinate P is obtained, that is:
the value of the P coordinate point will generally appear as a decimal, assuming P (x + u, y + u), where x, y represent the integer part, respectively, and u, v represent the fractional part, respectively. The positions of the 16 pixels closest to P are represented by a (i, j) (i, j is 0,1,2,3) as shown in fig. 4.
After the coordinate positions of the 16 points closest to the P are obtained, the parameter x in the bicubic interpolation basis function is obtained next, so as to obtain the weight w (x) corresponding to the 16 points. The bicubic interpolation basis function is a one-dimensional function, and the image is two-dimensional, so that the coordinates of pixel points in the image are decomposed into rows and columns for independent calculation. The parameter X represents the distance from the pixel point to the P point, for example, the distance between a (0,0) and P (X + u, Y + u) is (1+ u,1+ v), so the horizontal weight of a (0,0) is W (1+ u), the vertical weight of W (1+ v), and the contribution value of a (0,0) to B (X, Y) is a (0,0) × W (1+ u) × W (1+ v). Further deducing that the abscissa weights of a (0, i) are W (1+ u), W (1-u) and W (2-u), respectively; the ordinate weights of a (j,0) are respectively W (1+ v), W (1-v) and W (2-v); therefore, the B (X, Y) pixel values are:
where W (u +1-i) is the horizontal weight and W (u +1-j) is the vertical weight.
The values of all points in the bicubic interpolation image can be obtained according to the formula.
Step 103: calculating a gradient map of the image, performing Gaussian filtering on the gradient map to remove noise, and calculating a relaxation operator to add the relaxation operator into a projection formula;
image gradient is a common image information metric, which represents the difference between the current pixel and the surrounding pixels. The larger the gradient is, the larger the difference between the current pixel and the surrounding pixels is, the more the pixel is located in the edge area of the image, and the larger the information content is; the smaller the gradient, the smaller the difference between the current pixel and the surrounding pixels, indicating that the pixel contains a smaller amount of information.
A gaussian gradient map is introduced to measure the importance of the pixels. Setting original image
Has a size of M1×M2According to the neighborhood distribution of the current pixel, the weighting gradient of the image can be defined as shown in formula (4).
In the formula: g1，g2，g3，g4Are respectively shown as formula (5), formula (6), formula (7) and formula (8).
In the formula: g1，g2，g3，g4The weighted gradient values of the current pixel point in all directions are used respectively, and the weighted gradient values are used for weakening the influence of noise. Meanwhile, the gradient with a longer distance takes a smaller weight, and the idea of local mean is also added.
And calculating the gradient value of each pixel in the whole frame image according to a formula to obtain a gradient map. Since the difference between the noise point and its neighborhood is also large, in order to further filter the noise, the calculated gradient map is subjected to a gaussian filtering once, and the remaining points with large gradients can basically be determined to be pixel points located near the edge. The traditional POCS method is used for correcting the initial reference frame in an undifferentiated mode, a smooth area and an edge area in an image are not distinguished, the gray value change of the smooth area is smaller than that of the edge area, but the correction degree of the smooth area is the same as that of the edge area, and therefore burrs can appear in the reconstructed image in the smooth area.
The different information amounts contained in the neighborhood of the pixel can be represented by a gaussian gradient map, and the property is that the gradient value is large near a strong edge and small near a weak edge, and the relaxation operator for defining the method also has the property. The relaxation operator is shown in equation (1) above. The relaxation operator is then added to the projection formula.
Step 104: and selecting one image from the rest low-resolution images, and finding out the corresponding position of a pixel point on the low-resolution image on the initial reference frame through motion estimation.
During the imaging process, slight shake of the image sensor and slight movement of the target object cause the same object in the two images to be displaced at a sub-pixel level. Therefore, before super-resolution reconstruction of the sequence images, motion parameter estimation must be performed on the sequence images.
The motion estimation of the sequence images refers to solving the displacement difference of the same objective object between the two images, namely the difference of the coordinate position of the object between the two images. The application of motion estimation in the process of image super-resolution reconstruction specifically means that each pixel in a low-resolution image is accurately positioned to the coordinate position of the corresponding pixel of a high-resolution image corresponding to the pixel. If the corresponding position coordinates are not found accurately, the constraint set defined by the POCS algorithm can process the pixels at the wrong positions, and the function of image super-resolution reconstruction cannot be achieved.
There are many methods for performing motion estimation operations on a sequence of images, and block matching based methods are used herein, in which each frame of a current sequence of low resolution images is divided into blocks. And searching the image block with the best matching result in the search window of the target high-resolution reference frame. The difference between the searched best image block and the position of the image block in the original image is called a motion vector. Specifically, if the position coordinate of the center pixel of the image block with the current size of 3 × 3 in one frame image is (m, n), and the position coordinate of the center pixel of the image block in the other frame image is (m + i, n + j), the displacement difference between the two images of the image block is (i, j), which is a motion vector. The criterion for finding the best matching image block is different, and the minimum mean square error criterion is adopted in the text, is intuitive in definition and relatively concise in calculation, and is the most common matching criterion at present.
Step 105: simulating a degradation process by using a PSF (particle swarm optimization), reducing the initial reference frame image to the same size as the low-resolution image, solving the residual error of the initial reference frame image and the low-resolution image, and correcting the initial reference frame by using an improved projection formula according to the residual error;
all imaging systems cannot be ideal optical imaging systems, and there is always image degradation for a particular imaging process. A particular imaging point may not be imaged exactly completely on a pixel grid in the image, but may produce some blurring, with some of the imaging signal overflowing to the surrounding imaging grid. This is caused by the point spread function PSF of the imaging system.
In the implementation process of the POCS algorithm, each pixel of the low-resolution image is mapped into the high-resolution imaging grid one by one, and the range of action of the PSF is found. And calculating an estimated value of the low-resolution image corresponding to the current pixel according to the PSF and the degradation model of the image, comparing the estimated value with an actual value of the low-resolution image, and correcting the related pixel points of the current high-resolution reference frame if the calculated residual exceeds a preset range until the residual is reduced to be within the preset range. According to the principle, the correction process of the POCS algorithm is not one-time correction, but needs to be iterated for multiple times, and the residual errors obtained by calculation of all the pixel points can be reduced to be within an allowable range.
In a specific implementation of the POCS algorithm, the image point spread function is determined by the specific imaging system, and h (x, y) represents a common gaussian model, which can be expressed as:
in the above formula, X0And Y0Coordinates of a center point representing a point spread function, x and y represent abscissa and ordinate of a pixel of the target image, ShThe support domain, which represents the point spread function, is typically 3 x 3 or 5 x 5 in size.
The modified projection formula is as follows:
in the formula: x is the initial reference frame, λ is the relaxation operator, r(y)As residual, h is the PSF template, δ0Is a noise related quantity. According to the formula (10), the gradation can be adjusted according to the imageThe image is adaptively corrected by the change characteristics, the correction degree is small in a smooth area, burrs are restrained, the correction degree is large in an edge area, and the convergence speed is accelerated.
Step 106: judging whether the sequence low-resolution images are used or not;
the correction of the initial reference frame is guided by using the sequence low-resolution images, the more guide images are used, the more spatial information is blended into the initial reference frame, and the higher spatial resolution of the high-resolution image after the final correction is finished is, so that the low-resolution images should be used as much as possible to guide the correction. If the sequence low-resolution images are used, entering the next step; if there are more sequential low resolution images that are not used, the process returns to step 104 where the low resolution image is again selected to guide the correction.
Step 107: solving the mean square error of the high-resolution image of the iterative reconstruction and the high-resolution image of the iterative reconstruction of the last time, and comparing the mean square error with a set threshold value;
the mean square error of the high-resolution images reconstructed by two iterations can be used to measure the similarity between the two images. If the mean square error is small enough, the two images are considered to be very similar, the algorithm is converged, the gray level image of the wave band is reconstructed, and the next step is carried out; otherwise, the algorithm is not converged and should return to step 104 for the next iteration.
Step 108: and (4) using the gray level images of all wave bands of the hyperspectral images from the step 101 to the step 107, and finally completing the super-resolution reconstruction of the hyperspectral images.
True hyperspectral data experiment
The improved POCS-based hyperspectral image super-resolution reconstruction method provided by the invention is subjected to application effect analysis and evaluation by adopting two sets of public and real hyperspectral image data sets.
1. Data set and parameter settings
(1) CAVE data set
Both sets of data from the CAVE database were used for the experiments, and are "face" and "face and real food" data. The spectral band range of the CAVE data set image is 400nm to 700nm, 31 spectral bands are provided, and the size of the hyperspectral image of each band is 512 multiplied by 512. FIG. 2a shows a grayscale image obtained from the 10 th band after 2-fold down-sampling of the face data set; FIG. 3a shows a grayscale image of the 10 th band obtained after 2-fold down-sampling of the fake and real food dataset.
Evaluation index of experiment
(1) Mean Square Error (Mean Square Error, MSE)
The mean square error of the image refers to the sum of absolute values of gray differences of all corresponding pixel points of the two images divided by the total number of pixels of the image. The smaller the mean square error, the smaller the difference between the two images, and the more similar the images. The mean square error MSE of an image is defined in the form:
where x and y represent the two images, respectively, and M and N represent the horizontal and vertical lengths of the images, respectively.
(2) Peak Signal-to-Noise Ratio (PSNR)
The peak signal-to-noise ratio PSNR is defined in the form:
(3) structural SIMilarity (SSIM)
The structural similarity is an index for measuring the similarity of two images, and the definition form is as follows:
where x and y are two images, μxIs the average value of x, μyIs the average value of y and is,
is the variance of x and is,
is the variance of y, σxyIs the covariance C of x and y1And C2Is a constant used to maintain stability.
2. Analysis and evaluation of test results
The results of an experiment using two groups of real hyperspectral image data are shown in tables 1-2, and the corresponding reconstruction result images are shown in attached figures 2b and 3 b.
The experiment will introduce the traditional POCS method, compare its reconstruction result with the improved POCS method:
compared with the traditional POCS method, the improved POCS method eliminates the problems of edge blurring and burr generation in a smooth area of a reconstructed image to a certain extent, adapts the iteration times and avoids the subjectivity of manually setting the iteration times.
TABLE 1 face data set super resolution reconstruction results
TABLE 2 super resolution reconstruction of fake and real food data sets
Aiming at the problem of low spatial resolution of the hyperspectral image, the invention provides an improved POCS-based super-resolution reconstruction method to improve the spatial resolution of the hyperspectral image. The theoretical basis of the method is set theory, a plurality of priori information of the image are mainly utilized, each priori information can be seen as a closed convex set, and the intersection of the closed convex sets is the solution space of the POCS method. The method comprises the steps of firstly, randomly selecting one gray image of a first wave band of a sequence low-resolution high-spectrum image, obtaining an initial reference frame through bicubic interpolation, and relieving the problem of edge blurring of a reconstructed image to a certain extent; then, the remaining gray level images of the first wave band are corrected according to a projection formula with a relaxation operator, and burrs in a smooth area of a reconstructed image are suppressed; after iteration is carried out for more than two times, the condition of exiting iteration is taken as whether the mean square error between the iteration reconstructed images of the previous iteration and the next iteration is less than a certain threshold value, so that the iteration process is self-adaptive, and the subjectivity of considering the set iteration times is avoided; and finally, repeating the process on the gray level image of each wave band of the hyperspectral image to obtain the hyperspectral image with improved spatial resolution. The experimental results of two groups of truly-disclosed hyperspectral data sets prove the effectiveness of the improved hyperspectral image super-resolution reconstruction method based on POCS provided by the invention.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (4)
1. An improved super-resolution reconstruction method of hyperspectral images based on POCS is characterized in that: the method comprises the following steps:
s1: selecting a gray level image of a certain wave band from the high spectrum images of the sequence low resolution to obtain a sequence low resolution gray level image of the wave band;
s2: randomly selecting any one of the sequential low-resolution gray level images and carrying out interpolation processing on the selected image in a bicubic interpolation mode to obtain an initial reference frame;
s3: calculating a gradient map of the image, performing Gaussian filtering processing on the gradient map, acquiring a relaxation operator according to the gradient map, and adding the relaxation operator into a projection formula to obtain an improved projection formula;
s4: selecting one image from the rest low-resolution images, and finding out the corresponding position of a pixel point on the low-resolution image on the initial reference frame through motion estimation;
s5: simulating a degradation process by adopting a point spread function PSF, reducing the initial reference frame image to the same size as the low-resolution image, then acquiring a residual error between the degraded image and the original low-resolution image, and modifying and optimizing the initial reference frame of the wave band on the initial reference frame by utilizing an improved projection formula according to the residual error;
s6: judging whether all the sequence low-resolution images are used for optimizing the initial reference frame, if so, entering S7, otherwise, returning to S4;
s7: calculating the mean square error of the high-resolution image of the iterative reconstruction and the high-resolution image of the iterative reconstruction last time, comparing the mean square error with a set threshold, if the mean square error is less than the set threshold, entering S8, and if the mean square error is greater than the set threshold, returning to S4;
s8: and repeating the steps from S1 to S7 until the gray level images of all wave bands of the hyperspectral image are reconstructed.
2. The improved POCS-based hyperspectral image super-resolution reconstruction method of claim 1, further characterized by: the following method is specifically adopted in S2:
s21: calculating the corresponding position (X + u, Y + v) of the point (X, Y) in the magnified image in the original image according to the multiple k of the original image A and the magnified image B;
s22: finding 16 pixel points closest to (x + u, y + v) in the original image;
s23: calculating the horizontal weight and the vertical weight of the 16 pixel points according to a bicubic interpolation basis function, wherein a bicubic interpolation basis function calculation formula is as follows:
s24: the pixel values of 16 pixel points are combined with the horizontal and vertical weights thereof to carry out weighted superposition to obtain the pixel value of a point (X, Y) in the amplified image, and the calculation formula is as follows:
3. the improved POCS-based hyperspectral image super-resolution reconstruction method of claim 1, further characterized by: the following method is specifically adopted in S3:
s31: and solving the weighted gradient value of the current pixel point in each direction according to the following formula:
s32: and calculating the total weighted gradient of each pixel in the image according to the following formula to obtain an image gradient map with the adaptive regional correction characteristic:
s33: performing Gaussian filtering processing on the obtained gradient map to remove noise;
s34: obtaining a relaxation operator according to the obtained gradient map:
s35: by residual error r(y)And noise coefficient delta0Taking the relative size of the original reference frame x as a criterion, subtracting the residual error from the noise coefficient, normalizing the PSF template h, and performing adaptive correction on each pixel point in the original reference frame x by combining a relaxation operator lambda, wherein the process is expressed by an improved projection formula as follows:
4. the improved POCS-based hyperspectral image super-resolution reconstruction method of claim 1, further characterized by: the following method is specifically adopted in S7:
calculating the mean square error of the two iteration reconstructed images according to the following formula:
and judging the relative magnitude of the mean square error and the set threshold, if the mean square error is small, exiting the iteration, otherwise, returning to S4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110558431.XA CN113506212B (en) | 2021-05-21 | 2021-05-21 | Improved hyperspectral image super-resolution reconstruction method based on POCS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110558431.XA CN113506212B (en) | 2021-05-21 | 2021-05-21 | Improved hyperspectral image super-resolution reconstruction method based on POCS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113506212A true CN113506212A (en) | 2021-10-15 |
| CN113506212B CN113506212B (en) | 2023-05-23 |
Family
ID=78008481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110558431.XA Active CN113506212B (en) | 2021-05-21 | 2021-05-21 | Improved hyperspectral image super-resolution reconstruction method based on POCS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113506212B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114037608A (en) * | 2021-10-20 | 2022-02-11 | 浙江大华技术股份有限公司 | Image registration method |
| CN116188305A (en) * | 2023-02-16 | 2023-05-30 | 长春理工大学 | Multispectral image reconstruction method based on weighted guided filtering |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005026765A1 (en) * | 2003-09-18 | 2005-03-24 | The Institute Of Cancer Research; Royal Cancer Hospital | Dynamic mr imaging involving constrained reconstruction using pocs algorithm |
| CN103136734A (en) * | 2013-02-27 | 2013-06-05 | 北京工业大学 | Restraining method on edge Halo effects during process of resetting projections onto convex sets (POCS) super-resolution image |
| CN108537728A (en) * | 2018-03-05 | 2018-09-14 | 中国地质大学（武汉） | High spectrum image super-resolution forming method and system based on spectrum fidelity |
| CN108765288A (en) * | 2018-05-25 | 2018-11-06 | 杭州电子科技大学 | A kind of POCS Image Super-resolution Reconstruction methods kept based on edge |
-
2021
- 2021-05-21 CN CN202110558431.XA patent/CN113506212B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005026765A1 (en) * | 2003-09-18 | 2005-03-24 | The Institute Of Cancer Research; Royal Cancer Hospital | Dynamic mr imaging involving constrained reconstruction using pocs algorithm |
| CN103136734A (en) * | 2013-02-27 | 2013-06-05 | 北京工业大学 | Restraining method on edge Halo effects during process of resetting projections onto convex sets (POCS) super-resolution image |
| CN108537728A (en) * | 2018-03-05 | 2018-09-14 | 中国地质大学（武汉） | High spectrum image super-resolution forming method and system based on spectrum fidelity |
| CN108765288A (en) * | 2018-05-25 | 2018-11-06 | 杭州电子科技大学 | A kind of POCS Image Super-resolution Reconstruction methods kept based on edge |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114037608A (en) * | 2021-10-20 | 2022-02-11 | 浙江大华技术股份有限公司 | Image registration method |
| CN116188305A (en) * | 2023-02-16 | 2023-05-30 | 长春理工大学 | Multispectral image reconstruction method based on weighted guided filtering |
| CN116188305B (en) * | 2023-02-16 | 2023-12-19 | 长春理工大学 | Multispectral image reconstruction method based on weighted guided filtering |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113506212B (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127688B (en) | A kind of super-resolution image reconstruction method and its system | |
| CN106204447A (en) | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance | |
| JP5524692B2 (en) | Information processing apparatus and method, and program | |
| CN106169174B (en) | Image amplification method | |
| CN107507135A (en) | Image reconstructing method based on coding aperture and target | |
| CN103020898B (en) | Sequence iris image super resolution ratio reconstruction method | |
| CN111783583B (en) | SAR image speckle suppression method based on non-local mean algorithm | |
| WO2014070489A1 (en) | Recursive conditional means image denoising | |
| CN113506212B (en) | Improved hyperspectral image super-resolution reconstruction method based on POCS | |
| CN110070539A (en) | Image quality evaluating method based on comentropy | |
| WO2023115801A1 (en) | Point-by-point correction and restoration method and system for large field-of-view degraded image having aero-optical effect | |
| US11145032B2 (en) | Image processing apparatus, method and storage medium for reducing color noise and false color | |
| WO2018225133A1 (en) | Image processing device, image processing method and image processing program | |
| CN105513033A (en) | Super-resolution reconstruction method based on non-local simultaneous sparse representation | |
| CN112184549B (en) | Super-resolution image reconstruction method based on space-time transformation technology | |
| Liu et al. | Image super-resolution based on adaptive joint distribution modeling | |
| CN112488920B (en) | Image regularization super-resolution reconstruction method based on Gaussian-like fuzzy core | |
| WO2016098323A1 (en) | Information processing device, information processing method, and recording medium | |
| JP2023003763A (en) | Learning apparatus, image processing apparatus, learning processing method, and program | |
| JP2017010094A (en) | Image processing apparatus, imaging device, image processing method, image processing program, and recording medium | |
| US20120038785A1 (en) | Method for producing high resolution image | |
| CN103618904B (en) | Motion estimation method and device based on pixels | |
| CN115937302A (en) | Hyperspectral image sub-pixel positioning method combined with edge preservation | |
| Chen et al. | Structural similarity-based nonlocal edge-directed image interpolation | |
| Milyukova et al. | Image restoration using two-dimensional variations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |