CN107292821B - A kind of super-resolution image reconstruction method and system - Google Patents

Abstract

The present invention provides a method and system for super-resolution image reconstruction, including performing K -means classification on all pixels of the training images in the training image library, and recording the cluster centers; The distance principle classifies the image blocks of all training images, records the category of each image block, and obtains the joint dictionary of each category respectively; divides the low-scoring test image to be reconstructed into blocks, and calculates the relationship between each image block and each cluster The distance between the centers is classified according to the principle of the minimum distance, and the joint dictionary of the corresponding category is adaptively selected for sparse modeling, and the sparse coefficient is obtained by solving; the joint dictionary and the sparse coefficient are used to reconstruct the corresponding high-resolution of each image block in the low-scoring test image The optimal reconstruction image corresponding to the entire low-scoring test image is obtained. The invention can adaptively train joint dictionaries, and adaptively select appropriate dictionaries for sparse modeling, with better reconstruction accuracy.

Description

A super-resolution image reconstruction method and system

technical field

The invention belongs to the field of computer vision, and in particular relates to a technical scheme for super-resolution reconstruction of images by training a joint dictionary and adaptive sparse modeling.

Background technique

Super-resolution reconstruction (SR) refers to the use of certain methods to reconstruct one or more low-resolution images into high-resolution images with higher pixel density and detailed information. It plays a very important role in the fields of biomedicine, remote sensing monitoring, and public safety.

The expression of signal sparsity has always been concerned by researchers. For example, the basic principle of the classic image compression algorithm JPEG is to use the image sparsity in the DCT domain to compress it. In 2006, after Candes et al. proposed the compressed sensing theory and demonstrated it systematically, the use of signal sparsity for analysis and application has become a research hotspot.

Image super-resolution reconstruction algorithms have always been a concern of researchers. High-resolution images are degraded to low-resolution images by warping, blurring, sampling, and adding system noise. How to reverse solve, restore or construct a clearer image has become the focus of everyone's research. The over-complete sparse representation of images has become the mainstream image representation model in recent years, and research on image denoising and super-resolution reconstruction using image sparsity continues. Since Yang et al. got inspiration from compressed sensing in 2008 and used sparse coding for super-resolution reconstruction, the research on this method has continued to heat up. This method establishes the corresponding relationship between high-resolution and low-resolution image blocks through dictionaries and sparse coefficients. First, the corresponding high-resolution and low-resolution image blocks are obtained from the image training library, and then the Feature-Sign Search algorithm (Feature-Sign Search, FSS ) to solve the l _{+ 1} constrained optimization problem when solving dictionaries and sparse coefficients, and through joint training of high-resolution and low-resolution dictionary pairs, LR image blocks and HR image blocks have the same sparse coefficients in the corresponding dictionaries. During reconstruction, the sparse coefficients are obtained through the LR image blocks and the LR dictionary, and then the corresponding HR image blocks are obtained by using the HR dictionary and the sparse coefficients. The classic super-resolution reconstruction based on sparse expression has two shortcomings: ① FSS linear programming is used to train the dictionary, which requires a large amount of calculation; ② only gradient features are used to distinguish image blocks when training the dictionary, and the dictionary has limited representation ability.

Contents of the invention

Aiming at the deficiencies of existing methods, a super-resolution reconstruction method and system based on sparse representation and adaptive multi-resolution joint dictionary was invented, which can effectively make up for the shortcomings of the original method’s insufficient representation of dictionary capabilities and improve image super-resolution reconstruction. precision.

In order to achieve the above object, the technical solution of the present invention provides a super-resolution image reconstruction method, comprising the following steps,

Step a: Carry out K-means classification to all pixels of the training image in the training image database, and record the clustering center; perform image segmentation on each training image, and classify the image blocks of all training images based on the clustering center according to the principle of minimum distance, Record the category to which each image block belongs;

Step b, according to the image block classification results in step a, respectively obtain the joint dictionary {D _h , D _l } of each category C _k , k=1,...,K, where D _h is a high-scoring dictionary, D _l is a low score dictionary;

Step c, divide the low-scoring test image y to be reconstructed into blocks, calculate the distance between each image block and each cluster center, classify according to the principle of minimum distance, adaptively select the joint dictionary of the corresponding category for sparse modeling, and solve Get the sparse coefficient α ^* ;

Step d, use the joint dictionary and the sparse coefficient α ^* to reconstruct the corresponding high-resolution image patch for each image patch y _i in the low-scoring test image y The optimal reconstructed image corresponding to the entire low-scoring test image y is obtained.

Moreover, the minimum distance principle is implemented using Euclidean distance.

Moreover, in step c, assuming that a certain image block y _i belongs to category C _k , select the corresponding k-th joint dictionary {D _h , D _l }, and perform sparse expression according to the following formula,

Among them, α is the sparse coefficient, the vector vector β is a preset parameter, the matrix P is a matrix for extracting the overlapping area between the current image block and the previously reconstructed high-resolution image block, and ω is the pixel of the overlapping area on the current image block and the previously reconstructed high-resolution image block value, F is a linear feature extraction operator.

The present invention also provides a super-resolution image reconstruction system, comprising the following modules,

The first module is used to classify all pixels of the training image in the training image library by K-means, and record the clustering center; perform image segmentation on each training image, and divide the image blocks of all training images based on the principle of minimum distance based on the clustering center Classify and record the category to which each image block belongs;

The second module is used to obtain the joint dictionary {D _h , D _l } of each category C _k according to the image block classification result in step a, k=1,...,K, where D _h is a high-scoring dictionary , D _l is a low score dictionary;

The third module is used to block the low-scoring test image y to be reconstructed, and calculate the distance between each image block and each cluster center, classify according to the principle of minimum distance, and adaptively select the joint dictionary of the corresponding category for sparse construction Modulus, solve to get the sparse coefficient α ^* ;

The fourth module is used to reconstruct the high-resolution image block corresponding to each image block y _i in the low-scoring test image y by using the joint dictionary and the sparse coefficient α ^* The optimal reconstructed image corresponding to the entire low-scoring test image y is obtained.

Moreover, the minimum distance principle is implemented using Euclidean distance.

Moreover, in the third module, assuming that a certain image block y _i belongs to category C _k , select the corresponding k-th joint dictionary {D _h , D _l }, and perform sparse expression according to the following formula,

Among them, α is the sparse coefficient, the vector vector β is a preset parameter, the matrix P is a matrix for extracting the overlapping area between the current image block and the previously reconstructed high-resolution image block, and ω is the pixel of the overlapping area on the current image block and the previously reconstructed high-resolution image block value, F is a linear feature extraction operator.

Aiming at the shortcomings of insufficient representation ability of dictionaries in the prior art, the present invention classifies all pixels of the training images in the training image library, and then determines the classification of each image block, and self-adaptively constructs a joint dictionary for improvement, hoping to better image to express. Compared with the existing methods, the present invention can adaptively train the joint dictionary, and can adaptively select the appropriate dictionary for sparse modeling according to the basic characteristics of the image block, with better precision, and can be used in medical images, satellite images and The popularization and use of video and other application fields has important market value.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are exemplary only, and are not intended to limit the scope of the present invention.

As shown in Figure 1, the embodiment of the present invention proposes to perform super-resolution reconstruction of the image based on an adaptive multi-resolution joint dictionary based on sparse expression, first perform adaptive joint dictionary training on the training image database, and then perform sparse modeling on the blurred image , reconstruct the image after obtaining the sparse coefficients and output it. The specific process is as follows:

Step a, performing K-means classification on all pixels of the training image in the training image library, recording the cluster center, and performing block classification on the training image;

In the embodiment, there are several high-resolution training images in the training image library, read the pixels of all the training images and use the K-means algorithm to obtain K cluster centers, and perform image segmentation on the training images, based on the cluster centers Classify the image blocks of all training images according to the principle of minimum distance, and record the category of each image block of each training image. During specific implementation, the user can preset the value of K and the size parameters of the divided image blocks. Preferably, the image block adopts overlapping division, that is, several pixels overlap between two adjacent blocks, for example, the size of each image block is 40×40, and 5 pixels overlap between adjacent image blocks.

For the convenience of implementation and reference, the K-means algorithm is provided as follows:

1) Randomly select K pixel values from all the pixels of the N training images as the cluster centers;

2) For each remaining pixel, calculate its distance to each centroid, and classify it into the nearest centroid class;

3) Recalculate the cluster center of each class that has been obtained;

4) Iterate 2) to 3) until the new cluster center is equal to the original cluster center or less than the preset specified threshold, and the algorithm ends.

In step a of the embodiment, it is assumed that z _n , n=1, . . . , N are images to be trained in the training image database. Use the K-means algorithm to cluster the pixels x _q in all high-resolution training images, q=1,...,Q, Q is the total number of pixels in all training images. Generate K categories, let a category be recorded as where C _k is the kth class, is the tth pixel sample in class C _k , q _k is the number of pixels in class C _k , satisfying Compute the class center of C _k and the radius

For a certain image block on any training image, compare the distance to each class center μ _k , and confirm the category according to the minimum value. During specific implementation, the user may select a distance measurement method, such as Euclidean distance, Mahalanobis distance, and the like. The embodiment preferably adopts the Euclidean distance.

Step b, according to the image block classification result of step a, carry out the training of joint dictionary class by class;

As a high-scoring image, each training image has a corresponding low-scoring image, and the low-scoring image is divided into low-resolution image blocks corresponding to the high-resolution image blocks on the training image.

According to the high-resolution image blocks and corresponding low-resolution image blocks belonging to each category C _k , the joint dictionary {D _h , D _l }, _k =1, . Among them, D _h is a high score dictionary, and D _l is a low score dictionary.

In step b of the embodiment, for different categories of C _k , the joint dictionary {D _h , D _l } between the high-resolution image block and the low-resolution image block is obtained, and the solution formula is as follows:

In the formula, Z represents a sparse coefficient matrix, X ^h is a high-resolution image block, and Y ^l is a corresponding low-resolution image block. N and M are respectively the dimensions of the high-resolution and low-resolution image blocks expanded into a one-dimensional vector form, and the image blocks can be expanded from top to bottom and from left to right during specific implementation. The parameter λ is used to balance the sparsity of the solution, and an empirical value can be used for specific implementation.

Step c: Divide the low-scoring test image y to be reconstructed into blocks, and calculate the distance between each image block y _i and each cluster center, classify the image block y _i according to the principle of minimum distance, and select the joint dictionary of the corresponding category to perform Sparse modeling, solving sparse coefficients;

In step c of the embodiment, the low-scoring test image y is divided into blocks, and the Euclidean distances from the image block to each cluster center μ _k are respectively calculated, and classified according to the minimum distance: that is, for a certain image block y in the low-scoring test image y _i , its class (d _k represents the distance from the current image block to the kth cluster center, means take the smallest d _k ).

The minimum distance principle in the embodiment is the same as step a, and the Euclidean distance is used to realize it. The way to block the low-score test image y to be reconstructed is consistent with the low-score image block method corresponding to the training image.

Select the corresponding k-th joint dictionary {D _h , D _l }, and perform sparse expression according to the following formula:

Among them, α is the sparse coefficient, the vector vector β is a preset parameter for maintaining the consistency between the high-resolution image block matching the low-resolution input and the neighborhood. In the embodiment, β is an empirical value (for example, β=1 in some documents). The matrix P is a matrix for extracting the overlapping area between the current image block and the previously reconstructed high-resolution image block, and ω contains the pixel values of the overlapping area on the current image block and the previously reconstructed high-resolution image block. F is a linear feature extraction operator. From this, the optimal sparse coefficient is obtained, denoted as α ^* .

Step d, use the joint dictionary and the sparse coefficient α ^* to reconstruct the corresponding high-resolution image patch for each image patch y _i in the low-scoring test image y Reconstruct each image block in the low-scoring test image y and put it into the corresponding position of the corresponding high-resolution image, and take the mean value for the overlapping part, so that the corresponding optimal reconstruction of the entire low-scoring test image y can be output image.

During specific implementation, the method provided by the present invention can realize the automatic operation process based on software technology, and can also realize the corresponding system in a modular manner.

The embodiment of the present invention also provides a super-resolution image reconstruction system, including the following modules,

The first module is used to classify all pixels of the training image in the training image library by K-means, and record the clustering center; perform image segmentation on each training image, and divide the image blocks of all training images based on the principle of minimum distance based on the clustering center Classify and record the category to which each image block belongs;

The second module is used to obtain the joint dictionary {D _h , D _l } of each category C _k according to the image block classification result in step a, k=1,...,K, where D _h is a high-scoring dictionary , D _l is a low score dictionary;

The third module is used to block the low-scoring test image y to be reconstructed, and calculate the distance between each image block and each cluster center, classify according to the principle of minimum distance, and adaptively select the joint dictionary of the corresponding category for sparse construction Modulus, solve to get the sparse coefficient α ^* ;

The fourth module is used to reconstruct the high-resolution image block corresponding to each image block y _i in the low-scoring test image y by using the joint dictionary and the sparse coefficient α ^* The optimal reconstructed image corresponding to the entire low-scoring test image y is obtained.

For the specific implementation of each module, reference may be made to the corresponding steps, which will not be described in detail in the present invention.

To sum up, the present invention first constructs multiple joint dictionaries adaptively through the training image database, and then divides the images into blocks. Each image block adaptively selects the corresponding dictionary for sparse reconstruction, and outputs a high-scoring image. The technical solution combines the advantages of joint dictionaries and self-adaptive selection of multiple dictionaries, which guarantees the reconstruction of higher quality images.

Claims (4)

1. A super-resolution image reconstruction method, characterized in that: comprising the following steps, Step a: Carry out K-means classification to all pixels of the training image in the training image database, and record the clustering center; perform image segmentation on each training image, and classify the image blocks of all training images based on the clustering center according to the principle of minimum distance, Record the category to which each image block belongs; Step b, according to the image block classification results in step a, respectively obtain the joint dictionary {D _h , D _l } of each category C _k , k=1,..., K, where D _h is a high-scoring dictionary, D _l is a low score dictionary; Step c, divide the low-scoring test image y to be reconstructed into blocks, calculate the distance between each image block and each cluster center, classify according to the principle of minimum distance, adaptively select the joint dictionary of the corresponding category for sparse modeling, and solve To get the sparse coefficient α ^* , the implementation is as follows, Assuming that a certain image block y _i belongs to category C _k , select the corresponding k-th joint dictionary {D _h , D _l }, and perform sparse expression according to the following formula, Among them, α is the sparse coefficient, the vector vector β is a preset parameter, the matrix P is a matrix for extracting the overlapping area between the current image block and the previously reconstructed high-resolution image block, and ω is the pixel of the overlapping area on the current image block and the previously reconstructed high-resolution image block value, F is a linear feature extraction operator, and the parameter λ is used to balance the sparsity of the solution; Step d, use the joint dictionary and the sparse coefficient α ^* to reconstruct the corresponding high-resolution image patch for each image patch y _i in the low-scoring test image y The optimal reconstructed image corresponding to the entire low-scoring test image y is obtained. 2. The super-resolution image reconstruction method according to claim 1, characterized in that: the principle of minimum distance is realized by Euclidean distance. 3. A super-resolution image reconstruction system, characterized in that: comprising the following modules, The first module is used to classify all pixels of the training image in the training image library by K-means, and record the clustering center; perform image segmentation on each training image, and divide the image blocks of all training images based on the principle of minimum distance based on the clustering center Classify and record the category to which each image block belongs; The second module is used to obtain the joint dictionary {D _h , D _l } of each category C _k according to the image block classification result in step a, k=1,..., K, where D _h is high sub-dictionary, D _l is a low-scoring dictionary; The third module is used to block the low-scoring test image y to be reconstructed, and calculate the distance between each image block and each cluster center, classify according to the principle of minimum distance, and adaptively select the joint dictionary of the corresponding category for sparse construction Modulus, solve to get the sparse coefficient α ^* , the implementation is as follows, Assuming that a certain image block y _i belongs to category C _k , select the corresponding k-th joint dictionary {D _h , D _l }, and perform sparse expression according to the following formula, Among them, α is the sparse coefficient, the vector vector β is a preset parameter, the matrix P is a matrix for extracting the overlapping area between the current image block and the previously reconstructed high-resolution image block, and ω is the pixel of the overlapping area on the current image block and the previously reconstructed high-resolution image block value, F is a linear feature extraction operator, and the parameter λ is used to balance the sparsity of the solution; The fourth module is used to reconstruct the high-resolution image block corresponding to each image block y _i in the low-scoring test image y by using the joint dictionary and the sparse coefficient α ^* The optimal reconstructed image corresponding to the entire low-scoring test image y is obtained. 4. The super-resolution image reconstruction system according to claim 3, characterized in that: the minimum distance principle is realized by Euclidean distance.