CN104021523B - A kind of method of the image super-resolution amplification based on marginal classification - Google Patents

Abstract

The invention discloses an image enlargement method based on edge classification. The method first performs edge detection on the low-resolution image to obtain a binarized image indicated by the edge of the low-resolution image. Then, an initial interpolation is performed on the low-resolution image to obtain an initial high-resolution image. Next, extract a 3×3 image block in the high-resolution image, classify the image block according to the direction of the edge in the binarized image corresponding to this block, and classify some pixels in the block according to its category Re-interpolate. The method of the present invention well considers the characteristics of the direction of the edge, and performs interpolation according to this, so that the details of the enlarged image are relatively clear, especially the edge part and the part close to the edge. Experiments show that the reconstructed image is very close to the original high-resolution image.

Description

A Method of Image Super-resolution Enlargement Based on Edge Classification

technical field

The invention relates to a new method for super-resolution image enlargement in image processing. Especially according to the edge information on the low-resolution image, a small image block on the initially enlarged high-resolution image is classified according to the edge points on the corresponding low-resolution image, and a certain image block in the image block is classified according to this classification. The method of re-interpolating some pixels.

Background technique

With the rapid development of digital cameras and Internet technology, people's demand for high-definition digital images is increasing day by day. However, due to the limitation of network bandwidth and storage space, the cost of storage and transmission of high-definition images is relatively high, which consumes system resources, such as consuming a very large storage space or a very large bandwidth of the system. For this reason, digital image compression technology has been proposed, resulting in international standards for image compression, such as JPEG, JPEG2000 and so on. In the current lossless image compression method, the compression factor is generally below 4. At the same time, the lossy image compression method will cause image distortion, resulting in block artifacts, ringing artifacts, and the like. For this reason, image super-resolution magnification technology has received widespread attention, and has become a hot research issue in the field of image processing in recent years.

The purpose of image super-resolution magnification is a method of obtaining high-resolution images from low-resolution images. In this way, only low-resolution images can be transmitted or stored during transmission and storage, and then high-resolution images can be obtained by using image super-resolution amplification technology on the receiver or display side. At the same time, the image super-resolution magnification technology can be combined with the existing image compression technology to further reduce the system resources consumed for storing and transmitting high-resolution images. Furthermore, the storage and transmission of high-definition video can also use the image super-resolution method to reduce the consumption of system resources.

At present, image super-resolution methods are divided into interpolation methods and sample-based methods. Among them, the example-based method needs to first establish a sample database composed of low-resolution image blocks and their corresponding high-resolution image blocks obtained by training, and its computational complexity is very large, making it difficult to apply in real time. Although the method based on interpolation has low computational complexity, it is easy to blur the image and make the edge of the image unclear. The human eye is more sensitive to the edge of the image, and its lower distortion will greatly reduce the visual effect of the image.

For this reason, the present invention proposes a method of modifying the values of edge pixels and nearby pixels in the enlarged image according to the edges of the image after the initial enlargement. Therefore, during interpolation, the interpolation direction is consistent with the edge direction of the image, and an enlarged image with clear edges is obtained.

Contents of the invention

In view of the above-mentioned defects in the prior art, the present invention proposes a method for classifying the edge conditions in an image, and then performing corresponding re-interpolation for different categories. Based on the edge information on the low-resolution image as heuristic information, interpolation is performed according to the direction of the edge, so that the edge of the enlarged image is clear, and the shortcomings of the blurred edge of the existing super-resolution interpolation method are overcome.

In order to achieve the above object, the present invention provides a method for performing 11 different classifications on edges after edge detection and then performing re-interpolation. When the magnification factor is 2×2, that is, the horizontal magnification is 2 times, and the vertical magnification is 2 times. Of course, the present invention is not limited to magnification of 2×2 times. The present invention includes:

Step 1, initial interpolation-based super-resolution upscaling to obtain an initial high-resolution image;

Step 2, performing edge detection and extraction on the low-resolution image;

Step 3, binarize the image after edge extraction, so that the binarized values on the non-edge points become 0, and the binarized values on the edge points become 1. That is, on the binarized image, the pixel value on the edge of the image is 1, and the values of other pixel points that are not edges are all 0;

Step 4, set x=0, y=0;

Step 5, take (x, y) as the upper left corner of the high-resolution image block, and extract a 3×3 image block from the high-resolution image. For a low-resolution binarized image, take (x/2, y/2) as the upper left corner and extract a 2×2 image block. According to the 2×2 edge binarized image blocks, the extracted high-resolution image blocks can be classified into the following 11 categories: upper edge class, lower edge class, left edge class, left edge class, Right edge class, left lower oblique edge class, right lower oblique edge class, upper left edge class, lower left edge class, upper right edge class, lower right edge class, other classes; then, according to the edge class, the initial enlarged image The edge pixels of the 3×3 image block extracted above and the nearby pixels are re-interpolated;

Step 6, put x=x+2, if x≤2W-5 (W is the width of low-resolution image), then skip to step 5 and operate on the next image block;

Step 7, set y=y+2, if y≤2H-5 (H is the height of the low-resolution image), then skip to step 5 and operate on the next image block;

Step eight, the super-resolution zoom-in of the current low-resolution image ends. A high-resolution image of (2W)×(2H) size is obtained.

Further, in the first step, a bilinear interpolation method is adopted. That is, for the following image blocks:

Among them, A, B, C, and D are the pixel values of the pixels on the low-resolution image respectively, and a, b, c, d, and e are the pixels to be interpolated on the high-resolution image. In the bilinear interpolation method, their pixel values are obtained by:

Wherein, the clip(x) function limits the value of x to the range of a pixel value, that is, clip(x)=max(I _min , min(x,I _max )). Here, I _min and I _max are the minimum and maximum possible values of a pixel, respectively.

Further, in the second step, the Canny operator is used for edge extraction. The specific calculation method of the Canny operator is as follows:

First, use a Gaussian filter to smooth the image, that is, usually choose a Gaussian smoothing function, and its impact function in the frequency domain is:

Wherein, what D (u, v) represented is the distance from the far point of the Fourier transform, σ is the curvature, and H (u, v) represents the degree of expansion of the Gaussian curve; for avoiding that the edge is too blurred, the present invention uses a small The convolutional template. Specifically, the present invention uses the following 5×5 template to perform convolution operations on low-resolution images:

Second, calculate the magnitude and direction of the gradient, that is, first select a template for first-order differential convolution:

Then define the gradients Ψ _{1 (} m, n), Ψ _{2 (} m, n) are respectively:

Ψ ₁ (m, n)=f(m, n)*H ₁ (m, n)

Ψ ₂ (m, n) = f(m, n)*H ₂ (m, n)

After further calculation, the strength and direction of the edge are obtained as follows:

In the original Canny operator method, the calculated gradient magnitude will be used for non-maximum (NMS) convergence; the present invention improves this step. First quantize the edge angle θ _Ψ as θ ₁ , θ ₁ ∈ {0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°}. Then, if θ ₁ =0°, then it must be Ψ(m,n)>Ψ(m,n+1) to judge the current point as the initial edge point; if θ ₁ =45°, then Ψ(m,n )>Ψ(m-1,n+1) to judge the current point as the initial edge point; if θ ₁ =90°, then Ψ(m,n)>Ψ(m-1,n) to judge the current point is the initial edge point; if θ ₁ =135°, then Ψ(m,n)>Ψ(m-1,n-1) is required to judge the current point as the initial edge point; if θ ₁ =180°, then Only Ψ(m, n)>Ψ(m,n-1) can judge the current point as the initial edge point; if θ ₁ =225°, then Ψ(m,n)>Ψ(m+1,n- 1) to determine the current point as the initial edge point; if θ ₁ =270°, then Ψ(m,n)>Ψ(m+1,n) is required to determine the current point as the initial edge point, if θ ₁ = 315°, then Ψ(m, n)>Ψ(m+1, n+1) must be used to determine that the current point is the initial edge point. In this way, the amount of calculation can be reduced and better results can be achieved.

Finally, a double-threshold algorithm is used to connect the detected image edges. That is, when the gradient magnitude Ψ(m, n) of the initial edge point detected in the previous step>T _h , this point is determined to be an image edge point. Then, take these edge points as seeds, scan the adjacent points around them, and when the gradient magnitude Ψ(m, n)>T _l , add this point to the set of image edge points. In the present invention, according to a large number of experiments and their results, the values of these two parameters _{Th h} and T _l are set as T _h =200 and T _l =100.

Further, in the fifth step, the process of classifying the image blocks according to the edges in the 3×3 image blocks can be described as follows. That is, for image blocks extracted from high-resolution images

In this image block, the pixels a ₁₁ , a ₁₃ , a ₃₁ , and a ₃₃ belong to the pixels in the low-resolution image, and the values of other pixels in this block are obtained by interpolation. According to the corresponding low-resolution binarized edge distribution map, the image block can be classified according to the position where the edge point appears in the block. Its categories are as follows:

(a) Upper edge class. At this time, a ₁₁ and a ₁₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₁ , a ₂₂ , and a ₂₃ . Its interpolation formula is

Wherein, clip(x) is defined as mentioned above, α and β are two weights, and the condition α+β=1 is satisfied.

(b) Lower edge class. At this time, a ₃₁ and a ₃₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₁ , a ₂₂ , and a ₂₃ . Its interpolation formula is

(c) Left edge class. At this time, a ₁₁ and a ₃₁ are edge points of the image. At this time, the pixel points that need to be re-interpolated are a ₁₂ , a ₂₂ , and a ₃₂ . Its interpolation formula is

(d) Right edge class. At this time, a ₁₃ and a ₃₃ are edge points of the image. At this time, the pixel points that need to be re-interpolated are a ₁₂ , a ₂₂ , and a ₃₂ . Its interpolation formula is

(e) Lower-left slanted edge class. At this time, a ₁₁ and a ₃₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , a ₂₁ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(f) Right lower sloping edge class. At this time, a ₁₃ and a ₃₁ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , a ₂₁ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(g) Top left box edge class. Lower left sloping edge class. At this time, a ₁₁ , a ₁₃ and a ₃₁ are image edge points. At this time, the pixel points that need to be re-interpolated are a ₂₂ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(h) The edge class of the lower left box. Lower left sloping edge class. At this time, a ₁₁ , a ₃₁ and a ₃₃ are image edge points. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , and a ₂₃ . Its interpolation formula is

(i) Top right box edge class. At this time, a ₁₁ , a ₁₃ and a ₃₃ are image edge points. At this time, the pixel points that need to be re-interpolated are a ₂₂ , a ₂₁ , and a ₃₂ . Its interpolation formula is

(j) The edge class of the lower right box. At this time, a ₃₁ , a ₁₃ and a ₃₃ are image edge points. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₂₁ , and a ₁₂ . Its interpolation formula is

(k) Other categories. All other cases fall into this category. For cases belonging to this category, no re-interpolation of the extracted high-resolution image blocks is performed in the present invention.

To sum up, the present invention first performs initial interpolation and amplification, here any existing interpolation and amplification method can be selected, and then the method of the present invention is used to improve it and improve its performance. Since the bilinear interpolation method has lower computational complexity and better performance, the present invention adopts the bilinear interpolation method for initial interpolation amplification. Then, the present invention uses the Canny operator to extract the edge of the low-resolution image. Next, extract a block with a size of 3×3 in the high-resolution image, and classify the block according to the position and direction of the edge of the block in the low-resolution image. When the block belongs to the top 10 categories, according to the edge The category of this block is re-interpolated; when this block belongs to other classes, the original interpolation is retained. Then, to extract the next block, do the same. Until this is done for all blocks in high resolution. Finally, a high-resolution image with sharp edges is obtained. Its innovation lies in the extraction of 3×3 blocks of high-resolution images, and using the edge information on low-resolution images to classify them, and perform targeted re-interpolation according to their categories, or retain the original Some values make the edges in the image stand out and clearer. Improve the effect of image enlargement.

The idea, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings, so as to fully understand the purpose, features and effects of the present invention.

Description of drawings

Fig. 1 is the flowchart of the super-resolution image reconstruction algorithm based on edge classification of the present invention;

Fig. 2 is a graph of experimental results of the method for super-resolution image reconstruction based on edge classification of the present invention.

detailed description

Below in conjunction with accompanying drawing, the embodiment of the present invention is described in detail: present embodiment implements under the premise of the technical scheme of the present invention, has provided detailed implementation and specific operation process, but protection scope of the present invention is not limited to the following the embodiment.

As shown in accompanying drawing 1, the method for the new image enlargement based on edge classification of the present invention is carried out according to the following steps:

Step 1, initial bilinear interpolation-based super-resolution upscaling to obtain an initial high-resolution image; that is, for the following image blocks:

Among them, A, B, C, and D are the pixel values of the pixels on the low-resolution image respectively, and a, b, c, d, and e are the pixels to be interpolated on the high-resolution image. In the bilinear interpolation method, their pixel values are obtained by:

Wherein, the clip(x) function limits the value of x to the range of a pixel value, that is, clip(x)=max(I _min , min(x,I _max )). For an image with 8-bit luminance, I _min =0, I _max =255.

Here, I _min and I _max are the minimum and maximum possible values of a pixel, respectively.

The second step is to perform edge detection and extraction on the low-resolution image; here, the Canny operator is used for edge extraction. The specific calculation method of the Canny operator is as follows:

First, use a Gaussian filter to smooth the image, that is, usually choose a Gaussian smoothing function, and its impact function in the frequency domain is:

Among them, D(u, v) represents the distance of Fourier transform, σ is the curvature, and H(u, v) represents the degree of Gaussian curve expansion; in order to avoid the edge being too blurred, the present invention utilizes a small convolution template .

Specifically, the present invention uses the following 5×5 template to perform convolution operations on low-resolution images:

Second, calculate the magnitude and direction of the gradient, that is, first select a template for first-order differential convolution:

Then define the gradients Ψ _{1 (} m, n), Ψ _{2 (} m, n) are respectively:

Ψ ₁ (m, n)=f(m, n)*H ₁ (m, n)

Ψ ₂ (m, n) = f(m, n)*H ₂ (m, n)

After further calculation, the strength and direction of the edge are obtained as follows:

In the original Canny operator method, the calculated gradient magnitude will be used for non-maximum (NMS) convergence; the present invention improves this step. First quantize the edge angle θ _Ψ as θ ₁ , θ ₁ ∈ {0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°}. Then, if θ ₁ =0°, then it must be Ψ(m,n)>Ψ(m,n+1) to judge the current point as the initial edge point; if θ ₁ =45°, then Ψ(m,n )>Ψ(m-1,n+1) to judge the current point as the initial edge point; if θ ₁ =90°, then Ψ(m,n)>Ψ(m-1,n) to judge the current point is the initial edge point; if θ ₁ =135°, then Ψ(m,n)>Ψ(m-1,n-1) is required to judge the current point as the initial edge point; if θ ₁ =180°, then Only Ψ(m, n)>Ψ(m,n-1) can judge the current point as the initial edge point; if θ ₁ =225°, then Ψ(m,n)>Ψ(m+1,n- 1) to determine the current point as the initial edge point; if θ ₁ =270°, then Ψ(m,n)>Ψ(m+1,n) is required to determine the current point as the initial edge point, if θ ₁ = 315°, then Ψ(m, n)>Ψ(m+1, n+1) must be used to determine that the current point is the initial edge point. In this way, the amount of calculation can be reduced and better results can be achieved.

Finally, a double-threshold algorithm is used to connect the detected image edges. That is, when the gradient magnitude Ψ(m, n) of the initial edge point detected in the previous step>T _h , this point is determined to be an image edge point. Then, take these edge points as seeds, scan the adjacent points around them, and when the gradient magnitude Ψ(m, n)>T _l , add this point to the set of image edge points. In the present invention, according to a large number of experiments and their results, the values of these two parameters _{Th h} and T _l are set as T _h =200 and T _l =100.

Step 3, binarize the image after edge extraction, so that the binarized values on the non-edge points become 0, and the binarized values on the edge points become 1. That is, on the binarized image, the pixel value on the edge of the image is 1, and the values of other pixel points that are not edges are all 0;

Step 4, set x=0, y=0;

Step 5, take (x, y) as the upper left corner of the high-resolution image block, and extract a 3×3 image block from the high-resolution image. For a low-resolution binarized image, take (x/2, y/2) as the upper left corner and extract a 2×2 image block. According to the 2×2 edge binarized image blocks, the extracted high-resolution image blocks can be classified into the following 11 categories: upper edge class, lower edge class, left edge class, left edge class, Right edge class, left lower oblique edge class, right lower oblique edge class, upper left edge class, lower left edge class, upper right edge class, lower right edge class, other classes; then, according to the edge class, the initial enlarged image The edge pixels of the extracted 3×3 image block and its nearby pixels are re-interpolated; according to the edge in the 3×3 image block, the process of classifying the image block can be described as follows. That is, for image blocks extracted from high-resolution images

In this image block, the pixels a ₁₁ , a ₁₃ , a ₃₁ , and a ₃₃ belong to the pixels in the low-resolution image, and the values of other pixels in this block are obtained by interpolation. According to the corresponding low-resolution binarized edge distribution map, the image block can be classified according to the position where the edge point appears in the block. Its categories are as follows:

(a) Upper edge class. At this time, a ₁₁ and a ₁₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₁ , a ₂₂ , and a ₂₃ . Its interpolation formula is

Wherein, the clip(x) function limits the value of x to the range of a pixel value, that is, clip(x)=max(I _min , min(x,I _max )). Here, I _min and I _max are the minimum and maximum possible values of a pixel, respectively. α and β are two weights, satisfying the condition α+β=1.

(b) Lower edge class. At this time, a ₃₁ and a ₃₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₁ , a ₂₂ , and a ₂₃ . Its interpolation formula is

(c) Left edge class. At this time, a ₁₁ and a ₃₁ are edge points of the image. At this time, the pixel points that need to be re-interpolated are a ₁₂ , a ₂₂ , and a ₃₂ . Its interpolation formula is

(d) Right edge class. At this time, a ₁₃ and a ₃₃ are edge points of the image. At this time, the pixel points that need to be re-interpolated are a ₁₂ , a ₂₂ , and a ₃₂ . Its interpolation formula is

(e) Lower-left slanted edge class. At this time, a ₁₁ and a ₃₃ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , a ₂₁ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(f) Right lower sloping edge class. At this time, a ₁₃ and a ₃₁ are edge points of the image. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , a ₂₁ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(g) Top left box edge class. Lower left sloping edge class. At this time, a ₁₁ , a ₁₃ and a ₃₁ are image edge points. At this time, the pixel points that need to be re-interpolated are a ₂₂ , a ₂₃ , and a ₃₂ . Its interpolation formula is

(h) The edge class of the lower left box. Lower left sloping edge class. At this time, a ₁₁ , a ₃₁ and a ₃₃ are image edge points. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₁₂ , and a ₂₃ . Its interpolation formula is

(i) Top right box edge class. At this time, a ₁₁ , a ₁₃ and a ₃₃ are image edge points. At this time, the pixel points that need to be re-interpolated are a ₂₂ , a ₂₁ , and a ₃₂ . Its interpolation formula is

(j) The edge class of the lower right box. At this time, a ₃₁ , a ₁₃ and a ₃₃ are image edge points. At this time, the pixel points to be re-interpolated are a ₂₂ , a ₂₁ , and a ₁₂ . Its interpolation formula is

(k) Other categories. All other cases fall into this category. For cases belonging to this category, no re-interpolation of the extracted high-resolution image blocks is performed in the present invention.

Here, the present invention determines α=0.7 and β=0.3 based on a large number of experiments.

Step 6, put x=x+2, if x≤2W-5 (W is the width of low-resolution image), then skip to step 5 and operate on the next image block;

Step 7, set y=y+2, if y≤2H-5 (H is the height of the low-resolution image), then skip to step 5 and operate on the next image block;

Step eight, the super-resolution zoom-in of the current low-resolution image ends. A high-resolution image of (2W)×(2H) size is obtained.

What the experiment of the present invention mainly selects is the face image in FERET database, wherein specifically selects four images that include skin color in different degrees for reconstruction, the size of the original high-resolution image is 120*120, according to every two pixels Taking the rule of one pixel, the size of the degraded image after downsampling becomes 60*60, that is, it is reduced to 1/4 of the original size, and the multiplier that needs to be interpolated should be set to 2 times in the length and width directions.

The present invention selects the human face image database as the experimental object to carry out the experiment, and then the experimental results are evaluated by subjective and objective evaluations to detect the quality of the experimental results. The evaluation is based on the feeling of the details of the image, and the objective evaluation is to use different formulas and algorithms to illustrate the quality of the image through data.

Figure 2 shows the results of the proposed method of the present invention. The part of column (a) in this figure is the low-resolution image obtained by downsampling the actual high-resolution image, and the part of column (b) is the bilinear interpolation of the low-resolution image of column (a). As a result, (c) column part is the edge information map extracted by the method of the present invention to the image of (a) column, and (d) column part is the high-resolution image obtained by the method of the present invention to (a) column image processing, Column (e) is the actual high-resolution image.

From this figure, it can be seen that the edge part and the part near the edge can be well reconstructed with the proposed method. At the same time, after zooming in, it can be seen that the overall smoothness of the image reconstructed by the algorithm of the present invention is very high, the edges are clearer, and the points near the edges are closer to the real image. Due to the difference in the number of detected edges due to the difference in skin color, the results of the reconstructed image are also slightly different.

Table 1 The PSNR of the output image based on the bilinear interpolation method and the method of the present invention in the actual FERET face database image

Table 2 The SSIM of the output image based on the bilinear interpolation method and the method of the present invention in the actual FERET face database image

The present invention mainly selects two objective evaluation methods of peak signal-to-noise ratio (PSNR) and feature similarity (SSIM) to evaluate the performance of the proposed method. The formula for calculating PSNR is as follows:

In the above formula, n is the number of bits used by the image brightness value, for example, n=8 for an 8-bit image brightness value. MSE is root mean square error, defined as:

Wherein the image size is (2W) * (2H), f ' (i, j) represents the pixel value of the high-resolution image reconstructed by the method of the present invention, and f (i, j) represents the original high-resolution image The value of the pixel.

The specific calculation formula of the feature similarity evaluation (SSIM) method is as follows:

Among them, μ _x , μ _y represent the mean value of the original image and the reconstructed image respectively, C ₁ , C ₂ represent the brightness of the two images before and after reconstruction, σ _x , σ _y represent the variance of the original image and the reconstructed image. Represents the contrast component of the original image before reconstruction and the reconstructed image, Represents the structural similarity of the two images before and after reconstruction.

Calculate PSNR and SSIM of the super-resolution image reconstructed by the traditional bilinear interpolation method and the actual high-resolution image rate for the 4 low-resolution images in the accompanying drawing 2 (a), and the reconstruction method proposed by the present invention Table 1 and Table 2 are obtained from the PSNR and SSIM of the super-resolution image and the actual high-resolution image. It can be seen from these tables that the proposed method has higher PSNR and SSIM than the traditional bilinear interpolation method, the reconstruction effect is better than the traditional bilinear interpolation reconstruction method, and the performance is superior.

From the data of Table 1 and Table 2 and Fig. 2, it can be seen that the more edges detected are more beneficial to the super-resolution reconstruction method based on edge classification of the present invention. At the same time, it can also be explained that edge pixels have an important influence on the method of interpolation and amplification, and the reconstruction algorithm based on edge classification proposed by the present invention can more accurately reconstruct edge points and nearby pixels, so it is beneficial to practical applications.

The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection defined by the claims.

Claims (4)

1. a kind of image super-resolution rebuilding method for adding rim detection, methods described carries out 11 after rim detection to edge Kind different classification carry out interpolation again again, when being 2 × 2 for the multiple of amplification, i.e. 2 times of horizontal magnification, 2 times of vertical magnification, Methods described includes：

Step one, the initial amplification of the super-resolution based on interpolation, to obtain initial high-definition picture；

Step 2, rim detection is carried out to low-resolution image with extracting；

Step 3, carries out binary conversion treatment to the image after edge extracting, the value of the binaryzation on non-edge point is changed into 0, The value of binaryzation on marginal point is 1, i.e. on the image of binaryzation, and the pixel point value on image border is 1, other not to be The pixel point value all 0 at edge；

Step 4, puts x=0, y=0；

Step 5, with the upper left corner that (x, y) is high-definition picture block, extracts the figure of 3 × 3 sizes on high-resolution image As block, to the binary image of low resolution, with (x/2, y/2) for the upper left corner, extract the image block of 2 × 2 sizes, according to this 2 Image block after the edge binaryzation of × 2 sizes, classifies to the high-resolution image block of extraction, particularly may be divided into as follows 11 classes：Respectively top edge class, lower edge class, left hand edge class, right hand edge class, lower-left beveled edge class, bottom right beveled edge class, upper left Corner edge class, lower-left corner edge class, upper right corner edge class, bottom right corner edge class, other classes；Then, it is right according to the classification at edge The edge pixel point of the image block for 3 × 3 sizes extracted on high-definition picture and its neighbouring pixel, carry out interpolation again；

Step 6, puts x=x+2, if x≤2W-5, W are the width of low-resolution image, then jumps to step 5 to next image Block is operated；

Step 7, puts y=y+2, if y≤2H-5, H are the height of low-resolution image, then jumps to step 5 to next image Block is operated；

Step 8, the super-resolution amplification to current low-resolution image terminates, and obtains the high-resolution of (2W) × (2H) sizes Image.

2. a kind of image super-resolution rebuilding method for adding rim detection as claimed in claim 1, wherein, the step 2 In, the specific computational methods of Canny operators are as follows：

First, Gaussian filter smoothed image is used, that is, chooses a Gaussian smoothing function under normal conditions, in frequency domain Its impulse function is：

Wherein, what D (u, v) was represented is the distance of Fourier transformation, and σ is curvature, and H (u, v) represents the degree of Gaussian curve extension； To avoid edge from excessively obscuring, the small convolution mask of utilization scope of the present invention；

Specifically, the present invention carries out convolution algorithm using 5 × 5 following template to the figure of low resolution：

Secondly, amplitude and the direction of gradient are calculated, that is, chooses the template of a first-order difference convolution first：

Then low-resolution image f (m, n) is defined, m is longitudinal coordinate, and n is lateral coordinates, in H₁、H₂On two orthogonal directions Gradient Ψ₁(m, n), Ψ₂(m, n) is respectively：

Ψ₁(m, n)=f (m, n) * H₁(m, n)

Ψ₂(m, n)=f (m, n) * H₂(m, n)

Passing through, further computing obtains the intensity at edge and direction is shown below：

In the method for original Canny operators, non-maximum convergence will be carried out with the gradient magnitude calculated；The present invention This step is improved, quantifies edge angle θ first_ΨFor θ₁, θ₁∈ 0 °, and 45 °, 90 °, 135 °, 180 °, 225 °, 270 °, 315 ° }, then, if θ₁=0 °, then must Ψ (m, n) ＞ Ψ (m, n+1) just judge current point for initial marginal point；If θ₁=45 °, then must Ψ (m, n) ＞ Ψ (m-1, n+1) just judge current point for initial marginal point；If θ₁=90 °, then must (m-1 n) just judges current point for initial marginal point to palpus Ψ (m, n) ＞ Ψ；If θ₁=135 °, then must Ψ (m, n) ＞ Ψ (m-1, n-1) just judges current point for initial marginal point；If θ₁=180 °, then must Ψ (m, n) ＞ Ψ (m, n-1) just sentence Disconnected current point is initial marginal point；If θ₁=225 °, then must Ψ (m, n) ＞ Ψ (m+1, n-1) just judge that current point is Initial marginal point；If θ₁=270 °, then must Ψ (m, n) ＞ Ψ (m+1, n) just judges current point for initial marginal point, If θ₁=315 °, then must Ψ (m, n) ＞ Ψ (m+1, n+1) just judge current point for initial marginal point；So, it can reduce Amount of calculation, and obtain preferable effect；

Finally, the image border having been detected by is connected with dual threashold value-based algorithm, i.e. when previous step detect it is initial Gradient magnitude Ψ (m, n) the ＞ T of marginal point_hWhen, it is image border point to determine the point；Then, using these marginal points as seed, sweep Point adjacent around it is retouched, as its gradient magnitude Ψ (m, n) ＞ T_lWhen, this point is added the set of image border point, the present invention In, according to substantial amounts of experiment and as a result, to the two parameters T_h、T_lValue be set as T_h=200, T_l=100.

3. a kind of image super-resolution rebuilding method for adding rim detection as claimed in claim 1,

According to the edge in the image block of 3 × 3 sizes, the process to image block classification can be described as follows, i.e. to high resolution graphics The image block extracted as in,

In this image block, a₁₁、a₁₃、a₃₁、a₃₃Pixel is remaining in the pixel belonged in low-resolution image, this block Pixel point value is obtained by interpolation, the edge distribution figure of the binaryzation of the low resolution according to corresponding to it, can be by this block The position that marginal point occurs is classified to this image block, and the classification that it is classified is as follows：

(a) top edge class, now a₁₁And a₁₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₁、a₂₂、a₂₃, its interpolation Formula is

Wherein, clip (x) functions are x value to be restricted within the scope of the codomain of a pixel point value, i.e. clip (x)=max (I_min, min (x, I_max)), here, I_minAnd I_maxIt is that a pixel may obtain minimum value and maximum respectively, α and β are two Individual weights, meet condition alpha+beta=1；

(b) lower edge class, now a₃₁And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₁、a₂₂、a₂₃, its interpolation Formula is

(c) left hand edge class, now a₁₁And a₃₁For image border point, at this time, it may be necessary to which the pixel of interpolation is a again₁₂、a₂₂、a₃₂, Its interpolation formula is

(d) right hand edge class, now a₁₃And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₁₂、a₂₂、a₃₂, its interpolation Formula is

(e) lower-left beveled edge class, now a₁₁And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₁₂、a₂₁、 a₂₃、a₃₂, its interpolation formula is

(f) bottom right beveled edge class, now a₁₃And a₃₁It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₁₂、a₂₁、 a₂₃、a₃₂, its interpolation formula is

(g) upper left corner edge class, now a₁₁、a₁₃And a₃₁It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₂₃、 a₃₂, its interpolation formula is

(h) lower-left corner edge class, now a₁₁、a₃₁And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₁₂、 a₂₃, its interpolation formula is

(i) upper right corner edge class, now a₁₁、a₁₃And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₂₁、 a₃₂, its interpolation formula is

(j) bottom right corner edge class, now a₃₁、a₁₃And a₃₃It is image border point, it is necessary to which the pixel of interpolation is a again₂₂、a₂₁、 a₁₂, its interpolation formula is

(k) other classifications, other situations belong to this classification, the situation to belonging to this classification, not to institute in the present invention The high-definition picture block of extraction carries out interpolation again.

4. the image super-resolution rebuilding method of rim detection is added as claimed in claim 3, wherein：Value to α and β is α=0.7, β=0.3.