CN104021523B - A kind of method of the image super-resolution amplification based on marginal classification - Google Patents
A kind of method of the image super-resolution amplification based on marginal classification Download PDFInfo
- Publication number
- CN104021523B CN104021523B CN201410193840.4A CN201410193840A CN104021523B CN 104021523 B CN104021523 B CN 104021523B CN 201410193840 A CN201410193840 A CN 201410193840A CN 104021523 B CN104021523 B CN 104021523B
- Authority
- CN
- China
- Prior art keywords
- image
- point
- interpolation
- resolution
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000003321 amplification Effects 0.000 title claims abstract description 25
- 238000003199 nucleic acid amplification method Methods 0.000 title claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 238000002474 experimental method Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000000205 computational method Methods 0.000 claims description 2
- 230000006835 compression Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002620 method output Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a kind of method of the image amplification based on marginal classification.This method carries out rim detection to low-resolution image first, with the figure for the binaryzation for obtaining the instruction of the image border in low resolution.Then, initial interpolation is carried out to the image of low resolution, obtains initial high-resolution image.Next, extracting 3 × 3 image block in high-definition picture, the trend at the edge in binary image corresponding with this block is classified to this image block, and carries out interpolation again to some pixels in block according to its classification.The method of the present invention has advantageously taken into account the feature of moving towards at edge, and row interpolation is entered according to this, so that the detail section of the image after amplification is than more visible, especially marginal portion and the part close to edge.Experiment shows that the image after reconstruction and original high-resolution image are closely.
Description
Technical field
The present invention relates in image procossing, a kind of new method of super-resolution image amplification.Differentiated especially in accordance with low
Marginal information on rate image, corresponding low is differentiated to a small image block on the high-definition picture that initially amplifies according to its
Marginal point on rate image is classified, and according to method of this classification to some pixel progress in image block again interpolation.
Background technology
With developing rapidly for digital camera and Internet technology, people are growing day by day to the demand of high-definition digital image.
But, limited by the network bandwidth and memory space, the cost ratio of storage and the transmission of high-definition image is larger, very consumption system
The resource of system, if desired for the very big memory space or very big bandwidth of consumption system.Therefore, the compression skill of digital picture
Art is proposed out, generates the international standard of compression of images, such as JPEG, JPEG2000 etc..Current lossless compression of images
Method, the multiple of compression is general below 4.Meanwhile, the compression method damaged of image can bring the distortion of image, produce block
Effect (block artifact), ringing effect (ringing artifact) etc..Therefore, the amplification skill of the super-resolution of image
Art has obtained the extensive concern of people, and as the hot research problem in image processing field in recent years.
The purpose of image super-resolution amplification is the method that high-definition picture is obtained by the image of low resolution.So,
In transmission and storage, the image of low resolution only can be transmitted or stored, then in reciever or display side using image
Super-resolution amplifying technique obtains high-resolution image.Meanwhile, the super-resolution amplifying technique of image can be with existing figure
As compress technique is combined, the system resource for storing and transmitting high-definition picture and consuming further is reduced to.Further
Ground, the storage and transmission of HD video can also reduce the consumption of system resource using the ultra-resolution method of image.
At present, the method for the super-resolution of image is divided into the method for interpolation and the method based on sample.Wherein, based on sample
Method need first to set up one and be made up of what training was obtained low-resolution image block high-definition picture block corresponding with its
Sample data storehouse, and its amount of calculation is very big, it is difficult to application in real time.Method based on interpolation, although its computation complexity is low,
But the fuzzy of image is easily caused, make the marginal portion of image unintelligible.And human eye is to compare for the marginal portion of image
Sensitive, its relatively low distortion will substantially reduce the visual effect of image.
Therefore, the present invention proposes a kind of after initial amplification, the edge according to image is changed in enlarged drawing
Edge pixel point and its near pixel value method.So that when interpolation, the edge of interpolation direction and image is walked
To consistent, sharp-edged enlarged drawing is obtained.
The content of the invention
In view of the drawbacks described above of prior art, the present invention proposes a kind of situation at the edge in image and divided
Class, the method that corresponding interpolation again is then carried out to different classifications.According to the marginal information on low-resolution image, to make
For didactic information, interpolation is always carried out according to walking for edge, makes the image edge clear of amplification, overcome existing at present super
The shortcoming of the edge blurry of the method for resolution ratio interpolation.
To achieve the above object, 11 kinds of different classification are carried out again to edge after rim detection the invention provides a kind of
The method for carrying out again interpolation.For amplification multiple be 2 × 2 when, i.e. 2 times of horizontal magnification, 2 times of vertical magnification.This certain hair
It is bright to be not limited to 2 × 2 times of amplification.The present invention includes：
Step one, the initial amplification of the super-resolution based on interpolation, to obtain initial high-definition picture；
Step 2, rim detection is carried out to low-resolution image with extracting；
Step 3, carries out binary conversion treatment to the image after edge extracting, the value of the binaryzation on non-edge point is become
For 0, the value of the binaryzation on marginal point is 1.That is, on the image of binaryzation, the pixel point value on image border is 1, other
It is not the pixel point value all 0 at edge；
Step 4, puts x=0, y=0；
Step 5, with the upper left corner that (x, y) is high-definition picture block, extracts 3 × 3 sizes on high-resolution image
Image block.To the binary image of low resolution, with (x/2, y/2) for the upper left corner, the image block of 2 × 2 sizes is extracted.According to
Image block after the edge binaryzation of this 2 × 2 size, classifies to the high-resolution image block of extraction, particularly may be divided into
Following 11 class：Respectively top edge class, lower edge class, left hand edge class, right hand edge class, lower-left beveled edge class, bottom right beveled edge class,
Upper left corner edge class, lower-left corner edge class, upper right corner edge class, bottom right corner edge class, other classes；Then, according to the class at edge
Not, to the edge pixel point and its neighbouring pixel of the image block of 3 × 3 sizes extracted on the image that initially amplifies, weighed
New interpolation；
Step 6, puts x=x+2, if x≤2W-5 (W is the width of low-resolution image), jumps to step 5 to next
Image block is operated；
Step 7, puts y=y+2, if y≤2H-5 (H is the height of low-resolution image), jumps to step 5 to next
Image block is operated；
Step 8, the super-resolution amplification to current low-resolution image terminates.Obtain the high score of (2W) × (2H) sizes
Resolution image.
Further, in the step one, bilinear interpolation method is used.That is, for following image block：
Wherein, A, B, C, D are the pixel value of the pixel on low-resolution image respectively, and a, b, c, d, e are high-resolution
The pixel of interpolation is needed on image.In bilinear interpolation method, their pixel value is obtained by following formula：
Wherein, clip (x) functions are x value to be restricted within the scope of the codomain of a pixel point value, i.e. clip (x)
=max (Imin, min (x, Imax)).Here, IminAnd ImaxIt is that a pixel may obtain minimum value and maximum respectively.
Further, in the step 2, edge extracting is carried out using Canny operators.The specific side of calculating of Canny operators
Method is as follows：
First, Gaussian filter smoothed image is used, that is, chooses a Gaussian smoothing function under normal conditions, in frequency
Its impulse function is in domain：
Wherein, what D (u, v) was represented is the distance apart from Fourier transformation compared with far point, and σ is curvature, and H (u, v) represents Gauss
The degree of curve extension；To avoid edge from excessively obscuring, the template of the small convolution of utilization scope of the present invention.Specifically, it is of the invention
Convolution algorithm is carried out to the figure of low resolution using 5 × 5 following template：
Secondly, amplitude and the direction of gradient are calculated, that is, chooses the template of a first-order difference convolution first：
Then low-resolution image f (m, n) (m is longitudinal coordinate, and n is lateral coordinates) is defined in H1、H2Two orthogonal directions
On gradient Ψ1(m, n), Ψ2(m, n) is respectively：
Ψ1(m, n)=f (m, n) * H1(m, n)
Ψ2(m, n)=f (m, n) * H2(m, n)
Passing through, further computing obtains the intensity at edge and direction is shown below：
In the method for original Canny operators, non-maximum (NMS) receipts will be carried out with the gradient magnitude calculated
Hold back；The present invention is improved to this step.Quantify edge angle θ firstΨFor θ1, θ1∈ 0 °, and 45 °, 90 °, 135 °, 180 °,
225 °, 270 °, 315 ° }.Then, if θ1=0 °, then must Ψ (m, n) ＞ Ψ (m, n+1) just judge current point to be initial
Marginal point；If θ1=45 °, then must Ψ (m, n) ＞ Ψ (m-1, n+1) just judge current point for initial marginal point；If θ1
=90 °, then (m-1 n) just judges current point for initial marginal point to necessary Ψ (m, n) ＞ Ψ；If θ1It is=135 °, then necessary
Ψ (m, n) ＞ Ψ (m-1, n-1) just judge current point for initial marginal point；If θ1=180 °, then must Ψ (m, n) ＞ Ψ
(m, n-1) just judges current point for initial marginal point；If θ1=225 °, then must Ψ (m, n) ＞ Ψ (m+1, n-1) just sentence
Disconnected current point is initial marginal point；If θ1=270 °, then (m+1 n) just judges that current point is first to necessary Ψ (m, n) ＞ Ψ
The marginal point of beginning, if θ1=315 °, then must Ψ (m, n) ＞ Ψ (m+1, n+1) just judge current point for initial marginal point.
So, amount of calculation can be reduced, and obtains preferable effect.
Finally, the image border having been detected by is connected with dual threashold value-based algorithm.That is, detected when previous step
Gradient magnitude Ψ (m, n) the ＞ T of initial edge pointshWhen, it is image border point to determine the point.Then, using these marginal points as kind
Son, scans point adjacent around it, as its gradient magnitude Ψ (m, n) ＞ TlWhen, this point is added the set of image border point.This
In invention, according to substantial amounts of experiment and as a result, to the two parameters Th、TlValue be set as Th=200, Tl=100.
Further, in the step 5, according to the edge in the image block of 3 × 3 sizes, to the process of image block classification
It can be described as follows.That is, the image block to being extracted in high-definition picture
In this image block, a11、a13、a31、a33Pixel is its in the pixel belonged in low-resolution image, this block
Remaining pixel point value is obtained by interpolation.The edge distribution figure of the binaryzation of low resolution according to corresponding to it, can be by this
The position that marginal point occurs in block is classified by this image block.The classification that it is classified is as follows：
(a) top edge class.Now a11And a13For image border point.At this time, it may be necessary to which the pixel of interpolation is a again21、a22、
a23.Its interpolation formula is
Wherein, clip (x) is defined as described above, and α and β are two weights, meet condition alpha+beta=1.
(b) lower edge class.Now a31And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is a again21、a22、
a23.Its interpolation formula is
(c) left hand edge class.Now a11And a31For image border point.At this time, it may be necessary to which the pixel of interpolation is a again12、a22、
a32.Its interpolation formula is
(d) right hand edge class.Now a13And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is a again12、a22、
a32.Its interpolation formula is
(e) lower-left beveled edge class.Now a11And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is again
a22、a12、a21、a23、a32.Its interpolation formula is
(f) bottom right beveled edge class.Now a13And a31For image border point.At this time, it may be necessary to which the pixel of interpolation is again
a22、a12、a21、a23、a32.Its interpolation formula is
(g) upper left box edge class.Lower-left beveled edge class.Now a11、a13And a31For image border point.At this time, it may be necessary to again
The pixel of interpolation is a22、a23、a32.Its interpolation formula is
(h) lower-left frame edge class.Lower-left beveled edge class.Now a11、a31And a33For image border point.At this time, it may be necessary to again
The pixel of interpolation is a22、a12、a23.Its interpolation formula is
(i) upper right box edge class.Now a11、a13And a33For image border point.At this time, it may be necessary to the pixel of interpolation again
For a22、a21、a32.Its interpolation formula is
(j) lower right box edge class.Now a31、a13And a33For image border point.At this time, it may be necessary to the pixel of interpolation again
For a22、a21、a12.Its interpolation formula is
(k) other classifications.In the case of other, this classification is belonged to.In situation to belonging to this classification, the present invention not
Again interpolation is carried out to the high-definition picture block extracted.
In summary, the present invention carries out initial interpolation amplification first, and any one existing interpolation can be selected to put here
Big method, then makes improvements and improves performance with the method for the present invention.Due to bilinear interpolation method have it is relatively low
Computation complexity and preferable performance, the present invention carry out initial interpolation amplification using bilinear interpolation method.Then, it is of the invention
Edge extracting is carried out to low resolution image using Canny operators.Next, extracting the block of 3 × 3 sizes in high resolution graphics, root
The position at edge of the block in low resolution figure and trend, sort out this block accordingly, when this block belongs to preceding 10 class, according to
The classification at edge carries out interpolation again to this block；When this block belongs to other classes, retain original interpolation.Then, extract next
Block, carries out identical operation.Untill all blocks have all carried out this operation in high-resolution.Finally, edge is obtained clear
Clear high-resolution image.Its innovation is the extraction of the block to 3 × 3 sizes of high-resolution image, and utilizes
Marginal information on low-resolution image is sorted out to it, and the interpolation again shot the arrow at the target according to its classification, or protects
Original numerical value is stayed, enables edge in image prominent, apparent.Lift the effect of image amplification.
The technique effect of the design of the present invention, concrete structure and generation is described further below with reference to accompanying drawing, with
It is fully understood from the purpose of the present invention, feature and effect.
Brief description of the drawings
Fig. 1 is the flow chart of the super-resolution image reconstruction algorithm based on marginal classification of the present invention；
Fig. 2 is the experimental result picture of the super-resolution image reconstruction method based on marginal classification of the present invention.
Embodiment
Embodiments of the invention are elaborated below in conjunction with the accompanying drawings：The present embodiment is with technical solution of the present invention premise
It is lower to be implemented, give detailed embodiment and specific operating process, but protection scope of the present invention be not limited to it is following
Embodiment.
As shown in Figure 1, the method for the new image amplification of the invention based on marginal classification is carried out in accordance with the following steps：
Step one, the initial amplification of the super-resolution based on bilinear interpolation, to obtain initial high-definition picture；
That is, for following image block：
Wherein, A, B, C, D are the pixel value of the pixel on low-resolution image respectively, and a, b, c, d, e are high-resolution
The pixel of interpolation is needed on image.In bilinear interpolation method, their pixel value is obtained by following formula：
Wherein, clip (x) functions are x value to be restricted within the scope of the codomain of a pixel point value, i.e. clip (x)
=max (Imin, min (x, Imax)).For the image of 8 bit intensities, Imin=0, Imax=255.
Here, IminAnd ImaxIt is that a pixel may obtain minimum value and maximum respectively.
Step 2, rim detection is carried out to low-resolution image with extracting；Here, edge is carried out using Canny operators to carry
Take.The specific computational methods of Canny operators are as follows：
First, Gaussian filter smoothed image is used, that is, chooses a Gaussian smoothing function under normal conditions, in frequency
Its impulse function is in domain：
Wherein, what D (u, v) was represented is the distance of Fourier transformation, and σ is curvature, and H (u, v) represents Gaussian curve extension
Degree；To avoid edge from excessively obscuring, the small convolution mask of utilization scope of the present invention.
Specifically, the present invention carries out convolution algorithm using 5 × 5 following template to the figure of low resolution：
Secondly, amplitude and the direction of gradient are calculated, that is, chooses the template of a first-order difference convolution first：
Then low-resolution image f (m, n) (m is longitudinal coordinate, and n is lateral coordinates) is defined in H1、H2Two orthogonal directions
On gradient Ψ1(m, n), Ψ2(m, n) is respectively：
Ψ1(m, n)=f (m, n) * H1(m, n)
Ψ2(m, n)=f (m, n) * H2(m, n)
Passing through, further computing obtains the intensity at edge and direction is shown below：
In the method for original Canny operators, non-maximum (NMS) receipts will be carried out with the gradient magnitude calculated
Hold back；The present invention is improved to this step.Quantify edge angle θ firstΨFor θ1, θ1∈ 0 °, and 45 °, 90 °, 135 °, 180 °,
225 °, 270 °, 315 ° }.Then, if θ1=0 °, then must Ψ (m, n) ＞ Ψ (m, n+1) just judge current point to be initial
Marginal point；If θ1=45 °, then must Ψ (m, n) ＞ Ψ (m-1, n+1) just judge current point for initial marginal point；If θ1
=90 °, then (m-1 n) just judges current point for initial marginal point to necessary Ψ (m, n) ＞ Ψ；If θ1It is=135 °, then necessary
Ψ (m, n) ＞ Ψ (m-1, n-1) just judge current point for initial marginal point；If θ1=180 °, then must Ψ (m, n) ＞ Ψ
(m, n-1) just judges current point for initial marginal point；If θ1=225 °, then must Ψ (m, n) ＞ Ψ (m+1, n-1) just sentence
Disconnected current point is initial marginal point；If θ1=270 °, then (m+1 n) just judges that current point is first to necessary Ψ (m, n) ＞ Ψ
The marginal point of beginning, if θ1=315 °, then must Ψ (m, n) ＞ Ψ (m+1, n+1) just judge current point for initial marginal point.
So, amount of calculation can be reduced, and obtains preferable effect.
Finally, the image border having been detected by is connected with dual threashold value-based algorithm.That is, detected when previous step
Gradient magnitude Ψ (m, n) the ＞ T of initial edge pointshWhen, it is image border point to determine the point.Then, using these marginal points as kind
Son, scans point adjacent around it, as its gradient magnitude Ψ (m, n) ＞ TlWhen, this point is added the set of image border point.This
In invention, according to substantial amounts of experiment and as a result, to the two parameters Th、TlValue be set as Th=200, Tl=100.
Step 3, carries out binary conversion treatment to the image after edge extracting, the value of the binaryzation on non-edge point is become
For 0, the value of the binaryzation on marginal point is 1.That is, on the image of binaryzation, the pixel point value on image border is 1, other
It is not the pixel point value all 0 at edge；
Step 4, puts x=0, y=0；
Step 5, with the upper left corner that (x, y) is high-definition picture block, extracts 3 × 3 sizes on high-resolution image
Image block.To the binary image of low resolution, with (x/2, y/2) for the upper left corner, the image block of 2 × 2 sizes is extracted.According to
Image block after the edge binaryzation of this 2 × 2 size, classifies to the high-resolution image block of extraction, particularly may be divided into
Following 11 class：Respectively top edge class, lower edge class, left hand edge class, right hand edge class, lower-left beveled edge class, bottom right beveled edge class,
Upper left corner edge class, lower-left corner edge class, upper right corner edge class, bottom right corner edge class, other classes；Then, according to the class at edge
Not, to the edge pixel point and its neighbouring pixel of the image block of 3 × 3 sizes extracted on the image that initially amplifies, weighed
New interpolation；According to the edge in the image block of 3 × 3 sizes, the process to image block classification can be described as follows.That is, to high-resolution
The image block extracted in rate image
In this image block, a11、a13、a31、a33Pixel is its in the pixel belonged in low-resolution image, this block
Remaining pixel point value is obtained by interpolation.The edge distribution figure of the binaryzation of low resolution according to corresponding to it, can be by this
The position that marginal point occurs in block is classified by this image block.The classification that it is classified is as follows：
(a) top edge class.Now a11And a13For image border point.At this time, it may be necessary to which the pixel of interpolation is a again21、a22、
a23.Its interpolation formula is
Wherein, clip (x) functions are x value to be restricted within the scope of the codomain of a pixel point value, i.e. clip (x)
=max (Imin, min (x, Imax)).Here, IminAnd ImaxIt is that a pixel may obtain minimum value and maximum respectively.α and
β is two weights, meets condition alpha+beta=1.
(b) lower edge class.Now a31And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is a again21、a22、
a23.Its interpolation formula is
(c) left hand edge class.Now a11And a31For image border point.At this time, it may be necessary to which the pixel of interpolation is a again12、a22、
a32.Its interpolation formula is
(d) right hand edge class.Now a13And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is a again12、a22、
a32.Its interpolation formula is
(e) lower-left beveled edge class.Now a11And a33For image border point.At this time, it may be necessary to which the pixel of interpolation is again
a22、 a12、a21、a23、a32.Its interpolation formula is
(f) bottom right beveled edge class.Now a13And a31For image border point.At this time, it may be necessary to which the pixel of interpolation is again
a22、a12、a21、a23、a32.Its interpolation formula is
(g) upper left box edge class.Lower-left beveled edge class.Now a11、a13And a31For image border point.At this time, it may be necessary to again
The pixel of interpolation is a22、a23、a32.Its interpolation formula is
(h) lower-left frame edge class.Lower-left beveled edge class.Now a11、a31And a33For image border point.At this time, it may be necessary to again
The pixel of interpolation is a22、a12、a23.Its interpolation formula is
(i) upper right box edge class.Now a11、a13And a33For image border point.At this time, it may be necessary to the pixel of interpolation again
For a22、a21、a32.Its interpolation formula is
(j) lower right box edge class.Now a31、a13And a33For image border point.At this time, it may be necessary to the pixel of interpolation again
For a22、a21、a12.Its interpolation formula is
(k) other classifications.In the case of other, this classification is belonged to.In situation to belonging to this classification, the present invention not
Again interpolation is carried out to the high-definition picture block extracted.
Here, the present invention determines α=0.7, β=0.3 according to substantial amounts of experiment.
Step 6, puts x=x+2, if x≤2W-5 (W is the width of low-resolution image), jumps to step 5 to next
Image block is operated；
Step 7, puts y=y+2, if y≤2H-5 (H is the height of low-resolution image), jumps to step 5 to next
Image block is operated；
Step 8, the super-resolution amplification to current low-resolution image terminates.Obtain the high score of (2W) × (2H) sizes
Resolution image.
What the experiment of the present invention was mainly chosen is the facial image in FERET databases, wherein what is specifically chosen is to include
Four different degrees of width images of the colour of skin are used to rebuild, and the size of original high-resolution image is 120*120, according to each two pixel
Point takes the degraded image size after the regulation of a pixel, down-sampling to be changed into 60*60, that is, narrows down to original 1/4, it is necessary to insert
The multiple of value amplification should be set on length and cross direction each 2 times.
The present invention selects face database for experimental subjects to be tested, and experiment acquired results then are passed through into subjectivity
Evaluation with objective two aspects carrys out the quality of test experience result, and wherein subjective assessment is overall to image according to human eye
Visual effect also has to the impression of image detail part to evaluate, and objective evaluation is exactly to be passed through using different formula algorithms
Data illustrate the quality of image.
Accompanying drawing 2 shows the result of method proposed by the invention.(a) row part in this figure is to actual high-resolution
Rate image carries out the image for the low resolution that down-sampling is obtained, and (b) row part is to carry out two-wire to the low-resolution image that (a) is arranged
Property interpolation result, (c) row part for the present invention method to the marginal information figure of (a) image zooming-out arrange, (d) row are partly
The high-resolution image that the method for the present invention is obtained to (a) row image procossing, (e) row part is actual high resolution graphics
Picture.
As can be seen that marginal portion can be reconstructed well and close to side by employing proposed method from this accompanying drawing
The part of edge.Meanwhile, as can be seen that inventive algorithm is reconstructed, the overall smoothness of the image come is very high after amplification, and edge is more
Clearly, the point close to edge is also more nearly real image.Because the difference of the colour of skin can cause the number of edges detected
Difference, the result for the image rebuild also is slightly different.
Based on bilinear interpolation method and the method output image of the present invention in the actual FERET face databases image of table 1
PSNR
Based on bilinear interpolation method and the method output image of the present invention in the actual FERET face databases image of table 2
SSIM
It is of the invention mainly to select Y-PSNR (PSNR) and two kinds of method for objectively evaluating of characteristic similarity (SSIM) to comment
The performance for the method that valency is proposed.The formula for wherein calculating PSNR is as follows：
In above formula, n is the bit number used in image brightness values, such as 8 bit image brightness value n=8.MSE is equal
Square error, is defined as：
Wherein image size is (2W) × (2H), the high-definition picture of the method reconstruct of f ' (i, j) expression present invention
Pixel value, f (i, j) represent original high-resolution image pixel value.
The specific formula for calculation that characteristic similarity evaluates (SSIM) method is as follows：
Wherein, μx, μyThe average of image, C after representing original image respectively and rebuilding1, C2Then represent two width figures before and after rebuilding
The brightness of picture, σx, σyThe variance of image after representing original image and rebuilding.What is represented is the original graph before rebuilding
The contrast component of image after picture and reconstruction,Represent be then rebuild before and after two images structure it is similar
Degree.
The super-resolution that traditional bilinear interpolation method of putting is reconstructed is calculated respectively to 4 width low-resolution images in accompanying drawing 2 (a)
Rate image and the PSNR and SSIM of actual full resolution pricture rate, and the super-resolution figure that method proposed by the invention is reconstructed
As the PSNR and SSIM with actual high-definition picture, Tables 1 and 2 is obtained.It can be seen that proposed method from these tables
There is higher PSNR and SSIM than traditional bilinear interpolation method, rebuild effect and be better than traditional bilinear interpolation reconstruction side
Method, performance is more superior.
From the data and Fig. 2 of Tables 1 and 2, while as can be seen that the edge detected is more more more be conducive to the present invention
The super resolution ratio reconstruction method based on marginal classification.It can also illustrate that edge pixel point has to the method for interpolation amplification simultaneously
Important influence, and the algorithm for reconstructing proposed by the invention based on marginal classification can more accurately rebuild marginal point and its near
Pixel, therefore be conducive to practical application.
Preferred embodiment of the invention described in detail above.It should be appreciated that the ordinary skill of this area is without wound
The property made work just can make many modifications and variations according to the design of the present invention.Therefore, all technical staff in the art
Pass through the available technology of logical analysis, reasoning, or a limited experiment on the basis of existing technology under this invention's idea
Scheme, all should be in the protection domain being defined in the patent claims.
Claims (4)
1. a kind of image super-resolution rebuilding method for adding rim detection, methods described carries out 11 after rim detection to edge
Kind different classification carry out interpolation again again, when being 2 × 2 for the multiple of amplification, i.e. 2 times of horizontal magnification, 2 times of vertical magnification,
Methods described includes：
Step one, the initial amplification of the super-resolution based on interpolation, to obtain initial high-definition picture；
Step 2, rim detection is carried out to low-resolution image with extracting；
Step 3, carries out binary conversion treatment to the image after edge extracting, the value of the binaryzation on non-edge point is changed into 0,
The value of binaryzation on marginal point is 1, i.e. on the image of binaryzation, and the pixel point value on image border is 1, other not to be
The pixel point value all 0 at edge；
Step 4, puts x=0, y=0；
Step 5, with the upper left corner that (x, y) is high-definition picture block, extracts the figure of 3 × 3 sizes on high-resolution image
As block, to the binary image of low resolution, with (x/2, y/2) for the upper left corner, extract the image block of 2 × 2 sizes, according to this 2
Image block after the edge binaryzation of × 2 sizes, classifies to the high-resolution image block of extraction, particularly may be divided into as follows
11 classes：Respectively top edge class, lower edge class, left hand edge class, right hand edge class, lower-left beveled edge class, bottom right beveled edge class, upper left
Corner edge class, lower-left corner edge class, upper right corner edge class, bottom right corner edge class, other classes；Then, it is right according to the classification at edge
The edge pixel point of the image block for 3 × 3 sizes extracted on high-definition picture and its neighbouring pixel, carry out interpolation again；
Step 6, puts x=x+2, if x≤2W-5, W are the width of low-resolution image, then jumps to step 5 to next image
Block is operated；
Step 7, puts y=y+2, if y≤2H-5, H are the height of low-resolution image, then jumps to step 5 to next image
Block is operated；
Step 8, the super-resolution amplification to current low-resolution image terminates, and obtains the high-resolution of (2W) × (2H) sizes
Image.
2. a kind of image super-resolution rebuilding method for adding rim detection as claimed in claim 1, wherein, the step 2
In, the specific computational methods of Canny operators are as follows：
First, Gaussian filter smoothed image is used, that is, chooses a Gaussian smoothing function under normal conditions, in frequency domain
Its impulse function is：
Wherein, what D (u, v) was represented is the distance of Fourier transformation, and σ is curvature, and H (u, v) represents the degree of Gaussian curve extension；
To avoid edge from excessively obscuring, the small convolution mask of utilization scope of the present invention；
Specifically, the present invention carries out convolution algorithm using 5 × 5 following template to the figure of low resolution：
Secondly, amplitude and the direction of gradient are calculated, that is, chooses the template of a first-order difference convolution first：
Then low-resolution image f (m, n) is defined, m is longitudinal coordinate, and n is lateral coordinates, in H1、H2On two orthogonal directions
Gradient Ψ1(m, n), Ψ2(m, n) is respectively：
Ψ1(m, n)=f (m, n) * H1(m, n)
Ψ2(m, n)=f (m, n) * H2(m, n)
Passing through, further computing obtains the intensity at edge and direction is shown below：
In the method for original Canny operators, non-maximum convergence will be carried out with the gradient magnitude calculated；The present invention
This step is improved, quantifies edge angle θ firstΨFor θ1, θ1∈ 0 °, and 45 °, 90 °, 135 °, 180 °, 225 °, 270 °,
315 ° }, then, if θ1=0 °, then must Ψ (m, n) ＞ Ψ (m, n+1) just judge current point for initial marginal point；If
θ1=45 °, then must Ψ (m, n) ＞ Ψ (m-1, n+1) just judge current point for initial marginal point；If θ1=90 °, then must
(m-1 n) just judges current point for initial marginal point to palpus Ψ (m, n) ＞ Ψ；If θ1=135 °, then must Ψ (m, n) ＞ Ψ
(m-1, n-1) just judges current point for initial marginal point；If θ1=180 °, then must Ψ (m, n) ＞ Ψ (m, n-1) just sentence
Disconnected current point is initial marginal point；If θ1=225 °, then must Ψ (m, n) ＞ Ψ (m+1, n-1) just judge that current point is
Initial marginal point；If θ1=270 °, then must Ψ (m, n) ＞ Ψ (m+1, n) just judges current point for initial marginal point,
If θ1=315 °, then must Ψ (m, n) ＞ Ψ (m+1, n+1) just judge current point for initial marginal point；So, it can reduce
Amount of calculation, and obtain preferable effect；
Finally, the image border having been detected by is connected with dual threashold value-based algorithm, i.e. when previous step detect it is initial
Gradient magnitude Ψ (m, n) the ＞ T of marginal pointhWhen, it is image border point to determine the point；Then, using these marginal points as seed, sweep
Point adjacent around it is retouched, as its gradient magnitude Ψ (m, n) ＞ TlWhen, this point is added the set of image border point, the present invention
In, according to substantial amounts of experiment and as a result, to the two parameters Th、TlValue be set as Th=200, Tl=100.
3. a kind of image super-resolution rebuilding method for adding rim detection as claimed in claim 1,
According to the edge in the image block of 3 × 3 sizes, the process to image block classification can be described as follows, i.e. to high resolution graphics
The image block extracted as in,
In this image block, a11、a13、a31、a33Pixel is remaining in the pixel belonged in low-resolution image, this block
Pixel point value is obtained by interpolation, the edge distribution figure of the binaryzation of the low resolution according to corresponding to it, can be by this block
The position that marginal point occurs is classified to this image block, and the classification that it is classified is as follows：
(a) top edge class, now a11And a13It is image border point, it is necessary to which the pixel of interpolation is a again21、a22、a23, its interpolation
Formula is
Wherein, clip (x) functions are x value to be restricted within the scope of the codomain of a pixel point value, i.e. clip (x)=max
(Imin, min (x, Imax)), here, IminAnd ImaxIt is that a pixel may obtain minimum value and maximum respectively, α and β are two
Individual weights, meet condition alpha+beta=1；
(b) lower edge class, now a31And a33It is image border point, it is necessary to which the pixel of interpolation is a again21、a22、a23, its interpolation
Formula is
(c) left hand edge class, now a11And a31For image border point, at this time, it may be necessary to which the pixel of interpolation is a again12、a22、a32,
Its interpolation formula is
(d) right hand edge class, now a13And a33It is image border point, it is necessary to which the pixel of interpolation is a again12、a22、a32, its interpolation
Formula is
(e) lower-left beveled edge class, now a11And a33It is image border point, it is necessary to which the pixel of interpolation is a again22、a12、a21、
a23、a32, its interpolation formula is
(f) bottom right beveled edge class, now a13And a31It is image border point, it is necessary to which the pixel of interpolation is a again22、a12、a21、
a23、a32, its interpolation formula is
(g) upper left corner edge class, now a11、a13And a31It is image border point, it is necessary to which the pixel of interpolation is a again22、a23、
a32, its interpolation formula is
(h) lower-left corner edge class, now a11、a31And a33It is image border point, it is necessary to which the pixel of interpolation is a again22、a12、
a23, its interpolation formula is
(i) upper right corner edge class, now a11、a13And a33It is image border point, it is necessary to which the pixel of interpolation is a again22、a21、
a32, its interpolation formula is
(j) bottom right corner edge class, now a31、a13And a33It is image border point, it is necessary to which the pixel of interpolation is a again22、a21、
a12, its interpolation formula is
(k) other classifications, other situations belong to this classification, the situation to belonging to this classification, not to institute in the present invention
The high-definition picture block of extraction carries out interpolation again.
4. the image super-resolution rebuilding method of rim detection is added as claimed in claim 3, wherein：Value to α and β is
α=0.7, β=0.3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410193840.4A CN104021523B (en) | 2014-04-30 | 2014-04-30 | A kind of method of the image super-resolution amplification based on marginal classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410193840.4A CN104021523B (en) | 2014-04-30 | 2014-04-30 | A kind of method of the image super-resolution amplification based on marginal classification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104021523A CN104021523A (en) | 2014-09-03 |
| CN104021523B true CN104021523B (en) | 2017-10-10 |
Family
ID=51438262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410193840.4A Active CN104021523B (en) | 2014-04-30 | 2014-04-30 | A kind of method of the image super-resolution amplification based on marginal classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104021523B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787912B (en) * | 2014-12-18 | 2021-07-30 | 南京大目信息科技有限公司 | Classification-based step type edge sub-pixel positioning method |
| CN104881842B (en) * | 2015-05-18 | 2019-03-01 | 浙江师范大学 | A kind of image super-resolution method based on picture breakdown |
| CN106169173B (en) * | 2016-06-30 | 2019-12-31 | 北京大学 | Image interpolation method |
| CN109345465B (en) * | 2018-08-08 | 2023-04-07 | 西安电子科技大学 | GPU-based high-resolution image real-time enhancement method |
| CN109410177B (en) * | 2018-09-28 | 2022-04-01 | 深圳大学 | Image quality analysis method and system for super-resolution image |
| CN109557101B (en) * | 2018-12-29 | 2023-11-17 | 桂林电子科技大学 | Defect detection device and method for non-elevation reflective curved surface workpiece |
| CN112348103B (en) * | 2020-11-16 | 2022-11-11 | 南开大学 | Image block classification method and device and super-resolution reconstruction method and device thereof |
| CN114863057A (en) * | 2022-03-25 | 2022-08-05 | 东南大学成贤学院 | Method for three-dimensional reproduction of point cloud reconstructed CT (computed tomography) image |
| CN117830104A (en) * | 2023-12-29 | 2024-04-05 | 摩尔线程智能科技(成都)有限责任公司 | Image super-resolution processing method, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499164A (en) * | 2009-02-27 | 2009-08-05 | 西安交通大学 | Image interpolation reconstruction method based on single low-resolution image |
| CN101957309A (en) * | 2010-08-17 | 2011-01-26 | 招商局重庆交通科研设计院有限公司 | All-weather video measurement method for visibility |
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7379625B2 (en) * | 2003-05-30 | 2008-05-27 | Samsung Electronics Co., Ltd. | Edge direction based image interpolation method |
| WO2012114574A1 (en) * | 2011-02-21 | 2012-08-30 | 三菱電機株式会社 | Image magnification device and method |
-
2014
- 2014-04-30 CN CN201410193840.4A patent/CN104021523B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499164A (en) * | 2009-02-27 | 2009-08-05 | 西安交通大学 | Image interpolation reconstruction method based on single low-resolution image |
| CN101957309A (en) * | 2010-08-17 | 2011-01-26 | 招商局重庆交通科研设计院有限公司 | All-weather video measurement method for visibility |
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
Non-Patent Citations (3)
| Title |
|---|
| Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation;Lingfeng et al.;《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》;20130831;第23卷(第8期);第1289-1299页 * |
| 一宗基于边缘预测的图像实时放大技术;黄彪 等;《红外与激光工程》;20130630;第42卷(第S1期);第268-273页 * |
| 一种基于细化边缘的图像放大方法;王东鹤;《微电子学与计算机》;20100228;第27卷(第2期);第122-125段 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104021523A (en) | 2014-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104021523B (en) | A kind of method of the image super-resolution amplification based on marginal classification | |
| Li et al. | Infrared and visible image fusion using a deep learning framework | |
| WO2021052261A1 (en) | Image super-resolution reconstruction method and apparatus for sharpening of label data | |
| Li et al. | Best-buddy gans for highly detailed image super-resolution | |
| CN107122826B (en) | Processing method and system and storage medium for convolutional neural networks | |
| Huang et al. | Framelet regularization for uneven intensity correction of color images with illumination and reflectance estimation | |
| Yoo et al. | Image restoration by estimating frequency distribution of local patches | |
| CN105447840B (en) | The image super-resolution method returned based on active sampling with Gaussian process | |
| CN103093444A (en) | Image super-resolution reconstruction method based on self-similarity and structural information constraint | |
| US11887218B2 (en) | Image optimization method, apparatus, device and storage medium | |
| CN108765280A (en) | A kind of high spectrum image spatial resolution enhancement method | |
| CN107610093B (en) | Full-reference image quality evaluation method based on similarity feature fusion | |
| CN105550989B (en) | The image super-resolution method returned based on non local Gaussian process | |
| Yilmaz et al. | Metaheuristic pansharpening based on symbiotic organisms search optimization | |
| CN110418139B (en) | Video super-resolution restoration method, device, equipment and storage medium | |
| CN111489333B (en) | No-reference night natural image quality evaluation method | |
| Cai et al. | Perception preserving decolorization | |
| Zheng et al. | Overwater image dehazing via cycle-consistent generative adversarial network | |
| Qu et al. | AIM-MEF: Multi-exposure image fusion based on adaptive information mining in both spatial and frequency domains | |
| CN117197166B (en) | Polyp image segmentation method and imaging method based on edge and neighborhood information | |
| CN109064394A (en) | A kind of image super-resolution rebuilding method based on convolutional neural networks | |
| Wang et al. | Infrared and visible image fusion based on Laplacian pyramid and generative adversarial network. | |
| CN108986027A (en) | Depth image super-resolution reconstruction method based on improved joint trilateral filter | |
| CN108876721A (en) | Super-resolution image reconstruction method and system based on course learning | |
| CN110390339B (en) | Image correction method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |