CN108171654B - Chinese character image super-resolution reconstruction method with interference suppression - Google Patents
Chinese character image super-resolution reconstruction method with interference suppression Download PDFInfo
- Publication number
- CN108171654B CN108171654B CN201711161572.8A CN201711161572A CN108171654B CN 108171654 B CN108171654 B CN 108171654B CN 201711161572 A CN201711161572 A CN 201711161572A CN 108171654 B CN108171654 B CN 108171654B
- Authority
- CN
- China
- Prior art keywords
- image
- low
- resolution
- chinese character
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000001629 suppression Effects 0.000 title claims abstract description 23
- 239000013598 vector Substances 0.000 claims abstract description 67
- 238000001914 filtration Methods 0.000 claims abstract description 43
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 29
- 238000005286 illumination Methods 0.000 claims abstract description 20
- 238000009499 grossing Methods 0.000 claims abstract description 18
- 238000000926 separation method Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 15
- 239000011159 matrix material Substances 0.000 claims description 41
- 238000012360 testing method Methods 0.000 claims description 36
- 238000012549 training Methods 0.000 claims description 26
- 238000011084 recovery Methods 0.000 claims description 10
- 230000015572 biosynthetic process Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000003786 synthesis reaction Methods 0.000 claims description 9
- 230000003190 augmentative effect Effects 0.000 claims description 7
- 230000000903 blocking effect Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000001604 Rao's score test Methods 0.000 claims description 3
- 230000003416 augmentation Effects 0.000 claims description 3
- 230000002902 bimodal effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000005728 strengthening Methods 0.000 abstract 1
- 238000005192 partition Methods 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000003706 image smoothing Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
Abstract
The invention belongs to the field of text image processing, and discloses a super-resolution reconstruction method for Chinese character images with interference suppression. The invention uses the image layer separation method to realize the separation of the Chinese character image illumination layers and reserve the reflection layer reflecting the essential attributes of the Chinese character image illumination layers. Then, the wavelet decomposition is carried out on the reflection layer image, and L with different scale factors is adopted for different sub-images0And smoothing filtering, and then realizing binarization separation of an image background and characters based on a self-adaptive text image binarization algorithm. And then, performing super-resolution reconstruction based on neighborhood embedding on the preprocessed Chinese character image, strengthening the weight of horizontal and vertical strokes in the composition of key characteristic vectors, and simultaneously fully considering diagonal strokes such as left falling strokes, right falling strokes and the like to design a super-resolution reconstruction method with interference suppression suitable for the Chinese character image.
Description
Technical Field
The invention belongs to the field of text image processing, and particularly relates to a super-resolution reconstruction method for a Chinese character image with interference suppression.
Background
In the current information age, electronic texts, which are an important information carrier spreading at high speed, are becoming one of the most intuitive ways for people to transmit and receive information. The Chinese characters are one of the earliest characters in the world, the Chinese characters are inherited for thousands of years, the cultural connotation contained in the Chinese characters is great and profound, and the Chinese character image is used as a special type image and has wide application in the fields of image processing, machine identification and the like. In many practical situations, the Chinese character images acquired by people are not all under ideal conditions, such as a dark illumination environment, and underlining added to the bottom of the Chinese characters for neatness, which can be an interference factor in super-resolution reconstruction. In order to suppress the above interference in the subsequent reconstruction process and improve the image reconstruction quality, it is necessary to perform interference suppression preprocessing before reconstruction.
In addition, the resolution (image size) of the obtained chinese character image is limited by the technical limitations of the imaging device, the scanning device, and the like, and in practical applications, in order for people or intelligent machines to correctly recognize and understand the information transmitted in the chinese character image, it is often necessary to effectively improve the resolution. However, the resolution of the image is directly improved by adopting a common image interpolation method, and although the size of the obtained Chinese character image is increased, the edge of the Chinese character image is not clear, and the definition of the image is not effectively improved. The visual clarity of the image should be maintained to the maximum extent while increasing the resolution of the image. Therefore, the method adopts an image super-resolution reconstruction method to amplify the resolution of the Chinese character image and improve the visual quality of the amplified image.
In recent years, a sample-based image super-resolution reconstruction method gradually becomes a research hotspot due to unique advantages, and the method learns the relation between high-resolution and low-resolution image blocks in a training set by means of a machine learning algorithm through constructing the training set or directly utilizes information in the training set to predict or reconstruct high-frequency detail information in a single-frame low-resolution test image, thereby realizing the super-resolution reconstruction of the image. In the method, a representative method is a super-resolution reconstruction method based on neighborhood embedding, and is characterized in that information of sample image blocks in a training set can be fully utilized under the condition of a small-scale training set, and finally a super-resolution reconstruction result with good quality can be obtained. Therefore, the super-resolution method based on neighborhood embedding is adopted to amplify the resolution of the low-resolution Chinese character image subjected to interference suppression in the early stage.
The invention is mainly based on the following theoretical or technical methods: illumination layer separation method, L0Norm smoothing filtering theory, guiding filtering theory and super-resolution reconstruction method based on neighborhood embedding. In order to achieve a better reconstruction effect, a low-rank matrix recovery and iterative back projection method is added. Meanwhile, a method for reconstructing the super-resolution of the Chinese character image with interference suppression is provided by combining methods of wavelet decomposition and synthesis, self-adaptive text image binarization and the like.
The illumination Layer Separation method is proposed by Yu Li et al in Single Image Layer Separation using Relative smoothening, which can solve the problem of extracting two layers of images from an Image, wherein one Layer is smoother than the other Layer. And L is0The norm Smoothing filter theory is based on the Image Smoothing via L by Li Xu et al0GradThe method provided by the patent Minimization can highlight the main boundary of the image, eliminate the image structure with low pixel value to a certain extent, and is particularly suitable for the applications of image edge extraction, cut-and-paste JPEG artifact removal and the like. The Guided Filtering theory is proposed by Kaiming He et al in Guided Image Filtering, and is a novel edge-preserving filter based on a local linear model. An Adaptive text Image Binarization algorithm is proposed by J.Sauvula et al in Adaptive Document Image Binarization.
The method for reconstructing the neighborhood Embedding Super-Resolution image is proposed by Hong Chang et al in Super-Resolution thread neighbor Embedding, and the method assumes that manifolds respectively formed by paired high-Resolution and low-Resolution image blocks are similar in local structure, so that the neighborhood relationship of each point in the low-Resolution data can be mapped to the corresponding point in the high-Resolution data. For the composition of the feature vector of the low-Resolution Image, see Image Super-Resolution Via Sparse Representation by Jianchao Yang et al, four gradient operators for extracting edge features in the first-order, second-order horizontal and vertical directions of the low-frequency sub-Image.
In addition, a low-rank matrix recovery and iterative back projection method is added in the reconstruction process so as to further improve the reconstruction effect. The low rank matrix recovery method is given by Emmanuel J. cand des et al in Robust Principal Component analysis? The method divides the matrix affected by the noise into the sum of a low-rank matrix and a sparse matrix. The iterative back projection algorithm is proposed by Michal Irani et al in Improving resolution by Image Registration, and aiming at the problem that a result Image obtained by neighborhood embedding based reconstruction has a blocking effect, global reconstruction constraint is added to the result Image in an iterative mode, so that the reconstruction quality is further improved.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a Chinese character image super-resolution reconstruction method with interference suppression.
The invention is realized in such a way that Chinese character images with interference are displayedSeparating the line illumination layer to obtain a reflection layer reflecting the essential attributes of the line illumination layer, performing wavelet decomposition on the reflection layer image to obtain low-frequency sub-images and high-frequency sub-images, and performing L with different scale factors on the sub-images0And (6) smoothing and filtering. And then, a self-adaptive text image binarization algorithm is applied, and binarization separation of an image background and characters is further realized by combining a guide filter. According to the structural characteristics of the Chinese character image, the horizontal and vertical stroke weights are increased, and the diagonal stroke characteristics such as left falling, right falling and the like are also fully considered, so that the neighborhood embedding super-resolution reconstruction method more suitable for the Chinese character image is generated.
Further, the method for reconstructing the super-resolution Chinese character image with interference suppression comprises the following steps:
step one, separating an illumination layer with interference Chinese character images: for the input dark Chinese character image, a layer separation method based on relative smooth characteristics, which is proposed by Yu Li and the like, is applied to separate an illumination layer, and a reflection layer reflecting the essential attributes of the illumination layer is reserved; for the input underlined Chinese character image, the above layer separation method is also applied, so that the interference of an illumination layer is removed, and simultaneously, the function of pre-removing the underline with the color lighter than that of the character is also played;
step two, wavelet decomposition and L of reflection layer image0Smoothing and filtering: and (4) performing wavelet decomposition on the reflection layer image obtained in the step one to obtain a low-frequency sub-image and three high-frequency sub-images. Because the low-frequency sub-image and the high-frequency sub-image respectively keep different structural features in one image, and then L is utilized according to the difference of the two different interference factors0According to the smooth filtering theory, smooth filtering processing of different scale factors is carried out on each sub-image, and then wavelet synthesis is carried out on the processed sub-images;
and step three, binarization and guided filtering of the wavelet-synthesized image: according to the gray value distribution characteristics of the Chinese character image, namely the whole image presents bimodal distribution with the gray values of characters and a background as the center, the image after wavelet synthesis is subjected to binarization processing by utilizing a self-adaptive text image binarization algorithm provided by J.Sauvola and the like, so that the character image only comprises two gray values of black and white. Because the preprocessed Chinese character image still possibly has interference, filtering processing is carried out again by using a guide filter so as to realize preprocessing before reconstruction of the super-resolution of the Chinese character image with the interference;
step four, generating a training set of super-resolution reconstruction: selecting a certain number of clear high-resolution images without interference, obtaining low-resolution images corresponding to the high-resolution images one by a method of downsampling and interpolation, and forming a training set by the paired high-resolution images and the paired low-resolution images. And performing wavelet decomposition on the low-resolution images in the training set, and combining first-order and second-order gradient operators to generate a characteristic graph group corresponding to the low-resolution images. And then, the feature map group corresponding to each low partial image is blocked, so that each low partial image block corresponds to a plurality of feature image blocks, and high partial images corresponding to the low partial images one by one are also correspondingly blocked. And combining a plurality of characteristic image blocks corresponding to each low-resolution image block into a column vector, namely a characteristic vector. Selecting N nearest to each low-score feature vector according to Euclidean distance between the low-score image block feature vectorstrainA low-score feature vector, which is organized into a feature group, corresponding to NtrainThe high-resolution image blocks are correspondingly grouped. In this way, all the low-partition characteristic blocks and the corresponding high-partition image blocks are coded and grouped;
step five, grouping of low-score test characteristic image blocks and low-rank matrix recovery: and performing interpolation processing on the test image, and then performing feature image group extraction, blocking and feature vector generation according to a low-resolution image training mode. For each test low-score feature vector, searching a low-score feature vector with the minimum Euclidean distance to the test low-score feature vector in the training set, and using N in the group where the feature vector is locatedtrainA low score feature vector, together with the test feature vector, forms an augmented matrix. N corresponding to the packettrainAnd the high-resolution image blocks also form a high-resolution matrix (without adding the high-resolution block characteristic vector corresponding to the test low-resolution block). And then decomposing the amplification matrix by using a low-rank matrix recovery algorithm proposed by Emmanuel J.Cand. For the high-resolution matrix,obtaining a corresponding high-grade low-rank matrix by adopting the same algorithm;
step six, reconstruction and image splicing of the high-resolution image block: selecting Euclidean distance nearest neighbor N of the reference vector from the low-rank matrix by using a neighborhood embedding reconstruction method and taking the corresponding test characteristic vector in the augmented low-rank matrix as a referencetestA low rank feature vector. And based on the nearest neighbor low-rank feature vectors, reconstructing the test low-rank feature vector by linear combination, and calculating a reconstruction coefficient which minimizes the reconstruction error. Combining the reconstruction coefficient with N in high-rank low-rank matrixtestAnd reconstructing a high-grade image block corresponding to the low-grade image block to be tested according to the high-grade low-grade vector corresponding to the low-grade vector. All blocks of the input test image are reconstructed according to the method, the reconstructed high-resolution image blocks are spliced according to the average fusion of the overlapped areas, and then the blocking effect is further eliminated through an iterative back projection algorithm. Therefore, the Chinese character image super-resolution reconstruction with interference suppression based on neighborhood embedding is realized.
Further, the specific method for performing smooth filtering on each sub-image after wavelet decomposition in the second step is as follows:
l proposed by Li Xu et al0The smoothing filtering theory assumes an image S, the filtered image
The gradient at any pixel point j is:
wherein
And
respectively representing the gradient values of the pixel point j in the horizontal direction x and the vertical direction y, and aiming at the image
Number of pixels of medium non-zero gradientCounting the amount:
wherein
To define a counter, filtering the processed image
This can be further estimated by minimizing:
wherein
As data fidelity terms to ensure L0The smooth filtered image has the maximum similarity with the original image, and lambda is a filtering parameter. By means of L0The smoothing filter is used for preprocessing the image super-resolution reconstruction, so that most of interference information in the Chinese character image can be filtered to a certain extent, namely noise components contained in a reflecting layer of the dark Chinese character image are filtered; for the Chinese character image with underlines, underlines which are lighter in color relative to strokes of Chinese characters in the reflecting layer are filtered to a certain extent.
Wavelet decomposition is carried out on the image to obtain a low-frequency sub-image of an image and high-frequency sub-images in the horizontal direction, the vertical direction and the diagonal direction. Because the low-frequency sub-image and each high-frequency sub-image respectively retain the structural information of different frequencies in one image, and the L with different scale factors is adopted for each sub-image by combining the two different interferences0Smoothing and filtering: for the dark Chinese character image, the low-frequency subimages are not filtered, and the high-frequency subimages are subjected to weaker smooth filtering; for the underlined Chinese character image, the low-frequency and horizontal high-frequency subimages are strongly smoothedIt filters the higher frequency sub-image more weakly. Finally, performing wavelet synthesis on the filtered sub-images;
further, the specific method for extracting the low-resolution feature image in the fourth step and the fifth step is as follows:
in the aspect of selecting key low-resolution image features for grouping and matching, the structural characteristics of Chinese character images are fully considered, the proportional weight of horizontal and vertical edge features represented by horizontal and vertical strokes is increased, and meanwhile, diagonal edge features represented by left-falling and right-falling strokes are fully considered. It should be noted that the wavelet decomposition used here is different from the previous image preprocessing, that is, a wavelet decomposition method without image down-sampling is used here, so as to better perform the block matching between the high and low resolution images.
After wavelet decomposition, four sub-images with the same size as the original image are obtained: a low frequency sub-image and high frequency sub-images in the horizontal, vertical and diagonal directions. Further, convolution operation is carried out on the low-frequency sub-images through four gradient operators used by Jianchao Yang et al shown in a formula (3), edge features of the first order and the second order of the low-frequency sub-images in the horizontal direction and the vertical direction are extracted, and three high-frequency sub-images obtained through wavelet decomposition are combined to generate important low-resolution image feature vectors.
Wherein f is2、f4Are respectively f1And f3Transposing;
further, in the fourth step, the fifth step and the sixth step, grouping with the nearest euclidean distance, searching nearest neighbor feature blocks, constructing an amplification matrix, recovering a low rank, linearly combining and reconstructing high and low resolution image blocks and the like are all calculated by respectively forming column vectors from the low resolution feature blocks and the high resolution image blocks. Meanwhile, for convenience of description, the high resolution is also referred to as high score and the low resolution is also referred to as low score. This is achieved byIn general, NtrainMuch greater than Ntest；
Further, the super-resolution reconstruction based on neighborhood embedding in the sixth step is proposed by Hong Chang et al, and the specific implementation steps are as follows:
(a) for low resolution test image XtEach image block in (1)
In the training set
Find its k nearest neighbors
Wherein N isg(j) A neighborhood set representing j;
(b) using these nearest neighbors
Reconstructing the low resolution image block
Calculating a weight ω for minimizing a reconstruction errorijThe calculation formula is as follows:
(c) according to the assumption that the manifold of high-resolution image blocks also has a similar local geometry, those high-resolution image blocks in the training set that correspond to nearest neighbors are used
And weight omegaijEstimating a high resolution image block by linear combination of
Drawings
Fig. 1 is a flowchart of a method for reconstructing super-resolution of a chinese character image with interference suppression according to an embodiment of the present invention.
FIG. 2 is a detailed flowchart of the method flowchart shown in FIG. 1, based on the neighborhood embedding Chinese character image super-resolution reconstruction.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1 and fig. 2, the method for reconstructing super-resolution chinese character images with interference suppression according to the embodiment of the present invention includes the following steps:
step one, separating an illumination layer with interference Chinese character images: for the input dark Chinese character image, a layer separation method based on relative smooth characteristics, which is proposed by Yu Li and the like, is applied to separate an illumination layer, and a reflection layer reflecting the essential attributes of the illumination layer is reserved; for the input underlined Chinese character image, the above layer separation method is also applied, so that the interference of an illumination layer is removed, and simultaneously, the function of pre-removing the underline with the color lighter than that of the character is also played;
step two, wavelet decomposition and L of reflection layer image0Smoothing and filtering: and (4) performing wavelet decomposition on the reflection layer image obtained in the step one to obtain a low-frequency sub-image and three high-frequency sub-images. Because the low-frequency sub-image and the high-frequency sub-image respectively keep different structural features in one image, and then L is utilized according to the difference of the two different interference factors0According to the smooth filtering theory, smooth filtering processing of different scale factors is carried out on each sub-image, and then wavelet synthesis is carried out on the processed sub-images;
and step three, binarization and guided filtering of the wavelet-synthesized image: according to the gray value distribution characteristics of the Chinese character image, namely the whole image presents bimodal distribution with the gray values of characters and a background as the center, the image after wavelet synthesis is subjected to binarization processing by utilizing a self-adaptive text image binarization algorithm provided by J.Sauvola and the like, so that the character image only comprises two gray values of black and white. Because the preprocessed Chinese character image still possibly has interference, filtering processing is carried out again by using a guide filter so as to realize preprocessing before reconstruction of the super-resolution of the Chinese character image with the interference;
step four, generating a training set of super-resolution reconstruction: selecting a certain number of clear high-resolution images without interference, obtaining low-resolution images corresponding to the high-resolution images one by a method of downsampling and interpolation, and forming a training set by the paired high-resolution images and the paired low-resolution images. And performing wavelet decomposition on the low-resolution images in the training set, and combining first-order and second-order gradient operators to generate a characteristic graph group corresponding to the low-resolution images. And then, the feature map group corresponding to each low partial image is blocked, so that each low partial image block corresponds to a plurality of feature image blocks, and high partial images corresponding to the low partial images one by one are also correspondingly blocked. And combining a plurality of characteristic image blocks corresponding to each low-resolution image block into a column vector, namely a characteristic vector. Selecting N nearest to each low-score feature vector according to Euclidean distance between the low-score image block feature vectorstrainA low-score feature vector, which is organized into a feature group, corresponding to NtrainThe high-resolution image blocks are correspondingly grouped. In this way, all the low-partition characteristic blocks and the corresponding high-partition image blocks are coded and grouped;
step five, grouping of low-score test characteristic image blocks and low-rank matrix recovery: and performing interpolation processing on the test image, and then performing feature image group extraction, blocking and feature vector generation according to a low-resolution image training mode. For each test low-score feature vector, searching a low-score feature vector with the minimum Euclidean distance to the test low-score feature vector in the training set, and using N in the group where the feature vector is locatedtrainA low score feature vector, together with the test feature vector, forms an augmented matrix. N corresponding to the packettrainA high-resolution image block also forms a high-resolution matrix (without adding the high-resolution block corresponding to the test low-resolution block)A feature vector). And then decomposing the amplification matrix by using a low-rank matrix recovery algorithm proposed by Emmanuel J.Cand. For the high-resolution matrix, obtaining a corresponding high-resolution low-rank matrix by adopting the same algorithm;
step six, reconstruction and image splicing of the high-resolution image block: selecting Euclidean distance nearest neighbor N of the reference vector from the low-rank matrix by using a neighborhood embedding reconstruction method and taking the corresponding test characteristic vector in the augmented low-rank matrix as a referencetestA low rank feature vector. And based on the nearest neighbor low-rank feature vectors, reconstructing the test low-rank feature vector by linear combination, and calculating a reconstruction coefficient which minimizes the reconstruction error. Combining the reconstruction coefficient with N in high-rank low-rank matrixtestAnd reconstructing a high-grade image block corresponding to the low-grade image block to be tested according to the high-grade low-grade vector corresponding to the low-grade vector. All blocks of the input test image are reconstructed according to the method, the reconstructed high-resolution image blocks are spliced according to the average fusion of the overlapped areas, and then the blocking effect is further eliminated through an iterative back projection algorithm. Therefore, the Chinese character image super-resolution reconstruction with interference suppression based on neighborhood embedding is realized.
Further, the specific method for performing smooth filtering on each sub-image after wavelet decomposition in the second step is as follows:
l proposed by Li Xu et al0The smoothing filtering theory assumes an image S, the filtered image
The gradient at any pixel point j is:
wherein
And
respectively represent imagesGradient values in the horizontal direction x and the vertical direction y of the pixel j for the image
Counting the number of pixels with medium non-zero gradient:
wherein
To define a counter, filtering the processed image
This can be further estimated by minimizing:
wherein
As data fidelity terms to ensure L0The smooth filtered image has the maximum similarity with the original image, and lambda is a filtering parameter. By means of L0The smoothing filter is used for preprocessing the image super-resolution reconstruction, so that most of interference information in the Chinese character image can be filtered to a certain extent, namely noise components contained in a reflecting layer of the dark Chinese character image are filtered; for the Chinese character image with underlines, underlines which are lighter in color relative to strokes of Chinese characters in the reflecting layer are filtered to a certain extent.
Wavelet decomposition is carried out on the image to obtain a low-frequency sub-image of an image and high-frequency sub-images in the horizontal direction, the vertical direction and the diagonal direction. Because the low-frequency sub-image and each high-frequency sub-image respectively retain the structural information of different frequencies in one image, and the L with different scale factors is adopted for each sub-image by combining the two different interferences0Smoothing and filtering: to pairIn the dark Chinese character image, the low-frequency subimages are not filtered, and the high-frequency subimages are subjected to weaker smooth filtering; for the underlined Chinese character image, the low-frequency and horizontal-direction high-frequency sub-images are subjected to stronger smooth filtering, and the other-direction high-frequency sub-images are subjected to weaker filtering. Finally, performing wavelet synthesis on the filtered sub-images;
further, the specific method for extracting the low-resolution feature image in the fourth step and the fifth step is as follows:
in the aspect of selecting key low-resolution image features for grouping and matching, the structural characteristics of Chinese character images are fully considered, the proportional weight of horizontal and vertical edge features represented by horizontal and vertical strokes is increased, and meanwhile, diagonal edge features represented by left-falling and right-falling strokes are fully considered. It should be noted that the wavelet decomposition used here is different from the previous image preprocessing, that is, a wavelet decomposition method without image down-sampling is used here, so as to better perform the block matching between the high and low resolution images.
After wavelet decomposition, four sub-images with the same size as the original image are obtained: a low frequency sub-image and high frequency sub-images in the horizontal, vertical and diagonal directions. Further, convolution operation is carried out on the low-frequency sub-images through four gradient operators used by Jianchao Yang et al shown in a formula (3), edge features of the first order and the second order of the low-frequency sub-images in the horizontal direction and the vertical direction are extracted, and three high-frequency sub-images obtained through wavelet decomposition are combined to generate important low-resolution image feature vectors.
Wherein f is2、f4Are respectively f1And f3Transposing;
further, the super-resolution reconstruction based on neighborhood embedding in the sixth step is proposed by Hong Chang et al, and the specific implementation steps are as follows:
(a) for low resolutionTest image XtEach image block in (1)
In the training set
Find its k nearest neighbors
Wherein N isg(j) A neighborhood set representing j;
(b) using these nearest neighbors
Reconstructing the low resolution image block
Calculating a weight ω for minimizing a reconstruction errorijThe calculation formula is as follows:
(c) according to the assumption that the manifold of high-resolution image blocks also has a similar local geometry, those high-resolution image blocks in the training set that correspond to nearest neighbors are used
And weight omegaijEstimating a high resolution image block by linear combination of
In FIG. 2, the Euclidean distance nearest grouping, the nearest neighbor feature block searching, the composition of the augmentation matrix and the low rank recovery are used as the linear combination of the high and low resolution image blocksAnd in the processes of reconstruction and the like, the low-resolution feature block and the high-resolution image block are respectively formed into column vectors for calculation. Meanwhile, for convenience of description, the high resolution is also referred to as high score and the low resolution is also referred to as low score. In addition, N is generallytrainMuch greater than Ntest。
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (5)
1. A Chinese character image super-resolution reconstruction method with interference suppression is characterized in that the Chinese character image super-resolution reconstruction method with interference suppression uses an image layer separation method based on relative smooth characteristics, firstly, separation of an illumination layer of a Chinese character image with interference is realized, and a reflection layer reflecting essential attributes of the Chinese character image is obtained; then, the reflection layer is subjected to wavelet decomposition to obtain a low-frequency sub-image and three high-frequency sub-images, and L with different scale factors is carried out on each sub-image according to different interference factors borne by the image0Smoothing and filtering; finally, a self-adaptive text image binarization algorithm is adopted to realize binarization separation of the background and characters of the Chinese character image, and interference suppression pretreatment before super-resolution reconstruction of the Chinese character image is realized through a guiding filter with excellent performance; in the super-resolution reconstruction stage, according to the structural characteristics of the Chinese character image, the horizontal and vertical stroke weights are increased, and meanwhile, the characteristics of left-falling and right-falling diagonal strokes are fully considered, so that a neighborhood embedding super-resolution reconstruction method more suitable for the Chinese character image is generated;
the Chinese character image super-resolution reconstruction method with interference suppression comprises the following steps:
step one, separating an illumination layer with interference Chinese character images: for the input dark Chinese character image, the separation treatment of the illumination layer is carried out based on the layer separation method with the relative smooth characteristic, and the reflecting layer reflecting the essential attribute is reserved; for the input underlined Chinese character image, the above layer separation method is also applied, so that the interference of an illumination layer is removed, and simultaneously, the function of pre-removing the underline with the color lighter than that of the character is also played;
step two, wavelet decomposition and L of reflection layer image0Smoothing and filtering: performing wavelet decomposition on the reflection layer image obtained in the step one to obtain a low-frequency sub-image and three high-frequency sub-images; the low-frequency sub-image and the high-frequency sub-image respectively reserve different structural features in one image, and then L is utilized according to the difference of two different interference factors, namely the dark Chinese character image and the underline with the color lighter than the character per se0According to the smooth filtering theory, smooth filtering processing of different scale factors is carried out on each sub-image, and then wavelet synthesis is carried out on the processed sub-images;
and step three, binarization and guided filtering of the wavelet-synthesized image: according to the bimodal distribution characteristics of the gray value of the Chinese character image, carrying out binarization processing on the wavelet-synthesized image by using a self-adaptive text image binarization algorithm; because the preprocessed Chinese character image still possibly has interference, filtering processing is carried out again by using a guide filter so as to realize preprocessing before reconstruction of the super-resolution of the Chinese character image with the interference;
step four, generating a training set of super-resolution reconstruction: selecting a certain number of clear high-resolution images without interference, obtaining low-resolution images corresponding to the high-resolution images one by a method of downsampling and interpolation, and forming a training set by the paired high-resolution images and the paired low-resolution images; performing wavelet decomposition on the low partial images in the training set, and combining first-order and second-order gradient operators to generate a characteristic graph group corresponding to the low partial images; then, the feature map group corresponding to each low partial image is blocked, so that each low partial image block corresponds to a plurality of feature image blocks, and high partial images corresponding to the low partial images one by one are also correspondingly blocked; combining a plurality of characteristic image blocks corresponding to each low-resolution image block into a column vector, namely a characteristic vector; selecting N nearest to each low-score feature vector according to Euclidean distance between the low-score image block feature vectorstrainA low-score feature vector, which is organized into a feature group, corresponding to NtrainCorrespondingly compiling the high-resolution image blocks into a group; in this manner, all low-scoring feature blocks and corresponding high scores are scoredThe image blocks are all encoded and grouped;
step five, grouping of low-score test characteristic image blocks and low-rank matrix recovery: firstly, carrying out interpolation processing on a test image, and then carrying out feature image group extraction, blocking and feature vector generation according to a low-resolution image training mode; for each test low-score feature vector, searching a low-score feature vector with the minimum Euclidean distance to the test low-score feature vector in the training set, and using N in the group where the feature vector is locatedtrainA low score feature vector, which forms an augmentation matrix with the test feature vector; n corresponding to the packettrainThe high-resolution image blocks also form a high-resolution matrix, wherein the high-resolution block characteristic vectors corresponding to the low-resolution blocks are not added; then decomposing the augmented matrix by using a low-rank matrix recovery algorithm, removing a sparse matrix containing noise and illumination interference, and obtaining a corresponding augmented low-rank matrix; for the high-resolution matrix, obtaining a corresponding high-resolution low-rank matrix by adopting the same algorithm;
step six, reconstruction and image splicing of the high-resolution image block: selecting Euclidean distance nearest neighbor N of the reference vector from the low-rank matrix by using a neighborhood embedding reconstruction method and taking the corresponding test characteristic vector in the augmented low-rank matrix as a referencetestA low rank eigenvector; based on the nearest neighbor low-rank feature vectors, the testing low-rank feature vectors are reconstructed by linear combination, and a reconstruction coefficient enabling the reconstruction error to be minimum is calculated; combining the reconstruction coefficient with N in high-rank low-rank matrixtestReconstructing a high-grade image block corresponding to the test low-grade image block by using the high-grade low-grade vectors corresponding to the low-grade vectors; reconstructing all blocks of an input test image, splicing the reconstructed high-resolution image blocks according to the average fusion of overlapping areas, and further eliminating the blocking effect by an iterative back projection algorithm; and realizing the Chinese character image super-resolution reconstruction with interference suppression based on neighborhood embedding.
2. The method for reconstructing the super-resolution Chinese character image with interference suppression as claimed in claim 1, wherein the specific method for smoothing the sub-images after wavelet decomposition in the second step is as follows:
L0the smoothing filtering theory assumes an image S, the filtered image
The gradient at any pixel point j is:
wherein
And
respectively representing the gradient values of the pixel point j in the horizontal direction x and the vertical direction y, and aiming at the image
Counting the number of pixels with medium non-zero gradient:
wherein
To define a counter, filtering the processed image
This can be further estimated by minimizing:
wherein
Is a number ofAccording to fidelity terms, to ensure L0The smooth filtered image has the maximum similarity with the original image, and lambda is a filtering parameter; by means of L0The smoothing filter is used for preprocessing the image super-resolution reconstruction, so that most of interference information in the Chinese character image can be filtered to a certain extent, namely noise components contained in a reflecting layer of the dark Chinese character image are filtered; for the Chinese character image with underlines, underlines which are lighter in color relative to strokes of Chinese characters in the reflecting layer are filtered to a certain extent;
performing wavelet decomposition on the image to obtain a low-frequency sub-image of an image and high-frequency sub-images in the horizontal direction, the vertical direction and the diagonal direction; because the low-frequency sub-image and each high-frequency sub-image respectively retain the structural information of different frequencies in one image, and the L with different scale factors is adopted for each sub-image by combining the two different interferences0Smoothing and filtering: for the dark Chinese character image, the low-frequency subimages are not filtered, and the high-frequency subimages are subjected to weaker smooth filtering; for the Chinese character image with underline lines, carrying out stronger smooth filtering on the low-frequency and horizontal-direction high-frequency subimages of the Chinese character image, and carrying out weaker filtering on the other-direction high-frequency subimages; and finally, performing wavelet synthesis on the filtered sub-images.
3. The method for reconstructing the super-resolution Chinese character image with interference suppression as claimed in claim 1, wherein the specific method for extracting the low-resolution feature image in the fourth step and the fifth step is as follows:
in the aspect of selecting key low-resolution image features for grouping and matching, the structural characteristics of Chinese character images are fully considered, the proportional weights of horizontal and vertical edge features represented by horizontal and vertical strokes are increased, and meanwhile, the diagonal edge features represented by left-falling and right-falling strokes are fully considered; after wavelet decomposition, four sub-images with the same size as the original image are obtained: a low-frequency sub-image and high-frequency sub-images in the horizontal direction, the vertical direction and the diagonal direction; further, carrying out convolution operation on the low-frequency sub-image and four gradient operators shown in the formula (3), extracting first-order and second-order edge features in the horizontal and vertical directions of the low-frequency sub-image, and combining three high-frequency sub-images obtained by wavelet decomposition to generate important low-resolution image feature vectors in a combined manner:
wherein f is2、f4Are respectively f1And f3The transposing of (1).
4. The method for reconstructing super-resolution of chinese character images with interference suppression according to claim 1, wherein in the fourth, fifth and sixth steps, the linear combination reconstruction of high and low resolution image blocks by grouping with the closest euclidean distance, searching nearest neighbor feature blocks, constructing an augmentation matrix and low rank restoration are performed by respectively constructing the low resolution feature blocks and the high resolution image blocks into column vectors for calculation; meanwhile, the high resolution is referred to as high score for short, and the low resolution is referred to as low score for short; in addition, N is generallytrainMuch greater than Ntest。
5. The method for reconstructing the super-resolution of the Chinese character image with the interference suppression function according to claim 1, wherein the super-resolution reconstruction based on the neighborhood embedding in the sixth step is implemented by the following steps:
(a) for low resolution test image XtEach image block in (1)
In the training set
Find its k nearest neighbors
Wherein N isg(j) A neighborhood set representing j;
(b) by usingThese nearest neighbors
Reconstructing the low resolution image block
Calculating a weight ω for minimizing a reconstruction errorijThe calculation formula is as follows:
(c) according to the assumption that the manifold of high-resolution image blocks also has a similar local geometry, those high-resolution image blocks in the training set that correspond to nearest neighbors are used
And weight omegaijEstimating a high resolution image block by linear combination of
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711161572.8A CN108171654B (en) | 2017-11-20 | 2017-11-20 | Chinese character image super-resolution reconstruction method with interference suppression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711161572.8A CN108171654B (en) | 2017-11-20 | 2017-11-20 | Chinese character image super-resolution reconstruction method with interference suppression |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108171654A CN108171654A (en) | 2018-06-15 |
| CN108171654B true CN108171654B (en) | 2021-06-22 |
Family
ID=62527136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711161572.8A Expired - Fee Related CN108171654B (en) | 2017-11-20 | 2017-11-20 | Chinese character image super-resolution reconstruction method with interference suppression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108171654B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109410239B (en) * | 2018-11-07 | 2021-11-16 | 南京大学 | Text image super-resolution reconstruction method based on condition generation countermeasure network |
| CN109543669A (en) * | 2018-12-04 | 2019-03-29 | 龙马智芯（珠海横琴）科技有限公司 | The processing method and processing device of text, storage medium, electronic device |
| CN111879791B (en) * | 2020-07-30 | 2023-06-20 | 西湖大学 | Machine vision system and method for enhancing raised features on pattern surface |
| CN112183335A (en) * | 2020-09-28 | 2021-01-05 | 中国人民大学 | Handwritten image recognition method and system based on unsupervised learning |
| CN113660444A (en) * | 2021-10-19 | 2021-11-16 | 杭州太美星程医药科技有限公司 | Image generation method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635047A (en) * | 2009-03-25 | 2010-01-27 | 湖南大学 | Texture synthesis and image repair method based on wavelet transformation |
| CN102880859A (en) * | 2012-08-30 | 2013-01-16 | 华南理工大学 | Method for recognizing number plate |
| US8938118B1 (en) * | 2012-12-12 | 2015-01-20 | Rajiv Jain | Method of neighbor embedding for OCR enhancement |
| CN104809461A (en) * | 2015-05-08 | 2015-07-29 | 内蒙古科技大学 | License plate recognition method and system combining sequence image super-resolution reconstruction |
| CN105046651A (en) * | 2015-08-04 | 2015-11-11 | 深圳信息职业技术学院 | Super-resolution reconstruction method and apparatus for image |
| CN106127688A (en) * | 2016-06-30 | 2016-11-16 | 北京大学 | A kind of super-resolution image reconstruction method and system thereof |
-
2017
- 2017-11-20 CN CN201711161572.8A patent/CN108171654B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635047A (en) * | 2009-03-25 | 2010-01-27 | 湖南大学 | Texture synthesis and image repair method based on wavelet transformation |
| CN102880859A (en) * | 2012-08-30 | 2013-01-16 | 华南理工大学 | Method for recognizing number plate |
| US8938118B1 (en) * | 2012-12-12 | 2015-01-20 | Rajiv Jain | Method of neighbor embedding for OCR enhancement |
| CN104809461A (en) * | 2015-05-08 | 2015-07-29 | 内蒙古科技大学 | License plate recognition method and system combining sequence image super-resolution reconstruction |
| CN105046651A (en) * | 2015-08-04 | 2015-11-11 | 深圳信息职业技术学院 | Super-resolution reconstruction method and apparatus for image |
| CN106127688A (en) * | 2016-06-30 | 2016-11-16 | 北京大学 | A kind of super-resolution image reconstruction method and system thereof |
Non-Patent Citations (5)
| Title |
|---|
| Image Super-Resolution Via Sparse Representation;Jianchao Yang 等;《IEEE TRANSACTIONS ON IMAGE PROCESSING》;20101130;第19卷(第11期);第2868页 * |
| Super-Resolution Through Neighbor Embedding;Hong Chang 等;《Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition》;20040702;第2.4节 * |
| 基于低秩矩阵恢复和联合学习的图像超分辨率重建;陈晓璇 等;《计算机学报》;20140630;第37卷(第6期);全文 * |
| 张万绪 等.弱光背景下文本图像二值化提取算法.《西北大学学报(自然科学版)》.2017,第47卷(第2期), * |
| 弱光背景下文本图像二值化提取算法;张万绪 等;《西北大学学报(自然科学版)》;20170430;第47卷(第2期);第192-194页和图1-5 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108171654A (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108171654B (en) | Chinese character image super-resolution reconstruction method with interference suppression | |
| CN111062872B (en) | Image super-resolution reconstruction method and system based on edge detection | |
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| CN110992262B (en) | Remote sensing image super-resolution reconstruction method based on generation countermeasure network | |
| CN108257095B (en) | System for processing images | |
| CN105678723A (en) | Multi-focus image fusion method based on sparse decomposition and differential image | |
| Yue et al. | Unsupervised moiré pattern removal for recaptured screen images | |
| Chen et al. | Single depth image super-resolution using convolutional neural networks | |
| CN106157240A (en) | Remote sensing image super resolution method based on dictionary learning | |
| Zou et al. | Joint wavelet sub-bands guided network for single image super-resolution | |
| Liu et al. | Facial image inpainting using multi-level generative network | |
| Song et al. | Dual perceptual loss for single image super-resolution using esrgan | |
| CN116977651B (en) | Image denoising method based on double-branch and multi-scale feature extraction | |
| Nascimento et al. | Super-resolution of license plate images using attention modules and sub-pixel convolution layers | |
| CN113763300A (en) | Multi-focus image fusion method combining depth context and convolution condition random field | |
| Quan et al. | Deep Learning-Based Image and Video Inpainting: A Survey | |
| CN116188273A (en) | Uncertainty-oriented bimodal separable image super-resolution method | |
| CN111899166A (en) | Medical hyperspectral microscopic image super-resolution reconstruction method based on deep learning | |
| Xiu et al. | Double discriminative face super-resolution network with facial landmark heatmaps | |
| CN111275620A (en) | Image super-resolution method based on Stacking ensemble learning | |
| Huang et al. | A non-local enhanced network for image restoration | |
| Zhao et al. | Structure-Texture Dual Preserving for Remote Sensing Image Super-Resolution | |
| Gupta et al. | Super-resolution-based GAN for image processing: Recent advances and future trends | |
| Cao et al. | Improved wavelet prediction superresolution reconstruction based on U‐Net | |
| Bogdan et al. | DDocE: Deep Document Enhancement with Multi-scale Feature Aggregation and Pixel-Wise Adjustments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210622 Termination date: 20211120 |