CN110148087B - Image compression and reconstruction method based on sparse representation - Google Patents
Image compression and reconstruction method based on sparse representation Download PDFInfo
- Publication number
- CN110148087B CN110148087B CN201910430149.6A CN201910430149A CN110148087B CN 110148087 B CN110148087 B CN 110148087B CN 201910430149 A CN201910430149 A CN 201910430149A CN 110148087 B CN110148087 B CN 110148087B
- Authority
- CN
- China
- Prior art keywords
- image
- block
- dictionary
- resolution
- resolution image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000006835 compression Effects 0.000 title claims abstract description 20
- 238000007906 compression Methods 0.000 title claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000005457 optimization Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 7
- 230000000903 blocking effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 238000005192 partition Methods 0.000 claims description 3
- 238000006731 degradation reaction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Image Processing (AREA)
Abstract
The invention provides an image compression and reconstruction method based on sparse representation, which adopts a quadtree idea to divide an image into blocks, divides a smooth area of a low-resolution image into image blocks with the sizes of 64 x 64 and 32 x 32, divides a non-smooth image area containing detail information into image blocks with the size of 4*4, and shortens the dictionary training time by reducing the number of training samples on the basis of ensuring the accuracy of dictionary training; when the decoding end reconstructs a decoded image, the smooth image block is directly reconstructed by using a bicubic interpolation mode, and the image feature block containing detail information is reconstructed by using a sparse representation mode, so that the high-resolution image reconstruction time is shortened on the premise of ensuring the subjective quality of the image reconstruction; and the image compression ratio is improved.
Description
Technical Field
The invention relates to the field of image compression and reconstruction, in particular to an image compression and reconstruction method based on sparse representation.
Background
The image information has obvious advantages of intuitiveness, liveness, concrete and the like, and is one of important ways for people to acquire the information. However, high quality images often occupy a large amount of memory space, and classical image compression techniques, such as JPEG, JPEG2000, can compress the amount of data to the maximum extent possible while guaranteeing the image quality as much as possible. However, with the advent of high definition and ultra high definition images, conventional image compression techniques sometimes fail to meet the information acquisition requirements, along with limitations in network bandwidth and storage media.
Because of the bandwidth of the network and the storage space constraints of the computer, network transmissions often employ low resolution, compressed images. However, people often prefer clear images in real life. Therefore, the image compression method for compressing the image data as large as possible on the premise of ensuring that the reconstructed image does not influence subjective quality, so that the image data occupies smaller network bandwidth and storage space is quite significant.
Based on the existing mainstream image compression standard, the invention discloses a new image compression and reconstruction framework. The framework combines the existing image compression coding method JPEG2000 and the existing image super-resolution method based on sparse representation, and aims to improve the image compression rate and the subjective quality of image reconstruction.
Disclosure of Invention
The invention provides an image compression and reconstruction method based on sparse representation on the basis of JPEG2000 compression standard, aiming at improving the image compression ratio and simultaneously guaranteeing the subjective quality of decompressed images.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
s1, downsampling an original high-resolution image X to be compressed to obtain a downsampled low-resolution image Y L ；
S2, for the low resolution image Y L Upsampling to obtain Y of the same resolution as X H Image Y is divided by using quadtree block division method H Block division is performed to divide the image Y H Dividing into image blocks of 64×64, 32×32 and 16×16, for image block of 16×16, further dividing into image block of 4*4 and extracting features to obtain low resolution image feature block F L ；
S3, utilizing the one-to-one correspondence of the high-low resolution image blocks and Y H Corresponding division is carried out on the high-resolution image X to obtain a high-resolution image block X after block division H ；
S4, respectively training the high-resolution image blocks X by using K-SVD (K times singular value decomposition) H And a low resolution image feature block F L Corresponding dictionary D H And D L ；
S5, coding the low-resolution image Y at the coding end L Through JPEG2000 standard compressionShrinking to obtain Y L And a high-low resolution dictionary D H And D L Packaging and transmitting to a decoding end;
s6, firstly de-multiplexing the code stream at the decoding end to obtain a low-resolution image Y L Compressed code stream and high-low resolution dictionary D of (2) H And D L ；
S7, pair Y L The compressed code stream of (2) is decoded by JPEG2000 standard to obtain low resolution image Y' L Then for Y' L Upsampling to obtain an image Y 'with the same resolution as the original image' H ；
S8, for the image Y' H Performing the same block division as in step S2, and extracting the characteristics of the 4*4 image block to obtain corresponding image characteristic block F' L ；
S9, after block division is completed, the block divided by 64 x 64 and 32 x 32 is directly adopted as a reconstruction block by the up-sampling result in the step S7; for image feature block F' L Using sparse representation method and high-low resolution dictionary D H And D L Performing super-resolution reconstruction to obtain a reconstructed image block X' H ；
S10, splicing all reconstructed image blocks to corresponding positions to obtain a reconstructed high-resolution image X 0 Then, the iterative back projection technology is utilized for X 0 Global optimization is carried out to obtain a final reconstructed image X * 。
As one of the preferred embodiments of the present invention, the downsampling method in step S1 may be any of a variety of existing downsampling methods, such as a bicubic downsampling method, a simple extraction method, and the like.
As one of the preferred embodiments of the present invention, the upsampling method of step S2 is a bicubic upsampling method.
Step S2 is to divide the image Y by using a quadtree block H The specific implementation mode of the division is as follows:
s2.1 image Y H Divided into a plurality of mutually non-overlapping image blocks of size 64 x 64, called CTU for short, which is Y H A divided basic unit;
s2.2 each CTU mayThree sub-picture blocks (CUs) are selected and divided into 64X 64, 32X 32 and 16X 16, wherein the CUs with the sizes of 64X 64 and 32X 32 correspond to Y H Is a smooth region of CU size 16 x 16 corresponds to Y H Is not smooth.
S2.3, determining a threshold T of block division, wherein the specific method is as follows:
for an image Y of size m x n H The average value of the gray differences between the adjacent pixels in the ith row and the jth column is respectivelyAnd->The calculation formula is as follows:
the threshold T of the block division is defined by the whole m x n image Y H The average value of the gray differences between adjacent pixels is determined, namely:
s2.4 for each CTU of 64 x 64, the block partitioning procedure is as follows:
s2.4.1 the average value T of the gray differences between adjacent pixels of each CTU is calculated in the same manner as S2.3 CTU And compared with a blocking threshold T. When T is CTU >T, dividing the CTU into four CUs of 32×32, and executing step S2.4.2; otherwise, the CTU does not continue to divide, i.e. the size of 64×64;
s2.4.2 calculates the average value T of the gray differences between adjacent pixels of 4 32X 32CU sequentially according to the same method as S2.3 CU And compared with a blocking threshold T. When T is CU >T, the CU isDividing into 4 CUs 16×16; otherwise, the CU does not continue to partition, i.e. the size is 32 x 32.
The step S2 is further divided into image blocks of size 4*4 for the image blocks of size 16×16, and features are extracted to obtain a low resolution image feature block F L The specific method of (2) is as follows:
s2.5 for a CU 16 x 16, it is further divided into a plurality of 4*4 tiles, adjacent 4*4 tiles having k (0 < k < 4) pixels overlapping.
S2.6 the following 4 one-dimensional filters are used for extracting features of the image block of 4*4.
f 1 ＝[-1,0,1],f 2 ＝f 1 T ,f 3 ＝[1,0,-2,0,1],f 4 ＝f 3 T Where T represents the transpose. Applying the four filters to obtain 4 eigenvectors of each image block, and connecting the 4 eigenvectors into a 64 x 1 eigenvector, denoted as image eigenvector F L 。
As one of the preferable schemes of the invention, the step S4 is to train by using K-SVD to obtain the low-resolution image characteristic block F L Corresponding dictionary D L The specific implementation mode of the method is as follows:
s4.1 with all F L A sample dataset matrix F is composed.
S4.2 matrix F may be derived from dictionary matrix D L And coefficient matrix A, the aim of dictionary training is to find dictionary matrix D L And a coefficient matrix a. The linearity is expressed as:
F≈D L A
the dictionary training process described above may be translated into solving the following optimization problem:
epsilon is the maximum allowed by the reconstruction error.
S4.3, solving the optimization problem as follows:
s4.3.1 the Image super-resolution via sparse represen by YangDictionary published in the station paper as initial D L 。
S4.3.2 in the known D L Under the condition, the sparse coefficient of each sample is obtained to obtain the D L A corresponding coefficient matrix A;
s4.3.3 under the condition of knowing A, updating D by adopting K-SVD method L ；
S4.3.4 steps S4.3.2 and S4.3.3 are repeated until the reconstruction error meets the requirements, at which point D L Namely F L A corresponding training dictionary.
As one of the preferable schemes of the invention, the step S4 trains the high resolution image block X by using K-SVD H Corresponding dictionary D H In the process of (a), an image block X with a size of 4*4 is processed H Converted into a column vector of 16X 1, all X H Forming a matrix of sample data sets, and dictionary D L The dictionary D is obtained by using K-SVD as in the training process H 。
As one of the preferable schemes of the invention, the step S9 utilizes a sparse representation method and a high-low resolution dictionary D H And D L The super-resolution reconstruction process comprises the following specific steps:
s9.1 for each image feature block F' L Using dictionary D L The corresponding sparse coefficient alpha can be solved, and the calculation formula is as follows:
F′ L ＝D L α ||α|| 0 ≤N
wherein alpha is E R N N is the number of atoms in the sparse dictionary.
S9.2 corresponding reconstructed high resolution image block X' H Can use dictionary D H And the solved sparse coefficient alpha are linearly combined to represent:
X′ H ＝D H α ||α 0 ≤N
as one of the preferable schemes of the invention, the step 10 uses the iterative back projection method to perform the X 0 The specific steps for global optimization are as follows:
s10.1 reconstruction of high resolution image X 0 Downsampling D and blurringH, simulating a high-resolution image degradation process, and solving a degraded image X' t And a low resolution image Y L Error of (2) to reconstruct high resolution image X t And performing iterative updating. The calculation formula of the iterative back projection method is as follows:
X t+1 ＝X t +((Y L -X′ t )↑s)*p
wherein X 'is' t ＝DHX t ，X t For the t iteration process high resolution image, p is a back projection filter, ∈r is upsampling, and S is an upsampling factor.
S10.2 As final reconstructed high resolution image X * Stopping iterative updating when the reconstruction constraint is met, wherein the reconstruction constraint formula is as follows:
compared with the prior art, the invention has the following advantages:
(1) The invention adopts the quadtree idea to divide the image into blocks, divides the smooth area of the low-resolution image into image blocks with the sizes of 64 x 64 and 32 x 32, and divides the non-smooth image area containing detail information into image blocks with the size of 4*4, thereby shortening the dictionary training time by reducing the number of training samples on the basis of ensuring the accuracy of the dictionary training;
(2) When the decoding end reconstructs a decoded image, the smooth image block is directly reconstructed by using a bicubic interpolation mode, and the image feature block containing detail information is reconstructed by using a sparse representation mode, so that the high-resolution image reconstruction time is shortened on the premise of ensuring the subjective quality of image reconstruction;
(3) The invention carries out joint dictionary training on high-resolution image blocks and low-resolution image feature blocks, adds additional information contained in images into an initial dictionary trained by an external image library, updates the training dictionary and obtains a training dictionary D corresponding to the corresponding high-resolution image blocks H Corresponding to low resolution image feature blocksTraining dictionary D L Features contained in the dictionary are more in line with the image reconstruction requirements, and the image reconstruction quality is further improved.
Drawings
FIG. 1 is a block diagram of a sparse representation based image compression and reconstruction method of the present invention;
FIG. 2 is a block partitioning diagram of a quadtree of the present invention;
FIG. 3 is a schematic diagram of the image quadtree block division result of the present invention;
fig. 4 is a schematic diagram of the 16×16 image block division result according to the present invention.
Detailed Description
The invention is further explained with reference to the drawings and examples, wherein fig. 1 is a block diagram of image compression and reconstruction based on sparse representation, and the main steps include:
step 1: taking 640X 640 images as high-resolution images X to be compressed, and performing 4 times downsampling on the high-resolution images X to obtain images Y with spatial resolution of 160X 160 L ；
Step 2: for image Y L Performing bicubic upsampling with a sampling factor of 4 to obtain an image Y with a resolution of 640 x 640 H ；
Further, for image Y H The concrete operation of block division by using the quadtree idea is as follows:
image Y H The CTU is a basic unit of image block division, and each CTU can be selectively divided into three sub-image blocks (CUs) of 64 x 64, 32 x 32 and 16 x 16, and fig. 2 is a schematic diagram of quadtree block division. The threshold T of block division is defined by the whole 640 x 640 image Y H The average value of the gray differences between adjacent pixels is determined, namely:
in the aboveAnd->Respectively are images Y H The average value of the gray differences between the i-th row and j-th column adjacent pixels is calculated as follows:
for each 64×64ctu, the block partitioning procedure is as follows:
(1) Calculating the average value T of gray differences between adjacent pixels of each CTU CTU And compared with a blocking threshold T. When T is CTU >T, dividing the CTU into four CUs of 32×32, and executing step (2); otherwise, the CTU does not continue to divide, i.e. the size of 64×64;
(2) Sequentially calculating the average value T of gray differences between 4 adjacent pixels of 32X 32CU CU And compared with a blocking threshold T. When T is CU >T, dividing the CU into 4 16 x 16 CUs; otherwise, the CU does not continue to partition, i.e. the size is 32 x 32.
FIG. 3 is a schematic diagram showing the division result of the image quadtree blocks. For Y H After the above division of all CTUs, Y H Is divided into CUs of sizes 64 x 64 and 32 x 32, and the non-smooth region is divided into CUs of 16 x 16.
Further, for the image blocks with the size of 16×16, the image blocks are further divided into a plurality of image blocks with the size of 4*4, and k (0 < k < 4) pixels are overlapped on adjacent image blocks. When k=1, that is, adjacent image blocks overlap by one pixel, the image block division result is that as shown in fig. 4, an image block with a size of 16×16 is divided into 25 image blocks 4*4. The following 4 one-dimensional filters are used for the image block of 4*4 to extract its features:
f 1 ＝[-1,0,1],f 2 ＝f 1 T ,f 3 ＝[1,0,-2,0,1],f 4 ＝f 3 T where T represents the transpose. Applying the four filters to obtain 4 eigenvectors of each image block, and connecting the 4 eigenvectors into a 64 x 1 eigenvector, denoted as image eigenvector F L 。
Step 3: original high resolution image block X to be compressed H Dividing:
utilizing a low resolution image Y according to a one-to-one correspondence of high-low resolution image blocks H Corresponding division is made for the high resolution image X, the smooth region is divided into 64X 64 and 32X 32 image blocks, the non-smooth region is divided into a plurality of 4*4 image blocks, and the adjacent image blocks have k (0<k<4) Overlap of the individual pixels.
Step 4: training high resolution image blocks X separately using K-SVD (K-th singular value decomposition) H And a low resolution image feature block F L Corresponding dictionary D H And D L 。
Further, for the low resolution image feature block F L Corresponding dictionary D L Is trained as follows:
obtaining the low-resolution image characteristic block F from the step 2 L As dictionary D L Training samples, all samples are formed into a sample dataset matrix F. The sample dataset matrix F may be composed of a dictionary matrix D L And coefficient matrix A, the aim of dictionary training is to find dictionary matrix D L And a coefficient matrix a. The linearity is expressed as:
F≈D L A
the dictionary learning described above may be translated into solving the following optimization problem:
epsilon is the maximum value allowed by reconstruction errors, and the solving process of the optimization problem is as follows:
(1) Dictionary published in Image super-resolution via sparse representation of Yang as initial D L . At a known D L Under the condition, the sparse coefficient of each sample is obtained to obtain the D L A corresponding coefficient matrix A;
(2) Under the condition of the known A, the D is updated by adopting a K-SVD method L K-SVD updates the dictionary by adopting a column-by-column updating mode, and when the kth column d of the dictionary is updated k When the other columns of the dictionary are fixed, the current error matrix E is calculated:
adjustment d k And alpha k The error is made as small as possible, and a training dictionary after the first iteration is obtained after each column of the initial dictionary is updated;
(3) Continuously updating the coefficient matrix and the training dictionary, and stopping iterative updating until the reconstruction error of the sample meets the requirement to obtain a low-resolution image feature block F L Corresponding training dictionary D L 。
Further, high resolution image block X H Corresponding dictionary D H Is trained as follows:
obtaining a high-resolution image block X with the size of 4*4 from the step 3 H Converted into a column vector of 16X 1, all X H Forming a matrix of sample data sets, and dictionary D L The training process is the same, and the K-SVD algorithm is utilized to obtain the high-resolution image block X H Corresponding dictionary D H 。
Step 5: at the encoding end, the low resolution image Y L Compression by JPEG2000 standard to obtain Y L And a high-low resolution dictionary D H And D L Packaging and transmitting to a decoding end;
step 6: de-multiplexing at decoding end to obtain low resolution image Y L Compressed code stream and high-low resolution dictionary D of (2) H And D L ；
Step 7: for low resolution image Y L The compressed code stream of (2) is decoded by JPEG2000 standard to obtain low resolution image Y' L The spatial resolution is 160 x 160, then for Y' L The image Y 'with 640 x 640 resolution is obtained by 4 times up-sampling' H ；
Step 8: for image Y' H The same block division as in step 2 is performed, the smooth region is divided into blocks with the sizes of 64 x 64 and 32 x 32, the non-smooth region is divided into blocks with the size of 4*4, and the characteristic is extracted to obtain an image characteristic block F' L ；
Step 9: for the image blocks divided in the step 8, the image blocks with the sizes of 64 x 64 and 32 x 32 are directly used as reconstructed image blocks by using the 4 times up-sampling result in the step 7; for image feature block F' L The reconstruction is carried out by using a sparse representation method, and the specific implementation mode is as follows:
(1) Using image feature blocks F' L And dictionary D L Can solve for the corresponding sparse coefficient α:
F′ L ＝D L α ||α|| 0 ≤N
wherein alpha is E R N N is the number of atoms in the sparse dictionary.
(2) Corresponding reconstructed high resolution image block X' H Using dictionary D H And the solved sparse coefficient alpha are linearly combined to represent:
X′ H ＝D H α ||α|| 0 ≤N
step 10: splicing all the reconstructed image blocks to corresponding positions to obtain a reconstructed high-resolution image X 0 Then, the iterative back projection technology is utilized for X 0 Global optimization is carried out to remove possible saw tooth or ringing effect of the reconstructed image, and a final reconstructed image X is obtained * And satisfies reconstruction constraints:
finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (5)
1. The image compression and reconstruction method based on sparse representation is characterized by comprising the following steps of: the method is completed according to the following steps:
s1, downsampling an original high-resolution image X to be compressed to obtain a downsampled low-resolution image Y L ；
S2, for the low resolution image Y L Upsampling to obtain Y of the same resolution as X H Image Y is divided by using quadtree block division method H Block division is performed to divide the image Y H Dividing into image blocks of 64×64, 32×32 and 16×16, for image block of 16×16, further dividing into image block of 4*4 and extracting features to obtain low resolution image feature block F L ；
S3, utilizing the one-to-one correspondence of the high-low resolution image blocks and Y H Corresponding division is carried out on the high-resolution image X to obtain a high-resolution image block X after block division H ；
S4, respectively training high-resolution image blocks X by using K-SVD H Corresponding dictionary D H And a low resolution image feature block F L Corresponding dictionary D L ；
S5, coding the low-resolution image Y at the coding end L Compressing to obtain Y L And a high resolution dictionary D H And a low resolution dictionary D L Packaging and transmitting to a decoding end;
s6, firstly de-multiplexing the code stream at the decoding end to obtain a low-resolution image Y L Compressed code stream of (D) and high resolution dictionary D H And a low resolution dictionary D L ；
S7, pair Y L Decoding the compressed code stream of (2) to obtain low resolution image Y' L Then for Y' L Upsampling to obtain an image Y 'with the same resolution as the original image' H ；
S8, for the image Y' H Performing block division identical to S2, and extracting features of 4*4 image block to obtain corresponding image feature block F' L ；
S9, after block division is completed, the block divided by 64 x 64 and 32 x 32 is directly adopted as a reconstruction block by an up-sampling result in S7; for image feature block F' L Using sparse representation and high resolution dictionary D H And a low resolution dictionary D L Performing super-resolution reconstruction to obtain a reconstructed image block X' H ；
S10, splicing all reconstructed image blocks to corresponding positions to obtain a reconstructed high-resolution image X 0 Then, the iterative back projection technology is utilized for X 0 Global optimization is carried out to obtain a final reconstructed image X * ；
The up-sampling method in the step S2 is a bicubic up-sampling method;
in step S2, the image Y is divided by using a quadtree block division method H The specific implementation mode of the division is as follows:
s2.1, image Y H Divided into a plurality of mutually non-overlapping image blocks of size 64 x 64, called CTUs, which is Y H A divided basic unit;
s2.2, each CTU is selectively divided into three types of sub-image blocks CU 64×64, 32×32 and 16×16, wherein a CU with a size of 64×64 and 32×32 corresponds to Y H Is a smooth region of CU size 16 x 16 corresponds to Y H Is a non-smooth region of (1);
s2.3, determining a threshold T of block division, wherein the specific method comprises the following steps:
for an image Y of size m x n H The average value of the gray differences between the adjacent pixels in the ith row and the jth column is respectivelyAndthe calculation formula is as follows:
the threshold T of the block division is defined by the whole m x n image Y H The average value of the gray differences between adjacent pixels is determined, namely:
s2.4, for each CTU of 64×64, the block division flow is as follows:
s2.4.1 calculating the average value T of gray differences between adjacent pixels of each CTU according to the same method as S2.3 CTU And comparing with a blocking threshold T; when T is CTU >T, dividing the CTU into four CUs of 32×32, and executing step S2.4.2; otherwise, the CTU does not continue to divide, i.e. the size of 64×64;
s2.4.2 calculating average value T of gray differences between adjacent pixels of 4 32X 32CU according to the same method as S2.3 CU And compared with a blocking threshold T when T CU >T, dividing the CU into 4 16 x 16 CUs; otherwise, the CU does not continue to partition, i.e. the size is 32×32;
step S2, for the image block with the size of 16 x 16, further dividing into image blocks with the size of 4*4, extracting features, and obtaining a low-resolution image feature block F L The specific method of (2) is as follows:
s2.5, for a CU 16 x 16, further dividing into a plurality of 4*4 image blocks, where adjacent 4*4 image blocks have k pixels overlapping, 0< k <4;
s2.6, extracting the characteristics of the image block of 4*4 by adopting the following 4 one-dimensional filters;
f 1 ＝[-1,0,1],f 2 ＝f 1 T ,f 3 ＝[1,0,-2,0,1],f 4 ＝f 3 T
where T represents the transpose, applying the four filters to obtain 4 feature vectors for each image block, and concatenating the 4 feature vectors into a 64 x 1 feature vector, denoted as image feature block F L 。
2. The method according to claim 1, characterized in that: s4, obtaining a low-resolution image feature block F by using K-SVD training L Corresponding dictionary D L The specific implementation mode of the method is as follows:
s4.1 using all F L Forming a sample dataset matrix F;
s4.2, matrix F is formed by dictionary matrix D L And a coefficient matrix a, the linear representation being:
F≈D L A
the dictionary training process described above translates into solving the following optimization problem:
epsilon is the maximum allowed by reconstruction errors;
s4.3, solving the optimization problem as follows:
s4.3.1, select initial D L ；
S4.3.2, in the known D L Under the condition, the sparse coefficient of each sample is obtained to obtain the D L A corresponding coefficient matrix A;
s4.3.3 under the condition of the known A, adopting a K-SVD method to update D L ；
S4.3.4 repeating steps S4.3.2 and S4.3.3 until the reconstruction error meets the requirements, at which point D L Namely F L A corresponding training dictionary.
3. The method according to claim 2, characterized in that: in step S4, training high resolution image block X using K-SVD H Corresponding dictionary D H In the process of (a), an image block X with a size of 4*4 is processed H Converted into a column vector of 16X 1, all X H Forming a matrix of sample data sets, and dictionary D L The dictionary D is obtained by using K-SVD as in the training process H 。
4. The method according to claim 1, characterized in that:
step S9 utilizes sparse representation method and high resolution dictionary D H And a low resolution dictionary D L The super-resolution reconstruction process comprises the following specific steps:
s9.1 for each image feature block F' L Using dictionary D L The corresponding sparse coefficient alpha is solved, and the calculation formula is as follows:
F′ L ＝D L α ||α|| 0 ≤N
wherein alpha is E R N N is the number of atoms in the sparse dictionary;
s9.2 corresponding reconstructed high resolution image block X' H Using dictionary D H And the solved sparse coefficient alpha are linearly combined to represent:
X′ H ＝D H α ||α|| 0 ≤N。
5. the method according to claim 1, characterized in that: step S10 of utilizing iterative back projection method to X 0 The specific steps for global optimization are as follows:
s10.1 reconstruction of high resolution image X 0 Downsampling D and blurring H, simulating a high-resolution image degradation process, and solving a degraded image X' t And a low resolution image Y L Error of (2) to reconstruct high resolution image X t Performing iterative updating; the calculation formula of the iterative back projection method is as follows:
X t+1 ＝X t +((Y L -X′ t )↑s)*p
wherein X 'is' t ＝DHX t ，X t For the high-resolution image of the t iteration process, p is a back projection filter, ∈r is upsampling, and S is an upsampling factor;
s10.2, when the final reconstructed high resolution image X * Stopping iterative updating when the reconstruction constraint is met, wherein the reconstruction constraint formula is as follows:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910430149.6A CN110148087B (en) | 2019-05-22 | 2019-05-22 | Image compression and reconstruction method based on sparse representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910430149.6A CN110148087B (en) | 2019-05-22 | 2019-05-22 | Image compression and reconstruction method based on sparse representation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110148087A CN110148087A (en) | 2019-08-20 |
| CN110148087B true CN110148087B (en) | 2023-08-04 |
Family
ID=67592697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910430149.6A Active CN110148087B (en) | 2019-05-22 | 2019-05-22 | Image compression and reconstruction method based on sparse representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110148087B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114519661A (en) * | 2022-02-10 | 2022-05-20 | Oppo广东移动通信有限公司 | Image processing method, device, terminal and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581687A (en) * | 2013-09-11 | 2014-02-12 | 北京交通大学长三角研究院 | Self-adaptive depth image coding method based on compressed sensing |
| EP2846533A1 (en) * | 2013-09-10 | 2015-03-11 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Device for compressive image acquisition |
| CN105554501A (en) * | 2015-05-03 | 2016-05-04 | 李峰 | Image acquisition and compression method and device |
| CN106101725A (en) * | 2016-06-28 | 2016-11-09 | 电子科技大学 | A kind of based on compressive sensing theory with the method for compressing image of spatial domain down-sampling technology |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2552323B (en) * | 2016-07-18 | 2020-04-29 | Imagination Tech Ltd | Mip map compression |
-
2019
- 2019-05-22 CN CN201910430149.6A patent/CN110148087B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2846533A1 (en) * | 2013-09-10 | 2015-03-11 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Device for compressive image acquisition |
| CN103581687A (en) * | 2013-09-11 | 2014-02-12 | 北京交通大学长三角研究院 | Self-adaptive depth image coding method based on compressed sensing |
| CN105554501A (en) * | 2015-05-03 | 2016-05-04 | 李峰 | Image acquisition and compression method and device |
| CN106101725A (en) * | 2016-06-28 | 2016-11-09 | 电子科技大学 | A kind of based on compressive sensing theory with the method for compressing image of spatial domain down-sampling technology |
Non-Patent Citations (1)
| Title |
|---|
| 图像压缩算法发展概述;王明伟等;《空间电子技术》;20160425;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110148087A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110087092B (en) | Low-bit-rate video coding and decoding method based on image reconstruction convolutional neural network | |
| CN104378644B (en) | Image compression method and device for fixed-width variable-length pixel sample string matching enhancement | |
| CN107018422B (en) | Still image compression method based on depth convolutional neural networks | |
| CN102164282B (en) | Coefficient-random-permutation-based compressive sensing method and system for image coding | |
| CN110351568A (en) | A kind of filtering video loop device based on depth convolutional network | |
| CN109889846B (en) | Method and device for compressing and decompressing Demura table data and coding and decoding system | |
| CN110062232A (en) | A kind of video-frequency compression method and system based on super-resolution | |
| CN109547784A (en) | A kind of coding, coding/decoding method and device | |
| CN103473797B (en) | Spatial domain based on compressed sensing sampling data correction can downscaled images reconstructing method | |
| US20230100615A1 (en) | Video processing method and apparatus, and device, decoder, system and storage medium | |
| CN111885280A (en) | Hybrid convolutional neural network video coding loop filtering method | |
| CN115131675A (en) | Remote sensing image compression method and system based on reference image texture migration | |
| CN109922339A (en) | In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology | |
| CN111726614A (en) | HEVC (high efficiency video coding) optimization method based on spatial domain downsampling and deep learning reconstruction | |
| CN113132729B (en) | Loop filtering method based on multiple reference frames and electronic device | |
| CN105590296B (en) | A kind of single-frame images Super-Resolution method based on doubledictionary study | |
| CN109257608B (en) | Image processing method, device and system | |
| CN110148087B (en) | Image compression and reconstruction method based on sparse representation | |
| CN114640857A (en) | Image compression method and 3D noise reduction method based on improved Huffman coding | |
| CN110677644A (en) | Video coding and decoding method and video coding intra-frame predictor | |
| CN110728726B (en) | Image compression method based on user interaction and deep neural network | |
| CN111080729A (en) | Method and system for constructing training picture compression network based on Attention mechanism | |
| CN115131254A (en) | Constant bit rate compressed video quality enhancement method based on two-domain learning | |
| CN113822801B (en) | Compressed video super-resolution reconstruction method based on multi-branch convolutional neural network | |
| CN105611288B (en) | A kind of low bit rate image sequence coding method based on Constrained interpolation technique |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230414 Address after: 518000 1002, Building A, Zhiyun Industrial Park, No. 13, Huaxing Road, Henglang Community, Longhua District, Shenzhen, Guangdong Province Applicant after: Shenzhen Wanzhida Technology Co.,Ltd. Address before: 310018 no.1158, No.2 street, Baiyang street, Hangzhou Economic and Technological Development Zone, Zhejiang Province Applicant before: HANGZHOU DIANZI University |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230630 Address after: Room 207, Building 6, Science and Technology Creative Park, Technopole, Yuelu Mountain University, Changsha City, 410000, Hunan Province Applicant after: Hunan Huakai Digital Technology Co.,Ltd. Address before: 518000 1002, Building A, Zhiyun Industrial Park, No. 13, Huaxing Road, Henglang Community, Longhua District, Shenzhen, Guangdong Province Applicant before: Shenzhen Wanzhida Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |