CN109922339A - In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology - Google Patents
In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology Download PDFInfo
- Publication number
- CN109922339A CN109922339A CN201711314711.6A CN201711314711A CN109922339A CN 109922339 A CN109922339 A CN 109922339A CN 201711314711 A CN201711314711 A CN 201711314711A CN 109922339 A CN109922339 A CN 109922339A
- Authority
- CN
- China
- Prior art keywords
- sampling
- image
- block
- super
- rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses the image coding frameworks of a kind of combination multi-sampling rate down-sampling and super-resolution rebuilding technology.Mainly comprise the steps that image to be compressed is divided into 32 × 32 block, then to each piece of progress down-sampling under plurality of sampling rates；Rough JPEG encoding and decoding are carried out to every kind of down-sampling block, and up-sample decoded low resolution block；Each piece of optimal sample rate and quantization parameter is selected by rate-distortion optimization；According to selected parameter, using JPEG to each piece of progress encoding and decoding, and decoding block is rebuild using the super-resolution technique based on deep learning；Reconstructed block is combined into image.Higher compression ratio can may be implemented and under identical decoded image quality in the distortion performance that all-key rate section promotes JPEG in the present invention.This method can be adapted for a variety of mainstream image encoders, realize the efficient storage and transmission of image data.
Description
Technical field
The present invention relates to image super-resolution rebuildings and Image Compression, and in particular to adopts under a kind of combination multi-sampling rate
The image coding framework of sample and super-resolution rebuilding technology, belongs to field of picture communication.
Background technique
Compression of images is a kind of to reduce the image procossing skill of data volume by reducing the redundancy between raw image data
Art, the problems such as can solve the memory space inadequate and limited transmission bandwidth of vision facilities.JPEG is encoded as the image of mainstream
One of standard, since it has many advantages, such as lower computation complexity, so that it is in the lossy compressions neck such as network and wireless communication
Domain is widely used.However, the compression performance of JPEG will be greatly reduced in the case where limited bit rate, so that decoding image
There are serious pinch effects, to reduce the visual quality of compressed images.
Universal with high definition and ultra high-definition equipment, demand of the people to the resolution ratio of image and video is higher and higher.So
And it is limited to higher equipment cost and different use environments etc., so that the image and video quality that obtain, which are not able to satisfy still, to be needed
It asks.Super-resolution rebuilding is a kind of technology for carrying out increase resolution to known low-resolution image by way of software, can
To get higher-quality image under the premise of not needing to update hardware device, there is very high practicability.In recent years,
With the fast development of machine learning, the super resolution ratio reconstruction method based on deep learning enters the public visual field.Compared to
Traditional super-resolution rebuilding technology, the super-resolution rebuilding based on deep learning can obtain higher-quality image, simultaneously
In phase of regeneration, has and rebuild speed faster.
Down-sampling is carried out to image to be compressed in coding side, while obtaining original point by super-resolution technique in decoding end
The scheme of resolution image solves the problems, such as that existing compression standard is ineffective in low bit- rate section to a certain extent.But it should
Scheme is only applicable to compared with low bit- rate section, and with the promotion of code rate, compression performance, which has, significantly to be declined, therefore is actually being answered
There is significant limitation in.
Summary of the invention
The image coding framework of combination multi-sampling rate down-sampling proposed by the present invention and super-resolution rebuilding technology, for mentioning
JPEG coding standard is risen in the distortion performance of all-key rate section.The frame proposed simultaneously can also be applied to other third parties and compile
Code device is to promote its coding efficiency.
The image coding framework of combination multi-sampling rate down-sampling proposed by the present invention and super-resolution rebuilding technology, it is main to wrap
Include following operating procedure:
(1) original image is subjected to piecemeal by 32 × 32 sizes, the down-sampling of plurality of sampling rates is then carried out to each piece；
(2) the pre- encoding and decoding of more quantization parameters are carried out to the fritter after down-sampling, and after statistic quantification DCT coefficient non-zero
The number of value；
(3) decoded image fritter is interpolated into original resolution with interpolation method, and calculates reconstructed block and original block
Between mean square error；
(4) rate-distortion optimization algorithm is used, optimal sample rate and its corresponding down-sampling mode and quantization parameter are selected；
(5) according to the optional sampling mode and quantization parameter of acquisition, using JPEG to original picture block carry out down-sampling with
Encoding and decoding；
(6) model gone out by using the super-resolution rebuilding technique drill based on deep learning, to decoded image
Fritter carries out super-resolution rebuilding；
(7) it combines the image block of reconstruction according to corresponding mode, forms final decoding image.
Detailed description of the invention
Fig. 1 is the block diagram for the image coding framework that the present invention combines multi-sampling rate down-sampling and super-resolution rebuilding technology
Fig. 2 is the specific aim model training process of the super resolution ratio reconstruction method based on deep learning and its block diagram of reconstruction
Fig. 3 is that the distortion performance of the present invention and JPEG compression algorithm to ' Bike ' test image in all-key rate section compares
Fig. 4 is that the distortion performance of the present invention and JPEG compression algorithm to ' Lena ' test image in all-key rate section compares
Fig. 5 is the vision of ' Lena ' decoding image of the present invention and JPEG in low bit- rate section and when code rate is all 0.2bpp
Effect compares: figure (a) is that JPEG decodes image, and figure (b) is present invention decoding image
Fig. 6 is the view of ' Bike ' decoding image of the present invention and JPEG in middle high code rate section and when code rate is all 1.06bpp
Feel that effect compares: figure (a) is that JPEG decodes image, and figure (b) is present invention decoding image
Specific embodiment
The present invention will be further explained below with reference to the attached drawings:
In Fig. 1, in conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology, can specifically it be divided into
Seven steps below:
(1) original image is subjected to piecemeal by 32 × 32 sizes, the down-sampling of plurality of sampling rates is then carried out to each piece；
(2) the pre- encoding and decoding of more quantization parameters are carried out to the fritter after down-sampling, and after statistic quantification DCT coefficient non-zero
The number of value；
(3) decoded image fritter is interpolated into original resolution with interpolation method, and calculates reconstructed block and original block
Between mean square error；
(4) rate-distortion optimization algorithm is used, optimal sample rate and its corresponding down-sampling mode and quantization parameter are selected；
(5) according to the optional sampling mode and quantization parameter of acquisition, using JPEG to original picture block carry out down-sampling with
Encoding and decoding；
(6) the multiple models gone out by using the super-resolution rebuilding technique drill based on deep learning, to decoded
Image fritter carries out super-resolution rebuilding；
(7) it combines the image block of reconstruction according to corresponding mode, forms final decoding image.
Specifically, in the step (1), by original image segmentation to be compressed at the image for 32 × 32 sizes not overlapped
Block；Then it is directed to each image block, devises 9 kinds of different sample rates (the down-sampling modes different corresponding to 16 kinds) table 1
In give plurality of sampling rates and its corresponding down-sampling mode.Down-sampling is all made of the optimal down-sampling side based on least square
Case, whole process can be formulated as
Wherein, Hh,vFor low resolution blockCorresponding up-sampling matrix, h and v respectively indicate horizontal and vertical direction
On sample rate, Y is original picture block.Optimal down-sampling block according to least-squares algorithm, under available current sample rate
For
When current procedures carry out down-sampling, need to find out the up-sampling matrix under all sample rates, and use above-mentioned public affairs
Formula obtains the optimal down-sampling block under different sample rates.
1 sample rate of table and its sampling configuration
In the step (2), 8 different quantization parameters are given, to distribute more bits to each down-sampling block,
To weaken the loss of down-sampling process bring high-frequency information, standard JPEG then is carried out to the image block of 16 kinds of different resolutions
Encoding and decoding.In an encoding process, the number (NZC) for counting nonzero value of each image fritter DCT coefficient after quantization, is used for
Code rate in step (4) in rate-distortion optimization is to reduce calculation amount.Meanwhile it is worth noting that pre- encoding and decoding in current procedures
It does not need to carry out Huffman encoding and decoding, need to only carry out the positive and negative transform and quantization of DCT and inverse quantization processes.It is used in the present invention
Quantization parameter QP is
Wherein QF is that present image carries out the quality factor used when standard JPEG coding.
In the step (3), decoded down-sampled images fritter is up-sampled by simple interpolation method, and
The mean square error (MSE) between image block and original picture block after calculating up-sampling, as in rate-distortion optimization in step (4)
Distortion value.
In the step (4), in conjunction under the different sample rates and quantization parameter being calculated in step (2) and step (3)
NZC and MSE, by Rate-distortion optimization method obtain current image block optimal sample rate and quantization parameter, rate distortion it is excellent
Change process is defined as
Min (J), wtihJ=MSE+ λ NZC
The smallest cost function value under plurality of sampling rates and quantization parameter is calculated, the corresponding sample rate of the smallest J is selected
Under sampling configuration and quantization parameter, as optimal sampling configuration and quantization parameter.Simultaneously in optimization process, MSE and amount
It is directly proportional to change factor Q F, and NZC and quantizing factor QF are inversely proportional, is fitted by mass data and obtains its functional relation, and led to
Crossing the value for asking local derviation to obtain Lagrange multiplier λ is
In the step (5), the down-sampling rate selected using step (4) adopt to each 32 × 32 image block
Then sample carries out encoding and decoding under selected quantization parameter to the image fritter after down-sampling using JPEG, in encoding-decoding process
In, use the quantization parameter of step (4) acquisition as the quality factor of JPEG.
In the step (6), using the super-resolution rebuilding technology based on deep learning to decoded image fritter into
Row super-resolution rebuilding obtains the final image block of corresponding 32 × 32 size.Fig. 2 gives super resolution ratio reconstruction method instruction
Experienced and reconstruction process block diagram.
In the training stage, the training image of input is subjected to down-sampling under same sample rate, this process does not consider more
Then sample rate mode carries out JPEG to the training image after down-sampling at different Q P to simplify the network model trained
Encoding and decoding, then by interpolation method up-sample decoding image, obtain with an equal amount of degraded image of training image, finally will be former
Degraded image after beginning training image and up-sampling carries out dictionary respectively as the high-resolution and low-resolution image during model training
Training uses VDSR (Super Resolution Using Very Deep Convolutional in the training stage
Networks) algorithm is trained at each QP.Each decoded image fritter is quantified by it in phase of regeneration
Parameter QP selects corresponding network model to carry out super-resolution rebuilding.
In the step (7), the image block rebuild in step (6) is carried out accordingly according to the partitioned mode in step (1)
Combination, form final decoding image.
Two width gray level images ' Bike ' are selected at random from test picture library, ' Lena ' is tested with above-mentioned steps, and with
Joint Photographic Experts Group compares distortion performance and decodes the visual quality of image.Fig. 3 and Fig. 4 gives the rate distortion of two width pictures
Can, wherein horizontal axis is code rate, and unit is bpp；The longitudinal axis is Y-PSNR (PSNR), and unit is dB.When code rate is identical, PSNR
It is higher, illustrate that distortion performance is better.Fig. 5 is when code rate is 0.2bpp, and standard JPEG and the present invention compress ' Lena ' and tie
The subjective vision effect contrast figure of fruit.Fig. 6 is when code rate is 1.06bpp, and standard JPEG and the present invention compress ' Bike ' and tie
The subjective vision effect contrast figure of fruit.Experimental result has universality to other test images.
Claims (5)
1. combining the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology, it is characterised in that including following step
It is rapid:
Step 1: original image is subjected to piecemeal by 32 × 32 sizes, the down-sampling of plurality of sampling rates is then carried out to each piece；
Step 2: carrying out the pre- encoding and decoding of more quantization parameters to the fritter after down-sampling, and after statistic quantification DCT coefficient non-zero
The number of value；
Step 3: decoded image fritter is interpolated into original resolution with interpolation method, and calculates reconstructed block and original block
Between mean square error；
Step 4: rate-distortion optimization algorithm is used, optimal sample rate and its corresponding down-sampling mode and quantization parameter are selected；
Step 5: according to the optional sampling mode and quantization parameter of acquisition, using JPEG to original picture block carry out down-sampling with
Encoding and decoding；
Step 6: the multiple models gone out by using the super-resolution rebuilding technique drill based on deep learning, to decoded
Image fritter carries out super-resolution rebuilding；
Step 7: combining the image block of reconstruction according to corresponding mode, forms final decoding image.
2. the image coding framework of combination multi-sampling rate down-sampling according to claim 1 and super-resolution rebuilding technology,
It is characterized in that described in step 1 carry out each 32 × 32 image block using least square method under plurality of sampling rates
Optimal down-sampling,
In formula, Y is original picture block,For the low resolution block after down-sampling, h and v respectively indicate horizontal and vertical direction
On sample rate, this method acquires the corresponding up-sampling matrix H under every kind of sample rateh,v, then solved using least square method
Up-sampling block and original block in above formula minimize the error problem,
Known optimal down-sampling block of the original picture block under different sample rates can be obtained by this method.
3. the image coding framework of combination multi-sampling rate down-sampling according to claim 1 and super-resolution rebuilding technology,
It is characterized in that walking pre- encoding-decoding process constructed in two and step 3, compiled in the pre- encoding-decoding process without Huffman
Code only carries out the positive and negative transform and quantization of DCT and inverse quantization processes.
4. the image coding framework of combination multi-sampling rate down-sampling according to claim 1 and super-resolution rebuilding technology,
It is characterized in that being that rate distortion used in each image block selection optional sampling rate and quantization parameter is excellent described in step 4
Change algorithm, the purpose of rate-distortion optimization is to find so that the smallest mode of majorized function J,
Min (J), wtih J=MSE+ λ NZC
In formula, MSE and NZC be respectively the non-zero quantised DCT coefficient obtained in step 2 and step 3 under each pattern with
And the mean square error between reconstructed block and original block；Simultaneously in optimization process, MSE is directly proportional to quantizing factor QF, and NZC with
Quantizing factor QF is inversely proportional, and is fitted by mass data and obtains its functional relation, and by asking local derviation to obtain Lagrange multiplier λ
Value be
Wherein QF is that present image carries out the quality factor used when standard JPEG coding.
5. the image coding framework of combination multi-sampling rate down-sampling according to claim 1 and super-resolution rebuilding technology,
It is characterized in that use described in step 6 is based on the super-resolution rebuilding technology of deep learning to decoded down-sampling block
Super-resolution rebuilding: training stage is carried out, down-sampling is carried out to training image under same sample rate, then uses standard JPEG
Then encoding and decoding use VDSR by this process come the encoding-decoding process under sample rates different in phantom frame and quantization parameter
(Super Resolution Using Very Deep Convolutional Networks) algorithm trains decoded drop
Mapping relations between matter image and original image generate convolutional network model；When rebuilding, joined according to the quantization of current decoding block
Number, selects corresponding network model to be rebuild.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711314711.6A CN109922339A (en) | 2017-12-12 | 2017-12-12 | In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711314711.6A CN109922339A (en) | 2017-12-12 | 2017-12-12 | In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109922339A true CN109922339A (en) | 2019-06-21 |
Family
ID=66957354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711314711.6A Pending CN109922339A (en) | 2017-12-12 | 2017-12-12 | In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109922339A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110740350A (en) * | 2019-10-31 | 2020-01-31 | 北京金山云网络技术有限公司 | Image processing method, image processing device, terminal equipment and computer readable storage medium |
| CN112449140A (en) * | 2019-08-29 | 2021-03-05 | 华为技术有限公司 | Video super-resolution processing method and device |
| CN112767503A (en) * | 2021-01-15 | 2021-05-07 | 北京航空航天大学 | JPEG compression coding-based digital pathology full-slice image rapid analysis method |
| CN117058002A (en) * | 2023-10-12 | 2023-11-14 | 深圳云天畅想信息科技有限公司 | Video frame super-resolution reconstruction method and device and computer equipment |
| CN117221547A (en) * | 2023-11-07 | 2023-12-12 | 四川新视创伟超高清科技有限公司 | CTU-level downsampling-based 8K video coding method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060120614A1 (en) * | 2004-12-08 | 2006-06-08 | Markus Flierl | Method for spatially scalable video coding |
| CN103607591A (en) * | 2013-10-28 | 2014-02-26 | 四川大学 | Image compression method combining super-resolution reconstruction |
| CN105392009A (en) * | 2015-11-27 | 2016-03-09 | 四川大学 | Low bit rate image coding method based on block self-adaptive sampling and super-resolution reconstruction |
| CN107018422A (en) * | 2017-04-27 | 2017-08-04 | 四川大学 | Still image compression method based on depth convolutional neural networks |
-
2017
- 2017-12-12 CN CN201711314711.6A patent/CN109922339A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060120614A1 (en) * | 2004-12-08 | 2006-06-08 | Markus Flierl | Method for spatially scalable video coding |
| CN103607591A (en) * | 2013-10-28 | 2014-02-26 | 四川大学 | Image compression method combining super-resolution reconstruction |
| CN105392009A (en) * | 2015-11-27 | 2016-03-09 | 四川大学 | Low bit rate image coding method based on block self-adaptive sampling and super-resolution reconstruction |
| CN107018422A (en) * | 2017-04-27 | 2017-08-04 | 四川大学 | Still image compression method based on depth convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 何小海: "集成超分辨率重建的图像压缩编码新型框架及其实现", 《数据采集与处理》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112449140A (en) * | 2019-08-29 | 2021-03-05 | 华为技术有限公司 | Video super-resolution processing method and device |
| CN112449140B (en) * | 2019-08-29 | 2021-09-14 | 华为技术有限公司 | Video super-resolution processing method and device |
| US11922599B2 (en) | 2019-08-29 | 2024-03-05 | Huawei Technologies Co., Ltd. | Video super-resolution processing method and apparatus |
| CN110740350A (en) * | 2019-10-31 | 2020-01-31 | 北京金山云网络技术有限公司 | Image processing method, image processing device, terminal equipment and computer readable storage medium |
| CN112767503A (en) * | 2021-01-15 | 2021-05-07 | 北京航空航天大学 | JPEG compression coding-based digital pathology full-slice image rapid analysis method |
| CN112767503B (en) * | 2021-01-15 | 2022-08-12 | 北京航空航天大学 | JPEG compression coding-based digital pathology full-slice image rapid analysis method |
| CN117058002A (en) * | 2023-10-12 | 2023-11-14 | 深圳云天畅想信息科技有限公司 | Video frame super-resolution reconstruction method and device and computer equipment |
| CN117058002B (en) * | 2023-10-12 | 2024-02-02 | 深圳云天畅想信息科技有限公司 | Video frame super-resolution reconstruction method and device and computer equipment |
| CN117221547A (en) * | 2023-11-07 | 2023-12-12 | 四川新视创伟超高清科技有限公司 | CTU-level downsampling-based 8K video coding method and device |
| CN117221547B (en) * | 2023-11-07 | 2024-01-23 | 四川新视创伟超高清科技有限公司 | CTU-level downsampling-based 8K video coding method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107018422B (en) | Still image compression method based on depth convolutional neural networks | |
| CN109495741B (en) | Image compression method based on self-adaptive down-sampling and deep learning | |
| CN110087092B (en) | Low-bit-rate video coding and decoding method based on image reconstruction convolutional neural network | |
| CN109922339A (en) | In conjunction with the image coding framework of multi-sampling rate down-sampling and super-resolution rebuilding technology | |
| CN103607591B (en) | Video image compression method combining super-resolution reconstruction | |
| CN105392009B (en) | Low bit rate image sequence coding method based on block adaptive sampling and super-resolution rebuilding | |
| CN110751597B (en) | Video super-resolution method based on coding damage repair | |
| CN103489203A (en) | Image coding method and system based on dictionary learning | |
| WO2020238439A1 (en) | Video quality-of-service enhancement method under restricted bandwidth of wireless ad hoc network | |
| CN102281446B (en) | Visual-perception-characteristic-based quantification method in distributed video coding | |
| CN101668196B (en) | Low code rate image compression method based on down sampling and interpolation | |
| JP5301097B2 (en) | Transform domain sub-sampling for video transcoding | |
| KR102312337B1 (en) | AI encoding apparatus and operating method for the same, and AI decoding apparatus and operating method for the same | |
| WO2013067949A1 (en) | Matrix encoding method and device thereof, and matrix decoding method and device thereof | |
| CN113079378B (en) | Image processing method and device and electronic equipment | |
| WO2023000179A1 (en) | Video super-resolution network, and video super-resolution, encoding and decoding processing method and device | |
| CN110545426B (en) | Spatial domain scalable video coding method based on coding damage repair (CNN) | |
| CN101964910B (en) | Video spatial resolution conversion method based on code-rate type transcoding assistance | |
| CN111726614A (en) | HEVC (high efficiency video coding) optimization method based on spatial domain downsampling and deep learning reconstruction | |
| CN104780383B (en) | A kind of 3D HEVC multi-resolution video coding methods | |
| CN111726638A (en) | HEVC (high efficiency video coding) optimization method combining decompression effect and super-resolution | |
| CN111080729B (en) | Training picture compression network construction method and system based on Attention mechanism | |
| CN102724495A (en) | Wyner-Ziv frame quantification method based on rate distortion | |
| CN106559668B (en) | A kind of low code rate image compression method based on intelligent quantization technology | |
| Tan et al. | Image compression algorithms based on super-resolution reconstruction technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190621 |