CN101710993B - Block-based self-adaptive super-resolution video processing method and system - Google Patents
Block-based self-adaptive super-resolution video processing method and system Download PDFInfo
- Publication number
- CN101710993B CN101710993B CN 200910241349 CN200910241349A CN101710993B CN 101710993 B CN101710993 B CN 101710993B CN 200910241349 CN200910241349 CN 200910241349 CN 200910241349 A CN200910241349 A CN 200910241349A CN 101710993 B CN101710993 B CN 101710993B
- Authority
- CN
- China
- Prior art keywords
- sampling
- piece
- block
- rate distortion
- decoded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses block-based self-adaptive super-resolution video processing method and system. The method comprises the following steps of: selecting a block-based down-sampling mode for a video image; carrying out down-sampling on the current video image according to the down-sampling mode and acquiring a down-sampling block; selecting a basis rate anamorphic coding mode by combining a block coding forecasting mode and the down-sampling mode to code the acquired down-sampling block; decoding the coded down-sampling block, acquiring a decoded down-sampling block, carrying out type division on the decoded down-sampling block and carrying out up-sampling on the down-sampling block on the basis of the type of the down-sampling block to acquire an up-sampling reestablishment block. The invention can improve the compression efficiency of a vide code at a middle and low code rate and ensure that the code performance is not lower than the performance of the traditional mixed code at a high code rate at the same time.
Description
Technical field
The present invention relates to data compression and image processing field, relate in particular to a kind of block-based self-adaptive super-resolution video processing method and system.
Background technology
Along with network and development of Communication Technique, the increasing needs of vision signal pass through Network Transmission.But under the situation of limited bandwidth, network is merely able to transmit the encoded video of low code check.General method only reduces code check through the adjustment quantization parameter, and this will inevitably cause the reduction of video encoding quality.Therefore, improve the quality of hanging down encoded video under the code check and receive increasing concern.Developing into of super resolution technology addresses this problem, and a kind of feasible method is provided.Super-resolution technique is meant the nonredundancy information of utilizing multiframe to exist to exist between the low-resolution image of sub-pix displacement, generates a frame or multiframe high-definition picture.Along with the proposition of the super-resolution algorithms of some low complex degrees, super-resolution has also obtained application gradually in video compression.
Existing down-sampling coding techniques will carry out the area characteristic information of down-sampling coding usually encodes as load information, rebuilds to adopt corresponding up-sampling technology at coding side.These load informations are unfavorable for improving code efficiency.
Summary of the invention
The object of the present invention is to provide a kind of block-based self-adaptive super-resolution video processing method and system.Based on the present invention, the compression efficiency of video coding can guarantee that under the high code check, coding efficiency is not less than the performance of traditional mixing coding framework simultaneously in can improving, under the low code check.
The self-adaptive super-resolution video processing method that the present invention is based on piece comprises the steps: that the down-sampling pattern confirms step, to video image, selects block-based down-sampling pattern; The down-sampling step according to said down-sampling pattern, is carried out down-sampling to current video image, obtains the down-sampling piece; Coding step, combined block coded prediction pattern and said down-sampling pattern are selected the coding mode according to rate distortion, and the said down-sampling piece that obtains is encoded; Decoding step is decoded to the down-sampling piece of coding, obtains decoded down-sampling piece, and it is carried out type divide, and based on the type of said down-sampling piece, said down-sampling piece is carried out up-sampling, obtains the up-sampling reconstructed block; Wherein, in the said coding step, said selection is specially according to the coding mode of rate distortion: confirm the set of first rate distortion and second rate distortion, said first rate distortion is not for carrying out down-sampling, the resulting rate distortion of direct coding down-sampling piece; The set of said second rate distortion comprises that a plurality of second rate distortions, said second rate distortion are the different down-sampling pattern of employing to the encode rate distortion that produces and said down-sampling piece carried out the rate distortion sum that up-sampling produced of said down-sampling piece; The minimum pairing coding mode of rate distortion is selected in relatively first rate distortion and second rate distortion set.
In above-mentioned self-adaptive super-resolution video processing method, in the preferred said down-sampling step, said down-sampling pattern comprises: pattern 0 is to mode 3.Wherein, pattern 0 is the maintainance block original pixels; Pattern 1 is the vertical direction down-sampling, obtains 16 * 8 piece; Pattern 2 is the horizontal direction down-sampling, obtains 8 * 16 piece; Mode 3 is horizontal direction and vertical direction while down-sampling, obtains 8 * 8 piece.
In above-mentioned self-adaptive super-resolution video processing method; In the preferred said decoding step; The said division that the down-sampling piece is carried out type comprises the steps: the 1-norm calculation step of AC coefficient; Calculate in the DCT coefficient of j 8 * 8 of down-sampling piece of said decoding and rebuilding the 1-norm S of AC coefficient
jThe smooth block determining step is judged in the DCT coefficient of individual 8 * 8 of j the 1-norm S of AC coefficient
jWhether less than threshold value T
F,If,, then j 8 * 8 is smooth block; Otherwise, be moving mass or non-motion texture block; Wherein, said threshold value T
F,For being provided with in advance; Moving mass, non-motion texture block determining step, down-sampling piece after computes decoded is rebuild and the movement differential between the predict blocks judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not, said j 8 * 8 is non-motion texture block.
In above-mentioned self-adaptive super-resolution video processing method; In the preferred said decoding step; According to following rule said down-sampling piece is carried out up-sampling, obtain the up-sampling reconstructed block:, utilize spatial domain interpolation algorithm decoding and rebuilding down-sampling piece if individual 8 * 8 of j is smooth block; If j 8 * 8 is non-motion texture block, utilize the edge to keep interpolation algorithm to rebuild the down-sampling piece of decoding; If j 8 * 8 is moving mass, utilize the super-resolution rebuilding algorithm to rebuild the down-sampling piece of decoding; Wherein, said super-resolution rebuilding algorithm is:
The low-resolution image piece that said down-sampling piece is formed projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y′)-R(x″，y″)＝R(x，y)+e(x′，y′)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the said high-definition picture piece, SR (x y) is the value of the missing pixel of rebuilding.; R (x; Y) be the pixel value .g that lacks in the corresponding current reconstructed block in the reference image block of deriving by the motion vector of down-sampling piece (x '; Y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x "; y ") be and g (x ', y ') corresponding reference piece in pixel value.
In above-mentioned self-adaptive super-resolution video processing method, in the preferred said rate distortion calculation procedure, according to said second rate distortion of rate-distortion optimization Model Calculation, said rate-distortion optimization model is:
JD
i＝D
i+λ×R
i+D
u
Wherein, D
iThe distortion of presentation code down-sampling piece, R
iThe bit number that presentation code down-sampling piece is used, D
uDistortion in the expression up-sampling reconstructed block between missing pixel and original pixels.I is the down-sampling coding mode.
On the other hand, the present invention also provides a kind of block-based adaptive super-resolution processing system for video, comprising: down-sampling pattern determination module, down sample module, coding module, decoder module, rate distortion computing module.Wherein, be used for, select block-based down-sampling pattern video image; Be used for current video image being carried out down-sampling, obtain the down-sampling piece according to said down-sampling pattern; Coding module is used for combined block coded prediction pattern and said down-sampling pattern, selects the coding mode according to rate distortion, and the said down-sampling piece that obtains is encoded; Decoder module is used for the down-sampling piece of coding is decoded, and obtains said down-sampling piece, and said down-sampling piece is carried out type divide, and based on the type of said down-sampling piece, said down-sampling piece is carried out decoding and rebuilding; Wherein, in the said coding module, said selection is specially according to the coding mode of rate distortion: confirm the set of first rate distortion and second rate distortion, said first rate distortion is not for carrying out down-sampling, the resulting rate distortion of direct coding down-sampling piece; The set of said second rate distortion comprises that a plurality of second rate distortions, said second rate distortion are the different down-sampling pattern of employing to the encode rate distortion that produces and said down-sampling piece carried out the rate distortion sum that up-sampling produced of said down-sampling piece; The minimum pairing coding mode of rate distortion is selected in relatively first rate distortion and second rate distortion set.In above-mentioned adaptive super-resolution processing system for video, preferred said down-sampling pattern comprises pattern 0 to mode 3, and wherein, pattern 0 is the maintainance block original pixels; Pattern 1 is the vertical direction down-sampling, obtains 16 * 8 piece; Pattern 2 is the horizontal direction down-sampling, obtains 8 * 16 piece; Mode 3 is horizontal direction and vertical direction while down-sampling, obtains 8 * 8 piece.
In above-mentioned adaptive super-resolution processing system for video; In the preferred said decoder module; Comprise type division submodule; Said type is divided submodule and comprised: the 1-norm calculation unit of AC coefficient is used for calculating j 8 * 8 DCT coefficient of the down-sampling piece of said decoding and rebuilding, the 1-norm S of AC coefficient
jThe smooth block judging unit is used for judging j 8 * 8 DCT coefficient, the 1-norm S of AC coefficient
jWhether less than threshold value T
F,If,, then j 8 * 8 is smooth block; Otherwise, be moving mass or non-motion texture block; Wherein, said threshold value T
F,For being provided with in advance; Moving mass, non-motion texture block judging unit, down-sampling piece after being used for computes decoded and rebuilding and the movement differential between the predict blocks judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not, said j 8 * 8 is non-motion texture block, wherein, and said threshold interval [T
Ml, T
Mu] be provided with in advance.
In above-mentioned adaptive super-resolution processing system for video; Preferred said decoder module comprises up-sampling reconstruction submodule; Said reconstruction submodule is rebuild said down-sampling piece according to following rule: if j 8 * 8 is smooth block, utilize spatial domain interpolation algorithm decoding and rebuilding down-sampling piece; If j 8 * 8 is non-motion texture block, utilize the edge to keep interpolation algorithm to rebuild the down-sampling piece of decoding; If j 8 * 8 is moving mass, utilize the super-resolution rebuilding algorithm to rebuild the down-sampling piece of decoding; And said super-resolution rebuilding algorithm is: the low-resolution image piece that said down-sampling piece is formed projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y′)-R(x″，y″)＝R(x，y)+e(x′，y′)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the said high-definition picture piece, SR (x y) is the value of the missing pixel of rebuilding.; R (x; Y) be the pixel value .g that lacks in the corresponding current reconstructed block in the reference image block of deriving by the motion vector of down-sampling piece (x '; Y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x "; y ") be and g (x ', y ') corresponding reference piece in pixel value.
In above-mentioned adaptive super-resolution processing system for video, in the preferred said coding module, according to said second rate distortion of rate-distortion optimization Model Calculation, said rate-distortion optimization model is:
JD
i＝D
i+λ×R
i+D
u
Wherein, D
iThe distortion of presentation code down-sampling piece, R
iThe bit number that presentation code down-sampling piece is used, D
uDistortion in the expression up-sampling reconstructed block between missing pixel and original pixels.I is the down-sampling coding mode.
The present invention has following beneficial effect:
The first, utilization is selected the down-sampling pattern based on the optimum mode decision algorithm of rate distortion, has set up new rate-distortion model, and distortion (Distortion) part has comprised the distortion of the missing pixel of rebuilding, and has guaranteed that coding efficiency can not reduce.
The second, utilize reconstructed blocks characteristics of signals behind the coding, with 8 * 8 be base unit, vision signal is divided into smoothly, move and three kinds of signals of texture region of non-motion.Smooth region utilizes the DCT coefficient feature of 8 * 8 correspondences to judge, moving region and non-motion texture region utilize the movement differential of reconstructed blocks and predict blocks to judge.This method has been avoided the side information of coding zones of different video signal characteristic, has improved coding efficiency.
Three, when high-resolution reconstruction, the present invention has adopted based on the block type adaptive super-resolution reconstructing method, adopts simple interpolation algorithm for smooth region; Adopt super-resolution technique to rebuild in the moving region; At the texture region of non-motion, the interpolation algorithm that the present invention has adopted the edge to keep has kept the image high-frequency information preferably, has not only improved the subjective and objective quality of encode video image, has also reduced the complexity of algorithm simultaneously.
Description of drawings
Fig. 1 the present invention is based on the flow chart of steps of the self-adaptive super-resolution video processing method of piece for basis;
Fig. 2 the present invention is based in the self-adaptive super-resolution video processing method of piece for basis, when carrying out decoding step, and the flow chart of steps that block type is divided;
Fig. 3 the present invention is based in the self-adaptive super-resolution video processing method of piece for basis, when carrying out decoding step, and low complex degree super-resolution rebuilding algorithm sketch map;
Fig. 4 a is that resolution is 1080p, the RD curve synoptic diagram of cycle tests Bluesky;
Fig. 4 b is that resolution is 1080p, the RD curve synoptic diagram of cycle tests Pedestrian_area;
Fig. 5 the present invention is based on the simple structure block diagram of the adaptive super-resolution processing system for video of piece for basis;
Fig. 6 the present invention is based in the adaptive super-resolution processing system for video of piece for basis, and the type of decoder module is divided the simple structure block diagram of submodule.
Embodiment
For make above-mentioned purpose of the present invention, feature and advantage can be more obviously understandable, below in conjunction with accompanying drawing and embodiment the embodiment of the invention done further detailed explanation.
Basic thought of the present invention: utilize picture characteristics to select suitable down-sampling pattern and corresponding method for reconstructing; The piece of the video image behind the coding down-sampling; Up-sampling is rebuild the image block of original resolution during decoding, thereby reaches the purpose that improves the encoding compression performance.
With reference to Fig. 1, Fig. 1 comprises the steps: for according to the flow chart of steps that the present invention is based on the self-adaptive super-resolution video processing method of piece
Wherein, in the step 130, said selection is specially according to the coding mode of rate distortion:
Confirm the set of first rate distortion and second rate distortion, said first rate distortion is not for carrying out down-sampling, the resulting rate distortion of direct coding down-sampling piece; The set of said second rate distortion comprises that a plurality of second rate distortions, said second rate distortion are the different down-sampling pattern of employing to the encode rate distortion that produces and said down-sampling piece carried out the rate distortion sum that up-sampling produced of said down-sampling piece; The minimum pairing coding mode of rate distortion is selected in relatively first rate distortion and second rate distortion set.
For example said process is described.
The first, calculate first rate distortion.
The coding original block, rate distortion is respectively R
1, D
1, first rate distortion is D
1+ λ * R
1
The two rate distortions set of the second, calculating
The set of second rate distortion comprises a plurality of second rate distortions.For a certain down-sampling pattern, the down-sampling piece is encoded the encoding rate distortion R of down-sampling piece
2, D
2, but the distortion D of the down-sampling piece also has up-sampling reconstructed block time disappearance part
3, promptly second rate distortion is D
2+ D
3+ λ * R
2Calculate pairing second rate distortion of each down-sampling pattern, obtain the set of second rate distortion.
Three, confirm minimum rate distortion corresponding codes pattern
Confirm rate distortion minimum in second distortion set, and rate distortion that should minimum compares with said first rate distortion, determine the rate distortion of minimum, thereby can confirm should minimum rate distortion corresponding codes pattern.
With the example that is embodied as on the AVS reference software, introduce implementation of the present invention in detail below.
About step 110, step 120 and step 130, the coding of down-sampling and down-sampling piece
As shown in table 1, the present invention has defined down-sampling pattern in 4, and down-sampling is not carried out in pattern 0 expression, pattern 1～3, and horizontal down-sampling, vertical down-sampling and both direction down-sampling are simultaneously adopted in expression respectively.With the example that is embodied as on AVS reference software P frame, with 3 kinds of down-sampling patterns, associating block encoding predictive mode, as shown in table 2, respectively these 8 kinds of patterns are encoded.For newly-increased down-sampling pattern, no longer proceed mode division, P_L0_16x8_down for example, the present invention only divides the piece of this 16x8 and encodes, and comprises the motion vector of 1 16x8 and the DCT coefficient of 2 8x8 pieces.
The block-based down-sampling pattern of table 1
| The down-sampling | Pattern description | |
| 0 | The maintainance block |
|
| 1 | The vertical direction down-sampling obtains 16 * 8 | |
| 2 | The horizontal direction down-sampling obtains 8 * 16 | |
| 3 | Level and vertical direction be down-sampling simultaneously, obtains 8 * 8 |
Block mode in the block-based adaptive super-resolution video coding of table 2
| Block mode | Describe |
| 0 | |
| 1 | P_L0_L016x8 |
| 2 | P_L0_L08x16 |
| 3 | P_8x8 |
| 4 | P_8x8ref0 |
| 5 | P_L0_16x8_down |
| 6 | P_L0_8x16_down |
| 7 | P_L0_8x8_down |
About in the step 140, the division of block type
Rebuilding in view of piece is on decoded vision signal, to carry out; Therefore the present invention directly analyzes the characteristic of decoding and rebuilding piece; 8x8 piece in the video image down-sampling piece is divided into 3 types: smooth block, the texture block of moving mass and non-motion (non-moving mass, but comprise the piece of more radio-frequency component).The algorithm flow chart that block type is divided is as shown in Figure 2, comprises the steps:
Step 210: calculate in the DCT coefficient of j 8 * 8 of down-sampling piece of said decoding and rebuilding the 1-norm S of AC coefficient
j
Step 220: judge in the DCT coefficient of individual 8 * 8 of j the 1-norm S of AC coefficient
jWhether less than threshold value T
F,If,, then j 8 * 8 is smooth block; If not, execution in step 230.
Step 230: down-sampling piece after computes decoded is rebuild and the movement differential between the predict blocks.
Step 240: judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not, said j 8 * 8 is non-motion texture block.
Be elaborated in the face of above-mentioned steps down.The 1-norm S of AC coefficient in the DCT coefficient of each the 8x8 piece in the down-sampling piece that computes decoded is rebuild
j,
Wherein, j is the index of 8x8 piece in the down-sampling piece, and i is the coefficient index that scans according to zig-zag in the 8x8 piece.If S
jLess than preset threshold T
F,Judge that this 8x8 piece is a smooth block; Otherwise, be non-flat slide block.For non-flat slide block, the present invention has further done the segmentation of moving mass and non-moving mass, calculates the movement differential of reconstructed block and predict blocks,
Y wherein
J, k, lBe the reconstructed block signal, X
J, k, lIt is the predict blocks of motion vector points.If SAD
jAt preset threshold [T
Ml, T
Mu] in the scope, judge that current block is a moving mass, otherwise, be non-motion texture block.
Because the division of video signal characteristic of the present invention is an information of utilizing the decoding and reconstituting image, so any additional side information of can encoding, decoding end can utilize identical judgment criterion to make the judgement of block type.
About in the step 140, the super-resolution rebuilding of down-sampling piece
For three kinds of above-mentioned different video picture signals, the super-resolution algorithms reconstruct high-resolution piece that the present invention samples different respectively.For smooth block,,, utilized bilinear interpolation like the present invention so utilize simple spatial domain interpolation algorithm just can well rebuild the piece of original resolution owing to mainly comprise low-frequency component; For the texture region of non-motion, interpolation algorithm (S.Battiato, G.Gallom and F.Stanco that the present invention has utilized the edge to keep; " Alocally-adaptive zooming algorithm for digital images, " Image andVision Computing, Vol.20; No.11, pp.805-812, September 20021; 2) detail textures that, has well kept reconstructed blocks; For moving mass, owing to can from reference block, obtain movable information more accurately, help the reconstruction of super-resolution algorithms, so for the reconstruction of such down-sampling piece, the present invention has adopted the super-resolution rebuilding algorithm of low complex degree; Super-resolution algorithms is as shown in Figure 3, and the low-resolution image piece that at first down-sampling is obtained projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y′)-R(x″，y″)＝R(x，y)+e(x′，y′) (3)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the high-resolution piece; SR (x, y) be the missing pixel of rebuilding value .R (x, y) be the pixel value .g that lacks in the corresponding current reconstructed block in the reference image block of deriving by the motion vector of down-sampling piece (x '; Y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x "; y ") be and g (x ', y ') corresponding reference piece in pixel value.
Image block to different characteristic adopts different super-resolution algorithms, not only can improve the reconstruction quality of integral image, also can reduce the complexity of encoding and decoding simultaneously.
The rate-distortion optimization model of down-sampling piece
Owing to introduced the down-sampling block mode, the present invention has set up the rate-distortion optimization model of down-sampling piece,
JD
i＝D
i+λ×R
i+D
u (4)
Wherein, D
iThe distortion of presentation code down-sampling piece, R
iThe bit number that presentation code down-sampling piece is used, D
uDistortion in the expression up-sampling reconstructed block between missing pixel and original pixels.I is the down-sampling coding mode.
Original other 5 kinds of block modes among the associating AVS are selected the optimum block mode of RD at coding side, realize adaptive super-resolution coding.
Fig. 4 and table 3 have listed the result that the present invention realizes on AVS reference software P frame, its intermediate-resolution is respectively 720p and 1080p, and reference frame number is that 2, the first frames are I frame coding, and all the other are P frame coding.
With reference to Fig. 4 a, Fig. 4 b.Fig. 4 a is that resolution is 1080p, the RD curve synoptic diagram of cycle tests Bluesky; Fig. 4 b is that resolution is 1080p, the RD curve synoptic diagram of cycle tests Pedestrian_area.Transverse axis is encoder bit rate (kbps of unit) among the figure, and the longitudinal axis is image Y-PSNR (dB of unit), and curve a, c are original AVS coding efficiency, and curve b, c are the curve that has adopted the adaptive down-sampling coding techniques.As can be seen from Figure 4 the block-based adaptive super-resolution method for video coding of the present invention's proposition not only can significantly improve coding efficiency under the low code check; Also can improve simultaneously the coding efficiency under the high code check; This is because the present invention has adopted the rate-distortion optimization model as down-sampling mode decision criterion, has overcome D.Barreto, L.Alvarez; R.Molina; The method of A.Katsaggelos and the G.Callic ó shortcoming that performance reduces under high code check situation has guaranteed coding efficiency stability.In table 3, listed the result who on multisequencing more, tests, experiment shows that the present invention can effectively improve video coding performance.
Table 3 test result
The present invention adopt the super-resolution algorithms of low complex degree, simultaneously for non-moving region, adopted the lower interpolation algorithm of complexity, reduced the complexity of whole encoding and decoding framework, meet the real-time requirement of video coding.
Though the present invention is on the P of AVS reference software frame, it can be equally applicable to other encoding and decoding platform, as H.264/AVC, and VC-1 etc.The present invention is applicable to the coding of I frame and B frame too.
With reference to Fig. 5, Fig. 5 is according to the simple structure block diagram that the present invention is based on the adaptive super-resolution processing system for video of piece, comprising:
Down-sampling pattern determination module 510 is used for current video image, selects block-based current down-sampling pattern.
Down sample module 520 is used for according to said current down-sampling pattern current video image being carried out down-sampling, obtains the down-sampling piece.
Coding module 530 is used for combined block coded prediction pattern and said down-sampling pattern, and the said down-sampling piece that obtains is encoded.
Decoder module 540 is used for the down-sampling piece of coding is decoded, and obtains said down-sampling piece, and said down-sampling piece is carried out type divide, and based on the type of said down-sampling piece, said down-sampling piece is carried out decoding and rebuilding.
In coding module 530; Selection is specially according to the coding mode of rate distortion: calculate first rate distortion and second rate distortion; First rate distortion is not for carrying out down-sampling; The resulting rate distortion of direct coding down-sampling piece, second rate distortion is for encoding to the down-sampling piece and the down-sampling piece being carried out the rate distortion that said up-sampling reconstructed block that up-sampling obtains is produced; Relatively first rate distortion and second rate distortion are selected the less pairing coding mode of rate distortion.
Wherein, said down-sampling pattern comprises:
Pattern 0: maintainance block original pixels;
Pattern 1: the vertical direction down-sampling obtains 16 * 8 piece;
Pattern 2: the horizontal direction down-sampling obtains 8 * 16 piece;
Mode 3: horizontal direction and vertical direction be down-sampling simultaneously, obtains 8 * 8 piece.
With reference to Fig. 6, Fig. 6 the present invention is based in the adaptive super-resolution processing system for video of piece for basis, and the type of decoder module is divided the simple structure block diagram of submodule.
The type of decoder module is divided submodule and is comprised 1-norm calculation unit 610, smooth block judging unit 620 and the moving mass of AC coefficient, non-motion texture block judging unit 630.The 1-norm calculation unit 610 of AC coefficient is used for calculating j 8 * 8 DCT coefficient of the down-sampling piece of said decoding and rebuilding, the 1-norm S of AC coefficient
iSmooth block judging unit 620 is used for judging j 8 * 8 DCT coefficient, the 1-norm S of AC coefficient
iWhether less than threshold value T
F,If,, then j 8 * 8 is smooth block; Otherwise, be moving mass or non-motion texture block; Wherein, said threshold value T
F,For being provided with in advance; Down-sampling piece that moving mass, non-motion texture block judging unit 630 are used for computes decoded after rebuilding and the movement differential between the predict blocks judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not, said j 8 * 8 is non-motion texture block, wherein, and said threshold interval [T
Ml, T
Mu] be provided with in advance.
In the said decoder module, according to following regular reconstructing undersampled:, utilize spatial domain interpolation algorithm decoding and rebuilding down-sampling piece if j 8 * 8 is smooth block; If j 8 * 8 is non-motion texture block, utilize the edge to keep interpolation algorithm to rebuild the down-sampling piece of decoding; If j 8 * 8 is moving mass, utilize the super-resolution rebuilding algorithm to rebuild the down-sampling piece of decoding; And said super-resolution rebuilding algorithm is: the low-resolution image piece that said down-sampling piece is formed projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y)-R(x″，y″)＝R(x，y)+e(x′，y′)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the said high-definition picture piece, SR (x y) is the value of the missing pixel of rebuilding.; R (x; Y) be the pixel value .g that lacks in the corresponding current reconstructed block in the reference image block of deriving by the motion vector of down-sampling piece (x '; Y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x "; y ") be and g (x ', y ') corresponding reference piece in pixel value.
In the rate distortion computing module, according to the said rate distortion of rate-distortion optimization Model Calculation, said rate-distortion optimization model is:
JD
i＝D
i+λ×R
i+D
u
Wherein, D
iThe distortion of presentation code down-sampling piece, R
iThe bit number that presentation code down-sampling piece is used, D
uDistortion in the expression up-sampling reconstructed block between missing pixel and original pixels.I is the down-sampling coding mode.
More than the adaptive super-resolution processing system for video that the present invention is based on piece has been done simple explanation, its principle is similar with block-based self-adaptive super-resolution video processing method, the explanation that relevant part can the reference mass evaluation method is repeated no more at this.
More than a kind of block-based self-adaptive super-resolution video processing method provided by the present invention and system have been carried out detailed introduction; Used concrete example among this paper principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that on embodiment and range of application, all can change, in sum, this description should not be construed as limitation of the present invention.
Claims (8)
1. a block-based self-adaptive super-resolution video processing method is characterized in that, comprises the steps:
The down-sampling pattern is confirmed step, to video image, selects block-based down-sampling pattern;
The down-sampling step according to said down-sampling pattern, is carried out down-sampling to current video image, obtains the down-sampling piece;
Coding step, combined block coded prediction pattern and said down-sampling pattern are selected the coding mode according to rate distortion, and the said down-sampling piece that obtains is encoded;
Decoding step is decoded to the down-sampling piece of coding, obtains decoded down-sampling piece; And said decoded down-sampling piece is carried out type divide; Based on the type of said decoded down-sampling piece, said decoded down-sampling piece is carried out up-sampling, obtain the up-sampling reconstructed block;
Wherein, in the said coding step, said selection is specially according to the coding mode of rate distortion:
Confirm the set of first rate distortion and second rate distortion, said first rate distortion is not for carrying out down-sampling, the resulting rate distortion of the original block of the said current video image of direct coding; The set of said second rate distortion comprises that a plurality of second rate distortions, said second rate distortion are the different down-sampling pattern of employing to the encode rate distortion that produces and said decoded down-sampling piece carried out the rate distortion sum that up-sampling produced of the down-sampling piece that obtains in the said down-sampling step;
The minimum pairing coding mode of rate distortion is selected in relatively first rate distortion and second rate distortion set.
2. self-adaptive super-resolution video processing method according to claim 1 is characterized in that, in the said down-sampling step, said down-sampling pattern comprises:
Pattern 0: maintainance block original pixels;
Pattern 1: the vertical direction down-sampling obtains 16 * 8 piece;
Pattern 2: the horizontal direction down-sampling obtains 8 * 16 piece;
Mode 3: horizontal direction and vertical direction be down-sampling simultaneously, obtains 8 * 8 piece.
3. self-adaptive super-resolution video processing method according to claim 2 is characterized in that, in the said decoding step, the said division that the down-sampling piece is carried out type comprises the steps:
The 1-norm calculation step of AC coefficient is calculated in j 8 * 8 the DCT coefficient of said decoded down-sampling piece the 1-norm S of AC coefficient
j
The smooth block determining step is judged in the DCT coefficient of individual 8 * 8 of j the 1-norm S of AC coefficient
jWhether less than threshold value T
fIf,, then j 8 * 8 is smooth block; Otherwise, be moving mass or non-motion texture block; Wherein, said threshold value T
f, for being provided with in advance;
Moving mass, non-motion texture block determining step calculate the movement differential between said decoded down-sampling piece and the predict blocks, judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not,
Said j 8 * 8 is non-motion texture block.
4. self-adaptive super-resolution video processing method according to claim 3 is characterized in that, in the said decoding step, according to following rule said decoded down-sampling piece is carried out up-sampling, obtains the up-sampling reconstructed block:
If j 8 * 8 is smooth block, utilize the spatial domain interpolation algorithm to rebuild said decoded down-sampling piece;
If j 8 * 8 is non-motion texture block, utilize the edge to keep interpolation algorithm to rebuild said decoded down-sampling piece;
If j 8 * 8 is moving mass, utilize the super-resolution rebuilding algorithm to rebuild said decoded down-sampling piece;
Wherein, said super-resolution rebuilding algorithm is:
The low-resolution image piece that said decoded down-sampling piece is formed projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y′)-R(x″，y″)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the said high-definition picture piece, and (x y) is the value of the missing pixel of rebuilding to SR; R (, y) be the pixel value that lacks in the corresponding current reconstructed block in the reference image block of deriving by the motion vector of said decoded down-sampling piece; G (x ', y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x ", y ") be and g (x ', y ') corresponding reference piece in pixel value.
5. a block-based adaptive super-resolution processing system for video is characterized in that, comprising:
Down-sampling pattern determination module is used for video image, selects block-based down-sampling pattern;
Down sample module is used for according to said down-sampling pattern current video image being carried out down-sampling, obtains the down-sampling piece;
Coding module is used for combined block coded prediction pattern and said down-sampling pattern, selects the coding mode according to rate distortion, and the said down-sampling piece that obtains is encoded;
Decoder module is used for the down-sampling piece of coding is decoded, and obtains decoded down-sampling piece; And said decoded down-sampling piece is carried out type divide; Based on the type of said decoded down-sampling piece, said decoded down-sampling piece is carried out up-sampling, obtain the up-sampling reconstructed block;
Wherein, in the said coding module, said selection is specially according to the coding mode of rate distortion:
Confirm the set of first rate distortion and second rate distortion, said first rate distortion is not for carrying out down-sampling, the resulting rate distortion of the original block of the said current video image of direct coding; The set of said second rate distortion comprises that a plurality of second rate distortions, said second rate distortion are the different down-sampling pattern of employing to the encode rate distortion that produces and said decoded down-sampling piece carried out the rate distortion sum that up-sampling produced of the down-sampling piece that obtains in the said down-sampling step;
The minimum pairing coding mode of rate distortion is selected in relatively first rate distortion and second rate distortion set.
6. adaptive super-resolution processing system for video according to claim 5 is characterized in that, comprising: said down-sampling pattern comprises:
Pattern 0: maintainance block original pixels;
Pattern 1: the vertical direction down-sampling obtains 16 * 8 piece;
Pattern 2: the horizontal direction down-sampling obtains 8 * 16 piece;
Mode 3: horizontal direction and vertical direction be down-sampling simultaneously, obtains 8 * 8 piece.
7. adaptive super-resolution processing system for video according to claim 6 is characterized in that, in the said decoder module, comprises type division submodule, and said type is divided submodule and comprised:
The 1-norm calculation unit of AC coefficient is used for calculating j 8 * 8 DCT coefficient of said decoded down-sampling piece, the 1-norm S of AC coefficient
j
The smooth block judging unit is used for judging j 8 * 8 DCT coefficient, the 1-norm S of AC coefficient
jWhether less than threshold value T
f,, if then j 8 * 8 is smooth block; Otherwise, be moving mass or non-motion texture block; Wherein, said threshold value T
f, for being provided with in advance;
Moving mass, non-motion texture block judging unit are used to calculate the movement differential between said decoded down-sampling piece and the predict blocks, judge whether said movement differential is positioned at threshold interval [T
Ml, T
Mu], if said j 8 * 8 is moving mass, if not, said j 8 * 8 is non-motion texture block, wherein, and said threshold interval [T
Ml, T
Mu] be provided with in advance.
8. adaptive super-resolution processing system for video according to claim 7 is characterized in that, said decoder module comprises up-sampling reconstruction submodule, and said reconstruction submodule is rebuild said down-sampling piece according to following rule:
If j 8 * 8 is smooth block, utilize the spatial domain interpolation algorithm to rebuild said decoded down-sampling piece;
If j 8 * 8 is non-motion texture block, utilize the edge to keep interpolation algorithm to rebuild said decoded down-sampling piece;
If j 8 * 8 is moving mass, utilize the super-resolution rebuilding algorithm to rebuild said decoded down-sampling piece; And said super-resolution rebuilding algorithm is:
The low-resolution image piece that said decoded down-sampling piece is formed projects to the correspondence position of high-definition picture piece, and the pixel of disappearance position utilizes the respective pixel of reference block to rebuild, and reconstruction formula is following,
SR(x，y)＝R(x，y)+g(x′，y′)-R(x″，y″)
Wherein, (x y) is the coordinate of missing pixel to be rebuild in the said high-definition picture piece, and (x y) is the value of the missing pixel of rebuilding to SR; (x y) is the pixel value that lacks in the corresponding current reconstructed block in the reference image block of being derived by the motion vector of said decoded down-sampling piece to R; G (x ', y ') be the high-definition picture piece neutralization of current down-sampling piece before sampling (x, y) corresponding nearest neighbor pixels value, R (x ", y ") be and g (x ', y ') corresponding reference piece in pixel value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910241349 CN101710993B (en) | 2009-11-30 | 2009-11-30 | Block-based self-adaptive super-resolution video processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910241349 CN101710993B (en) | 2009-11-30 | 2009-11-30 | Block-based self-adaptive super-resolution video processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101710993A CN101710993A (en) | 2010-05-19 |
| CN101710993B true CN101710993B (en) | 2012-04-11 |
Family
ID=42403749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910241349 Active CN101710993B (en) | 2009-11-30 | 2009-11-30 | Block-based self-adaptive super-resolution video processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101710993B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201225654A (en) | 2010-12-13 | 2012-06-16 | Sonix Technology Co Ltd | Image accessing apparatus and image data transmission method thereof |
| WO2013155899A1 (en) * | 2012-04-16 | 2013-10-24 | Mediatek Inc. | Method and apparatus for sample adaptive offset coding with separate sign and magnitude |
| CN102800069A (en) * | 2012-05-22 | 2012-11-28 | 湖南大学 | Image super-resolution method for combining soft decision self-adaptation interpolation and bicubic interpolation |
| US10147202B2 (en) * | 2013-03-15 | 2018-12-04 | Arm Limited | Methods of and apparatus for encoding and decoding data |
| CN103268589A (en) * | 2013-05-10 | 2013-08-28 | 中科创达软件股份有限公司 | Somatosensory interaction method based on front-facing camera of mobile terminal |
| KR102085270B1 (en) * | 2013-08-12 | 2020-03-05 | 삼성전자 주식회사 | Method for selecting resolution with minimum distortion value and devices performing the method |
| CN103632359B (en) * | 2013-12-13 | 2016-03-30 | 清华大学深圳研究生院 | A kind of video super-resolution disposal route |
| CN103903236B (en) * | 2014-03-10 | 2016-08-31 | 北京信息科技大学 | The method and apparatus of face image super-resolution rebuilding |
| CN104780383B (en) * | 2015-02-02 | 2017-09-19 | 杭州电子科技大学 | A kind of 3D HEVC multi-resolution video coding methods |
| CN107343208B (en) * | 2016-04-29 | 2019-10-11 | 掌赢信息科技（上海）有限公司 | A kind of control video code rate method and electronic equipment |
| CN106791837B (en) * | 2016-12-15 | 2019-11-26 | 北京数码视讯科技股份有限公司 | The forerunner's analysis method and device of Video coding |
| CN107018429A (en) * | 2017-04-26 | 2017-08-04 | 陈翟 | Internet video data compression and frame picture display process |
| CN109218727B (en) * | 2017-06-30 | 2021-06-25 | 书法报视频媒体(湖北)有限公司 | Video processing method and device |
| CN109905717A (en) * | 2017-12-11 | 2019-06-18 | 四川大学 | A kind of H.264/AVC Encoding Optimization based on Space-time domain down-sampling and reconstruction |
| CN108184116A (en) * | 2017-12-18 | 2018-06-19 | 西南技术物理研究所 | A kind of image rebuilding method suitable for the transmission of wireless data chain |
| JP7269257B2 (en) * | 2018-04-13 | 2023-05-08 | コニンクリーケ・ケイピーエヌ・ナムローゼ・フェンノートシャップ | Frame-level super-resolution-based video coding |
| WO2019237659A1 (en) * | 2018-06-11 | 2019-12-19 | Suzhou Aqueti Technology Co., Ltd. | Blind compressive sampling method, apparatus and imaging system |
| CN109379592B (en) * | 2018-10-26 | 2021-04-30 | 北京北方安恒利数码技术有限公司 | Image encoding method and apparatus thereof |
| CN110278415B (en) * | 2019-07-02 | 2020-04-28 | 浙江大学 | Method for improving video quality of network camera |
| EP3997879A4 (en) * | 2019-07-09 | 2023-03-22 | Alibaba Group Holding Limited | Resolution-adaptive video coding |
| CN111698503B (en) * | 2020-06-22 | 2022-09-09 | 深圳市迪威码半导体有限公司 | Video high-power compression method based on preprocessing |
| CN114363617A (en) * | 2022-03-18 | 2022-04-15 | 武汉大学 | Network lightweight video stream transmission method, system and equipment |
| WO2023198074A1 (en) * | 2022-04-12 | 2023-10-19 | Zhejiang Dahua Technology Co., Ltd. | Video processing method, electronic device, and computer-readable storage medium |
| CN117058002B (en) * | 2023-10-12 | 2024-02-02 | 深圳云天畅想信息科技有限公司 | Video frame super-resolution reconstruction method and device and computer equipment |
| CN117221547B (en) * | 2023-11-07 | 2024-01-23 | 四川新视创伟超高清科技有限公司 | CTU-level downsampling-based 8K video coding method and device |
-
2009
- 2009-11-30 CN CN 200910241349 patent/CN101710993B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN101710993A (en) | 2010-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101710993B (en) | Block-based self-adaptive super-resolution video processing method and system | |
| CN101778275B (en) | Image processing method of self-adaptive time domain and spatial domain resolution ratio frame | |
| JP5044057B2 (en) | Method and apparatus for selecting a filter | |
| CN101175209B (en) | Adaptive de-blocking filtering method for mpeg video decoder | |
| KR100716999B1 (en) | Method for intra prediction using the symmetry of video, method and apparatus for encoding and decoding video using the same | |
| JP5503747B2 (en) | Image encoding device, image decoding device, image encoding method, and image decoding method | |
| CN102065298B (en) | High-performance macroblock coding implementation method | |
| CN106131554B (en) | The point self-adapted compensation method of HEVC quick sample product based on major side direction | |
| CN103607590B (en) | Efficient video coding based on structural similarity perceives Rate-distortion optimization method | |
| CN106507116B (en) | A kind of 3D-HEVC coding method predicted based on 3D conspicuousness information and View Synthesis | |
| CN108989815A (en) | The method and decoding apparatus for merging candidate list are generated for multiview video signal | |
| CN108712648B (en) | Rapid intra-frame coding method for depth video | |
| CN104539962A (en) | Layered video coding method fused with visual perception features | |
| CN106067979A (en) | For the device that image is decoded | |
| CN101325711A (en) | Method for controlling self-adaption code rate based on space-time shielding effect | |
| CN103596004A (en) | Intra-frame prediction method and device based on mathematical statistics and classification training in HEVC | |
| CN105306957A (en) | Adaptive loop filtering method and device | |
| CN106937116A (en) | Low-complexity video coding method based on random training set adaptive learning | |
| CN109327703A (en) | A kind of depth map decoding method based on different sampling blocks | |
| CN107864380A (en) | 3D HEVC fast intra-mode prediction decision-making techniques based on DCT | |
| CN106688238A (en) | Improved reference pixel selection and filtering for intra coding of depth map | |
| CN102917226B (en) | Intra-frame video coding method based on self-adaption downsampling and interpolation | |
| CN109587503A (en) | A kind of 3D-HEVC depth map intra-frame encoding mode high-speed decision method based on edge detection | |
| CN1194544C (en) | Video encoding method based on prediction time and space domain conerent movement vectors | |
| Hu et al. | Deep inter prediction with error-corrected auto-regressive network for video coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |